# NEURONAL STOCHASTIC VARIABILITY: INFLUENCES ON SPIKING DYNAMICS AND NETWORK ACTIVITY

EDITED BY: Mark D. McDonnell, Joshua H. Goldwyn and Benjamin Lindner PUBLISHED IN: Frontiers in Computational Neuroscience

#### *Frontiers Copyright Statement*

*© Copyright 2007-2016 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.*

*The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.*

*Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.*

*Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.*

*As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.*

> *All copyright, and all rights therein, are protected by national and international copyright laws.*

*The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.*

ISSN 1664-8714 ISBN 978-2-88919-884-9 DOI 10.3389/978-2-88919-884-9

## About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

## Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

## Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

## What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

## **NEURONAL STOCHASTIC VARIABILITY: INFLUENCES ON SPIKING DYNAMICS AND NETWORK ACTIVITY**

Topic Editors:

**Mark D. McDonnell,** University of South Australia, Australia **Joshua H. Goldwyn,** Ohio State University, USA **Benjamin Lindner,** Bernstein Center for Computational Neuroscience and Humboldt University Berlin, Germany

Stochastic fluctuations are intrinsic to and unavoidable at every stage of neural dynamics. For example, ion channels undergo random conformational changes, neurotransmitter release at synapses is discrete and probabilistic, and neural networks are embedded in spontaneous background activity.

The mathematical and computational tool sets contributing to our understanding of stochastic neural dynamics have expanded rapidly in recent years. New theories have emerged detailing the dynamics and computational power of the balanced state in recurrent networks. At the cellular level, novel stochastic extensions to the classical Hodgkin-Huxley model have enlarged our understanding of neuronal dynamics and action potential initiation. Analytical methods have been developed that allow for the calculation of the firing statistics of simplified phenomenological integrate-and-fire models, taking into account adaptation currents or temporal correlations of the noise.

This Research Topic is focused on identified physiological/internal noise sources and mechanisms. By "internal," we mean variability that is generated by intrinsic biophysical processes. This includes noise at a range of scales, from ion channels to synapses to neurons to networks.

The contributions in this Research Topic introduce innovative mathematical analysis and/or computational methods that relate to empirical measures of neural activity and illuminate the functional role of intrinsic noise in the brain.

**Citation:** McDonnell, M. D., Goldwyn, J. H., Lindner, B., eds. (2016). Neuronal Stochastic Variability: Influences on Spiking Dynamics and Network Activity. Lausanne: Frontiers Media. doi: 10.3389/978-2-88919-884-9

# Table of Contents

*05 Editorial: Neuronal Stochastic Variability: Influences on Spiking Dynamics and Network Activity* Mark D. McDonnell, Joshua H. Goldwyn and Benjamin Lindner

## **Theme 1: Stochastic Variability at the Single-Cell Level**


Cian O'Donnell and Mark C W van Rossum

*34 Diffusion approximation-based simulation of stochastic ion channels: which method to use?*

Danilo Pezo, Daniel Soudry and Patricio Orio

*49 Modeling the influence of short term depression in vesicle release and stochastic calcium channel gating on auditory nerve spontaneous firing statistics*

Bahar Moezzi, Nicolangelo Iannella and Mark D. McDonnell


Aurel A. Lazar and Yiyin Zhou

## **Theme 2: Influence of Variability on Network Dynamics**

*97 Self-consistent determination of the spike-train power spectrum in a neural network with sparse connectivity*

Benjamin Dummer, Stefan Wieland and Benjamin Lindner

*109 Structured chaos shapes spike-response noise entropy in balanced neural networks*

Guillaume Lajoie, Jean-Philippe Thivierge and Eric Shea-Brown


*132 Heterogeneity of heterogeneities in neuronal networks*

Fabiano Baroni and Alberto Mazzoni

*134 A Markov model for the temporal dynamics of balanced random networks of finite size*

Fereshteh Lagzi and Stefan Rotter

## Editorial: Neuronal Stochastic Variability: Influences on Spiking Dynamics and Network Activity

#### Mark D. McDonnell <sup>1</sup> \*, Joshua H. Goldwyn<sup>2</sup> and Benjamin Lindner 3, 4

*<sup>1</sup> Computational and Theoretical Neuroscience Laboratory, Institute for Telecommunications Research, University of South Australia, Mawson Lakes, SA, Australia, <sup>2</sup> Department of Mathematics, Ohio State University, Columbus, OH, USA, <sup>3</sup> Theory of Complex Systems and Neurophysics, Bernstein Center for Computational Neuroscience, Berlin, Germany, <sup>4</sup> Department of Physics, Humboldt University Berlin, Berlin, Germany*

Keywords: balanced network, channel noise, heterogeneity, Hodgkin-Huxley model, neural networks, neuronal variability, stochastic dynamics

## **The Editorial on the Research Topic**

## **Neuronal Stochastic Variability: Influences on Spiking Dynamics and Network Activity**

Stochastic variability is present across all scales of brain activity. At the single-cell level, for instance, synaptic transmission is mediated by stochastic release of neurotransmitter and membrane potentials fluctuate due to random conformational changes of ion channels. When these cell-level sources of stochastic variability emerge at the network level, they generate fluctuating currents that drive complex network dynamics. Even if intrinsic cellular noise sources are neglected, the interaction of many nonlinear units in recurrent networks typically leads to an effective network noise which is often mathematically tractable in a stochastic framework.

This Research Topic brings together works that address the pressing challenges of developing computational tools and mathematical theories that advance our understanding of stochastic neural dynamics. Six contributions cover stochastic variability at the single-cell level. Moezzi et al. study synaptic coupling between inner hair cells and auditory nerve fibers. Three works update our understanding of ion channel noise in stochastic versions of the Hodgkin-Huxley equations (O'Donnell and Van Rossum; Pezo et al.; Rowat and Greenwood). Puzerey and Galán quantify information transmission in a stochastic Hodgkin-Huxley neuron model that receives barrages of balanced excitatory and inhibitory inputs. Lazar and Zhou communicate a modeling framework that includes dendritic processing of noisy inputs and channel-noise influenced spike generation.

Edited and reviewed by: *Si Wu, Beijing Normal University, China*

#### \*Correspondence:

*Mark D. McDonnell mark.mcdonnell@unisa.edu.au*

Received: *27 February 2016* Accepted: *01 April 2016* Published: *21 April 2016*

#### Citation:

*McDonnell MD, Goldwyn JH and Lindner B (2016) Editorial: Neuronal Stochastic Variability: Influences on Spiking Dynamics and Network Activity. Front. Comput. Neurosci. 10:38.*

*doi: 10.3389/fncom.2016.00038*

The remaining four studies offer new perspectives on network dynamics. Dummer et al. works out the requirements for self-consistent input/output statistics for neurons embedded in recurrent networks. Lagzi and Rotter develop a Markov chain model that clarifies the stochastic dynamics of balanced networks. Mejias and Longtin explore effects of neural heterogeneity on network response properties. Lajoie et al. make elegant use of random dynamical systems theory to analyse stimulus encoding in in chaotic networks.

Two commentary articles are also part of this research topic: the commentary of Thomas on Lajoie et al. and the commentary of Baroni and Mazzoni on Mejias and Longtin.

## STOCHASTIC VARIABILITY IN SINGLE NEURON DYNAMICS

Moezzi et al. study the sub-cellular origins of long and short-term correlations in inter-spikeintervals (ISIs) in spontaneously firing individual fibers of the auditory nerve, and the form of the distribution of ISIs. They hypothesize the existence of a pre-synaptic mechanism in auditory inner hair cells that causes randomly delays in the availability of synaptic vesicles immediately following prior release of a vesicle. They propose a model for this that produces a simulated ISI distribution equivalent to those in standard existing models. Building on this, the authors tackle the subcellular source of ISI correlations. They report that introducing a model of ion channel noise in the calcium channels of pre-synaptic inner hair cells, when combined with the random vesicle availability model, enables simulation results from the overall model to match the qualitiative nature of ISIs in auditory nerve fiber spontaneous spiking.

Rowat and Greenwood analyse the distributions of interspike intervals generated from simulations of stochastic Hodgkin-Huxley models using several available algorithms. They conclude that appropriate use of the diffusion approximation (e.g., Goldwyn and Shea-Brown, 2011) can yield accurate estimations of these distributions when compared to "microscale" simulations of Markov chain models of ion channel kinetics. The authors draw on their previous studies of Hodgkin-Huxley and related models (Baxendale and Greenwood, 2011; Rowat and Greenwood, 2011) to explain features of the computed interspike interval distributions including local maxima ("bumps") and exponential shape in the tails of of these distributions.

Pezo et al. concur that diffusion approximations can provide accurate and efficient approximations to Markov chain models of channel noise. They caution, however, against using these methods for simulations of fewer than ∼1000 channels. In this low channel number regime, Markov chain methods can be used without sacrificing speed or accuracy. Pezo et al. make the important point that membrane area is subdivided into compartments when simulating dynamics of spatially-extended neurons. The local channel count within each membrane compartment can be small, and thus Markov chain methods may be preferred in multi-compartment computations. Taken together, the Pezo et al. and Rowat and Greenwood articles provide a set of benchmark simulations that will guide researchers interested in studying channel noise in neurons.

O'Donnell and Van Rossum remind us that channel noise depends on the channel type. In stochastic versions of the Hodgkin-Huxley model, for instance, membrane fluctuations are driven by Na<sup>+</sup> and K<sup>+</sup> channels. Differences in the properties of these ion channels include the probabilities of being in the open states, the time scales of channels opening and closing, and the maximal conductances per channel. O'Donnell and Van Rossum introduce a method to quantify how each channel type contributes to membrane potential fluctuations and how these fluctuations trigger spontaneous action potentials. Importantly, their method can be applied to any channel type in a conductance-based model (as they illustrate with an analysis of a model of CA1 hippocampal neuron). As such, the methods presented by O'Donnell and Van Rossum should be essential to researchers seeking to estimate membrane potential fluctuations induced by diverse channel types.

Puzerey and Galán study responses of the stochastic Hodgkin-Huxley model to barrages of excitatory and inhibitory inputs. They quantify information transmission (measured from estimates of spike train entropy) for a range of synaptic time scales and delays between excitation and lagging inhibition. They find that synaptic kinetics modulate information transmission the most when synaptic currents are balanced and delays in inhibition are small (0.8 ms). The framework of these simulations (neuron model with channel noise, driven by balanced and noisy synaptic inputs) lays a foundation for future studies that will integrate sources of stochastic variability at the neuron and network levels.

Lazar and Zhou seek to identify the theoretical limits of precision with which noisy sensory neurons can encode and decode stimuli. To this end, they introduce an innovative abstract neuron model, comprised from a dendritic module and a spikegenerating module, and study small circuits of such neuron models. The model exhibits a strong grounding in physiological detail; it includes active dendrites; it can use biophysical models such as the Hodgkin-Huxley model for spike generation; and most pertinently to this research topic, includes two intrinsic noise sources. The first source of noise is dendritic variability; the second is ion channel noise due to a finite number of channels (also the focus of O'Donnell and Van Rossum; Pezo et al.; Rowat and Greenwood in this topic). The authors provide extensive mathematical analysis and simulation results, and based on this argue that a duality between stimulus decoding and functional identification holds.

## STOCHASTIC VARIABILITY IN NEURON NETWORK DYNAMICS

The study by Dummer et al. explores an important condition for the temporal correlations of spiking neurons within homogeneous recurrent networks. In such networks, the mean activity of every unit is determined in a self-consistent manner, i.e., its mean input (coming from similar neurons) is simply related to its mean output. Likewise, temporal input correlations are proportional to temporal output correlations and this fact can be used to determine the spike-train correlation function or, equivalently, the spike-train power spectrum, from iterative simulations of a single neuron. Dummer et al. compare two such iterative schemes to simulations of a sparse recurrent network and find excellent agreement. Moreover, their study proves that the emergent network noise can be strongly colored and that the shape of the power spectrum (the "noise color") depends in a nontrivial way on cellular and network parameters.

Although single-cell spike train variability is an important proxy of network stochasticity, it does not necessarily give reliable estimates of the multi-cell spike-train variability that is important for encoding of time-dependent stimuli at the population level. Lajoie et al. study the multi-cell variability of a recurrent network of deterministic quadratic integrate-andfire neurons (a highly chaotic system) under the influence of a set of frozen-noise stimuli. Results from the theory of random dynamical systems are used to estimate an upper bound of the noise entropy. This entropy quantifies the variability caused by different initial conditions of the chaotic system. Lajoie et al. show that, surprisingly, this upper bound of the multi-cell noise

entropy is an order of magnitude lower than a naïve estimate that is based on the single cell's noise entropy. These results may pave the way for the estimation of the mutual information between multicellular spike patterns and complex spatio-temporal stimuli (see also the commentary article of Thomas).

A significant open question in understanding information processing and learning in mammalian cortex is whether the ubiquitous heterogeneity in the anatomy and physiology of cortical neurons and their synaptic connections are necessary for function, or makes little difference. Mejias and Longtin systematically investigate this question for a physiological parameter. They introduce heterogeneity into a standard population model of sparsely-connected excitatory and inhibitory cortical neurons that traditionally assumes homogeneity in all parameters. The heterogeneity in the model takes the form of random distributions for the membranepotential threshold for action potential generation in individual neurons. The authors use simulations and mathematical analysis to explore the effects of heterogeneity, and report intriguing differences that result from whether it is excitatory neurons that are modeled as heterogeneous, or inhibitory neurons. Following publication, a commentary from Baroni and Mazzoni summarized the paper of Mejias and Longtin and discuss the many ways in which the work could be extended. They point out that different heterogeneities could interact in complex unpredictable ways, hence coining the phrase heterogeneity of heterogeneities.

Last but not least, Lagzi and Rotter also consider recurrently connected networks of excitatory and inhibitory neurons. They specifically consider the balanced-state of such networks, where population activity fluctuates but has a steady-state stable mean. Like Dummer et al. and Lajoie et al. the model considered has no external noise, and the fluctuations are a result of the underlying chaotic dynamics of the system. The central contribution of Lagzi and Rotter is to propose a two-state Markovian model which they show exhibits the same phenomenology and statistical properties

## REFERENCES


as simulations of recurrently connected cortical networks in the balanced regime. The authors additionally derive corresponding mean-field equations and provide associated analysis.

## OUTLOOK

The work collected in this Research Topic represents important advances in our understanding of stochastic variability of dynamics in neurons and networks. Methodological issues—such as preferred numerical methods for simulating channel noise in stochastic versions of Hodgkin-Huxley-type models—have been clarified. Moving forward, we expect to see more examples of models that are tightly constrained by known physiological data (as in Moezzi et al.). We have highlighted how variability, as interpreted in these works, can arise from stochastic cellular mechanisms (synapses, ion channels) and from nonlinear and chaotic network dynamics. Of great interest will be future studies that bridge these scales by exploring, for instance, how celllevel noise filters through populations of neurons to shape the dynamics and signal processing capabilities of neural networks.

## AUTHOR CONTRIBUTIONS

All authors listed have made substantial, direct and intellectual contribution to the work, and approved it for publication.

## FUNDING

MM's contribution was supported by an Australian Research Fellowship from the Australian Research Council (project number DP1093425).

## ACKNOWLEDGMENT

We thank all authors and reviewers of manuscripts submitted to this research topic.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 McDonnell, Goldwyn and Lindner. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

## The ISI distribution of the stochastic Hodgkin-Huxley neuron

#### *Peter F. Rowat <sup>1</sup> \* and Priscilla E. Greenwood2*

*<sup>1</sup> Institute for Neural Computation, University of California, San Diego, La Jolla, CA, USA <sup>2</sup> Mathematics Department, University of British Columbia, Vancouver, BC, Canada*

#### *Edited by:*

*Benjamin Lindner, Bernstein Center for Computational Neuroscience, Germany*

#### *Reviewed by:*

*Benjamin Lindner, Bernstein Center for Computational Neuroscience, Germany Sten Rüdiger, Humboldt University*

*at Berlin, Germany*

#### *\*Correspondence:*

*Peter F. Rowat, Institute for Neural Computation, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093, USA e-mail: prowat@ucsd.edu*

## The simulation of ion-channel noise has an important role in computational neuroscience. In recent years several approximate methods of carrying out this simulation have been published, based on stochastic differential equations, and all giving slightly different results. The obvious, and essential, question is: which method is the most accurate and which is most computationally efficient? Here we make a contribution to the answer. We compare interspike interval histograms from simulated data using four different approximate stochastic differential equation (SDE) models of the stochastic Hodgkin-Huxley neuron, as well as the exact Markov chain model simulated by the Gillespie algorithm. One of the recent SDE models is the same as the Kurtz approximation first published in 1978. All the models considered give similar ISI histograms over a wide range of deterministic and stochastic input. Three features of these histograms are an initial peak, followed by one or more bumps, and then an exponential tail. We explore how these features depend on deterministic input and on level of channel noise, and explain the results using the stochastic dynamics of the model. We conclude with a rough ranking of the four SDE models with respect to the similarity of their ISI histograms to the histogram of the exact Markov chain model.

**Keywords: ISI distribution, Hodgkin-Huxley, stochastic dynamics, stochastic differential equation, ISI histogram, Gillespie algorithm, Kurtz approximation**

## **1. INTRODUCTION**

Channel noise is important because it contributes to spike-time variability (Sigworth, 1980; White et al., 2000) and has been shown to be essential for subthreshold oscillations in stellate cells (White et al., 1998; Dorval and White, 2005), and for response variability in sensory cells (Fisch et al., 2012). In addition it contributes importantly to intrinsic irregular firing in cortical interneurons (Englitz et al., 2008; Stiefel et al., 2013), while in certain small neurons a single channel opening can initiate a spike (Lynch and Barry, 1989).

In this paper we compare published SDE approximation methods that simulate the stochastic Hodgkin-Huxley (HH) neuron model, by comparing the inter-spike-interval (ISI) distributions produced when driven by a constant DC current I. Theoretical work on the ISI distributions of stochastic neuron models was carried out by Chow and White (1996); Gerstein and Mandelbrot (1964); Gutkin and Ermentrout (1998); Tuckwell (2005), and Wilbur and Rinzel (1983).

In all cases the deterministic model used as a basis for the various stochastic schemes is the classical model of Hodgkin and Huxley (1952). This model was introduced to describe action potentials in the squid giant axon, and remains a foundation of modern neuroscience. Its dynamics comprise a subcritical Hopf bifurcation together with a switching region in phase space where a fixed point is near to a limit cycle, the two being separated by an unstable limit cycle (**Figures 1A,B**). Thus, the deterministic

HH model has a bistable range: when the input current, *I*, lies between 6.2 and 9.8µA/cm<sup>2</sup> (approximately) it is either spiking tonically—represented by the system traversing the locally stable limit cycle—or is quiescent—represented by the system spiraling inside the unstable limit cycle in toward the fixed point. When noise is present, and a trajectory traverses the switching region where the fixed point is close to the stable limit cycle, the system can switch between limit cycle behavior and quiescence. Thus, its overall behavior exhibits irregular switching between bursts of tonic spiking and periods of quiescence. This stochastic behavior continues to occur for a considerable range of the input, I, both below and above the deterministic bistable region *I* = [6.2, 9.8] (Yu and Lewis, 1989; Rowat, 2007).

Here we study the dependence of firing and quiescence patterns on the way noise is modeled, as reflected in the resulting distribution of inter-spike intervals (ISIs). The models we investigate are presented and studied in the papers by Fox and Lu (1994), Fox (1997), Goldwyn et al. (2011), Linaro et al. (2011), Orio and Soudry (2012), and Güler (2013).

In the standard stochastic model for the HH neuron each potassium channel has four binary gates, all of which must be open for potassium to be conducted. Each sodium channel has three activation gates and one inactivation gate, producing eight states, the channel being open only when it is in a particular one of these states [for complete details see

Rowat and Greenwood ISI distribution of the stochastic Hodgkin-Huxley neuron

**FIGURE 1 | (A)** Bifurcation diagram of the deterministic Hodgkin-Huxley model. Solid line, stable fixed point (SFP). Long-dashed line, unstable fixed point (UFP). Dot-dashed lines, extreme values of voltage on the stable limit cycle (SLC). Short-dashed lines, extreme voltage values on the unstable limit cycle (ULC). **(B)** The switching region in the two-dimensional Morris-Lecar model neuron. A similar, but four-dimensional, region is present in the Hodgin-Huxley phase-space. **(B1)** The fixed point is very close to the stable limit cycle with an unstable limit cycle (ULC) between them. Inset: the switching region, enlarged. **(B2)** This shows a noisy trajectory that emits one spike followed by sub-threshold oscillation inside the ULC and then another spike.

Rowat (2007)]. The voltage dependent rates of moving between states have been established from data by Hodgkin and Huxley (1952).

A Markov chain algorithm for keeping track of the number of channels in each state was developed by Chow and White (1996), Gillespie (1977), and Skaugen and Walløe (1979). The algorithm was used by Rowat (2007) to compute several aspects of HH stochastic dynamics. This "exact" method of simulation of the stochastic HH we call the Micro model.

Because of both computation speed and ease of analysis, it is useful to replace the Micro Markov chain model with a system of stochastic differential equations, an SDE model. In fact we were shown how to do this already in a paper of Kurtz (1978), where a system of SDE's is constructed that approximates a density dependent Markov chain at a rate depending on population size N, with error of order log (*N*)/*N*.

Without knowing about the results of Kurtz (1978), authors of a number of papers, Fox (1997); Güler (2013); Linaro et al. (2011), and Orio and Soudry (2012), have devised systems of SDEs to approximate the Micro model. In fact Orio and Soudry (2012), using heuristics, derived the same set of approximating SDEs which a theorem of Kurtz (1978) defines for the Micro model. Here we call this model the Orio-Kurtz or simply Orio model, and sometimes the Kurtz approximation. However, it should be kept in mind that Kurtz proved the approximation for general Markov chain models in 1978. The complete Langevin system of SDEs proposed by Fox (1997), which requires taking two matrix square-roots at every time-step, was implemented for the first time by Goldwyn et al. (2011) so we sometimes refer to this as the Fox-Goldwyn model.

In neuroscience it is widely accepted that the distribution of spike timing, not simply mean spike frequency, is important. While the Micro, Fox-Goldwyn, Güler, Linaro, and Orio-Kurtz models all produce nearly the same mean spike frequency, it is not known how well these models capture the inter-spike-interval (ISI) distributions of the Micro model. Here we generate and compare the ISI distributions of the four SDE models with the ISI distributions of the Micro model, for a range of input current, I, that includes the region of bistability in the deterministic Hodgkin-Huxley model.

We find that, in fact, the ISI distribution is quite similar for all these models over a large range of (constant) deterministic inputs, I, and over a large range of channel numbers, which are proportional to A, the membrane area used. This result is, in fact, expected from analytic considerations, since, arguably, all the models are at least fairly well-approximated by the same diffusion process (Kurtz, 1978), which is based on a approximation theorem with a known error rate and can be regarded as a "gold standard." We will see that both the Fox-Goldwyn and the Orio-Kurtz approximations give ISI distributions which are both quite close to that of the approximated Markov chain model. The Güler model compares well with the Fox-Goldwyn and Orio-Kurtz approximations while the Linaro model is somewhat less successful.

Our second main result is the first detailed description of the form of the ISI distribution of the stochastic HH, which appears in Section 3. In addition we explore how each of the features of the ISI distribution depends on I, the input to the model neuron, and on the number of channels in play, which is functionally related to the standard deviation of the noise in the system.

#### **1.1. FOUR SDE MODELS OF HODGKIN-HUXLEY NOISE**

The current conservation equation for voltage V (mV) and applied current *I* (µA/cm2) in the deterministic HH model is

$$C\dot{V} = I - \left[\mathbf{g}\_{Na}(V - V\_{Na}) + \mathbf{g}\_{K}(V - V\_{K}) + \mathbf{g}\_{L}(V - V\_{L})\right] \quad (1)$$

where the constants are given in **Table 1**.

Equation (1) is the deterministic basis of the four stochastic differential equation (SDE) models we study: the Fox approximation, the Orio-Kurtz approximation, and the models of Güler

**Table 1 | Hodgkin-Huxley parameters for model simulations.**


**Table 2 | Correspondence between si variables and states** *m***jh***k***.**


**Table 3 | Changes in ISI distribution parameters with changes in noise level and applied current.**


(2013) and Linaro et al. (2011). Simulation of the Markov chain model called " Micro" is detailed in Rowat (2007). A potassium channel has four activation *n*-gates, where each gate has the (opening, closing) rates (α*n*(*V*), β*n*(*V*)). The corresponding Markov network is

where a channel in state *ni*, *i* = 0, 1,..., 4, has *i* open *n*-gates. The channel is closed except when in state *n*4.

A sodium channel has 3 activation *m*-gates and one inactivation *h*-gate, where the *m*-gates have (opening, closing) rates (α*m*(*V*), β*m*(*V*)), and the *h*-gates have rates (α*h*(*V*), β*h*(*V*)). The corresponding Markov network is

and the channel is open only in state m3h1. In state m*j*h*k*, there are *j* open *m*-gates and *k* open *h*-gates. The gate transition rates are given by the following functions:

$$\begin{aligned} \alpha\_m(V) &= \frac{0.1(V+40)}{1 - e^{-(V+40)/10}}, & \beta\_m(V) &= 4e^{-(V+65)/18},\\ \alpha\_h(V) &= 0.07 \, e^{-(V+65)/20}, & \beta\_h(V) &= \frac{1}{1 + e^{-(V+35)/10}},\\ \alpha\_n(V) &= \frac{0.01(V+55)}{1 - e^{-(V+55)/10}}, & \beta\_n(V) &= 0.125 \, e^{-(V+65)/80}, \end{aligned} (4)$$

In the Orio and Linaro models one writes an SDE for the proportion of channels in each of the states shown above. Because the K<sup>+</sup> channels have 5 states and the Na<sup>+</sup> channels have 8 states, these approximations consist of a system of 13 SDEs for Orio or 11 SDEs for Linaro, plus Equation (1), which has no explicit noise term.

In the K+-channel equations that follow, each variable *ni* represents the proportion of channels in state *ni*, for *i* = 0, 1,..., 4. In the subsequent Na+-channel equations, for ease of notation we use variables *s*0, *s*1,..., *s*<sup>7</sup> to stand for the proportions of channels in states m0h0, m0h1, ..., m3h1. The correspondence is given in the **Table 2**.

When Equation (1) is integrated, the values of the K+- and Na+-conductances are given by the definitions:

$$\mathbf{g}\_{\mathbf{K}} = n\_4 \mathbf{\bar{g}}\_{\mathbf{K}}, \quad \mathbf{g}\_{\mathbf{Na}} = s\_7 \mathbf{\bar{g}}\_{\mathbf{Na}} \tag{5}$$

Here we write, in algorithmic form, how the SDEs which, in fact, follow from the Kurtz approximation (Kurtz, 1978), are formulated by Orio and Soudry (2012). A few words about the proof of the Kurtz approximation are in Section 1.2. To obtain each of the 13 equations for proportions of channels in each state, we first write the equation as an ordinary differential equation (ODE), thinking of the dynamics for a particular state as deterministic, i.e., the rates are deterministic input and outputs. For each state that is directly linked to the current state, we add to the right hand side a 2-component deterministic term with a positive input component and a negative output component. Then, for each of these deterministic terms on the right hand side, we add a noise term which has the form <sup>√</sup>*x dW* where *<sup>x</sup>*, in each case, is the deterministic term with any '−' signs changed to '+,' and *dW* is a Brownian increment. This gives the effective variance when the rates are considered as Poisson rates instead of deterministic rates. In each pair of directly linked states, the stochastic mass going out of one state is the same as the stochastic mass going into the other state. In these cases the terms <sup>√</sup>*xdW* are kept separate and have opposite signs in the two equations. We see examples in Equations (6) and (7) below. This simple description of the procedure for obtaining the Kurtz approximation (Kurtz, 1978) together with its justification is sketched in Greenwood and Gordillo (2009). Another version is in Orio and Soudry (2012), and its supplement S1.

The full system of SDEs for the potassium channel is given by:

$$
\dot{n}\_0 = (-4\alpha\_n n\_0 + \beta\_n n\_1) + \frac{\dot{\xi}\_1}{\sqrt{N\_K}} \sqrt{4\alpha\_n n\_0 + \beta\_n n\_1}
$$

$$
\dot{n}\_1 = (4\alpha\_n n\_0 - \beta\_n n\_1) + (2\beta\_n n\_2 - 3\alpha\_n n\_1)
$$

$$
$$

$$
\dot{n}\_2 = (3\alpha\_n n\_1 - 2\beta\_n n\_2) + (3\beta\_n n\_3 - 2\alpha\_n n\_2)
$$

$$
$$

$$
\dot{n}\_3 = (2\alpha\_n n\_2 - 3\beta\_n n\_3) + (4\beta\_n n\_4 - \alpha\_n n\_3)
$$

$$
$$

$$
\dot{n}\_4 = (\alpha\_n n\_3 - 4\beta\_n n\_4) - \frac{\dot{\xi}\_4}{\sqrt{N\_K}} \sqrt{\alpha\_n n\_3 + 4\beta\_n n\_4}
\tag{6}
$$

Here ξ*i*, *i* = 1,..., 4 are Gaussian noise terms with mean 0 and standard deviation 1. Note that there are 5 SDEs but only 4 noise terms.

The SDEs for the sodium Markov network (3) can be read off directly from the network using the recipe described above between Equations (5) and (6):

˙*s*<sup>0</sup> <sup>=</sup> (β*ms*<sup>2</sup> <sup>−</sup> <sup>3</sup>α*ms*0) <sup>+</sup> (β*hs*<sup>1</sup> <sup>−</sup> <sup>α</sup>*hs*0) <sup>+</sup> <sup>ξ</sup><sup>20</sup> <sup>√</sup>*NNa* β*ms*<sup>2</sup> + 3α*ms*<sup>0</sup> <sup>+</sup> <sup>ξ</sup><sup>10</sup> <sup>√</sup>*NNa* β*hs*<sup>1</sup> + α*hs*<sup>0</sup> ˙*s*<sup>1</sup> <sup>=</sup> (α*hs*<sup>0</sup> <sup>−</sup> <sup>β</sup>*hs*1) <sup>+</sup> (β*ms*<sup>3</sup> <sup>−</sup> <sup>3</sup>α*ms*1) <sup>−</sup> <sup>ξ</sup><sup>10</sup> <sup>√</sup>*NNa* β*hs*<sup>1</sup> + α*hs*<sup>0</sup> <sup>+</sup> <sup>ξ</sup><sup>31</sup> <sup>√</sup>*NNa* β*ms*<sup>3</sup> + 3α*ms*<sup>1</sup> ˙*s*<sup>2</sup> = (3α*ms*<sup>0</sup> − β*ms*2) + (2β*ms*<sup>4</sup> − 2α*ms*2) + (β*hs*<sup>3</sup> − α*hs*2) <sup>−</sup> <sup>ξ</sup><sup>20</sup> <sup>√</sup>*NNa* <sup>3</sup>α*ms*<sup>0</sup> <sup>+</sup> <sup>β</sup>*ms*<sup>2</sup> <sup>+</sup> <sup>ξ</sup><sup>42</sup> <sup>√</sup>*NNa* 2β*ms*<sup>4</sup> + 2α*ms*<sup>2</sup> <sup>+</sup> <sup>ξ</sup><sup>23</sup> <sup>√</sup>*NNa* α*hs*<sup>2</sup> + β*hs*<sup>3</sup> ˙*s*<sup>3</sup> = (3α*ms*<sup>1</sup> − β*ms*3) + (2β*ms*<sup>5</sup> − 2α*ms*3) − (α*hs*<sup>2</sup> − β*hs*3) <sup>−</sup> <sup>ξ</sup><sup>31</sup> <sup>√</sup>*NNa* <sup>3</sup>α*ms*<sup>1</sup> <sup>+</sup> <sup>β</sup>*ms*<sup>3</sup> <sup>+</sup> <sup>ξ</sup><sup>53</sup> <sup>√</sup>*NNa* 2β*ms*<sup>5</sup> + 2α*ms*<sup>3</sup> <sup>−</sup> <sup>ξ</sup><sup>23</sup> <sup>√</sup>*NNa* α*hs*<sup>2</sup> + β*hs*<sup>3</sup> ˙*s*<sup>4</sup> = (2α*ms*<sup>2</sup> − 2β*ms*4) + (3β*ms*<sup>6</sup> − α*ms*4) + (β*hs*<sup>5</sup> − α*hs*4) <sup>−</sup> <sup>ξ</sup><sup>42</sup> <sup>√</sup>*NNa* <sup>2</sup>α*ms*<sup>2</sup> <sup>+</sup> <sup>2</sup>β*ms*<sup>4</sup> <sup>+</sup> <sup>ξ</sup><sup>64</sup> <sup>√</sup>*NNa* 3β*ms*<sup>6</sup> + α*ms*<sup>4</sup> <sup>+</sup> <sup>ξ</sup><sup>54</sup> <sup>√</sup>*NNa* β*hs*<sup>5</sup> + α*hs*<sup>4</sup> ˙*s*<sup>5</sup> = (2α*ms*<sup>3</sup> − 2β*ms*5) + (3β*ms*<sup>7</sup> − α*ms*5) + (α*hs*<sup>4</sup> − β*hs*5) <sup>−</sup> <sup>ξ</sup><sup>35</sup> <sup>√</sup>*NNa* <sup>2</sup>α*ms*<sup>3</sup> <sup>+</sup> <sup>2</sup>β*ms*<sup>5</sup> <sup>+</sup> <sup>ξ</sup><sup>75</sup> <sup>√</sup>*NNa* 3β*ms*<sup>7</sup> + α*ms*<sup>5</sup>

$$\begin{aligned} -\frac{\dot{\xi}\_{54}}{\sqrt{N\_{\text{Na}}}} & \sqrt{\alpha\_{\text{h}}s\_{4} + \beta\_{\text{h}}s\_{5}} \\\\ \dot{s}\_{6} &= (\alpha\_{\text{m}}s\_{4} - 3\beta\_{\text{m}}s\_{6}) + (\beta\_{\text{h}}s\_{7} - \alpha\_{\text{h}}s\_{6}) - \frac{\dot{\xi}\_{64}}{\sqrt{N\_{\text{Na}}}} \sqrt{\alpha\_{\text{m}}s\_{4} + 3\beta\_{\text{m}}s\_{6}} \\ & + \frac{\dot{\xi}\_{76}}{\sqrt{N\_{\text{Na}}}} \sqrt{\beta\_{\text{h}}s\_{7} + \alpha\_{\text{h}}s\_{6}} \\\\ \dot{s}\_{7} &= (\alpha\_{\text{m}}s\_{5} - 3\beta\_{\text{m}}s\_{7}) - (\alpha\_{\text{h}}s\_{6} - \beta\_{\text{h}}s\_{7}) - \frac{\dot{\xi}\_{75}}{\sqrt{N\_{\text{Na}}}} \sqrt{\alpha\_{\text{m}}s\_{5} + 3\beta\_{\text{m}}s\_{7}} \\ & - \frac{\dot{\xi}\_{76}}{\sqrt{N\_{\text{Na}}}} \sqrt{\alpha\_{\text{h}}s\_{6} + \beta\_{\text{h}}s\_{7}} \end{aligned} \tag{7}$$

Güler (2013) presented a different stochastic Hodgkin-Huxley model. This model also approximates the stochastic dynamics of the membrane potential, arising from random opening and closing of sodium and potassium channels, by a system of seven differential equations, five of them stochastic, together with a modified version of Equation (1), appearing here as Equation (8). The stochastic dynamics, which follow, in a sense, more directly from the approach pioneered by Fox and Lu (1994) than the others, are approximated using carefully constructed diffusion coefficients. In addition, the drift coefficients contain stochastic components, *qK* and *qNa*, designed to capture "non-trivial cross-correlation persistence" (NCCP) effects, namely correlations between transmembrane voltage fluctuations and the component of open channel fluctuations due to gate multiplicities (Güler, 2011). Since properties of the NCCP effects are similar to those of a harmonic Brownian oscillator, the equations that describe *qK* and *qNa* are written as those of a Brownian oscillator. Güler argues that NCCP effects have a major influence on excitability, spontaneous firing, and spike coherence. Güler reports that his model captures very accurately the functional correspondence between input current and mean spike frequency as obtained from the Micro structure (Markov network) model, as well as the mean spike frequency obtained from the Linaro model.

In the Güler SDE model, the current conservation Equation (1) is modified to read:

$$C\dot{V} = -\overline{\mathfrak{g}}\_K \psi\_K (V - V\_K) - \overline{\mathfrak{g}}\_{\text{Na}} \psi\_{\text{Na}} (V - V\_{\text{Na}})$$

$$-\mathfrak{g}\_L (V - V\_L) + I \tag{8a}$$

$$\text{where} \quad \psi\_K = n^4 + \sqrt{\frac{n^4(1-n^4)}{N\_K}} \, q\_{Na} \tag{8b}$$

$$\text{and} \quad \psi\_{\text{Na}} = m^3 h + \sqrt{\frac{m^3 (1 - m^3)}{N\_{\text{Na}}}} \, h \, q\_{\text{Na}} \tag{8c}$$

and the periodic stochastic variables *qK* and *qNa* satisfy two second-order linear SDEs written as four first-order SDEs:

$$
\pi \dot{q}\_{\mathbb{K}} = p\_{\mathbb{K}} \tag{8d}
$$

$$\tau \dot{p} \mathbf{x} = -\varkappa \mathbf{x} p \mathbf{x} - \omega\_K^2 \left[ \alpha\_n (1 - n) + \beta\_n n \right] q \mathbf{x} + \xi \mathbf{x} \tag{8e}$$

$$
\pi \dot{q}\_{\rm Na} = p\_{\rm Na} \tag{8f}
$$

$$\tau \dot{\rho}\_{\rm Na} = -\varphi\_{\rm Na} p\_{\rm Na} - \omega\_{\rm Na}^2 \left[ \alpha\_m (1 - m) + \beta\_m m \right] q\_{\rm Na} + \xi\_{\rm Na} \tag{8g}$$

The gating variables n,m, and h are given by three more SDEs as in Fox and Lu's (1994) paper:

$$
\dot{n} = \alpha\_n (1 - n) - \beta\_n n + \eta\_n \tag{8h}
$$

$$
\dot{m} = \alpha\_m (1 - m) - \beta\_m m + \eta\_m \tag{8i}
$$

$$
\dot{h} = \alpha\_h (1 - h) - \beta\_h h + \eta\_h \tag{8\text{j}}
$$

The Gaussian noise terms have zero means, with variances given by

$$\text{Var}(\xi\_K) = \gamma\_K T\_K \left[ \alpha\_n (1 - n) + \beta\_n n \right] \tag{9a}$$

$$\text{Var}(\xi\_{\text{Na}}) = \wp\_{\text{Na}} T\_{\text{Na}} \left[ \alpha\_m (1 - m) + \beta\_m m \right] \tag{9b}$$

$$Var(\eta\_n) = \frac{\alpha\_n(1-n) + \beta\_n n}{4N\_K},\tag{9c}$$

$$\text{Var}(\eta\_m) = \frac{\alpha\_m (1 - m) + \beta\_m m}{\Re \text{N}\_{\text{Na}}},\tag{9d}$$

$$\text{Var}(\eta\_h) = \frac{\alpha\_h (1 - h) + \beta\_h h}{N\_{\text{Na}}} \tag{9e}$$

where the values of the fixed parameters in the Equations (8e,g) and (9a,b) are:

$$
\gamma\_K = \gamma\_{Na} = 10, \,\omega\_K^2 = 150, \,\omega\_{Na}^2 = 200, \, T\_K = 400, \, T\_{Na} = 800.
$$

The functions α*<sup>x</sup>* and β*x*, *x* = *n*, *m*, *h*, were given earlier in Equation (4). There are similarities between the Kurtz approximation and the Güler model, e.g., the Güler Equations (9c–e) specify that the diffusion terms of the SDEs (8h,i,j) are similar to the drift terms with '−' changed to '+' just as in the Kurtz approximation. There are significant differences seen in Güler's (8b,c) and in the fact that his SDEs (8d,e) form a second order SDE with a single noise term, and similarly for his SDEs (8f,g). Still it may be that Güler's model is an approximation to the Micro model in the same sense as the Kurtz approximation, or nearly so.

The Linaro model (Linaro et al., 2011) starts from the same current conservation Equation (1), appearing as Linaro et al. (2011; Equation 18). As in the Kurtz approximation, 11 SDEs are introduced Linaro Equation (19), but were obtained through the introduction of Orstein-Uhlenbeck processes for M-1 of the M elements of an M-state Markov process and applying this to the K- and Na- Markov processes. Hence both the drift terms and the diffusion coefficients take a different form from the Kurtz approximation Linaro Equation (19). In view of these differences it is perhaps surprising that the Linaro model produces ISI distributions which are rather close to those produced by the Micro model, the Fox-Goldwyn model, the Orio-Kurtz approximation, and the Güler model.

Diffusion approximations for this stochastic HH Markov chain model have also been studied by Bruce (2009); Goldwyn et al. (2011), and Huang et al. (2013). Engel et al. (2008) and Verechtchaguina et al. (2007) also study ISI histograms for a different modeled neuron and by a different approach.

## **1.2. KURTZ'S STRONG APPROXIMATION THEOREM FOR MARKOV CHAINS**

Here we describe briefly a theorem of Kurtz (1978) and how it applies to approximate the stochastic HH model by the system of SDE's consisting of Equations (1) and (4–7). A more complete version, including an alternate approach using a van Kampen expansion, is described by Baxendale and Greenwood (2011).

In fact one can approximate any normed density dependent Markov process, *XN*(*t*) = *X*(*t*)/*N*, with values in Z*d*, for large population size *N*, by a diffusion process with small error. The method of Kurtz (1978) represents a Z*<sup>d</sup>* -valued Markov process as a sum of Poisson processes. The essential step is replacing each normed compensated, or conditionally centered, Poisson process with a scalar Brownian motion, where an error of order log (*N*)/*N* is introduced. The resulting stochastic system can be written as

$$d\tilde{X}^N(t) = F(\tilde{X}^N(t))dt + \frac{1}{\sqrt{N}}C(\tilde{X}^N(t))dW(t),\qquad(10)$$

where *F* is the vector field of conditional means of the terms in Kurtz's sum, and the *d* × *d* diffusion coefficient matrix function *C*(*z*) is chosen so that *C*(*z*)*C*(*z*)<sup>∗</sup> = *B*(*z*), the covariance function arising from interactions of the terms. One avoids computing the square root of the matrix *B* by retaining the conditional centerings as separate Brownian increments as in Orio and Soudry (2012), Equation (13). We see these terms written out in Equations (5) and (6). This produces a sum of noise terms in each equation so that in distribution the system is the same as (9). The paper (Allen et al., 2008) gives the details of this process.

#### **1.3. THE FORM OF THE ISI DISTRIBUTION**

Suppose we have a recording of membrane potential from a neuron firing in response to a fixed input current I, or we are looking at the output of a simulation of a neuron firing model such as one of those we are considering. An inter-spike interval (ISI) is the time between two successive downward crossings of the recording across a potential level chosen to be well above the range of sub-threshold oscillations. In general the successive ISIs of a simulation of a stochastic model are regarded as independent whereas those of a real neuron are not necessarily so. However, we do not pursue this question here. We are interested only in the distribution of the random ISIs.

The mean spiking frequencies of three models, Micro, Fox-Lu, and Güler are compared for a range of input currents in Figures 6–8 by Güler (2013). These means are not exactly the same but are rather similar. Here we look instead at the entire distribution of ISIs. In Figure 1A of Rowat (2007) we find, already, with Area = 100µm<sup>2</sup> and *I* = 0µA/cm2, ISI histograms for the stochastic HH model considered by Chow and White (1996). Figure 16 of Rowat (2007) shows that the histogram of **Figure 1A** is nearly identical to that obtained when Gaussian noise is added to the HH current balance equation for a particular level of noise and a particular constant deterministic input. The effects of carefully modeled channel noise and an equivalent level of Gaussian noise added to the current balance Equation (1) are found to be nearly indistinguishable on the basis of the resulting ISI histograms. This observation motivates the present study where we compare ISI distributions more systematically and for additional recently studied SDE models of the stochastic HH equation.

The form of the ISI histograms indicates that the mean of the ISI distribution is in fact an inadequate parameter to use for comparison of stochastic models. The distribution is not unimodal but instead has the following characteristic form (see **Figures 2**, **3**). For short time intervals there is a tall, narrow peak even on a log scale which represents the distribution of times taken by those individual spike firings which are preceded by one or more spikes, i.e., the times taken by the simulated stochastic path to traverse the locally stable limit cycle of the dynamics when there was a preceding spike. The fact that this first peak is narrow indicates that the variance of the time taken by a stochastic firing is small. The

**FIGURE 2 | Each panel compares the ISI distributions generated by the Micro, Güler, Linaro, Orio, and Fox methods, for a particular area, applied current combination.** Here, the current is constant at 6.0µA/cm2 while the area takes values 100µm2 **(A)**, 500 **(B)**, 900 **(C)**. Equivalently, the noise amplitude decreases from **(A to C)**.

area under this first peak indicates the proportion of ISIs in runs, or "bursts," of two or more spikes. As was found by Rowat (2007), **Figure 4**, the height of this spike increases with the deterministic input, I.

The second obvious feature of the histogram, plotted on a log scale, as in Figure 16B of Rowat (2007) and **Figures 2**, **3**, is that the right tail of the distribution is exponential, as indicated in our linear-log plots by a straight line, starting soon, though not immediately, after the initial peak. The histogram appears increasingly noisy as the length of the ISI time interval increases since the amount of data for estimation decreases, and because short histogram bars are enlarged by the log-scale.

The fact that the right tail of the ISI histogram is exponentially decreasing for the stochastic Morris Lecar model is studied in detail and explained by Rowat and Greenwood (2011).

**FIGURE 3 | Each panel compares the ISI distributions generated by the algorithmic methods Micro, Güler, Linaro, Orio, and Fox, for a particular area, applied current combination.** Here, the area is constant at 400µm2 while the applied current takes values 10.0µA/cm2 **(A)**, 6.0µA/cm2 **(B)**, 2.0µA/cm2 **(C)**.

The explanation is based on the dynamics of the sub-threshold stochastic process which is in a conditional equilibrium beginning soon after a sub-threshold interval begins. The exit time from such an equilibrium must be exponentially distributed. This argument will apply to all stochastic HH models.

A third feature, less obvious but quite distinct, is that there are one or more small bumps or local maxima in the ISI histogram just after the initial peak, and before the exponential tail begins.

The one or more small local maxima in the ISI histogram are explained by considering the dynamics of the stochastic HH model just at the end of firing, i.e., as the stochastic path crosses into the basin of attraction of the locally stable fixed point of the deterministic HH model. The first orbit begins near the outer edge of the basin of attraction—the unstable limit cycle—and so the probability that the next firing (traverse of the locally stable limit cycle) comes after just one sub-threshold orbit is relatively high: see **Figure 1B**. Hence the probability that the ISI ends at the time taken by one such orbit from the end of the previous firing is relatively high. This produces the first small local maximum of the ISI distribution (see **Figures 2**, **3**). Given that the next firing does not occur in this first small orbit of the fixed point, the next subthreshold orbit takes again a similar length of time, and again the probability of firing near the end of this second orbit is somewhat increased, producing the second local maximum, and so on. When there is less noise the pattern is more distinct. After one or two, or at most three such random orbits, the stochastic path is very nearly in its temporary, conditional stationary distribution concentrated close around the fixed point, so the remainder of the ISI distribution is exponentially distributed as discussed above. This description is made more explicit in the Morris Lecar twodimensional system since the "inside" and "outside" of a limit cycle are well defined. In the four-dimensional HH system, the argument is made plausible by examining the relationships in 4 dimensions between the stable limit cycle, the unstable limit cycle, and the fixed point, and by projections onto a 2D plane (e.g., Rowat (2007), Figure 10, and the discussion).

The ISI distribution of any stochastic HH neuron model can, therefore, be resolved into three sections, an initial peak representing the distribution of times taken up by contiguous spike firings, followed by one or more local maxima representing the additional times taken up by each of a few subthreshold orbits of the fixed point immediately after firing, followed by an exponential tail representing time until escape from the subthreshold state once the process is in its conditionally stationary distribution. Thus, a complete comparison of ISI histograms for simulations of stochastic HH models built from different noise models can be made by comparing the defining parameters of these three components: the center, height, and width of the initial peak, the shapes and placement of the local maxima, and the parameters of the exponential tails. We can use these criteria for comparing the ISI histograms produced by the four stochastic HH models described in Section 1.1.

## **2. METHODS AND IMPLEMENTATION DETAILS**

All model computer runs used the standard Hodgkin-Huxley parameter values, as in **Table 1**, and all data sets used for the ISI histograms had 105 elements. All histograms were normalized so that their bars sum to 1, and all are displayed with a log scale on the y-axis because the first peak is often an order of magnitude higher than the second and third peaks. The integration time-step was 0.005 ms. An SDE was used for each state. If a potassium variable became negative the random number generator was called again. If a sodium variable became negative it was immediately reset to zero. At the end of each integration step, but before integrating Equation (1), the potassium variables *ni*, *i* = 0,..., 4 were normalized to satisfy *ni* = 1, and the sodium variables *sj*, *j* = 0,..., 7 were normalized by *sj* = 1. This seems more correct than defining the last variable (*n*<sup>0</sup> or *s*0) in terms of the others, since it preserves the relative values of the variables, but has the disadvantage of using two extra SDEs. However, for the Orio method and any Kurtz-type approximation it does not increase the number of random number generator calls.

## **3. RESULTS**

In **Figures 2**, **3** we see ISI histograms from simulations of the Markov chain model and the four SDE models, which are labeled Micro, Güler, Linaro, Orio, and Fox, to refer to the five ways of modeling HH channel noise described in Section 1.1. The noise level is proportional to Area−1/<sup>2</sup> since the standard deviation of the Na+-channel noise (K+-channel noise) is ∝ *N*−1/<sup>2</sup> *Na* ∝ *N*−1/<sup>2</sup> *K* and the number of channels is a constant times the area. In **Figure 5** where the area A = 400µm2, *NNa* = 60 × *A* = 24000 and *NK* = 18 × *A* = 7200 Separate sets of plots show the results for fixed applied current *I* = 6µA/cm<sup>2</sup> and area *A* = 100, 200, ..., 1000 µm2, and for fixed area A = 400µm<sup>2</sup> and applied currents *I* = 2, 3,..., 12µA/cm2. See **Figures 4**, **5**. All the histograms show the features detailed in Section 3: an initial peak, followed by local maxima, followed by an exponential tail, that appears linear on a log scale. The Güler, Micro, Orio, and Fox histograms are nearly identical in all respects and the Linaro plots are very similar.

The histograms computed by the Güler method all have a slightly higher proportion of area under the initial peak, corresponding to runs of successive spikes, than Micro-generated histograms, while Orio-generated histograms always have slightly

lower proportion of ISIs in runs than Micro (differing by no more than 4%), while the Linaro histograms have proportions of ISIs in runs that are 8% lower than in Micro histograms.

Since the plots for the different SDE models are so similar it may not be worth dwelling on the differences. The exponential parameters of the tails are very close as evidenced by the nearly parallel plots. Also the timing of all features is nearly identical, showing that the different ways of modeling noise by SDEs have had little effect on the pattern of firing of the simulated neuron.

In **Figures 2**, **3** we see what happens to the three parts of the ISI histogram as the area A increases and the applied current *I* increases. Since A is proportional to the numbers of channels, in fact the standard deviation of the noise per unit time is proportional to A−1/2. We see in **Figure 2** that the slope of the exponential part of the histogram, i.e., the slope of the last part, becomes less negative as A increases, equivalently, as the noise decreases. The height of the initial peak barely changes, but we see that the bumps in the middle part of the histogram become more prominent as noise decreases. When A is fixed and I, the deterministic input, decreases, in **Figure 3** we see that the negative slope, i.e., the negative exponent of the exponential tail also decreases but, opposite to the case when noise decreases, the bumps (only one can be seen) become less prominent. The height of the initial peak decreases considerably, and its position moves right as *I* decreases. These effects are studied in more detail for the Orio model in **Figures 4**, **5**. We discuss their interpretation in the next section, but give a summary in **Table 3**.

## **4. DISCUSSION**

The models we have simulated produce similar histograms with the same basic features in good alignment. Here we discuss further how these features depend on two important parameters of the stochastic Hodgkin-Huxley model, the deterministic input, I, and the strength of the stochastic input which is proportional to A−1/2. Notice that these two parameters can be regarded as measures of deterministic and stochastic input, respectively.

First let us focus on how these two inputs affect the parameter of the ISI tail distribution. We see from the log-linear plots

**FIGURE 6 | Comparison of exponential tail exponents generated by the five methods, for fixed current** *<sup>I</sup>* **<sup>=</sup> <sup>6</sup>***.***<sup>0</sup> <sup>µ</sup>A/cm2, plotted as a function of Area−1***/***<sup>2</sup> where Area takes the values 1000, 900, . . . , 100** µ**m2.**

in **Figures 2**–**5** that the negative slope of the final segment of the ISI distribution, which is the negative exponent of the exponential tail of the distribution, increases with increasing deterministic I, as well as with increasing stochastic input, A−1/2, i.e., these become steeper with decreasing A and with increasing I. In **Figures 6**, **7** the tail exponents are plotted as functions of A−1/<sup>2</sup> and of I, respectively.

To understand this result we return to the state space picture of the stochastic dynamics of the neuron model, represented by **Figure 1B2**, which shows the dynamics for the analogous Morris-Lecar model. Between firings the state of the neuron is in the subthreshold region centered on the fixed point but well inside the unstable limit cycle except while traversing the switching region. The probability that it moves out of this region and fires is greater if either the noise has greater standard deviation or if the deterministic input to which the noise is added is greater. Hence an increase of either *I* or A−1/<sup>2</sup> should have a similar effect on the parameter of the ISI exponential tail distribution. Furthermore, as *I* changes the configuration pictured in **Figure 1B** changes. As *I* moves toward the bifurcation at *I* ≈ 9.8, the bifurcation diagram in **Figure 1A** shows us that the stability of the fixed point decreases, becoming zero at *I* = 9.8, while the unstable limit cycle shrinks and disappears. Correspondingly, the subthreshold region shrinks in size, but does not disappear since one sees short intervals of subthreshold behavior for values of *I* ranging at least as high as 12.0. In **Figure 8** one might note that when *I* = 12 none of the curves have reached 1. Both these effects cause the probability of firing to increase. Reduction in stability means it is easier to escape from the fixed point, while reduction in size of the unstable limit cycle means the size of the subthreshold regime is smaller, thus reducing the expected time to reach the deterministic basin of attraction of the stable limit cycle. The combination of these effects seem to cause the relation between *I* and the exponential tail to be concave, as in **Figure 7**, instead of nearly linear as in the case of noise, as in **Figure 6**. Switching to spiking and maintenance of spiking become more probable, and the exponential tail of the ISI distribution becomes steeper.

Next we consider the effect of increasing *I* or A−1/<sup>2</sup> on the bumps in the middle part of the ISI histograms. To understand this we need to review some theory about the behavior of a similar neuron model during its subthreshold phase. This was studied in detail for the Morris-Lecar model by Ditlevsen and Greenwood (2013). The same analysis applies here with some alterations because the deterministic HH model is 4-dimensional. If we linearize the stochastic model at the fixed point we obtain a linear stochastic system of the form

$$dX = -AX\,dt + C\,dW.\tag{11}$$

The deterministic matrix C is obtained by evaluating the stochastic diffusion coefficient matrix at the fixed point. The matrix A will have a pair of complex eigenvalues, −λ ± *i*ω with negative real part λ ω and other negative eigenvalues −β, −γ where β, γ are greater than λ. This means the system, started near the fixed point, moves rapidly toward the plane defined by the eigenvectors corresponding to the complex eigenvalues. Thus, the 4-dimensional system can be studied in terms of a 2-dimensional system. Other examples are found in Baxendale and Greenwood (2011).

When the neuron fires and then becomes subthreshold, the stochastic path enters the region "inside" the unstable limit cycle at its edge and proceeds to roughly circle the fixed point at a frequency ω, the orbit being damped at a rate λ while also being restrained from damping by the stochastic aspect of the

model. The path of this process can be approximated in terms of a fixed rotation multiplied by a 2-dimensional Ornstein-Uhlenbeck process as shown by Baxendale and Greenwood (2011). The sample path of the process orbits the fixed point for some time, with frequency ω, until it arrives at its stationary distribution. Before stationarity sets in we see one or more decreasing "bumps" in the ISI histogram, with frequency ω, and after stationarity sets in we have the exponential tail, being the escape distribution from a stationary distribution by a standard argument.

As examples, according to computation by Hassard (1978): for *I* = 5, λ ± *i*ω = −0.097 ± 0.521*i*, −β = −0.129, −γ = −4.60; for *I* = 9, λ ± *i*ω = −0.015 ± 0.578*i*, −β = −0.137, −γ = −4.73. We find that the spacing between the second and third bumps in **Figure 4**, where *I* = 6, and also for larger areas (not shown), is approximately 12 ms, which is in rough agreement with 2π/ω ≈ 12.05 ms for *I* = 5 above. It is notable that the eigenvalue frequency ω, and thus the bump spacing, bears no particular relationship to the frequency of the unstable limit cycle (ULC), as computed by Rinzel and Miller (1980). Let *I*<sup>1</sup> ≈ 9.8 be the subcritical Hopf bifurcation. For *I* close to *I*1, *I* < *I*1, the eigenvalue frequency and the ULC frequency are the same at approximately 90 Hz, but as *I* decreases toward *Iv* ≈ 6.26, ω decreases by only 9% while the ULC frequency decreases steeply from 55% from 90 Hz at *I*<sup>1</sup> to approximately 40 Hz at *I* = 7.5 then smoothly reverses direction and increases back up to about 50 Hz at *I*υ. One might also note the reduction in λ as *I* increases from 5 to 9, while β and γ are both much larger than λ, as predicted by Equation (10).

Here we make a comment on the existence of exponential tails for *I* > *I*1. The underlying mechanism of this has been discussed in Rowat and Greenwood (2011). Numerically, it has been shown by Rowat (2007) and Tateno and Pakdaman (2004) that the probability *p*(*I*) that a spike is followed by a non-spike is continuous across *I*1. Note that 1 − *p*(*I*) is the proportion of ISIs in runs of two or more spikes (see **Figures 8**, **9**).

When *I* > *I*1, say with *I* = 11 or *I* = 12, the equilibrium point is now unstable with dominant eigenvalues λ ± *i*ω, where λ is small, 0 < λ ω. Since λ is small one sees numerically that a deterministic trajectory started very close to the unstable

**FIGURE 9 | Proportion of ISIs in runs of two or more spikes, compared by method, for fixed current** *<sup>I</sup>* **<sup>=</sup> <sup>6</sup>***.***<sup>0</sup> <sup>µ</sup>A/cm2, as a function of Area−1***/***2.** Area takes values 1000, 900, . . . , 100µm2.

equilibrium makes several very tight, small, slowly expanding spirals around the equilibrium before switching out to the stable limit cycle (SLC)—i.e., the spiking cycle.

When *I* is below the Hopf bifurcation, with dominant eigenvalues λ ± *i*ω where λ < 0 and −λ ω, Baxendale and Greenwood (2011) identify the stochastic process whereby deterministic damped oscillations, with the addition of noise, show sustained oscillations at an amplitude well above the expected noise level. The exit time from this stochastic equilibrium process is what creates the exponential tail when *I* < *I*1.

In view of the observation above, that when *I* > *I*1, several small tight spirals may occur before a deterministic trajectory, started close to the unstable FP, switches out to the SLC, it seems reasonable to propose that in the presence of noise there is a short-term stochastic process that tends to contract the slowly expanding deterministic spirals, thus creating a conditional equilibrium for a short time before the trajectory switches out to the SLC. Thus, the exit time from this conditional stochastic equilibrium has an exponential distribution that creates the exponential tail when *I* > *I*1.

The number and pronounced definition of the bumps become less as the noise increases because the onset of stationarity is hastened by more noise. We see this effect in **Figure 4**. In **Figure 5** we observe that changing *I* with the noise level fixed has much less effect on the size and definition of the "bumps."

Finally, how do the parameters *I* and *A*−1/<sup>2</sup> affect the height, width, and location of the main peak of the ISI histogram? In **Figure 8** we have plotted the proportion of ISI mass in runs of two or more spikes as a function of input *I* for fixed A = 400µm2. This is represented in the histogram as the area under the first peak. It increases roughly linearly with *I* except for saturation at 0 and 1. The main peak moves right as *I* decreases and as A increases.

Note that any occurrence of a run of two or more spikes corresponds to the occurrence of a spike immediately followed by another spike, hence the proportion of ISI histogram mass under the first peak is in fact the probability that a spike is followed by another spike. Equivalently, the histogram mass or area under the tail (including any "bumps") is the probability that a spike is followed by a period of quiescent behavior.

**Figures 8**, **9** show that the proportion of ISIs occurring in runs of two or more spikes increase roughly linearly with *I* and with *A*−1/2, respectively, when the other variable is fixed. This is reflected in the ISI distribution as an increase in the height and width of the initial peak. The reasons for increasing steepness of the exponential tail apply equally to the increase we observe in **Figures 8**, **9**. A large negative tail exponent implies less area, or "probability mass," under the tail and more mass in the main peak. Thus, the increases seen in **Figures 8**, **9** correlate well with the increases in negative tail exponent in **Figures 6**, **7**.

In **Table 4**, we give numerical values for the exponential tail exponent, the proportion of ISIs in runs of two or more spikes, and the running times, across all the models, for one specific (Area, I) combination, namely Area = 400µm2, *I* = 6.0µA/cm2. These computer simulations were all run consecutively on the same hardware. We see that for these parameters, the Fox-Goldwyn and Orio-Kurtz methods are equally close (within a few percent), the Güler method a little further away, and the Linaro method further away again.

ISI densities were also computed by Verechtchaguina et al. (2007) and Engel et al. (2008) by a different method and for a different neuron. An electrical circuit was used to capture the frequency-dependent subthreshold dynamics in stellate and pyramidal cells of the entorhinal cortex, which was converted to a noise-driven harmonic oscillator; from this they analytically computed ISI densities.

## **5. CONCLUSION**

**Figures 6**–**9** and **Table 4** together show that the Fox-Goldwyn, and Orio-Kurtz methods both generate ISI histograms very close to those of Micro. The Güler histograms are not quite as close and the Linaro histograms are only a little further off.

According to Kurtz' theorem the Orio method gives an error of at most log (*N*)/*N* which is 0.0001 for the data sets computed here (*N* = 105). Hence it should be regarded as a "gold standard" for producing a good approximation to the ISI distribution of the Markov chain model. However, when computation time is an issue, one might well prefer to use the Güler model which runs about three times as fast as the Linaro and Orio models. This was true for our Python implementations on a 2.5 GHz Intel Core i5 and will no doubt generalize to other languages and systems. We used the same basic code framework for the Güler, Linaro, and Orio methods. The main reason for the increased speed is that



*Column 2 lists the exponential exponent of the histogram tail, while column 3 lists the probability that a spike is immediately followed by another spike. This is equivalent to the proportion of spikes that occur in a run of two or more spikes. In the lower four rows of columns 2 and 3, the first figure is the actual parameter value while the second figure is its difference from the corresponding value for the Micro simulation. The fourth column gives the running time on our hardware and the fifth the computer language used. Both the Fox-Goldwyn and Linaro code were retrieved from the ModelDB repository.*

the Güler simulation calls the random number generator much less often than the others. In addition, the Güler method uses considerably fewer algebraic operations. Unfortunately the Fox-Goldwyn model was implemented in Fortran so its computation time cannot reasonably be compared with the other three SDE models.

Although the Güler method generates histogram parameters further away from the Micro histogram parameters than either the Fox-Goldwyn or Orio-Kurtz histogram parameters, one must bear in mind that when introducing harmonic Brownian oscillator-type SDEs, there are six phenomenological parameters in the Güler method that were carefully chosen by examination of simulations of Micro voltage data in a subthreshold regime, with *I* = −4µA/cm<sup>2</sup> (to avoid spikes). It may be that if these parameters were chosen with reference to Micro simulation voltage data generated with another *I*-value, e.g., in the middle of the bistability interval [6.2, 9.8], the parameters of the Güler histograms could be much closer to the parameters of the Micro histograms.

## **REFERENCES**


Yu, X., and Lewis, E. R. (1989). Studies with spike initiators: linearization by noise allows continuous signal modulation in neural networks. *IEEE Trans. Biomed. Eng.* 36, 36–43. doi: 10.1109/10. 16447

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 26 May 2014; accepted: 25 August 2014; published online: 08 October 2014.*

*Citation: Rowat PF and Greenwood PE (2014) The ISI distribution of the stochastic Hodgkin-Huxley neuron. Front. Comput. Neurosci. 8:111. doi: 10.3389/fncom. 2014.00111*

*This article was submitted to the journal Frontiers in Computational Neuroscience. Copyright © 2014 Rowat and Greenwood. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Systematic analysis of the contributions of stochastic voltage gated channels to neuronal noise

#### *Cian O'Donnell 1,2\* and Mark C. W. van Rossum2 \**

*<sup>1</sup> Computational Neurobiology Laboratory, Salk Institute for Biological Studies, La Jolla, CA, USA*

*<sup>2</sup> School of Informatics, Institute for Adaptive and Neural Computation, University of Edinburgh, Edinburgh, UK*

#### *Edited by:*

*Benjamin Lindner, Bernstein Center for Computational Neuroscience, Germany*

#### *Reviewed by:*

*Andre Longtin, University of Ottawa, Canada Eric Shea-Brown, University of Washington, USA Richard Naud, University of Ottawa, Canada*

#### *\*Correspondence:*

*Cian O'Donnell, Computational Neurobiology Laboratory, Salk Institute for Biological Studies, 10010 N Torrey Pines Road, La Jolla, CA 92037, USA e-mail: cian@salk.edu; Mark C. W. van Rossum, School of Informatics, 10 Crichton Street, Edinburgh EH8 9AB, UK e-mail: mvanross@inf.ed.ac.uk*

## **1. INTRODUCTION**

An obvious characteristic of behavior is the variability that one observes from trial to trial in even the most controlled settings. This behavioral variability is reflected at the neural level in the noisy character of spike trains. Various hypotheses have been put forward for a potential functional role of neural variability, such as stochastic resonance (McDonnell and Abbott, 2009), prevention of synchrony (van Rossum et al., 2002), and probabilistic sampling (Buesing et al., 2011). A number of factors can contribute to trial-to-trial variability: non-stationarity and unobserved modulation of the nervous system; chaotic network dynamics resulting from deterministic single neuron dynamics (van Vreeswijk and Sompolinsky, 1996); and biophysical noise. In this paper we concentrate on the latter, and in particular on the noise from voltage-gated ion channels.

Ion channels are pore-forming macromolecular proteins that allow the selective passage of ionic currents in and/or out of the cell (Hille, 2001). Each ion channel can, at any given moment, occupy only one of multiple discrete conformational states; at least one of which is an open/conducting state, and at least one of which is a closed/non-conducting state. Transitions between states are exceedingly rapid (<1µs) and, like all molecular reactions, stochastic in nature—they are driven by thermal agitation. In the case of voltage-gated channels (VGCs) considered here, the transition probabilities depend on the cell's membrane potential. Channels are commonly modeled as Markov processes, which

Electrical signaling in neurons is mediated by the opening and closing of large numbers of individual ion channels. The ion channels' state transitions are stochastic and introduce fluctuations in the macroscopic current through ion channel populations. This creates an unavoidable source of intrinsic electrical noise for the neuron, leading to fluctuations in the membrane potential and spontaneous spikes. While this effect is well known, the impact of channel noise on single neuron dynamics remains poorly understood. Most results are based on numerical simulations. There is no agreement, even in theoretical studies, on which ion channel type is the dominant noise source, nor how inclusion of additional ion channel types affects voltage noise. Here we describe a framework to calculate voltage noise directly from an arbitrary set of ion channel models, and discuss how this can be use to estimate spontaneous spike rates.

**Keywords: channel noise, voltage-gated ion channels, Hodgkin–Huxley, spontaneous firing, simulation**

lead to accurate predictions of the noise in macroscopic currents recorded from neurons (Hille, 2001).

Because spike generation appears reliable during somatic current injection (Calvin and Stevens, 1968; Bryant and Segundo, 1976; Mainen and Sejnowski, 1995) and the number of VGCs is large, it is typically believed that the stochastic gating of VGC contributes little to the total observed variability in neuronal spiking. However, such a conclusion might be premature. First, during somatic current injection the dendrites are typically more hyperpolarized compared to the realistic case where the neuron receives synaptic input, and the noise typically decreases strongly with hyper-polarization (see below). Moreover, the total number of channels in a neuron might be large, but in spatial compartments such as narrow axons or dendrites, the number of channels is typically small.

Several experimental studies have focused on the physiological consequences of ion channel noise. Sigworth (1980) used fluctuation analysis to estimate the number of Na<sup>+</sup> channels at a single frog node of Ranvier ∼30000, and subsequently used formulae from Lecar and Nossal (1971a,b) to estimate fluctuations in the current threshold of action potential generation due to channel noise. Johansson and Arhem (1994) found that the stochastic opening of a small number of channels in cultured hippocampal neurons were sufficient to trigger spontaneous action potentials. White et al. (1998) recorded subthreshold membrane potential oscillations in stellate cells of layer II entorhinal cortex (EC) and found that they could only reproduce the co-existence of both oscillations and spiking in a computational model if they included stochastic gating of Na<sup>+</sup> channels, suggesting a form of periodic stochastic resonance. Subsequently, Dorval and White (2005) used the dynamic clamp technique to inject a "virtual" Na<sup>+</sup> conductance which was either deterministic or stochastic to EC stellate cells *in vitro*. Only the stochastic conductance could reproduce the observed membrane potential oscillations. Similarly, Dudman and Nolan (2009) used computational models of the same cell type to demonstrate that stochastic channel gating can also account for the clustered firing patterns displayed by these cells when stimulated by steady current pulses *in vitro*. Diba et al. (2004) characterized somatic subthreshold voltage noise in cultured hippocampal neurons due to stochastic ion channel gating. Voltage fluctuations were small, with a standard deviation <0.3 mV and based on pharmacological experiments appeared to arise primarily from K<sup>+</sup> channels. Jacobson et al. (2005) reported similar results from neocortical pyramidal cells from layer IV/V of rat somatosensory cortex brain slices, with similar amplitude (submillivolt) voltage fluctuations. Finally, Kole et al. (2006) used fluctuation analysis to measure the properties and distribution of hyperpolarization-activated cation (Ih) channels in LV neocortical pyramidal cells *in vitro*. They found that although the Ih single-channel conductance was exceedingly small (<1 pS), Ih channels contribute substantially to voltage noise in the distal dendrites of these cells.

A great deal of theoretical and numerical studies have looked at membrane noise from stochastic ion channels, beginning with Lecar and Nossal, who used stochastic differential equations and a reduced dynamical system model of the action potential to attempt to quantify the magnitude of membrane noise on action potential threshold fluctuations (Lecar and Nossal, 1971a,b). Skaugen and Walløe (1979) were the first to examine the consequences of stochastic gating of ion channels through numerical simulation. They found that in the stochastic version of Hodgkin– Huxley (HH) squid giant axon model the current threshold was lowered compared to deterministic models, the membrane could spike spontaneously, and that the frequency-current curve was smeared around the threshold. Subsequent simulation work by DeFelice and colleagues (Clay and DeFelice, 1983; Strassberg and DeFelice, 1993) further elaborated on the direct link between the microscopic (stochastic) and macroscopic (deterministic) versions of the HH model. Rubinstein (1995) simulated a model of the frog node of Ranvier and reproduced the spread in action potential firing threshold due to stochastic channel gating predicted by Lecar and Nossal, in agreement with earlier experiments (Verveen, 1960). Chow and White (1996) examined the dependence of spontaneous firing rate in the stochastic HH model on membrane patch area and found it to decrease exponentially with area. They approximated the system as a boundary escape problem, with stochastic gating of the activation subunit of the Na<sup>+</sup> channel as the noise source. They found that the mean escape time as a function of area agreed well with numerical simulation results (we will comment on this finding below). Manwani and Koch (1999) used a perturbative approach to compare the contributions of thermal noise, channel noise and synaptic "noise" (from Poissonian inputs) to total membrane voltage noise in a single compartment. Steinmetz et al. (2000) used similar methods to demonstrate the voltage and channel type dependence of ion channel noise spectra for both the HH model and a commonly used neocortical pyramidal cell model (Mainen et al., 1995). In the present study we employ methods very similar to both of these works, but toward a different goal: we aim to systematically separate all of the contributing factors that determine the contribution of an ion channel type to voltage noise and spontaneous firing.

In general detailed simulation of stochastic channels will give the most accurate answer regarding the noise and the contribution of the different channels. But as stochastic simulation of the full channel kinetics is very involved, several recent studies have developed approximate stochastic-differential equation models that efficiently capture the essence of the noise statistics of discrete ion channels (Goldwyn and Shea-Brown, 2011; Goldwyn et al., 2011; Linaro et al., 2011; Orio and Soudry, 2012). Our objective here is complementary but different: rather than developing a precise model for the noise we seek to *estimate* the contribution of the various channel types. Intuitively it is not clear what properties of a given channel type are important determinants for noise. This is relevant when a full state diagram of a certain channel type is not available, but nevertheless a coarse estimate of its contribution to noise is desired. At the same time by breaking down the various factors that determine the magnitude of the noise of a certain channel type, a deeper insight in the results from simulations and experiments can be obtained.

Our study is split in four parts: We use simulations to demonstrate that in the well-studied stochastic Hodgkin–Huxley model, most spikes are due to stochastic K<sup>+</sup> channel gating and not the Na<sup>+</sup> channel. This is of interest as in the literature conflicting findings can be found (see Discussion). Next, we review the different factors that explain how stochastic channel noise leads to noise in membrane voltage, using a linear, weak noise analysis and explain why the K<sup>+</sup> noise is dominant. While these individual contributing factors are well-known, a concise account was in our opinion lacking. Third, we examine the relation between voltage noise and spontaneous spikes using an approach recently introduced for integrate-and-fire neurons. We show that this relation is complex, but that nevertheless rough estimates are possible. Finally, we apply the same methods to analyze a CA1 pyramidal neuron model to show that the approach is easily generalizable to other neuron models.

## **2. MATERIALS AND METHODS**

All stochastic channel simulations were implemented using the Parallel Stochastic Ion Channel Simulator (PSICS) which is specifically designed for efficient simulations of stochastic ion channel gating in single neuron models (see Cannon et al., 2010 and http://psics.org/ for algorithmic details). Current noise injection in deterministic HH models was done using NEURON (Carnevale and Hines, 2006). Analysis was done using MATLAB (The Mathworks).

## **2.1. HODGKIN–HUXLEY MODELS**

All simulations of the Hodgkin–Huxley model used the standard voltage-dependent equations for Na<sup>+</sup> and K<sup>+</sup> gating schemes, at 6.3◦C (Hodgkin and Huxley, 1952). For completeness the standard parameters are given in **Table 1**.

In the HH squid axon model, the sodium conductance obeys

$$\mathcal{g}\_{\rm Na}(V, t) = \mathcal{y}\_{\rm Na} \rho\_{\rm Na} A m^3(V, t) h(V, t)$$

where γ*Na* is the single channel sodium conductance, ρ*Na* is the density of channels per area, and *A* is the area. The gating variables *m* and *h* move between off and on-states with a voltage-dependent rate α*m*, *<sup>h</sup>*(*V*) from the off-state to the on-state, and back with a rate β*m*, *<sup>h</sup>*(*V*). These rates have been empirically established as

$$\alpha\_m(V) = \frac{0.1(V+40)}{1 - e^{-(V+40)/10}} \quad \beta\_m(V) = 4e^{-(V+65)/18}$$

$$\alpha\_h(V) = 0.07e^{-(V+65)/20} \quad \beta\_h(V) = \frac{1}{1 + e^{-(V+35)/10}}$$

where *V* is membrane voltage in mV and the transition rates have units 1/ms.

In the limit of very many channels the gating variables are a continuous quantity, namely the probability to find them in the on-state. The dynamics of gating variable *m*(*V*, *t*) can be written as

$$\frac{dm(V,t)}{dt} = \frac{m\_{\infty}(V) - m(V,t)}{\mathfrak{r}\_{m}(V)}\tag{1}$$

where *m*∞(*V*) is the steady-state value for the activation variable,

$$m\_{\infty}(V) = \frac{\alpha\_m(V)}{\alpha\_m(V) + \beta\_m(V)}\tag{2}$$

and τ*m*(*V*) is its time constant

$$\pi\_m(V) = \frac{1}{\alpha\_m(V) + \beta\_m(V)}\tag{3}$$

and analogous for the inactivation variable *h*(*V*, *t*). The dynamics in the continuum limit is fully deterministic.

When the system is described stochastically, the gating variables of each channel are binary variables that switch between off-state and on-state. For the Na<sup>+</sup> channel to be open, all its 3 *m*'s and the *h* switch need to be in the on-state. To (naively) simulate

**Table 1 | The Hodgkin–Huxley parameters for model simulations.**


this case the transitions are drawn stochastically using a random number generator, using a time-step δ*t* such that αδ*t*,βδ*t* 1.

Likewise, the K<sup>+</sup> conductance is given by

$$\mathbf{g}\_{\mathcal{K}}(V, t) = \mathcal{Y}\_{\mathcal{K}} \rho\_{\mathcal{K}} \mathcal{A} n^4(V, t)$$

It has four identical activation variables, labeled *n*, with rates

$$\alpha\_n(V) = \frac{0.01(V+55)}{1 - e^{-(V+55)/10}} \quad \beta\_n(V) = 0.125e^{-(V+65)/80}$$

In all simulations the single channel conductance for both Na<sup>+</sup> and K<sup>+</sup> was 20 pS. Although this value is close to that reported experimentally for the squid giant axon K<sup>+</sup> conductance (Llano et al., 1988), it is slightly larger than experimental estimates for the Na<sup>+</sup> conductance (Bezanilla, 1987). These values were chosen for simplicity (it removes one confounding factor when comparing channel type noise contributions) and to enable comparison with the literature (Strassberg and DeFelice, 1993; Chow and White, 1996; Schneidman et al., 1998). In line with the literature, leak channels were modeled deterministically, although in more realistic models they should be made stochastic as well.

For the simulations and analysis of the hippocampal pyramidal cell model (**Figure 6**), we use the channel models for active Na+, delayed rectifier K<sup>+</sup> (Kdr), and A-type K<sup>+</sup> channel (Ka) exactly as previously published by Migliore et al. (1999), Jarsky et al. (2005). However, our model was single-compartment while these previous studies looked at multi-compartment model neurons. For consistency with the HH simulations we also choose a single-channel conductance of 20 pS. The channel densities were matched to the macroscopic conductance densities of the soma in the model of Jarsky et al. (2005), implying Na+: 20 channels /µm2, Kdr: 20/µm2, and Ka: 24/µm2. In addition to these active channels, we added two voltage-independent leak channels, one permeable to Na<sup>+</sup> and one permeable to K+, which we simulated deterministically. We chose their densities 0.0065 mS/cm<sup>2</sup> (Na+) and 0.0185 mS/cm<sup>2</sup> (K+), to fit two constraints: a total leak conductance of 0.025 mS/cm<sup>2</sup> (Jarsky et al., 2005), and a resting voltage of −65 mV. Finally, as according to Migliore et al. (1999) we set reversal potentials *ENa* = +55 mV and *EK* = −90 mV. When attempting to analytically calculate the membrane impedance for this model we unfortunately found that it diverged upward at around −60 mV. This singularity is problematic because it would break our small voltage noise assumption. Hence we instead estimated the impedance for this model empirically as is done in experiments. We injected sine wave currents over a large frequency range to the deterministic version of the model, and measured the amplitude of the resulting voltage responses.

#### **2.2. CALCULATION OF POWERSPECTRA OF K+ AND NA+ NOISE**

As is well known, the current noise power spectrum from a population of ion channels follows directly from the channel kinetic scheme and its autocovariance function (DeFelice, 1981). For ease of presentation we first summarize this calculation for a two-state channel. The conditional probability that a two-state channel is open at time *t* given that it was open at time 0, *po*<sup>|</sup>*o*(*t*) is

$$p\_{o|o}(t) = p\_{\infty} + (1 - p\_{\infty})e^{-t/\tau}$$

where *p*∞ is the steady-state open probability and τ is the correlation time. The autocorrelation *po*(*t*0)*po*(*t*) = *p*∞*po*<sup>|</sup>*o*(*t*). The autocovariance of a single channel *Co*(*t*) open probability can then be written as

$$\begin{aligned} C\_o(t) &= \langle \left[ p\_o(t\_0) - p\_\infty \right] \left[ p\_o(t) - p\_\infty \right] \rangle \\ &= p\_\infty p\_{o|o}(t) - p\_\infty^2 \\ &= p\_\infty (1 - p\_\infty) e^{-t/\tau} \end{aligned}$$

The autocovariance of the current through a population of *N* such channels, *CI*(*t*), is simply given by

$$C\_I(t) = \text{Ni}^2 C\_o(t)$$

where *i* is the single-channel current. Note that the autocovariance function at *t* = 0 is equal to the variance, *CI*(0) = *Ni*2*p*∞(1 − *p*∞) = σ2, and decays exponentially with time constant τ , so that when *t* τ , *CI*(*t*) → 0.

The Wiener-Khinchin theorem states that the power spectrum is equal to the real part of the Fourier transform of the autocovariance function

$$S\_I(\omega) = 4 \int\_0^\infty \mathcal{C}\_I(t) e^{-2\pi i \theta t} dt$$

$$= S\_I(0) \frac{1}{1 + \left(2\pi f \tau\right)^2} \tag{4}$$

where *SI*(0) = 4*Ni*2*p*∞(1 − *p*∞)τ and the pre-factor 4 arises from our definition of spectral density. Thus for the two-state channel population, the power spectrum is a single Lorenztian function with a corner frequency *fc* = 1/(2πτ ). Above this corner frequency, the power of the signal falls off ∝ 1/*f* 2.

In an analogous way we can calculate the power spectra of the HH Na<sup>+</sup> and K<sup>+</sup> channel populations. For the HH K<sup>+</sup> channel composed of four identical and independent sub-units, the conditional probability that the channel is open at time *t* given that it was open at time 0 is

$$p\_{K,o|o}(t) = \begin{pmatrix} n\_{\infty} + (1 \ - \ n\_{\infty})e^{-t/\tau\_n} \end{pmatrix}^4 \tag{5}$$

Hence, the autocovariance of the current noise from the HH K<sup>+</sup> channel population is a sum of exponentials,

$$\begin{split} C\_{IK}(t) &= N\_K i\_K^2 \left( n\_{\infty}^4 p\_{K, o \mid o}(t) - \left( n\_{\infty}^4 \right)^2 \right) \\ &= N\_K i\_K^2 n\_{\infty}^4 \left[ (1 - n\_{\infty})^4 e^{-4t/\tau\_{\pi}} + 4n\_{\infty} (1 - n\_{\infty})^3 e^{-3t/\tau\_{\pi}} \right] \\ &+ 6n\_{\infty}^2 (1 - n\_{\infty})^2 e^{-2t/\tau\_{\pi}} + 4n\_{\infty}^3 (1 - n\_{\infty}) e^{-t/\tau\_{\pi}} \right] \end{split}$$

The corresponding power spectrum of the current noise is

$$\begin{split} S\_{IK}(\omega) &= 4N\_k n\_{\infty}^4 i\_K^2 \tau\_n \left[ (1 - n\_{\infty})^4 \frac{4}{16 + \omega^2 \tau\_n^2} \right. \\ &\left. + n\_{\infty} (1 - n\_{\infty})^3 \frac{12}{9 + \omega^2 \tau\_n^2} + n\_{\infty}^2 (1 - n\_{\infty})^2 \frac{12}{4 + \omega^2 \tau\_n^2} \right. \end{split}$$

$$\left[ + n\_{\infty}^{3} (1 - n\_{\infty}) \frac{4}{1 + \omega^{2} \tau\_{n}^{2}} \right]$$

This is the sum of four Lorenztians with corner frequencies equal to 4/(2πτ*n*), 3/(2πτ*n*), 2/(2πτ*n*), and 1/(2πτ*n*). Because at the resting potential *n*∞ is close to zero, the first term in the square brackets with correlation time τ*n*/4 will dominate the power spectrum.

Similarly, for the Na<sup>+</sup> current noise power spectrum one has

$$p\_{\rm Na, o|o} = \left(m\_{\infty} + (1 - m\_{\infty})e^{-t/\tau\_m}\right)^3 \left(h\_{\infty} + (1 - h\_{\infty})e^{-t/\tau\_h}\right)^3$$

and the Na<sup>+</sup> current noise autocovariance is

$$\begin{split} C\_{\text{Na}}(t) &= N\_{\text{Na}}i\_{\text{Na}}^{2} \left( m\_{\infty}^{3}h\_{\infty}p\_{\text{Na},o|o} - \left( m\_{\infty}^{3}h\_{\infty} \right)^{2} \right) \\ &= N\_{\text{Na}}i\_{\text{Na}}^{2}m\_{\infty}^{3}h\_{\infty} \left[ 3m\_{\infty}^{2}h\_{\infty}(1 - m\_{\infty})e^{-t/\tau\_{m}} \right. \\ &\left. \left. + 3m\_{\infty}h\_{\infty}(1 - m\_{\infty})^{2}e^{-2t/\tau\_{m}} \right. \right. \\ &\left. + h\_{\infty}(1 - m\_{\infty})^{3}e^{-3t/\tau\_{m}} \right. \\ &\left. + m\_{\infty}^{3}(1 - h\_{\infty})e^{-t/\tau\_{h}} \right. \\ &\left. + 3m\_{\infty}^{2}(1 - m\_{\infty})(1 - h\_{\infty})e^{-t/\tau\_{m} - t/\tau\_{h}} \right. \\ &\left. + 3m\_{\infty}(1 - m\_{\infty})^{2}(1 - h\_{\infty})e^{-2t/\tau\_{m} - t/\tau\_{h}} \right. \\ &\left. + (1 - m\_{\infty})^{3}(1 - h\_{\infty})e^{-3t/\tau\_{m} - t/\tau\_{h}} \right] \end{split}$$

The corresponding power spectrum of the Na<sup>+</sup> current noise is

*SINa*(ω) = 4*NNai* 2 *Na*(*m*<sup>3</sup> <sup>∞</sup>*h*∞) 2 1 − *m*<sup>∞</sup> *m*∞  3τ*<sup>m</sup>* 1 + ω2τ <sup>2</sup> *m* + 1 − *m*<sup>∞</sup> *m*∞ <sup>2</sup> 6τ*<sup>m</sup>* 4 + ω2τ <sup>2</sup> *m* + 1 − *m*<sup>∞</sup> *m*∞ <sup>3</sup> 3τ*<sup>m</sup>* 9 + ω2τ <sup>2</sup> *m* + 1 − *h*<sup>∞</sup> *h*∞  τ*<sup>h</sup>* 1 + ω2τ <sup>2</sup> *h* + 3 1 − *h*<sup>∞</sup> *h*∞  1 − *m*<sup>∞</sup> *m*∞  τ*m*τ*<sup>h</sup>* τ*<sup>m</sup>* + τ*<sup>h</sup>* 1 1 + (ωτ*m*τ*h*/(τ*<sup>m</sup>* + τ*h*)) 2 + 3 1 − *h*<sup>∞</sup> *h*∞  1 − *m*<sup>∞</sup> *m*∞ <sup>2</sup> τ*m*τ*<sup>h</sup>* τ*<sup>m</sup>* + 2τ*<sup>h</sup>* 1 1 + (ωτ*m*τ*h*/(τ*<sup>m</sup>* + 2τ*h*)) 2 + 1 − *h*<sup>∞</sup> *h*∞  1 − *m*<sup>∞</sup> *m*∞ <sup>3</sup> τ*m*τ*<sup>h</sup>* τ*<sup>m</sup>* + 3τ*<sup>h</sup>* 1 1 + (ωτ*m*τ*h*/(τ*<sup>m</sup>* + 3τ*h*)) 2 

Near rest *m*<sup>∞</sup> 1 and *h*<sup>∞</sup> ≈ 0.6, so that the third and last Lorentzians dominate the powerspectrum. The corresponding corner-frequencies are 3/(2πτ*m*) and (τ*<sup>m</sup>* + 3τ*h*)/(2πτ*m*τ*h*), which are virtually identical.

We calculate the quasi-active (linearized) membrane impedance using standard methods (Mauro et al., 1970; Koch, 1999).

## **3. RESULTS**

## **3.1. STOCHASTIC POTASSIUM CHANNELS CAN TRIGGER SPONTANEOUS ACTION POTENTIALS**

Previously, it has been demonstrated that a Hodgkin–Huxley (HH) type neural model with discrete Markovian stochastic ion channels instead of the classic continuous deterministic rate equations can fire spontaneous action potentials if the membrane patch is small (Skaugen and Walløe, 1979; Clay and DeFelice, 1983; Strassberg and DeFelice, 1993; Chow and White, 1996; Schneidman et al., 1998). However, the relative contributions of the Na<sup>+</sup> and K<sup>+</sup> channel populations to spontaneous activity are less well understood. To investigate, we simulate the HH squid axon model using the PSICS simulator (Cannon et al., 2010) with stochastic Markovian ion channels while varying the membrane patch area under three different conditions: first, both sodium (Na+) and potassium (K+) channels stochastic ("all stochastic"), second, Na<sup>+</sup> stochastic but K<sup>+</sup> deterministic, and third, Na<sup>+</sup> deterministic but K<sup>+</sup> stochastic. Comparing the spontaneous firing rate between the three conditions allows us to find whether Na<sup>+</sup> or K<sup>+</sup> channels contribute most to spontaneous activity.

As observed previously (Chow and White, 1996), if a fixed density of ion channels is assumed, then the firing rate decreases approximately exponentially with increasing membrane patch area (**Figure 1**) such that membrane areas greater than ∼400µm<sup>2</sup> produce almost no spontaneous action potentials approximating the deterministic model. This exponential dependence of spontaneous rate with membrane area is consistent with a stochastic barrier-escape problem (Chow and White, 1996). As the channels are independent, the voltage variance is proportional to the number of channels *N*, but inversely proportional to the square of the area *A*, since the input impedance decreases linearly with area. Combining these two opposing factors, the spontaneous rate scales as exp ( − *A*2/*N*) ∝ exp ( − *A*).

When either Na<sup>+</sup> or K<sup>+</sup> channels are switched to deterministic mode, spontaneous firing rate is reduced compared to the fully stochastic mode. Surprisingly, however, stochastic K+ channel gating alone triggers greater spontaneous firing rates than stochastic Na+-channel gating alone (**Figure 1B**). At first impression, this result might be counter-intuitive because the opening of Na<sup>+</sup> channels is necessary for the initiation of an action potential, while the much slower K<sup>+</sup> channels are conventionally considered responsible for the re-polarizing phase. A simple conceptual model for spontaneous spike generation might therefore be that the chance opening of a few Na<sup>+</sup> channels depolarizes the membrane and activates the runaway Na<sup>+</sup> channel opening underlying the action potential. However, stochastic *closure* of K<sup>+</sup> channels can also depolarize the membrane, similarly activating Na<sup>+</sup> channels to trigger an action potential. We test this possibility by examining the dynamics of Na<sup>+</sup> and K<sup>+</sup> currents. We adapt the 'spike-triggered average' (STA) measure from the neural coding literature. Here we determine the average total current of a given ion channel population *x* at time interval *t* prior to a spontaneous action potential at time *ti*, averaged over *n* such events,

$$I\_{\mathbf{x}}^{\mathrm{STA}}(t) = \frac{1}{n} \left[ \sum\_{i=1}^{n} I\_{\mathbf{x}}(t\_i - t) \right]$$

In the "all stochastic" mode (**Figure 2A**, solid curves), we find that the STA potassium current *ISTA <sup>K</sup>* (*t*), drops between 8 and 2

ms before the spike, while there is a simultaneous increase in the STA sodium current *ISTA Na* (*t*). Nearer to the spike the Na<sup>+</sup> and K<sup>+</sup> currents grow rapidly but in opposite directions as the action potential forms. A positive current corresponds to depolarization. This depolarizing action of the K<sup>+</sup> current change be clearly seen in **Figure 2B** where we plot *IK*(*t*) and *INa*(*t*), the change in Na<sup>+</sup> and K<sup>+</sup> relative to resting current. Importantly, the change in *ISTA <sup>K</sup>* precedes the increase in *<sup>I</sup>STA Na* (**Figure 2B**), suggesting that spontaneous action potential firing in this model is primarily driven by K<sup>+</sup> channel fluctuations, not Na<sup>+</sup> noise. We test this explanation by simulating K<sup>+</sup> channel conductance in deterministic mode and repeating the STA measurement. As expected, in this case spontaneous spikes are not preceded by a drop in K<sup>+</sup> conductance, but instead driven by an elevated Na<sup>+</sup> conductance fluctuation (**Figure 2A**, dotted curves).

Further examination of the Na<sup>+</sup> and K<sup>+</sup> current dynamics confirm these findings. In **Figure 2C** we plot the timecourse of *INa*(*t*) (y-axis) vs. *IK*(*t*) (x-axis) from 10 to 1 ms prior to each recorded action potential together on the same plot. Regions to the lower-right of the identity diagonal (dashed black line) indicate timepoints where K<sup>+</sup> current fluctuations are contributing more to voltage depolarization than Na<sup>+</sup> current fluctuations. The mean STA curve for the "all stochastic" model (solid curve) initially takes off into this right-hand region. In contrast, the Na+ only stochastic (dotted curve) moves only slightly to the right of the origin before taking off in the vertical (Na+-driven) direction.

**Figures 2A–C** plot the average behavior. As the system is stochastic, we expect that occasionally Na<sup>+</sup> fluctuations trigger spikes as well. To get a sense of the spike-to-spike variability in **Figure 2D** we plot the percentage of cases where the potassium current exceeds the sodium current, *IK*(*t*) > *INa*(*t*), as a function of time before spike. In general, this quantity is timedependent because the depolarization due to K<sup>+</sup> fluctuations will most likely be maximal at some time between 8 and 2 ms before the spike. Once the full spike upswing begins (∼1 ms before *t* = 0) Na<sup>+</sup> always dominates. In the all-stochastic case, the earliest phase of most action potentials are K+-driven. In contrast, in the Na<sup>+</sup> stochastic case (dotted line) the majority of spikes are driven primarily by Na<sup>+</sup> fluctuations.

A slightly different picture appears for the K<sup>+</sup> stochastic simulations. In this mode, Na<sup>+</sup> fluctuations are removed so all spikes must by initially triggered by K<sup>+</sup> fluctuations. Consequently, the spike-triggered average K<sup>+</sup> current fluctuation amplitude is even greater than in the all-stochastic model (**Figures 2A–C**) and an even larger percentage of spikes are driven by momentary fluctuations in K<sup>+</sup> currents (**Figure 2D**). In summary, these simulations show that K<sup>+</sup> channel noise is the dominant driver of spontaneous spiking in the stochastic Hodgkin–Huxley model.

Schneidman et al. (1998) looked at stochastic Na<sup>+</sup> and K<sup>+</sup> channel trajectories during spike initiation to address whether a sufficiently strong stimulus can override the intrinsic channel noise. However, in contrast to our STA analysis for determining the contributions of Na<sup>+</sup> vs. K<sup>+</sup> noise to spontaneous spiking, they examined stimulus-driven firing by injecting a fluctuating current into the model neuron. In this stimulus-driven case, the observed trajectories of the Na<sup>+</sup> vs. K<sup>+</sup> currents combine the effect of the stimulus current dynamics and the effect of the channel noise. Our analysis however shows that the drop in noisy K<sup>+</sup> current occurs naturally before the spike.

#### **3.2. THE FACTORS DETERMINING A CONDUCTANCE'S CONTRIBUTION TO MEMBRANE NOISE**

These results lead to the questions: What properties of the HH K<sup>+</sup> conductance cause it to trigger more spontaneous action potentials than the Na<sup>+</sup> conductance? And how can the contribution of an arbitrary channel type be estimated? We proceed by first calculating the resulting noise in the membrane voltage. In the limit of large areas (small noise), this can be calculated exactly. In the subsequent section we relate the voltage noise to spontaneous spike rates. This will turn out to be only approximately possible.

There are at least five possible factors that determine a channel population's contribution to membrane noise:

## *3.2.1. Open probability (11* **×** *noisier for K***+***)*

First, Na<sup>+</sup> and K<sup>+</sup> have different steady-state open probabilities at resting membrane potential. The steady-state probability of a single ion channel being open is identical to the steady-state permeability fraction of the corresponding macroscopic conductance in the classic HH formalism. The steady-state Na<sup>+</sup> conductance is obtained from the product of the steady-state values of the *m*∞ and *h*∞ gating variables:

$$\operatorname{g\_{Na\infty}}(V) = \bar{\operatorname{g}}\_{\mathrm{Na}}\left[m\_{\infty}(V)\right]^{\mathfrak{F}}h\_{\infty}(V)$$

where *g*¯*Na* is the maximal conductance through the Na<sup>+</sup> channel population. Hence the open probability *po* = (*m*∞)3*h*∞. The gating variables *m*∞ and *h*∞ can in turn be expressed in terms of the forward and backward gating rates α*<sup>m</sup>* and β*m*, and α*<sup>h</sup>* and β*h*, see Equation 2.The steady-state K<sup>+</sup> open probability equals *po* = [*n*∞(*V*)] 4. At the resting potential of −65 mV in the HH squid axon model, the steady-state open probabilities are ∼0.000089 for the Na<sup>+</sup> and and ∼0.010 for the K<sup>+</sup> channels. At any instant the open probability follows a binomial distribution so that the variance in the single channel current σ<sup>2</sup> *<sup>i</sup>* = *i* <sup>2</sup>*po*(1 − *po*), where *i* is the single-channel current. The variance is parabolic in *po*: zero when *po* = 0 or 1, and maximal at *po* = 0.5. As below spike threshold, most ion channels have very low open probabilities, the standard deviation can be approximated by

$$
\sigma\_i = i\sqrt{p\_o}
$$

Therefore, ion channel populations with greater *po* at resting membrane potential tend to have larger current fluctuations than populations with lesser *po*. This effect predicts that the standard deviation of the K<sup>+</sup> channel noise is 10.7 times larger than Na<sup>+</sup> channel noise.

## *3.2.2. Number of channels (1.8* **×** *noisier for Na***+***)*

Second, because the channels act independently, the standard deviation of the number of open channels grows proportional to <sup>√</sup>*N*. Thus channel populations with greater *<sup>N</sup>* have greater fluctuations in their absolute number of open channels. The Na<sup>+</sup> population has 3.33× more channels in the standard HH model than the K<sup>+</sup> population (**Table 1**), yielding Na standard deviation larger by a factor <sup>√</sup>10/<sup>3</sup> <sup>≈</sup> <sup>1</sup>.8.

## *3.2.3. Driving force (10* **×** *noisier for Na***+***)*

The third factor is the difference in driving force for each conductance. As the HH model assumes that these ion channel current-conductance relationships are Ohmic (linear), the current through an open channel is proportional to the difference between the membrane potential and the channel's driving force,

$$i\_{\mathbf{x}} = \gamma\_{\mathbf{x}} (V\_{\mathbf{m}} - E\_{\mathbf{x}})$$

where *ix* is the single-channel current, γ*<sup>x</sup>* is the single-channel conductance, *Vm* is the membrane potential and *Ex* is the conductance's reversal potential, given by the Nernst equation. In the HH model, *ENa* = +50 mV, *EK* = −77 mV, and *Vrest* = −65 mV, giving Na<sup>+</sup> a driving force of +115 mV and K<sup>+</sup> a driving force of −12 mV. This means that the driving force for the Na<sup>+</sup> current is 9.6× greater than the K<sup>+</sup> current at *Vrest*.

## *3.2.4. Single-channel conductance (identical for Na***+** *and K***+***)*

Fourth, the single-channel conductance γ*<sup>x</sup>* is another important factor determining a channel's contribution to membrane noise. For the same channel population current per unit squared cell membrane, a larger γ*<sup>x</sup>* implies smaller *N*, and a larger *ix*. Hence channels with a larger single-channel conductance will have greater current fluctuations. In our implementation of the stochastic HH model, however, we assume, like most other stochastic models, the same single-channel conductance for both Na<sup>+</sup> and K<sup>+</sup> (20 pS). The value of 20 pS is close to experimentally measured estimates for Na+ (14 pS) (Bezanilla, 1987), while the K<sup>+</sup> conductance in the squid axon is probably made up of multiple different channel types, with single-channel conductances estimated at 10, 20, and 40 pS (Llano et al., 1988).

These four factors can be put together to construct a binomial model of the amplitude of channel noise at steady state. This model does not have any notion of dynamics or channel kinetics. We calculate the steady-state open probabilities at rest directly from the Hodgkin–Huxley equations, and test the binomial model's ability to reproduce simulated voltage-clamp data and probe its predictions on the relative magnitudes of Na<sup>+</sup> and K<sup>+</sup> channel noise (**Figure 3**).

We find that, as expected, the binomial model exactly predicts the conductance and current fluctuations from voltage-clamp simulation data at resting potential of −65 mV (**Figures 3C,F**). We use the binomial model to estimate the steady-state standard deviation in open channel numbers and total current from the Na<sup>+</sup> and K<sup>+</sup> populations at a range of membrane potentials (**Figures 3A–B,D–E**). As expected from the above analysis the Na<sup>+</sup> current standard deviation is about 1.7× that of the K<sup>+</sup> current (matched in simulations, see circle symbols in **Figure 3E**).

## *3.2.5. Channel gating kinetics and membrane filtering (3* **×** *noisier for K***+***)*

The fifth factor is that the Na<sup>+</sup> and K<sup>+</sup> conductances have different gating kinetics. These differences are important because the current fluctuations from each ion channel populations are filtered differentially by the membrane impedance, hence altering each channel's contributions to membrane voltage noise.

In the Methods we calculate the power-spectra of the Na<sup>+</sup> and K<sup>+</sup> current noise assuming a constant membrane potential and small noise. Both powerspectra are sums of multiple Lorentzians,

$$S\_I(f) = \sum\_k \frac{a\_k}{1 + \left(f/f\_c^k\right)^2}$$

where the *f <sup>k</sup> <sup>c</sup>* are the corner frequencies of the Lorentzians (the frequency at which the powerspectrum is half of the zero frequency magnitude), and *ak* are (voltage-dependent) coefficients. The full expressions are given in the Methods, but the K<sup>+</sup> noise spectrum is dominated by a Lorentzian with corner frequency *fc* = 4/(2πτ*n*). At the resting potential τ*<sup>n</sup>* ∼ 5.5 ms (Equation 3), so that the dominant Lorentzian has a corner frequency of 115 Hz. The Na<sup>+</sup> spectrum is dominated by a Lorentzian with *fc* = 3/(2πτ*m*). At rest τ*<sup>m</sup>* ∼ 0.24 ms, τ*<sup>h</sup>* ∼ 8.5 ms, so that the

dominant Lorentzian for Na<sup>+</sup> has a much higher corner frequency of 1980 Hz—note however that we include all Lorentzian terms in the presented results, not just the dominant one. The analytically calculated spectra match well the estimated spectra of the simulated stochastic currents, **Figure 4A**.

To calculate the voltage response to the current noise, we approximate the active membrane by a linear impedance. The voltage noise power spectrum *SV*(*f*) is given by the generalized Ohm's law

$$S\_V(f) = S\_I(f)|Z(f)|^2$$

where *SI*(*f*) is the power spectrum of a current noise source and *Z*(*f*) is the membrane impedance. The impedance relates

colored curves are theoretical, derived from channel kinetic schemes. **(B)** Variance of Na<sup>+</sup> and K<sup>+</sup> current noise from both simulation and theory. Note Na<sup>+</sup> channel noise variance is greater than K<sup>+</sup> channel noise. **(C)** Total HH membrane impedance at −65 mV as a function of signal frequency. **(D)** Voltage noise of Na<sup>+</sup> and K<sup>+</sup> channels calculated from current noise spectra and membrane impedance. Gray curve is sum of Na<sup>+</sup> and K<sup>+</sup> noise, while

K<sup>+</sup> channels contribute more to voltage noise than Na<sup>+</sup> channels. **(F)** Comparison of voltage noise variance from theory and simulation as a function of membrane area. Note curves substantially diverge only for areas < 500µm. For those small areas spontaneous spiking occurs and the associated large voltage fluctuations are not part of the theory. All other panels use a membrane area of 1000µm2.

changes in voltage to changes in currents. In active membranes the input impedance is not given by just the capacitance and leak conductance, but also by any other channels open at rest and their reaction to small changes in the voltage. The impedance can be found by linearizing the four-dimensional (*V*, *m*, *h*, and *n*) HH equations around the resting state (e.g., Mauro et al., 1970; Koch, 1984; Carnevale and Hines, 2006). In the HH model the presence of Na<sup>+</sup> and K<sup>+</sup> conductances introduce a resonance in the impedance at ∼100 Hz, but the 1/*f* behavior still dominates at higher frequencies (**Figure 4C**). The resonant peak in the impedance is the electrical signature of an inductor. Although such an inductor has no physical counterpart in the biological cell membrane, the delayed-rectifier K<sup>+</sup> conductance opposes changes in membrane potential and for small currents behaves as a phenomenological inductance (Mauro et al., 1970; Koch, 1984, 1999).

Now we combine the membrane impedance with the current noise spectra to calculate each channel population's contribution to voltage noise. In **Figure 4D** we plot the theoretical power spectra of the voltage noise from the HH Na<sup>+</sup> and K<sup>+</sup> channel populations, calculated at *Vm* = *Vrest* = −65 mV. The sum of the Na<sup>+</sup> and K<sup>+</sup> power spectra give the total voltage noise power spectrum (gray line in **Figure 4D**). This predicts almost exactly the power spectrum measured from simulation (dashed line in **Figure 4D**). The voltage noise variance from each channel population equals the integrated power spectrum:

$$
\sigma\_\mathbf{x}^2 = \int\_0^\infty \mathcal{S}\_\mathbf{x}(f) df
$$

where subscript *x* indicates the relevant channel population. These are graphed in **Figure 4E**. It is clear the K<sup>+</sup> channel fluctuations contributes ∼ 4× more *voltage* noise variance than Na+. While the Na<sup>+</sup> *current* noise has a greater variance than the K<sup>+</sup> current noise, it is filtered more strongly by the membrane impedance.

Can we summarize the total effects of membrane filtering on noise from the two channel types? One way to quantify this effect is to take the ratio of voltage and current noise standard deviations for each channel type, *rx* = σ*Vx*/σ*Ix*. Doing so we find that *rNa* = 44.5 M and *rK* = 141.7 M, implying that, after all other factors are accounted for, membrane filtering attenuates Na<sup>+</sup> noise ∼ 3× more than K<sup>+</sup> noise.

As both the noise spectra and the impedance are voltagedependent, these calculations assume that the voltage remains at a fixed potential, which holds if the fluctuations are small. Because here we simulate a large membrane area (1000µm2) with low membrane resistance, voltage changes are small and there is only a small discrepancy between the voltage noise calculated analytically and the estimate from simulation (**Figure 4E**). In particular for small membrane areas the voltage fluctuations will be large, nevertheless the approximation remains good down until areas where spontaneous spikes appear, **Figure 4F**. At this point, currents associated to the spike will dominate the measured current.

In summary, the contribution of each channel type to membrane noise is determined by their number, single-channel conductance, voltage dependencies and gating kinetics. This is true for any neuron model. In the case of the HH model, the sum properties of K<sup>+</sup> channels at subthreshold voltages make their contributions to membrane noise greater than that from Na<sup>+</sup> channels.

#### **3.3. EFFECT ON SPONTANEOUS SPIKE RATES**

So far we have seen that the channels in the HH model contribute differentially to the noise in the membrane voltage and that K<sup>+</sup> channels have the largest contribution. One would expect that therefore K<sup>+</sup> channels are the most important contributor of noise-driven spontaneous spike activity as well. This is indeed the case as we have seen in **Figure 1** but quantitatively the link between the spontaneous rate and the subthreshold membrane voltage fluctuations is not trivial.

The analysis of spontaneous spiking rate in the HH model to correlated noise is a complicated stochastic differential equation problem. Reducing the spiking mechanism to a one dimensional escape problem, Chow and White (1996) derived the spontaneous rate using multiplicative, white (uncorrelated) current noise to approximate the Na<sup>+</sup> noise. But it is not obvious how such an analysis can be extended to colored (correlated) noise. The timeconstants of the K<sup>+</sup> noise, Na<sup>+</sup> noise and the membrane are all of similar magnitude (**Figure 4**), complicating any perturbative expansion. Even in much simpler integrate-and-fire neuron models, the treatment of correlated noise is complicated, resulting in a two-dimensional Fokker-Planck equation that can only be solved in certain limits (Brunel and Sergi, 1998; Moreno-Bote and Parga, 2004; Alijani and Richardson, 2011).

We first examine how the correlation time in the Gaussian noise model affects spontaneous firing rates in a HH neuron. Traditionally, studies have kept the variance of the injected noise fixed while varying the correlation time. However, as shown above, this can lead to widely different voltage fluctuations due to the differential membrane filtering. The idea we examine here is that instead of the current variance, the voltage variance is a better predictor of the spontaneous rate. This was recently shown in exponential integrate-and-fire neurons: the noise driven firing rate was relatively independent of the noise correlation time when the voltage variance was kept constant (Alijani and Richardson, 2011).

In order to research this in the HH model, we simulated a deterministic HH neuron and injected a Gaussian noise current with a correlation time τ and with a correlation function

$$
\langle I(T)I(T+t) \rangle = \frac{\sigma\_V^2}{z^2(\pi)} e^{-|t|/\pi}
$$

The function *z*(τ ) is an impedance that relates the voltage variance to the current variance of injected colored noise with timeconstant τ . It is given by *z*2(τ ) = *SI*(*f*)|*Z*(*f*)| <sup>2</sup>*df* , where *SI*(*f*) = 4τ <sup>1</sup> <sup>+</sup> (2π*<sup>f</sup>* <sup>τ</sup> )<sup>2</sup> is the power-spectrum of the noise current, and *Z*(*f*) the linearized impedance of the HH model. Its shape reflects the resonance in the impedance, **Figure 5A**. As a result of this impedance correction the membrane voltage variance in the limit of small fluctuations equals σ<sup>2</sup> *<sup>V</sup>* and is thus independent of τ .

When the amplitude of the noise is scaled up, spontaneous spikes appear. The resulting spontaneous rate is shown in **Figures 5B** (black solid curve),**C**. Remarkably, across six orders of magnitude in the noise time-scale, ranging from noise much faster than any channel or membrane to extremely slow noise, the spontaneous rate varies only one order of magnitude (from 40 to 5 Hz for the noise in **Figure 5B**). With less noise this ratio can be larger as slow noise might then be unable to evoke the spontaneous spikes, **Figure 5C**. Furthermore, the relation is monotonic, which is useful when comparing two noise sources. The shape

**FIGURE 5 | Spontanous action potentials in the HH model driven with Gaussian noise with varying amplitude and correlation time. (A)** The impedance factor *z*(τ ) gives the voltage fluctuations resulting from a correlated noise current as a function of the noise correlation time. It is used to ensure the voltage variance is identical as the time constant of the noise is varied. Membrane area of 1000µm2. **(B)** Spontaneous activity as a result of a Gaussian current noise, as a function of the correlation time of the noise. When the variance of the current is fixed, the resulting rates vary strongly depending on the correlation time (dashed curve). However, when the noise is calibrated to yield the same variance in the

membrane voltage irrespective of the noise correlation time, the firing rate is much less variable even across six orders of magnitude (solid curve). **(C)** The firing rate vs. the impedance corrected current noise (expressed in σ*<sup>V</sup>* ) for various values of the noise time-constant. **(D)** The spontaneous firing rate vs. membrane area for K<sup>+</sup> (red) and Na<sup>+</sup> (blue) noise and their various approximations. Solid curve: full simulation, redrawn from **Figure 1B**. Thick dashed curve: approximation using Gaussian noise with identical variance and time-constant. Thin dashed curve: binomial approximation of the noise. In particular for Na<sup>+</sup> noise the approximations yield rates that are substantially too low.

of the curve highlights that fast fluctuating noise is typically an effective driver of neurons, while slow varying noise will tend to inactivate the Na channel close to threshold and is less effective.

Instead, the *current* variance is a much worse predictor of the firing rate (dashed line). Note that in the limit of very slow noise, the dynamics decouple and the spontaneous rate equals *P*(*I*)*f*(*I*)*dI*, where *P*(*I*) is the distribution of currents and *f*(*I*) is the neuron's deterministic f-I curve.

Finally, we ask if we can use these results to estimate the spontaneous rates caused by K+, Na+, or in fact any arbitrary channel noise. In the previous section the noise currents were approximated by colored Gaussian noise (an OU process) with variance and correlation time derived from the channel kinetics at the resting voltage, and filtered by the membrane linearized around rest. These approximations hold very well in the subthreshold regime, i.e., for small noise—equivalent to large membrane areas — see **Figure 4E**, but it is a priori unclear whether they also hold for larger noise amplitudes when spontaneous spikes appear.

We used a colored Gaussian current noise to model the K<sup>+</sup> and Na<sup>+</sup> noise, and injected this into a deterministic HH model. For example, in case of the K<sup>+</sup> channel the variance according to the above sections is

$$
\langle \delta I^2 \rangle = \rho\_K A [(V\_{\text{rest}} - E\_K) \gamma\_K n\_{\infty}^4 (V\_{\text{rest}})]^2
$$

and its correlation time is τ*n*/4. The spontaneous rate of the neuron driven by this noise was examined as a function of the membrane area. Although this could be accidental, for K<sup>+</sup> noise, the noise model gave an almost perfect fit to the fully stochastic simulations, **Figure 5D** compare solid and thick dashed curves. However, the approximated Na noise lead to far too few spontaneous spikes, **Figure 5D**. Its standard deviation had to be increased by some 50% to fit the simulated spontaneous rates. This need for a fudge factor shows that for smaller areas the Na noise is not well described by additive, colored Gaussian noise. There are many possible cause for this mismatch: the binomial instead of Gaussian current distribution (which additional simulations showed to be a small effect, **Figure 5D** thin dashed curve), the voltage dependence of the impedance, the dynamics of the full HH system, and likely most important, the strong dependence on the noise current on the membrane voltage, **Figure 3E**.

In summary then, while we find that sub-threshold noise can be estimated accurately, caution is needed when extrapolating to the spiking regime.

#### **3.4. APPLICATION TO CA1 PYRAMIDAL NEURON MODEL**

Above we showed how to break down the factors determining a given ion channel type's contribution to voltage noise and spontaneous spiking, using the Hodgkin–Huxley squid axon model as an example. However, our approach is completely general and could in principle be applied to any neuron model. To demonstrate its straightforward application, we now perform the same analysis on a well-studied mammalian cell type: the rodent hippocampal CA1 pyramidal neuron. We studied a single-compartment model of this cell type using a well-validated model from the literature (Migliore et al., 1999; Jarsky et al., 2005). The original model contained three active channel types: an Na<sup>+</sup> channel, an A-type K<sup>+</sup> channel, Ka, and a delayed rectifier K<sup>+</sup> channel, Kdr. We built stochastic versions of these active conductances with parameters exactly as previously used (Jarsky et al., 2005).

At small areas the model fired spontaneously (**Figure 6A**), similar to the HH model above (**Figure 1**). Also, similar to before, our theory well predicts the variance of voltage fluctuations for large membrane areas, but diverges from the simulation

Spontaneous firing rate as function of membrane area for CA1 pyramidal cell model. Qualitative behavior is similar to Hodgkin–Huxley model, **Figure 1**. **(B)** Comparison of voltage noise variance from theory and simulation as a function of membrane area. As with Hodgkin–Huxley model (**Figure 4F**), theory is a good predictor for large membrane areas when there is no spontaneous spiking. **(C)** Contributions to voltage variance from the different ion channel types in the model, for both theory and simulation. Two types of potassium channels dominate the noise in this model. Membrane area of 1000µm2.

results for small areas < 500µm<sup>2</sup> when the neuron spikes (**Figure 6B**).

In **Figure 6C** we plot the contributions to voltage variance from each of the three voltage-dependent ion channel types in the CA1 pyramidal neuron model. Similar to the HH model above, noise from K<sup>+</sup> channels dominates over noise from Na<sup>+</sup> channels. Of the two K<sup>+</sup> channel types in the model, the delayed rectifier K<sup>+</sup> channel contributes more voltage noise than the Atype K<sup>+</sup> channel. Interestingly, Ka has a larger current noise variance than Kdr (not shown), but because Ka has faster kinetics than Kdr, its current noise is more heavily filtered by the membrane capacitance leading to a switch in the relative magnitudes of their contributions to voltage variance.

These results demonstrate two points: first, our method is readily applicable to any neuron model; and second, the dominant source of ion channel noise depends on the physiological details of the neuron. However, it should not be concluded from these results that K<sup>+</sup> channels will always contribute more noise than Na<sup>+</sup> channels. Substantial channel noise can arise from channels permeable to any ion: Na+, K<sup>+</sup> or any other. The relative amplitudes and effects of channel noise simply depend on all of the earlier outlined factors and will need to be evaluated on a case-by-case basis.

## **DISCUSSION**

We have picked apart the various factors that determine a specific channel population's contribution to membrane noise. Although applied only to the Hodgkin Huxley and hippocampal pyramidal neuron models here, the method is applicable to any voltage gated channel model. In summary the factors are:


Another qualitative factor is the polarity of current flow. Open Na<sup>+</sup> channels further depolarize the cell, hence increasing the probability for other Na<sup>+</sup> channels to open and acting as a positive feedback loop. Hence, stochastic Na<sup>+</sup> channels increase excitability of the cell through regenerative depolarizing excursions in membrane potential (Dudman and Nolan, 2009). Open K<sup>+</sup> channels, in contrast, hyperpolarize the cell and act as negative feedback to changes in membrane potential. This negative feedback coupled with their relatively slow kinetics can, in some cases, enable stochastic K+channels to drive sub-threshold oscillations (Schneidman et al., 1998).

The combination of these factors yields an accurate prediction of the membrane voltage noise. While it is possible to obtain a coarse estimate of spontaneous firing rates, this is far from perfect and highlights two current hiatus in the theory, namely, the absence of an accurate phenomological model for channel noise when the noise can not be assumed to be small, and the lack of theory for colored noise driven spontaneous activity.

In the case of the stochastic HH model we have shown that the fluctuations from stochastic gating of potassium channels is the dominant source of noise by three different measures. First, a HH model where only K<sup>+</sup> channels gate stochastically spontaneous fires at higher rates than a HH model where only Na<sup>+</sup> channels gate stochastically (**Figure 1**) (Skaugen and Walløe, 1979; Schneidman et al., 1998; van Rossum et al., 2003). Second, examining the dynamics of Na<sup>+</sup> and K<sup>+</sup> currents in the milliseconds preceding a spontaneous action potential in the "all stochastic" HH model shows that, on average, spikes are generated by a drop in K<sup>+</sup> current that precedes the increase in Na<sup>+</sup> current (**Figure 2**). Third, direct calculation of the voltage noise spectra from each channel population at resting potential shows that K<sup>+</sup> channel fluctuations contribute ∼ 75% of the total membrane noise (**Figure 4**). This finding, although consistent with results reported by Schneidman et al. (1998); van Rossum et al. (2003), is in contrast with other simulations (Chow and White, 1996; Faisal et al., 2005). We discuss these two studies separately.

Chow and White (1996) used approximate analytical methods to directly calculate the spontaneous firing rate in the stochastic HH model, and compared the predictions to numerical simulations to find apparently very good agreement. Our own simulations produce quantitatively similar results to their simulations (data not shown), so it is likely that their simulated data are correct. However, their analytical calculations were based on the assumption that spontaneous spiking is driven *solely* by stochastic activation of Na<sup>+</sup> channels. No matter how elegant, their result can not be accurate as it ignores the K<sup>+</sup> noise, which is the main cause for spontaneous firing (**Figure 1**). If anything, their analytical model should be a better approximation of our simulations when K<sup>+</sup> channels are modeled deterministically. However, their calculations do not match this. The errors could have arisen in any of the multiple approximating steps necessary for their calculation. For example, they assume a static absolute voltage threshold when in reality the HH model has (1) no hard threshold for any type of stimulus (Izhikevich, 2007) and (2) different apparent spike thresholds for stimuli of different temporal structure (Koch, 1999). A more fruitful method for future studies could be to derive a higher-dimensional version of the spike threshold that incorporates both fast and slow channel variable states (Newby et al., 2013).

Faisal et al. (2005) find in cable axon HH models that Na<sup>+</sup> channels contribute more to spontaneous spiking than K<sup>+</sup> channels. We believe this to be a numerical simulation error as it is inconsistent with our simulations (when implemented with both PSICS and NEURON), and also those of Schneidman et al. (1998) and van Rossum et al. (2003)—implemented with the "NeuronC" simulator (Smith, 1992). Furthermore, we found the K<sup>+</sup> channels to be dominant not only in single compartment models but in cable structures as well. Without access to their simulator, it is difficult to tell where the discrepancy lies. Nevertheless, it is possible that Na<sup>+</sup> channel noise does drive spontaneous spiking in models other than the HH squid giant axon. Our general theory should help to quickly estimate such possibilities without resorting to full simulations.

#### **AUTHOR CONTRIBUTIONS**

Conception: Cian O'Donnell. Carried out analytical and computer calculations, wrote the paper: Cian O'Donnell and Mark C. W. van Rossum.

#### **FUNDING**

Cian O'Donnell was supported by EPSRC/BBSRC/MRC through the Doctoral Training Centre in Neuroinformatics.

#### **ACKNOWLEDGMENT**

Discussions with Matthew Nolan, Robert Cannon and Carson Chow are gratefully acknowledged.

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 14 March 2014; accepted: 17 August 2014; published online: 04 September 2014.*

*Citation: O'Donnell C and van Rossum MCW (2014) Systematic analysis of the contributions of stochastic voltage gated channels to neuronal noise. Front. Comput. Neurosci. 8:105. doi: 10.3389/fncom.2014.00105*

*This article was submitted to the journal Frontiers in Computational Neuroscience.*

*Copyright © 2014 O'Donnell and van Rossum. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Diffusion approximation-based simulation of stochastic ion channels: which method to use?

## *Danilo Pezo1, Daniel Soudry2 and Patricio Orio1,3\**

*<sup>1</sup> Centro Interdisciplinario de Neurociencia de Valparaíso, Universidad de Valparaíso, Valparaíso, Chile*

*<sup>2</sup> Department of Statistics and the Center for Theoretical Neuroscience at Columbia University, New York, NY, USA*

*<sup>3</sup> Instituto de Neurociencia, Facultad de Ciencias, Universidad de Valparaíso, Valparaíso, Chile*

#### *Edited by:*

*Joshua H. Goldwyn, New York University, USA*

#### *Reviewed by:*

*Daniele Linaro, University of Antwerp, Belgium Roberto Fernández Galán, Case Western Reserve University, USA*

#### *\*Correspondence:*

*Patricio Orio, Centro Interdisciplinario de Neurociencia de Valparaíso, Pasaje Harrington 287, Valparaíso 2360103, Chile e-mail: patricio.orio@uv.cl*

To study the effects of stochastic ion channel fluctuations on neural dynamics, several numerical implementation methods have been proposed. Gillespie's method for Markov Chains (MC) simulation is highly accurate, yet it becomes computationally intensive in the regime of a high number of channels. Many recent works aim to speed simulation time using the Langevin-based Diffusion Approximation (DA). Under this common theoretical approach, each implementation differs in how it handles various numerical difficulties—such as bounding of state variables to [0,1]. Here we review and test a set of the most recently published DA implementations (Goldwyn et al., 2011; Linaro et al., 2011; Dangerfield et al., 2012; Orio and Soudry, 2012; Schmandt and Galán, 2012; Güler, 2013; Huang et al., 2013a), comparing all of them in a set of numerical simulations that assess numerical accuracy and computational efficiency on three different models: (1) the original Hodgkin and Huxley model, (2) a model with faster sodium channels, and (3) a multi-compartmental model inspired in granular cells. We conclude that for a low number of channels (usually below 1000 per simulated compartment) one should use MC—which is the fastest and most accurate method. For a high number of channels, we recommend using the method by Orio and Soudry (2012), possibly combined with the method by Schmandt and Galán (2012) for increased speed and slightly reduced accuracy. Consequently, MC modeling may be the best method for detailed multicompartment neuron models—in which a model neuron with many thousands of channels is segmented into many compartments with a few hundred channels.

**Keywords: channel noise, stochastic simulation, ion channel, Langevin, conductance based models**

## **INTRODUCTION**

Understanding the effect of stochastic phenomena on the behavior of the nervous system requires stochastic simulation algorithms that effectively and accurately capture the dynamics of the underlying modeled phenomena. Among the sources for variability, the stochastic opening and closing of ion channels has caught the attention of several works over the past years. The best description of stochastic gating of ion channels is attained with the use of continuous time, discrete states Markov Chain (MC) processes (Neher and Stevens, 1977; Colquhoun and Hawkes, 1981), however this approach can be very slow in simulations with a large number of channels.

As an alternative to the explicit MC simulation, the Diffusion Approximation (DA) calculates the trajectory of a population of independent MCs using a Stochastic Differential Equation (SDE), sometimes called the Chemical Langevin Equation (Gillespie, 2000, 2007). Its application to the simulation of stochastic ion channels was suggested almost 20 years ago (Fox and Lu, 1994; Fox, 1997), but in the beginning there were some errors in the application of the scheme. This led to the belief that the approximation was not good enough (Mino et al., 2002; Bruce, 2007, 2009). Later, revised implementations of the algorithms were published showing that indeed the DA can reproduce the statistical properties of a population of discrete ion channel fluctuating between open and closed states (Goldwyn et al., 2011; Goldwyn and Shea-Brown, 2011). Other works (Dangerfield et al., 2012; Orio and Soudry, 2012) also offered a simplified description of the algorithm, making it easy to apply to any given kinetic scheme.

What the SDE system approach does is to approximate the trajectory in time of the fraction of channels at every state. For the simulation to remain physically meaningful, none of the fractions can be negative or greater than 1. We call this the "boundary constraint." This constraint would occasionally break in the numerical simulation of the SDE, if we use its naively discretized form (the Euler–Maruyama method). This is because stochastic fluctuations can make the variables leave the [0,1] interval. The problem amplifies when the number of channels is low and stochastic fluctuations increase.

If the boundary constraint is not maintained, this can generate additional technical problems in the simulation. Specifically, the calculation of the stochastic terms involves the square root of a term involving the fraction variables, which may yield complex values when the variables are negative. Such complex values must be avoided. Therefore, we get another constraint, which we call the "Real-valued Square Root" (RSR) constraint. This constraint is automatically fulfilled if the boundary constraint is maintained.

Finally, the sum of the fractions over all states must be equal to 1 at all times. We call this the "normalization constraint." Although this constraint is supposed to be guaranteed by the continuous-time equations (see below), machine rounding errors of the discretized equations can gradually break it. Additionally, any method that deals with the boundary constraint must also take normalization into account. For example, a naive truncation of any variable that leaves the [0,1] interval would break the normalization constraint.

To address these issues, a number of improvements have been proposed to the DA schemes.

Orio and Soudry (2012) proposed to allow the variables to freely change, ignoring the boundary constraint. In order to take care of the normalization constraint one of the variables was replaced with one minus the sum of the others. Additionally, an absolute value operation was added in the stochastic terms to maintain the RSR constraint. Previously, Goldwyn et al. (2011) also allowed the variables to freely change, but instead used a steady state approximation on the voltage to maintain the RSR constraint. However, such an approximation can be rather inaccurate (Dangerfield et al., 2012; Orio and Soudry, 2012; Huang et al., 2013a), even when the number of channels is relatively high.

Two methods aim to maintain the boundary constraint. Dangerfield et al. (2012) proposed that if variables break either the boundary or normalization constraint, they are "reflected" back into the valid region, in which all the constraints are kept. This is done using projection into a simplex (Chen and Ye, 2011). Huang et al. (2013a) proposed a different method. When the boundary constraint is broken, the variables should first be truncated. This breaks normalization, so the variables are then renormalized. Finally, in the next time step, the variables are incremented with the remainders from the truncation in the previous steps. As the boundary constraint is almost constantly broken, normalization is continuously corrected in both Dangerfield et al. (2012) and Huang et al. (2013a).

Other methods have been proposed with a different goal in mind. Schmandt and Galán (2012) aimed to reduce computational complexity and speeding up the simulation. They proposed to neglect stochastic noise terms in all state transitions, except those connecting the open state (or states), an approximation they call "Stochastic shielding." Güler (2013) introduced a stochastic HH model with colored noise in the conductance terms as well as in the current terms in order to capture the non-trivial crosscorrelation between the transmembrane voltage fluctuation and the component of open channel fluctuation attributed to multiple number of gates in individual ion channels. Another recently published method (Linaro et al., 2011), also used colored noise in the current terms (but not in the conductance terms). However, Linaro's method will not be examined here, because it involves a steady-state approximation in the stochastic terms (similarly to Goldwyn et al., 2011), which was shown to introduce inaccuracies (Orio and Soudry, 2012).

Despite the improvement in accuracy or computational efficiency that the new methods represent for the simulation of stochastic ion channel activity, we were concerned about the comparisons performed between them and the real benefit of implementing the numerical algorithms.

First, there is the computational cost issue. The initial motivation for developing DA methods was to make stochastic simulations faster than MC modeling. Therefore, if the extra computation needed to normalize and bound the variables makes it slower than MC then the purpose is defeated. Moreover, we already noticed that when the number of channels is low (when DA becomes more inaccurate) or with very small integration times, MC modeling can run faster than DA (see Figure 7 in Orio and Soudry, 2012). This, added to the fact that bounding and normalization of the DA requires more coding (and eventually, debugging), can render DA less attractive.

Second, we noted that the standard test employed to prove the accuracy of numerical methods for stochastic ion channels is the original Hodgkin and Huxley (HH) model. This was the only model used for testing in most previous papers, including a recent review (Rowat and Greenwood, 2014). This model, as standard and general as it is, reproduces the kinetics of ion channels of the squid axon at 6.3◦C, thus differing greatly from mammalian nerve excitable membranes. This difference can be very significant, as we noted (Orio and Soudry, 2012). There we found that the use of a steady-state approximation in the stochastic terms usually does not introduce severe inaccuracies in the context of the original HH model. However, deviations were detected in common current clamp-based simulations when the steady-state approximation is used in a model inspired in mammalian (therefore faster) ion channels. It is noteworthy that the difference between mammalian inspired and the squid axon model relies only in the parameters that describe the transition rate constants (and thus the time scale of the model), while the equations and the model framework are identical.

Thus, we see a necessity for testing the DA with and without the recently proposed corrections in a wider spectrum of simulation scenarios and taking into account other variables than simulation accuracy, namely:


In an attempt to test the real usability of the algorithms in the context of more complex neuronal models, we implemented them in one of the choice tools for biophysically-inspired modeling, the Neuron simulation environment (Hines and Carnevale, 2001; Carnevale and Hines, 2006). We conducted both singlecompartment and multi-compartment simulations using MC or DA algorithms and compared their performance as well as the ability of different DA implementations to reproduce the variability introduced by MC modeling.

In our results all DA algorithms deviate to some degree from the MC modeling when the number of channels falls below 1000, regardless of the attempts to deal with normalization and bounding of the variables. However, we see that in this condition MC modeling runs, in most of the scenarios, faster than DA implementations. Therefore, one of the most common motivations to use DA, which is to achieve faster computation times, is not accomplished when the number of channels is low. However, when the number of channels is high, DA algorithms can accurately reproduce MC, with improved speed. Specifically, in this regime, no inaccuracy was detected in both Orio and Soudry (2012) and Huang et al. (2013a); Schmandt and Galán (2012) was slightly inaccurate; Güler (2013) was somewhat inaccurate; and Dangerfield et al. (2012) was the least accurate. In terms of computational speed, the ranking is as follows (see **Figure 8**): (1) Stochastic Shielding (Schmandt and Galán, 2012) (2) Colored Noise (Güler, 2013) (3) Unbound DA (Orio and Soudry, 2012) (4) Reflected DA (Dangerfield et al., 2012) (5) Truncated and Restored DA (Huang et al., 2013a).

## **MATERIALS AND METHODS**

## **SIMULATIONS: MODELS EMPLOYED AND TESTS PERFORMED** *Original Hodgkin and Huxley model*

The original Hodgkin and Huxley (HH) model (Hodgkin and Huxley, 1952) was simulated with the equation:

$$C\_m \frac{dV(t)}{dt} = -\mathbf{g}\_{\rm Na}(t) \left( V(t) - E\_{\rm Na} \right) - \mathbf{g}\_{\rm K}(t) \left( V(t) - E\_{\rm K} \right)$$

$$- \lg \left( V(t) - E\_l \right) + I\_{\rm sim}(t) \tag{1}$$

With the exception of Güler's colored noise algorithm, sodium and potassium channels were treated as 8- and 5-state MCs, respectively. The corresponding kinetic schemes are:

$$\begin{array}{ccccccccc} m\_0 h\_0 & \stackrel{3\alpha\_m}{\underset{\beta\_m}{\rightleftharpoons}} & m\_1 h\_0 & \stackrel{2\alpha\_m}{\underset{2\beta\_m}{\rightleftharpoons}} & m\_2 h\_0 & \stackrel{\alpha\_m}{\underset{3\beta\_m}{\rightleftharpoons}} & m\_3 h\_0\\ \beta\_h \;\;\;\mid \;\alpha\_h & \beta\_h \;\;\mid \;\alpha\_h & \beta\_h \;\mid \mid \;\alpha\_h & \beta\_h \;\mid \mid \;\alpha\_h \; (\text{scheme1})\\ m\_0 h\_0 & \stackrel{3\alpha\_m}{\underset{\beta\_m}{\rightleftharpoons}} & m\_1 h\_0 & \stackrel{2\alpha\_m}{\underset{2\beta\_m}{\rightleftharpoons}} & m\_2 h\_0 & \stackrel{\alpha\_m}{\underset{3\beta\_m}{\rightleftharpoons}} & m\_3 h\_0\\ m\_0 & \stackrel{4\alpha\_n}{\underset{\beta\_n}{\rightleftharpoons}} & n\_1 & \stackrel{3\alpha\_n}{\underset{2\beta\_n}{\rightleftharpoons}} & n\_2 & \stackrel{2\alpha\_n}{\underset{4\beta\_n}{\rightleftharpoons}} & n\_3 & \stackrel{\alpha\_n}{\underset{4\beta\_n}{\rightleftharpoons}} & n\_4 \text{ (scheme2)} \end{array}$$

Sodium and potassium conductances at time *t* (*gNa*(*t*) and *gK*(*t*)) were calculated as the fraction of channels in the conducting states *m*3*h*<sup>1</sup> and *n*<sup>4</sup> multiplied by the maximum conductances *gNa* and *gK*, respectively. The kinetic rates α and β are given (in ms<sup>−</sup>1) by:

$$\alpha\_m(V) = \frac{0.1\left(V + 40\right)}{1 - \exp\left(-\frac{V + 40}{10}\right)}; \quad \beta\_m(V) = 4\exp\left(-\frac{V + 65}{18}\right)$$

$$\alpha\_h(V) = 0.07\exp\left(-\frac{V + 65}{20}\right); \beta\_h(V) = \frac{1}{1 + \exp\left(-\frac{V + 35}{10}\right)}$$

$$\alpha\_n(V) = \frac{0.01\left(V + 55\right)}{1 - \exp\left(-\frac{(V + 55)}{10}\right)}; \ \beta\_n(V) = 0.125\exp\left(-\frac{V + 65}{80}\right).$$

where the terms were corrected to adjust the resting potential to −65 mV. Correspondingly, the rest of parameters are: *Cm* = 1μF/cm2, *gNa* = 120mS/cm2, *gK* = 36mS/cm2, *gl* = 0.1mS/cm2, *ENa* = 50mV, *EK* = −77 mV, *El* = −54.3 mV.

With this model, the following tests were conducted:


## *Schmidt-Hieber and Bischofberger model—single compartment*

The Schmidt-Hieber and Bischofberger (SB) model was proposed after the characterization of sodium channels both at the soma and at the axon initial segment of granule cells of the hippocampus (Schmidt-Hieber and Bischofberger, 2010). Sodium channels are described by the following kinetic scheme:

$$\begin{array}{ccccccccc} m\_0 h\_0 & \stackrel{\alpha\_1}{\underset{\beta\_1}{\rightleftharpoons}} & m\_1 h\_0 & \stackrel{\alpha\_2}{\underset{\beta\_2}{\rightleftharpoons}} & m\_2 h\_0 & \stackrel{\alpha\_3}{\underset{\beta\_3}{\rightleftharpoons}} & m\_3 h\_0\\ \beta\_h \mid \mid \alpha\_h & \beta\_h \mid \mid \alpha\_h & \beta\_h \mid \mid \alpha\_h & \beta\_h \mid \mid \alpha\_h \text{ (scheme3)}\\ m\_0 h\_0 & \stackrel{\alpha\_1}{\underset{\beta\_1}{\rightleftharpoons}} & m\_1 h\_0 & \stackrel{\alpha\_2}{\underset{\beta\_2}{\rightleftharpoons}} & m\_2 h\_0 & \stackrel{\alpha\_3}{\underset{\beta\_3}{\rightleftharpoons}} & m\_3 h\_0 \end{array}$$

where the kinetic rates are given by

$$\alpha\_i(V) = \alpha\_{i,0} \exp\left(\alpha\_{i,1} V\right)$$

$$\beta\_i(V) = \beta\_{i,0} \exp\left(-\beta\_{i,1} V\right)$$

$$\alpha\_h(V) = \frac{\alpha\_{h,0}}{1 + \alpha\_{h,1} \exp\left(\alpha\_{h,2} V\right)}$$

$$\beta\_h(V) = \frac{\beta\_{h,0}}{1 + \beta\_{h,1} \exp\left(\beta\_{h,2} V\right)}$$

Parameters α*i*,*<sup>j</sup>* and β*i*,*<sup>j</sup>* are given in **Table 1** for both axonal and somatic channels. In single-compartment simulations, somatic parameters were employed.


**Table 1 | Activation parameters for somatic and axonal sodium channels in the Schmidt-Hieber and Bischofberger model.**

Potassium channels in SB model are simulated by the same kinetic scheme as HH model (scheme 2) with the following voltage dependent transition rates:

$$\alpha\_n(V) = s\varepsilon \times 0.01 \frac{V + 55}{1 - \exp\left(-\frac{V + 55}{10}\right)}$$

$$\beta\_n(V) = s\varepsilon \times 0.125 \exp\left(-\frac{V + 65}{80}\right)$$

where *sc* is a scale parameter that adjusts the kinetic constants according to the compartments in which the channels are being simulated. For single compartment simulations, equation (1) was used with the following parameters: *Cm* = 1μF/cm2, *gNa* = 20 mS/cm2, *gK* = 4 mS/cm<sup>2</sup> , *gl* = 0.1 mS/cm2, *ENa* = 75 mV, *EK* = −95mV, *El* = −70 mV. With this model, the number of channels was controlled by the membrane area, given a unitary conductance of 20 pS/cm2. We tested areas ranging from 15.7 to 628 μm2, resulting in 157 to 6283 sodium channels and 31 to 1257 potassium channels.

With this model, the following tests were conducted at different values of membrane area:


#### *Schmidt-Hieber and Bischofberger model—idealized multicompartment model*

An idealized model similar to the one described in Schmidt-Hieber and Bischofberger (2010, see **Figure 5A**) was simulated using stochastic algorithms for the ion channels. The parameters for ion channel densities and kinetic constants were used as described in the article with minor modifications, such as the absence of an axonal bleb and a longer axon for some simulations.

#### **SIMULATION ALGORITHMS**

The methods described and tested here are designed to simulate a number of independent and identical Markov Chains (MCs) with a discrete number of states, keeping track of the number of channels in any state at any given time. For the description of the algorithms, we denote S as the total number of states in a MC, *i* ∈ {1,..., *S*} are the individual states, and Ni is the number of MCs in state i.

#### *Markov chain simulations (MC)*

Markov chains were simulated using the Stochastic Simulation Algorithm (SSA) (Gillespie, 1976) with some modifications. Briefly, the method consists in:

At time *t*, calculate the effective transition rate λ(*t*) as

$$\lambda(t) = \sum\_{i}^{s} N\_i(t)\xi\_i(t)$$

where ζ*i*(*t*) is the sum of transition rates for transitions escaping from state *i*.

Calculate the time for the next transition *tn* as

$$t\_n = t\_{\mathcal{P}} - \frac{\log \xi\_1}{\lambda(t)}$$

where *tp* is the time of the previous transition (0 at the beginning of the simulation) and ξ<sup>1</sup> is a random number uniformly distributed within [0,1], drawn after the previous transition.

If *tn* > *t*, continue integrating the time and the membrane voltage equation.

If *tn* < *t*, perform a transition:

Calculate the probability of all *j* transitions:

$$P\_{\dot{j}}(t) = \frac{N\_{\dot{i}}(t)\alpha\_{\dot{j}}(t)}{\sum\_{j} N\_{\dot{i}}(t)\alpha\_{\dot{j}}(t)}$$

where *i* is the state originating transition *j* and α*<sup>j</sup>* its rate.

Build a cumulative sum of all transition probabilities. Draw a random number ξ<sup>2</sup> uniformly distributed in [0,1] and find the first term in the cumulative probability that is greater than ξ2.

Execute the transition indicated by the term found in the previous step, and draw a new random number ξ<sup>1</sup> to be used for the time of the next transition.

## *Unbounded diffusion approximation (UA) (Orio and Soudry, 2012)*

The DA algorithm was implemented with SDEs described previously (Orio and Soudry, 2012; see also Mélykúti et al., 2010). In matrix form, the equations for the sodium and potassium channels are, respectively:

$$\frac{dX\_{\rm Na}}{dt} = A\_{\rm Na} X\_{\rm Na} + \frac{1}{\sqrt{N\_{\rm Na}}} S\_{\rm Na}(X\_{\rm Na}) \xi(t)\_{\rm Na}$$

$$\frac{dX\_K}{dt} = A\_K X\_K + \frac{1}{\sqrt{N\_K}} S\_K(X\_K) \xi(t)\_K \tag{2}$$

where *XNa* = *m*0*h*<sup>0</sup> *m*1*h*<sup>0</sup> *m*2*h*<sup>0</sup> *m*3*h*<sup>0</sup> *m*0*h*<sup>1</sup> *m*1*h*<sup>1</sup> *m*2*h*<sup>1</sup> *m*3*h*<sup>1</sup> *T* and *XK* = *n*<sup>0</sup> *n*<sup>1</sup> *n*<sup>2</sup> *n*<sup>3</sup> *n*<sup>4</sup> *<sup>T</sup>* are column vectors with the fraction of channels at any given state, and ξ (*t*)*Na* and ξ (*t*)*<sup>K</sup>* are column vectors of independent normally distributed random variables (mean 0, variance 1) with length 10 and 4, respectively. *NNa* and *NK* are the number of sodium and potassium channels, respectively. The rate matrices *ANa* and *AK* and square root matrices *SNa*(*XNa*) and *SK*(*XK*) can be directly found from the state diagram of the corresponding ion channel type. This is explained in detail around equations 1 and 13 and the Supplemental Material in Orio and Soudry (2012). For example, in the case of Potassium channels we have

$$A\_K = \begin{bmatrix} -4\alpha\_n & \beta\_n & 0 & 0 & 0\\ 4\alpha\_n & -3\alpha\_n - \beta\_n & 2\beta\_n & 0 & 0\\ 0 & 3\alpha\_n & -2\alpha\_n - 2\beta\_n & 3\beta\_n & 0\\ 0 & 0 & 2\alpha\_n & -\alpha\_n - 3\beta\_n & 4\beta\_n\\ 0 & 0 & 0 & \alpha\_n & -4\beta\_n \end{bmatrix}$$

and

$$\begin{array}{c} \text{SK } (\text{X} \boldsymbol{\chi}) = \begin{bmatrix} \sqrt{4\alpha\_n n\_0 + \beta\_n n\_1} & 0\\ -\sqrt{4\alpha\_n n\_0 + \beta\_n n\_1} & \sqrt{3\alpha\_n n\_1 + 2\beta\_n n\_2} \\ 0 & -\sqrt{3\alpha\_n n\_1 + 2\beta\_n n\_2} \end{bmatrix} \\ \begin{bmatrix} 0 & 0\\ 0 & 0 \end{bmatrix} \\ \sqrt{2\alpha\_n n\_2 + 3\beta\_n n\_3} & 0 \\ -\sqrt{2\alpha\_n n\_2 + 3\beta\_n n\_3} & \sqrt{\alpha\_n n\_3 + 4\beta\_n n\_4} \\ 0 & -\sqrt{\alpha\_n n\_3 + 4\beta\_n n\_4} \end{bmatrix} . \end{array}$$

Note that in the case of the Schmidt-Hieber and Bischofberger model, the only difference from the HH model is that the rate constants for the sodium channel equations are different.

To take care of normalization, variables *m*1*h*<sup>0</sup> ... *m*3*h*<sup>1</sup> and *n*<sup>1</sup> ... *n*<sup>4</sup> were advanced by an Euler-Maruyama scheme and the remaining two were calculated as *m*0*h*<sup>0</sup> = 1 − *m*1*h*<sup>0</sup> − *m*2*h*<sup>0</sup> − *m*3*h*<sup>0</sup> − *m*0*h*<sup>1</sup> − *m*1*h*<sup>1</sup> − *m*2*h*<sup>1</sup> − *m*3*h*<sup>1</sup> and *n*<sup>0</sup> = 1 − *n*<sup>1</sup> − *n*<sup>2</sup> − *n*<sup>3</sup> − *n*4. As we do not control the bounding of the variables between 0 and 1, in order to ensure real-valued square roots we calculated the stochastic terms *S*(*X*) (and only those terms) taking the absolute value of the variables. Thus, we refer to this algorithm as UA – Unbounded with Absolute values in stochastic terms.

## *Reflected SDEs (Ref) (Dangerfield et al., 2012)*

This method aims to normalize the variables *m*1*h*<sup>0</sup> ... *m*3*h*<sup>1</sup> and *n*<sup>1</sup> ... *n*<sup>4</sup> and to keep them bounded in the interval [0, 1] using the reflected stochastic equation approach, described in section V of Dangerfield et al. (2012). We have used this method together with the DA equation system (Equations 2).

## *Truncated and restored DA (TR) (Huang et al., 2013a)*

We used the DA equations (Equations 2) with an additional residual term. This residual term was introduced to ensure the boundary and normalization constraints, as explained in section II.F in Huang et al. (2013a).

## *Stochastic shielding approximation (SSmc) (Schmandt and Galan, 2012)*

In this method, transitions not connecting to the conducting states are approximated to be deterministic and solved as ODEs. Transitions connecting conducting with non-conducting states are solved as Markov Chains with the already mentioned algorithm. As there is a mixture of continuous (ODEs) and discrete (MC) treatment of variables, violations of the constraints occur. In our implementation, and inspired by the code by Schmandt and Galan, variables going off the [0,N] boundary are manually corrected and normalization was performed as in the UA algorithm.

## *Stochastic shielding approximation with DA (SSda)*

We modified the Schmandt and Galán (2012) approach by calculating the stochastic transitions with a DA approach (Equations 2) rather than using MCs. Therefore, SSda similarly uses the same DA equations, but the stochastic terms related to transitions not connecting to the conducting states were neglected.

For example, for potassium channels, we now used

$$S\_K(X\_K) = \begin{bmatrix} 0 \ 0 \ 0 & 0 \\ 0 \ 0 \ 0 & 0 \\ 0 \ 0 \ 0 & 0 \\ 0 \ 0 \ 0 & \sqrt{\alpha\_n n\_3 + 4\beta\_n n\_4} \\ 0 \ 0 \ 0 & -\sqrt{\alpha\_n n\_3 + 4\beta\_n n\_4} \end{bmatrix},$$

Thus, we needed only 2 Brownian terms for sodium channels and 1 for potassium channels. Boundary and normalization constraints were again treated as in the UA algorithm, that is: there was no bounding of the variables; the absolute value of the variables was used in the square roots of the stochastic terms; and normalization was applied by calculating *m*0*h*<sup>0</sup> = 1 − *m*1*h*<sup>0</sup> − *m*2*h*<sup>0</sup> − *m*3*h*<sup>0</sup> − *m*0*h*<sup>1</sup> − *m*1*h*<sup>1</sup> − *m*2*h*<sup>1</sup> − *m*3*h*<sup>1</sup> and *n*<sup>0</sup> = 1 − *n*<sup>1</sup> − *n*<sup>2</sup> − *n*<sup>3</sup> − *n*4.

#### *HH with colored noise terms (CN) (Güler, 2013)*

We simulated the Güler's Brownian harmonic oscillator using the system of stochastic differential equations 6.1, 6.2 in Güler (2013), using the constant parameters given in Table 2 in Güler (2013). Note that these equations are very different from the standard DA equations.

To take care of the normalization constraint at each time step, if any of the variables *m*, *h* or *n*, left the [0,1] interval, then the stochastic term η was redrawn until the variable fulfilled the boundary constraint.

## **SOFTWARE**

All the models and simulations algorithms presented here were implemented and run in the Neuron simulation environment (Hines and Carnevale, 2001; Carnevale and Hines, 2006). The different algorithms were written inside the MOD files for each channel. With the exception of TR, all algorithms run fine regardless of the numeric integrator specified (cnexp or euler). They also produce the same results in Python using the Euler-Maruyama integration method (tested in some selected cases). The TR algorithm required the specification of the Euler integrator within the MOD file to produce the results presented here, otherwise a much lower firing rate was obtained. Simulation control and the recording of variables were specified with Python scripts (Hines et al., 2009). Sample codes and .mod files can be found in ModelDB http://senselab.med.yale.edu/ModelDB/ Accession 167772 .

Data analysis and plotting was performed using the Python libraries numpy, scipy, and matplotlib.

## **RESULTS**

To test the accuracy of the methods we performed a series of simulations, comparing the variability of the results to that obtained with explicit MCs solved by the exact Gillespie algorithm (Gillespie, 1976, 2007). We employed the original Hodgkin and Huxley model (Hodgkin and Huxley, 1952), in order to reproduce previously published comparisons of the algorithms. In addition we performed some tests with a faster model, based on sodium channels from granular cells in the hippocampus (Schmidt-Hieber and Bischofberger, 2010). Finally, we simulated a model neuron with multiple compartments (Schmidt-Hieber and Bischofberger, 2010) and measured the variability in the generation *and conduction* of action potentials.

## **SIMULATIONS WITH THE ORIGINAL HODGKIN AND HUXLEY MODEL** *Firing variability—15 ms simulation with stimulus*

A widely used test to compare stochastic simulation algorithms (Mino et al., 2002; Bruce, 2007; Orio and Soudry, 2012 and others) consists of a short simulation (15 ms) in which a 2-ms current stimulus is given after a 1-ms delay (**Figure 1A**). Depending on the amplitude of the stimulus the probability of eliciting an action potential increases, and this relationship depends on the number of channels. **Figure 1B** shows the probability of firing an action potential in 10,000 trials at different stimulus amplitudes, for the algorithms tested with *NNa* = 5000. **Figure 1C** plots the variance of action potential timing, a measure of jitter. The Reflection method produces a higher firing probability at all stimulus amplitudes. This entails a lower variability in action potential timing. Additionally, Güler's CN method produces a higher variability than MC and other methods. This result is repeated with higher number of channels, however at a lower number of channels the spontaneous firing of action potentials makes the comparison unreliable. To compare the behavior of the models with *NNa* ≤ 1600, we modified the protocol so the stimulus is sustained during the simulation (**Figure 1D**, inset) and explored negative values of current amplitude. As can be seen in **Figures 1D,E**, the behavior of all the DA algorithms (as well as of SSmc) deviates from MC considerably for *NNa* = 50 and to a minor degree for *NNa* = 500. Again, the Reflection method produces a higher firing probability than the other methods.

## *Spontaneous firing rate*

The original HH model with stochastic ion channel produces spontaneous firing activity that increases as the number of channels is decreased. With each simulation algorithm and with sodium channel number (N*Na*) ranging from 50 to 50,000, we simulated 500 s and recorded the occurrence of action potentials. **Figure 1F** shows the mean frequency of spikes that were detected in the simulations. With *NNa* ≥ 1600, almost all the methods reproduce the behavior of MC modeling. The sole exception is the Reflection method, that showed a higher firing probability at *NNa* = 1600 and below. To discard some incompatibility of the Neuron simulation environment with the reflection procedure, we repeated this simulation using an Euler-Maruyama integration procedure written in Python and obtained the same result. Huang's truncated and restored DA method seems to be the one that more closely follows MC modeling at extremely low number of channels, only slightly underestimating the firing rate. At *NNa* = 160, the Unbound DA and Stochastic Shielding methods overestimate the firing rate, dropping abruptly when *NNa* = 50. This latter behavior is actually due to numeric overflows that made the simulations run without producing action potentials. This was corrected using a smaller time step (**Figure 1F**, segmented lines). Thus, the Unbound DA (note that SSda is also unbound) becomes numerically unstable when the number of channels is too low.

#### *Voltage clamp—noisy voltage trace with action potential*

A third test, to check how the different DA methods can reproduce the variability of channel openings obtained with MC modeling, consists on recording the response of the model channels to a fixed voltage trajectory obtained from a stochastic simulation. The voltage trace is shown in **Figure 2A** and it contains an action potential as well as a noisy background (zoomed in **Figure 2B**). With each model and condition, 2000 independent simulations were run and the time evolution of open channels was recorded. At each point in time, the mean and variance of the open channels was calculated. In addition to the comparison with the behavior of MC simulation, we compared to the expected mean of open channels which is calculated by applying the same voltage clamp simulation to deterministic HH channels. Moreover, we can compute the expected variance as explained in Methods.

The results are shown in **Figure 2**. During the subthreshold regime, Reflection method overestimates the mean of open channels, both for sodium and potassium (**Figures 2C,D**, left). Huang's TR algorithm also overestimates the mean of open sodium channels (**Figure 2C**, left) to a minor extent. However, during the action potential any difference between the DA methods and the MC modeling or the exact solution appears to be negligible (**Figures 2C,D**, right). Regarding the variance of

pulse, for the same simulations shown in **(B)**. **(D,E)** Firing efficiencies obtained with a constant pulse (inset) of the indicated amplitudes with *NNa* = 50, *NK* = 15 **(D)** and *NNa* = 500, *NK* = 150 **(E)**. **(F)** Mean number

SSmc, stochastic shielding approximation (Schmandt and Galán, 2012); SSda, stochastic shielding with DA approximation; CN, colored noise (Güler, 2013).

the open channels (**Figures 2E,F**), the main deviation seems to occur with Güler's CN algorithm, which overestimates the variance of both open sodium and open potassium channels, during the subthreshold regime (left) and the action potential (right). Schmandt's stochastic shielding approximation (both SSmc and SSda) underestimates the variance of open channels during the action potential, when the voltage changes more rapidly. The results shown here are for *NNa* = 500, *NK* = 160; with higher number of channels (*NNa* = 5000, *NK* = 1600) we naturally observed less fluctuations but the results maintained: Reflected DA overestimates the mean of open channels in subthreshold regime; Colored Noise overestimates the variance of open channels in subthreshold regime; and Stochastic Shielding underestimates the variance of open channels during the action potential (not shown).

#### **SCHMIDT-HIEBER AND BISCHOFBERGER MODEL—SINGLE COMPARTMENT**

We decided to use a model with faster sodium channels, resembling mammalian ion channels, to test the accuracy of the DA methods when the transitions between states occur at faster rates. We chose a recently published model that focuses on the fast opening of sodium channel in the axon initial segment of granule cells from the hippocampus (Schmidt-Hieber and Bischofberger, 2010). We will refer to this model as the "SB" model. We noted that the SB model does not show spontaneous firing when simulated stochastically. Therefore, the 500-ms simulation test was not performed.

#### *Firing variability—20 ms with 1 ms stimulus*

We performed the test in which 20-ms were simulated with a 1-ms current stimulus (similar to **Figure 1A**). For each stimulus amplitude, 2000 simulations were run and the Firing Efficiency, mean firing time, and firing time variance were calculated. **Figure 3** shows that the results are similar to that obtained with HH. At high number of channels, most methods perform reasonably similar to MC with a higher excitability of the Reflection method (**Figures 3A,B**). However, when the number of channels is low all DA methods fail to approximate the results of MC, showing a much higher probability of firing at all amplitudes of the stimulus

scales. **(C)** Mean of open sodium channels during the subthreshold (left) and action potential (right) regimes, for the simulation algorithms tested. **(D)** Mean of open potassium channels. **(E)** Variance of open sodium channels. **(F)** Variance of open potassium channels. *NNa* = 500, *NK* = 160. Very similar results were obtained with *NNa* = 5000, *NK* = 1600 (see text).

(**Figure 3C**). Also, the Reflection method shows a lower firing time variability (**Figure 3D**).

#### *Voltage clamp*

We applied the voltage clamp test with the same voltage trace as the HH model to the SB model stochastic channels. Similar to what we observed for HH model, the Reflection method overestimates the mean of open channels (both sodium and potassium) during the subthreshold regime. Huang's truncated and restored method also overestimates it to a minor degree. During the action potential, the greatest deviation occurs with the stochastic shielding approximation, which underestimates the variance for both channels. With a higher number of channels, we observed similar results, with the exception of Huang's TR method performing better in the mean of open channels (not shown).

## **SCHMIDT-HIEBER AND BISCHOFBERGER MODEL—MULTI-COMPARTMENT SIMULATIONS**

To test the applicability of DA methods in more complex simulations of physiological relevance, we set up a multi-compartmental model of a neuron. We chose the idealized neuron described in Schmidt-Hieber and Bischofberger (2010) and shown in **Figure 4A**. Moreover, we kept the particular inhomogeneous sodium channel density for the axon that causes the action potentials to be initiated in the axon initial segment (AIS), about 10 μm from the soma (**Figure 4B**, bottom). The neuronal sections were spatially discretized according to their spatial constant λ, with a further increase in the number of segments in the AIS area. In total, the number of segments simulated were 895 with a 1500 μm axon and 2239 when the axon was extended to 7500μm. The distributions of segment areas is shown in **Figure 4C**. Together with the different ion channel densities and a unit conductance of 20 pS, the resulting distributions of number of channels per segment are shown in **Figures 4D,E**. It is noteworthy that the number of channels to be simulated in any given segment is rarely higher than 500 for sodium channels and never higher than 120 for potassium channels. In the model with the long axon, this adds up to 171189 sodium channels and 50670 potassium channels.

The model neuron with different stochastic channels was subject to a current clamp stimulus applied to the soma. The stimulus consisted in a 2 s noisy stimulus (**Figure 5A**) which in a deterministic simulation elicited 8 action potentials (represented by stars in the Figure). 400 independent simulations were performed with the same stimulus and **Figures 5B,C** show the raster plots (100 simulations) of spikes detected at the soma and at the tip of the axon, respectively. **Figure 5D** depicts a normalized firing probability calculated for the spikes at the tip of the axon. Both raster and firing probability plots show that the simulations with the Reflection method displayed a greater excitability, as several action potentials were only elicited with this algorithm and were not seen with the other DA methods or were seen with a much lower probability (i.e., around *t* = 1300 ms, *t* = 1700 ms and near the end of the trace). On the other hand, Stochastic Shielding with MC produced a lower excitability, firing near half of the action potentials per sweep than the other methods. The mean of spikes per sweep (**Figure 6A**) was significantly different to MC for all the algorithms, not only the Reflection and SSmc methods. With the Reflection method, however, the deviations from the other methods go beyond a higher excitability. Some spikes fired with high probability with all DA methods except for Reflection (see for instance around 800 ms), and some spikes had a slightly different timing with Reflection (700, 1500 ms). Therefore, the Reflection method in this test actually introduced a bias, producing spikes with different timings than the other DA method. The variability of the number of spikes elicited per trial varied with some DA algorithms compared to MC (**Figure 6B**) but only in the case of UA and SSmc a significant difference was observed.

We took the raster plots and searched for spikes that were repeated in at least 50% of the sweeps with the same timing ± 5 ms. These were called "common spikes" and for most algorithms 10–11 common spikes were found (**Figure 6C**), with the exception of Reflection and SSmc methods. Then, we measured how variable the timing of these spikes was at two axon locations, one near the initiation site and the other at the tip (**Figure 6D**). Although the TR method reproduced more closely the variability obtained with MCs, none of the observed differences was significant.

As a measure of variability with functional consequences, we examined how the duration of the action potential (duration measured at the detection threshold level of 0 mV) evolved as it propagates along the axon. The relevance of this measure is that the duration of action potentials at the release zone of a synapse will impact the amount of neurotransmitter released. **Figure 7A** plots the duration of all the action potentials recorded in a MC simulation at several sites of the axon, plotted against the duration at a site near the initiation. As a first observation, in the reference site there is a wide distribution of action potential durations, which gets narrower as the measurement site moves along the axon. Also, action potentials are shorter in the distal axon than in the initiation site and the Reflection method produces the shorter action potentials of all the methods (**Figure 7B**).

Besides getting shorter, action potential duration at the distal axon is completely uncorrelated to the duration at the initiation site (**Figure 7A**, right). We looked at how the correlation of action potential duration decays along the axon with the different stochastic simulation algorithms. Results are shown in **Figure 7C**. It is apparent that Reflection method produces a faster decay in the correlation, while the Unbound DA produces a longer propagation of correlation. To test for similarity, we fitted an exponential decay to the data points, obtaining a space constant λ. An extra sum of squares *F* test was performed to test the null hypothesis that the data points of each set could be fitted with the same λ as the MC data, showing that Unbound DA, Reflection and SSda methods produced a behavior significantly different to that obtained with MC (**Figure 7C**, inset). When the test was repeated with a fit to a double exponential decay, the same result was obtained.

## **COMPUTATION TIME**

To account for the usefulness of the simulation algorithms, we found important to compare the computational cost of each of them. **Figures 8A,B** show the time required to simulate 500 ms of the HH model with an integration time step (*dt*) of 5 and 0.5μs, respectively, for each algorithm used. As we reported previously (Orio and Soudry, 2012), DA methods are highly sensitive to *dt* but mostly insensitive to the number of channels. On the other hand, MC simulations are sensitive to both, but its sensitivity to the number of channels approaches a linear relationship as the number of channel increases. Importantly, as both *dt* and/or number of channel decrease, MC outperforms all of the DA methods, giving the best simulation times precisely in the condition where the DA methods showed to be more problematic.

In the case of the multi-compartment simulation, MC is faster than all DA methods except for Schmandt's stochastic shielding approximation (**Figure 8C**). This result is not surprising, given that the number of channels per simulated compartment was almost always below 1000 (**Figures 6C,D**).

## **DISCUSSION**

In this work we numerically tested five Diffusion Approximation algorithms proposed to reproduce the behavior of a number of simultaneous Markov Chains, in the context of stochastic ion channel modeling. Most of these algorithms (Dangerfield et al., 2012; Orio and Soudry, 2012; Schmandt and Galán, 2012; Huang et al., 2013a) are based on a Langevin Equation proposed for stochastic modeling of the Hodgkin and Huxley model (Fox and Lu, 1994; Fox, 1997). However, they deal in different ways with numerical issues that appear in the simulation of stochastic trajectories: the requirement that the variables be bounded in [0,1] ("boundary constraint") and the requirement that at any given time the sum of variables must be equal to 1 ("normalization constraint"). The boundary constraint breaks more frequently when the number of channels being simulated is low or when the integration time step increases, because the stochastic terms scale with *dt*/*N*. Therefore, special attention should be put to the performance of these simulation algorithms—and their capacity of reproducing MC behavior—with a low number of channels.

are elicited in a deterministic simulation.**(B)** Raster plots of action potentials detected at the soma during 100 of the simulations performed

Our tests were primarily aimed with a practical question in mind: what method should be used to study the contribution of channel stochasticity to neural excitability in any given specific context? To answer this question, the simple test of the numerical accuracy is not enough. Modeling algorithms should also be examined for applicability and simplicity of implementation in different contexts and for the computational efficiency for the intended model to be solved.

potential detected in a bin. The resulting vectors were then smoothed by

convolving with a Blackmann filter function of length 20.

### **ACCURACY**

Regarding accuracy, in brief we found that all DA algorithms fail in the reproduction of MC behavior when the number of channels is low (generally speaking, below 1000), with no clear "winner." For example, the TR procedure improved the results in some current clamp simulations (**Figures 1D–F**), but introduced higher deviations than the UA in others (**Figure 3**). The Reflection method in our simulations performed the worse, introducing a higher firing probability in current clamp simulations. Most likely, this is related to the higher mean of open sodium channels observed in voltage clamp (**Figure 2C**).

When the number of channels is 5000 or higher, the TR, UA and SS methods perform well in reproducing MC behavior in current clamp tests. This means that in the high number of channels regime, bounding the variables to [0,1] is not essential, as the UA and SSda implementations (both unbounded) give the same results than TR. Inaccuracies were observed (**Figure 1**) in the Reflection method, and, to a lesser extent, the CN method.

**FIGURE 6 | Firing statistics and variability in the multi-compartment model. (A)** Mean of spikes per sweep detected at the axon tip in each of the stochastic simulation algorithms. Data was compared to MC using a *t*-test. **(B)** Variance of the spikes per trial in each stochastic simulation. Data was compared to MC using a Bartlett's test for equal variance. **(C)** Number of

common spikes found in each simulation. A common spike is a spike present in more than 50% of the trials with the same timing, considering a window of ±5 ms. **(D)** For all the common spikes detected, the variance in their timing was calculated near the initiation site (blue bars) and at the axon tip (green bars). In **(A–B)**, ∗∗∗*p* < 0.001, ∗∗*p* < 0.01.

**FIGURE 7 | Variability and correlations in the duration of action potentials. (A)** Duration of all action potentials (measured at 0 mV) at different positions of the axon, plotted against the duration at a site near the initiation site. This reference location is designated as "0 μm." Data presented correspond to simulation with MC. *r* is the Pearson correlation coefficient and p is the associated *p*-value when testing for a correlation different to 0. **(B)** Evolution of action potential duration along the axon, for the different simulation algorithms. Data is mean ±

*SD*. **(C)** Correlation of action potential duration at different points in the axon, with the duration at the reference "0μm" location. Lines represent the fit to a single exponential decay. Inset: length constants λ obtained in the fit of the different data sets. Error bars represent the *SD* of the parameter estimation. The fits were compared to MC using an extra sum of squares *F* test, to test the hypothesis that each data set and the MC set could be fit with the same parameter λ. ∗∗∗*p* < 0.001, <sup>∗</sup>*p* < 0.05.

Additionally, both methods showed inaccuracy in the voltage clamp tests. Also on these tests, the SS methods showed some inaccuracy in the variance of open channels during the action potential, but the results in current clamp simulations suggest that this may not be relevant for the neural excitability.

The results with the multicompartment model deserve special attention, because some results were different to what was observed in a single compartment. Although the SSmc method introduced only minor inaccuracies in the single compartment test, this method severely altered the excitability of the multicompartment model. We could not identify the reason for this effect, and tested several alternatives of bounding and normalization which did not improve the results. Nevertheless, the overall effect is a reduced excitability, which is consistent with the deviation observed in single compartment (**Figure 1D**). We confirmed that this effect does not arise from our implementation of the algorithm, by repeating the single compartment test with the Matlab code published by Schmandt and Galán (ModelDB acc. 144468) and obtaining similar results (not shown). So, although it is an attractive method for increasing simulation speed in multicompartmental models, its use is not recommended until further testing is performed. The Reflection method altered the excitability of the neuron but the results were in agreement with the single compartment results. Curiously, the use of Stochastic Shielding in

a DA framework (SSda) improves the behavior of the simulations, bringing it closer to the behavior of MCs, like the TR does.

#### **APPLICABILITY AND SIMPLICITY**

In terms of applicability, a first observation is that the colored noise approach (Güler, 2013) can be used only if the ion channel is composed of independent subunits. This is sometimes true (e.g., the original HH model), but not always (e.g., the SB model). For example, channels with non-identical voltage sensors (Vandenberg and Bezanilla, 1991; Horn et al., 2000), cooperativity in the movement of voltage sensors (Bezanilla et al., 1994; Schoppa and Sigworth, 1998) or complex allosteric gating mechanisms (Horrigan and Aldrich, 2002) do not have independent subunits. Therefore, they cannot be modeled with the colored noise approach. Besides, Güler's equations have a number of constant parameters (γ*K*, γ*Na*, ω<sup>2</sup> *<sup>K</sup>*, <sup>ω</sup><sup>2</sup> *Na*, T*K*, T*Na*) that were estimated empirically to obtain an adequate level of channel noise (Güler, 2013). It is not clear how these parameters can be derived for other ion channels, even if they are composed of identical and independent gating subunits.

The other DA-based algorithms can be applied to any given kinetic scheme but first require to obtain the corresponding system of SDEs. Until recently, it seemed quite complicated to implement, since the original descriptions involved the calculation of a matrix root square (Fox and Lu, 1994; Goldwyn et al., 2011). However, alternative derivations of the Langevin equation (Mélykúti et al., 2010; Orio and Soudry, 2012) yield an explicit form that does not use complex matrix operations. This method can be derived for any given kinetic scheme using simple and intuitive rules without using matrix notation (see Supplemental Material in Orio and Soudry, 2012). Importantly, any kinetic scheme can be translated to an SDE system and the equations can be written explicitly. This makes it simpler to employ low-level or limited languages such as C or Neuron's NMODL, which are compiled prior to execution code and therefore run faster.

The DA algorithms differ in the treatment of boundary and normalization constraints. In this regard, the Unbound DA (UA) is the simplest, not taking care of the boundaries issue and doing a simple normalization by making one variable to depend on the others (alternative normalization procedures can be implemented, for instance dividing all the variable values by their sum). Finally, to avoid non-real square roots in the stochastic terms, a simple absolute value operation is performed. The Stochastic Shielding approximation, when used with DA equations (SSda), can simplify the code even further because it uses less stochastic terms. Huang's truncation and restoration is also rather simple to follow and implement. However, it requires several lines of code and a series of nested *if* and *for* blocks when written in simple languages. As a side note, we found that the restoration procedure requires the specification of the Euler integration method within Neuron's NMODL files. Failing to do so, using instead the default Crank-Nicholson integrator, results in severely distorted results such as a much lower firing rate. Dangerfield's reflection method takes similarly amount of lines of code as TR but it was more complicated to follow and implement.

#### **COMPUTATIONAL EFFICIENCY**

Our comparison of simulation time shows that DA-based methods are not the best choice for all the situations. It has already been noted that when the number of channels is low, MC modeling runs faster than DA (Orio and Soudry, 2012). The limit (number of channels) at which this happens is variable, depending not only on the time step used for numerical integration (as shown in **Figure 8A** vs. **Figure 8B**) but also on the kinetics of the channels being simulated, as this determines the number of transitions occurring in MCs (see below). Interestingly, the Stochastic Shielding in the context of MC (SSmc) behaves like the DA methods: it is faster than MC only with large numbers of channels and its performance depends mainly on the integration time step. This results from the increase in the number of (non-stochastic) ODEs that have to be solved at each integration step, regardless of the actual transitions that occur.

At first glance, it seems that a similar comparison carried out in another work (see Figure 8 in Huang et al., 2013b) produced very different results. In Huang et al. the MC method is faster than DA only if there are less than 20 ion channels. In contrast, we found that MC is more efficient if there are less than ∼500 channels (*dt* = 5μs) or ∼5000 channels (*dt* = 0.5μs). The main reason for this large difference is that in the MC simulations of Huang et al. the state dynamics of each gating particle were updated individually. This method is highly inefficient and should be avoided in stochastic simulations of neurons. In contrast, we used the standard efficient MC method by Gillespie (1976), which tracks only the total number of channels in each state.

Interestingly, in another work (Rowat and Greenwood, 2014) it was found that Güler's CN method is much faster than Unbound DA. However, we have found that it is only a little faster (**Figures 8A,B**). The difference might be attributed to the language (Neuron vs. Python), or some other difference in the implementation.

Within all DA methods, Stochastic Shielding is the fastest in all circumstances. Next comes the colored noise approach (as noted, not applicable to all kinetic schemes) closely followed by Unbounded DA and Reflection. Recall that Stochastic Shielding (SSda) produces a minor loss in accuracy when used with a large number of channels, being in almost all cases indistinguishable from MC modeling. Finally, in our simulations the less efficient (slowest) algorithm was the Truncated and Restored DA (Huang's).

The numerical stability issue also deserves to be considered. Although we did not perform a systematical assessment of numerical stability in our simulations, we noted that simulations with the UA and SSda algorithms produced unreliable results with *NNa* = 50, even at the lowest *dt* of 0.5μs. As the other algorithms did not show this problem, it is most likely due to the lack of variable bounding. We did not pursue in finding a fix for these fails as the low channel number is a condition where MC modeling becomes the fastest and most accurate method.

#### **THEORETICAL ESTIMATES OF ACCURACY AND SPEED**

For a given model, when is it better to use DA instead of MC? Specifically, we would like to know in advance when a DA approach will be accurate, and also faster than MC. A definite answer usually requires some preliminary simulations. However, as we explain next, a rough estimate could be obtained based on the following numbers: the simulation timestep, the number of channels to be simulated, and their typical constant rates.

Suppose we have N ion channels (of some particular type), with X of these ion channels in some state A, and α being a kinetic rate from state A to another B. In each simulation timestep dt, let be the number of channels switching from state A to state B. As different channels are independent, is distributed according to a binomial distriubtion with *n* = *X*, and *p* = α*dt*. Therefore, the average number of channels switching from A to B in that timestep is *np* = *X*α*dt*.

These quantities can be used to estimate the expected accuracy of DA. As explained in Orio and Soudry (2012), the key idea in DA is to use the central limit theorem and approximate the distribution of to be Gaussian. This approximation becomes accurate when *np* = *X*α*dt* 1. This also means that *N*α*dt* 1 since *X* < *N*. For example, in the HH model the slowest kinetic rates (in the relevant voltage range), are about α ∼ 0.1*ms*<sup>−</sup>1. Then if *dt* = 5μs DA should be expected to be accurate only when *N* > 2000, which is comparable to what we found in our simulations.

Using the same quantities we can also estimate the relative speed of the MC and DA algorithms. Simulation time is roughly proportional to the number of times the simulation variables (the fraction of channels in each) are being updated at each *dt* timestep (in which the voltage is updated). On the one hand, the MC algorithm performs updates each time a single channel switches between states. The number of these updates in each timestep, for each type of switch, is proportional to . Recall that the mean of is equal to *np* = *X*α*dt*. Therefore, in total, about *N*α*dt* updates are performed on each timestep (where α is the appropriate average over all the kinetic rates). On the other hand, the DA algorithms perform a single update at each timestep. Therefore, DA should become more efficient than MC only when *N*α*dt* > 1. For example, in the HH model most state transitions occur near rest voltage in the (fast) *m* kinetics, and so we get approximately α ∼ 2ms<sup>−</sup>1. This condition yields results comparable to what we found in our simulations (**Figure 8**): if *dt* = 5μs then N > 100, and if *dt* = 0.5μs then N > 1000.

### **CONCLUSIONS**

Suppose *dt* is the simulation timestep, *N* is the number of ion channels, and α is the "typical" transition rate of the channel. Our results suggest that, as a rule of a thumb,


#### **ACKNOWLEDGMENTS**

This work was supported by grants Fondecyt 1130862, ACT-1104, and ACT-1113 to Patricio Orio (Consejo Nacional de Investigación Científica y Tecnológica CONICyT, Chile). The Centro Interdisciplinario de Neurociencia de Valparaíso is a Millenium Science Institute supported by funds P09-022-F, Ministerio de Economía, Chile.

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 26 May 2014; accepted: 13 October 2014; published online: 03 November 2014.*

*Citation: Pezo D, Soudry D and Orio P (2014) Diffusion approximation-based simulation of stochastic ion channels: which method to use? Front. Comput. Neurosci. 8:139. doi: 10.3389/fncom.2014.00139*

*This article was submitted to the journal Frontiers in Computational Neuroscience. Copyright © 2014 Pezo, Soudry and Orio. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Modeling the influence of short term depression in vesicle release and stochastic calcium channel gating on auditory nerve spontaneous firing statistics

## *Bahar Moezzi\*, Nicolangelo Iannella and Mark D. McDonnell*

*Computational and Theoretical Neuroscience Laboratory, Institute for Telecommunications Research, University of South Australia, Mawson Lakes, SA, Australia*

#### *Edited by:*

*Joshua H. Goldwyn, New York University, USA*

#### *Reviewed by:*

*Ian Bruce, McMaster University, Canada John Wittig Jr., National Institutes of Health, USA*

#### *\*Correspondence:*

*Bahar Moezzi, Computational and Theoretical Neuroscience Laboratory, Institute for Telecommunications Research, University of South Australia, Building W, Mawson Lakes Boulevard, Mawson Lakes, SA 5095, Australia e-mail: bahar.moezzi@ mymail.unisa.edu.au*

We propose several modifications to an existing computational model of stochastic vesicle release in inner hair cell ribbon synapses, with the aim of producing simulated auditory nerve fiber spiking data that more closely matches empirical data. Specifically, we studied the inter-spike-interval (ISI) distribution, and long and short term ISI correlations in spontaneous spiking in post-synaptic auditory nerve fibers. We introduced short term plasticity to the pre-synaptic release probability, in a manner analogous to standard stochastic models of cortical short term synaptic depression. This modification resulted in a similar distribution of vesicle release intervals to that estimated from empirical data. We also introduced a biophysical stochastic model of calcium channel opening and closing, but showed that this model is insufficient for generating a match with empirically observed spike correlations. However, by combining a phenomenological model of channel noise and our short term depression model, we generated short and long term correlations in auditory nerve spontaneous activity that qualitatively match empirical data.

**Keywords: calcium dynamics, stochastic synapse, inner hair cell, auditory nerve, short term depression, neural variability, channel noise**

## **1. INTRODUCTION**

In the vertebrate auditory pathway, the inner hair cell and auditory nerve (IHC-AN) complex is the principal structure for the transduction of basilar membrane motion to stochastic trains of action potentials (Mulroy et al., 1974; Glowatzki and Fuchs, 2002; Johnson et al., 2009; Matthews and Fuchs, 2010). A computational model of the IHC-AN complex was proposed by Meddis (1986), and later modified by Sumner et al. (2002) to become a component in a larger computational model of the transformations of sounds by the middle ear. Unlike the Meddis (1986) model, in the Sumner et al. (2002) model, vesicle release from the IHC to the cleft was conceptualized as quantal and accruing with a probability that had a third power dependence on presynaptic calcium concentration. Later, the Sumner et al. (2002) model was modified by Meddis (2006) to take into account more physiological functions.

Here, we present a revised version of the Meddis (2006) model of the IHC-AN complex, with the aim of enhancing understanding of the biophysical sources of stochastic variability in the IHC-AN complex, by generating auditory nerve spontaneous spiking that provides an improved statistical match with empirical data.

The Meddis (2006) model includes a probabilistic "relative refractoriness" component, which is designed to replicate observed variation in the minimum time between spikes in AN fibers. Here we propose a pre-synaptic physiological explanation as the cause for what is attributed to post-synaptic relative refractoriness (note that we do not alter the original model's "absolute refractory" period, which models spike generation and membrane potential recovery). Specifically, we introduce a model of short term depression in pre-synaptic vesicle release, similar to short term plasticity models developed for cortical synapses (Tsodyks and Markram, 1997; Scott et al., 2012; Hennig, 2013; McDonnell et al., 2013). Unlike most such models, the conceptual model here is that there is a temporarily reduced probability of pre-synaptic vesicle release, following each actual release. Also unlike those models, the input to the synapse is not discrete spiking events, but instead the continuously valued membrane potential of the inner hair cell. The reason our model is suitable for capturing phenomena that have traditionally been attributed to post-synaptic relative refractoriness is that it introduces variability in the time between vesicle releases, which in turn leads to variability in the minimum time between post synaptic spikes. Our reasons for seeking this alternative conceptual model are given in the Discussion section.

We compare the resulting auditory nerve spontaneous firing statistics of our model with the firing statistics published by Heil et al. (2007). For spontaneous neural activity in auditory nerve fibers, inter-spike interval (ISI) distributions have been shown by Heil et al. (2007) to match empirical data better if the vesicle release inter-event interval (IEI) distribution was assumed to be a mixture of an exponential function and a gamma function with shape factor 2, both having the same scale parameters. We show that the probability density function (PDF) of ISI data obtained by Heil et al. (2007) fits PDF of ISI data obtained from our simulation if the time constant of short term depression is assumed to be around 2.5 ms.

Short and long term correlations have been observed in the spontaneous activity of auditory nerves (Teich, 1989; Lowen and Teich, 1992; Teich and Lowen, 1994). For individual auditory nerve fibers, it was shown that the Fano factor for spike counts increases for time scales from around 100 ms to tens of seconds indicating positive long term correlation and decreases slightly for time scales of around tens of milliseconds indicating short term negative correlation (Teich, 1989; Lowen and Teich, 1992; Teich and Lowen, 1994). Here we include a calcium channel noise model in the Meddis (2006) model. We show that for spontaneous activity, this biophysical noise model does not generate the short and long term correlations observed in the Teich and Lowen (1994) Fano factor curves.

However, we also modify the Meddis (2006) model to include a combination of a phenomenological model of IHC calcium channel noise and our model of short term depression in vesicle release. Using this model, for auditory nerve spontaneous activity, we generate Fano factor time curves that qualitatively match empirical Fano factor time curves of Teich and Lowen (1994); Teich (1989); Lowen and Teich (1992).

## **2. MATERIALS AND METHODS**

Firstly, in Section 2.1, we review the previous models that our research is built upon:


Then in Section 2.2, we provide a review of previous statistical analysis of empirical auditory nerve spontaneous activity data including research published by Heil et al. (2007), Teich and Lowen (1994), Teich (1989) and Lowen and Teich (1992). The final models we describe in Section 2.3 are our modifications to the Meddis (2006) model. These are designed to enhance understanding of the biophysical origin of stochastic variability in AN spiking, and to generate auditory nerve spontaneous spiking that provides an improved statistical match with empirical results, as described in Section 2.2.

## **2.1. PREVIOUS MODELS**

#### *2.1.1. Inner hair cell model*

Meddis (2006) describes a deterministic calcium-dependent model for converting the membrane potential of an inner hair cell, *v*(*t*), to a vesicle release rate, *k*(*t*). We use *c*(*t*) to describe the intra-cellular calcium concentration (relative to its rest concentration) as a function of time. In the model, the release-rate for available vesicles, *k*(*t*), is proportional to the cube of *c*(*t*). The calcium concentration depends on four constants, τ*c*, *Gc*, *Ec*, ν, on the membrane potential, *v*(*t*), and on an additional variable, *m*(*t*), where *m*3(*t*) represents the fraction of open channels at time *t* as well as the probability of a calcium channel to be open. This depends on three constants, γ , β and τ*m*, and on *v*(*t*). Note that *m*3(*t*) is bounded to the interval [0, 1], which is essential for it to physically represent a fraction of open channels. The maximum value of 1 occurs when *v*(*t*) is large and positive and the minimum value of 0 occurs when *v*(*t*) is large and negative.

In summary, the model has the following parameters:


The values of these parameters are summarized in **Table 1**. The equations describing conversion from *v*(*t*) to *k*(*t*) are

$$k(t) = \max(0, bc^3(t)),\tag{1}$$

$$\frac{dc(t)}{dt} = -\frac{c(t)}{\tau\_c} + \nu G\_c m^3(t)(E\_c - \nu(t)),\tag{2}$$

$$\frac{dm(t)}{dt} = -\frac{m(t)}{\tau\_m} + \frac{1}{\tau\_m \left(1 + \frac{e^{-\gamma\nu(t)}}{\beta}\right)},\tag{3}$$

where *k*(*t*) has units of releases per second. We have modified the Meddis (2006) and Sumner et al. (2002) models by introducing a constant ν with units of MA<sup>−</sup>1s <sup>−</sup><sup>1</sup> to ensure all terms in Equation (2) have units of Ms<sup>−</sup>1, where M is the unit of molar concentration. By fitting to the saccular hair cells of the bull-frog data, it has been shown (Hudspeth and Lewis, 1988) that

$$\nu = \frac{L}{2\mathcal{F}C\_{\nu}\xi},\tag{4}$$

where *F* is Faraday constant, *Cv* is the cell volume, ζ is the fraction of cell volume where calcium is accumulated to and *L* is the proportion of free calcium in the neuron. The values of these parameters are summarized in **Table 2**, with the result that ν = 2.3 × 10<sup>9</sup> MA<sup>−</sup>1s <sup>−</sup>1.

To confirm that our proposed model enhancements have no effect on previously established model features, in the Results section we compare the average vesicle release rates obtained from simulation of the proposed model to the average vesicle release

**Table 1 | Parameters for inner hair cell calcium levels.**


*The value of gc was obtained from Zampini et al. (2013). The value of v was obtained by running MAP BS with no stimulus present. All other values are identical to those used in publicly available Matlab source code MAP BS at http:// www*.*essexpsychology*.*macmate*.*me/ HearingLab/ modelling*.*html.*



*Values obtained from Hudspeth and Lewis (1988).*

rate obtained from simulation of the Meddis (2006) model. We introduce the notation *k* as the simulated average vesicle release rate. We show that the changes that we make to Meddis (2006) model result in *k* that are close to *k* obtained from the original model of Meddis (2006). The parameter *k* for the various proposed models are summarized in the tables.

A positive calcium current is required to increase the calcium concentration but in the Meddis (2006) and Sumner et al. (2002) models, calcium current is negative (i.e., inward) when *v*(*t*) < *Ec*. Therefore, we have used (*Ec* − *v*(*t*)) in Equation (2) instead of (*v*(*t*) − *Ec*) used in the Sumner et al. (2002) and Meddis (2006) models. The max( · ) function is included in Equation (1) since although it is possible for *c*(*t*) < 0 in the model (which represents calcium concentration less than its rest value), the rate *k*(*t*) cannot be negative. Note that the final term in Equation (2) has the form of the deterministic Hodgkin and Huxley (1952) voltagegated ion channel current model. Later, we replace this with a model of stochastically opening and closing ion channels.

### *2.1.2. Deterministic synapse model*

The input to the deterministic synapse model of Meddis (1986) is the rate at which the neurotransmitter is released to the cleft, *k*(*t*). There are three continuous-time-dependent variables that describe transport between a vesicle "factory," an "immediate store," the synaptic cleft, and a vesicle "recycling pool":


There are four parameters that have units of rate:


The values of these parameters are summarized in **Table 3**. The deterministic Meddis (1986) synapse model is of the following form

$$\frac{d\mathbf{x}(t)}{dt} = \mathbf{A}(t)\mathbf{x}(t) + \mathbf{B},\tag{5}$$

where

**Table 3 | Parameters for neurotransmitter release with values identical to those used in publicly available Matlab source code MAP BS at http://www.essexpsychology.macmate.me/HearingLab/ modelling.html.**


$$\mathbf{A}(t) = \begin{bmatrix} -r\_1 - k(t) & 0 & r\_2 \\ k(t) & -r\_3 - r\_4 & 0 \\ 0 & r\_4 & -r\_2 \end{bmatrix},$$

$$\mathbf{B} = \begin{bmatrix} r\_1 M \\ 0 \\ 0 \end{bmatrix} \text{and } \mathbf{x}(t) = \begin{bmatrix} x(t) \\ y(t) \\ z(t) \end{bmatrix}. \tag{6}$$

#### *2.1.3. Stochastic synapse model*

Subsequently, Sumner et al. (2002) and Meddis (2006) modified Meddis (1986) to build a model where movement of neurotransmitter is stochastic rather than deterministic and neurotransmitter in the immediate store is quantal rather than continuous. The stochastic Meddis (2006) synapse model is of the following form,

$$\frac{d\mathbf{x}}{dt} = \mathcal{B}(r\_1, (M - \mathbf{x}(t))) + \mathcal{B}(r\_2, \lfloor z(t) \rfloor) - \mathcal{B}(k(t), \mathbf{x}(t)), \tag{7}$$

$$\frac{dy}{dt} = \mathcal{B}(k(t), \mathbf{x}(t)) - r\_3 \mathbf{y}(t) - r\_4 \mathbf{y}(t),\tag{8}$$

$$\frac{dz}{dt} = r\_4 \mathbf{y}(t) - \mathcal{B}(r\_2, \lfloor z(t) \rfloor). \tag{9}$$

Stochastic movement of discrete vesicles of neurotransmitter is described by the binomial random variable, *B*(ρ , *n*): if there are *n* vesicles available during a small *dt*, each with equal probability of moving ρ*dt*, then *B*(ρ , *n*) is the number of vesicles moving during *dt*. Vesicles in the immediate store are quantal so *z*(*t*) is mapped to the largest previous integer, *z*(*t*).

### *2.1.4. Phenomenological synapse model*

It has been shown that by using rate estimates from a fractional Gaussian noise driven Poisson process model, the shape of published histograms of spontaneous discharge rate (Liberman, 1978) can be reproduced (Jackson and Carney, 2005). This has been incorporated into a phenomenological model of the synapse in the IHC-AN complex by Zilany et al. (2009); Zilany and Carney (2010); Zilany et al. (2014). This synapse model has both exponential and a power-law adaptation functions. The exponential adaptation is implemented using the diffusion model of Westerman and Smith (1988). The exponential adaptation path is followed by two parallel fast and slow power-law adaptation function. The fractional Gaussian noise is incorporated in the slow power-law adaptation path. The input to the synapse model is the relative membrane potential of the inner hair cell.

### *2.1.5. Models for converting vesicle release to AN spikes*

In the deterministic rate model of Meddis (1986), the amount of neurotransmitter in the cleft causes a post-synaptic spike at time *t* with probability,

$$p\_{\rm conv}(t) = hy(t)dt,\tag{10}$$

where *h* is a constant. An absolute refractory period of 1 ms during which no spike can occur is applied. A relative refractory period is not considered.

In the quantal stochastic model of Meddis (2006), each ejected vesicle to the cleft can generate a spike in the auditory nerve after an absolute refractory period (ARP) and relative refractory period (RRP) are considered. If a vesicle is released, a spike in the postsynaptic AN is generated if *p*conv(*t*) is greater than a uniformly distributed random number between 0 and 1.

$$p\_{\rm conv}(t) = \begin{cases} 0 & \text{if } \quad t - t\_l < t\_A, \\ 1 - C\_r e^{-(\frac{t - t\_l - t\_A}{t\_R})} & \text{if } \quad t - t\_l \ge t\_A, \end{cases}, \tag{11}$$

where *Cr* = 1, *tR* = 0.6 ms is the time constant of relative refractoriness, *tA* = 0.75 ms is the ARP, *t* is the current time, and *tl* is the time of the previous spike.

The conversion model of Sumner et al. (2002) is very similar to the conversion model in the Meddis (2006) model. The differences are that in the Sumner et al. (2002), *Cr* = 0.55 and *tR* = 0.8 ms.

In the Zilany et al. (2009), Zilany and Carney (2010) and Zilany et al. (2014) spike generator model, spike times in the auditory nerve are generated by a renewal process that simulate a non-homogeneous Poisson process driven by the output of the synapse model.

#### **2.2. PREVIOUS STATISTICAL ANALYSIS**

#### *2.2.1. Empirical vesicle release distribution*

Heil et al. (2007) has shown that the empirical ISI distribution for spontaneous neural activity in cat auditory nerve fibers is better described if the IEI distribution for vesicle release events is a mixture of an exponential distribution and a gamma distribution. The gamma distribution has a shape parameter equal to two, and both the gamma distribution and the exponential distribution have the same scale parameter.

To calculate the ISI parameters, ARP and RRP in the form of Equation (11) are used. Two additional parameters are involved:


Heil et al. (2007) obtained the following equation describing the ISI probability density function (PDF),

$$D(t) = \begin{cases} \frac{\theta}{t\_R(\theta - \frac{1}{t\_R})} ( (e^{-(\frac{t - t\_A}{t\_R})} - e^{-\theta(t - t\_A)} )(1 - \rho + \frac{\rho \theta}{\theta - \frac{1}{t\_R}}) \\ -\rho \theta (t - t\_A) e^{-\theta(t - t\_A)} ) & \text{for } t \ge t\_A, \\ 0 & \text{for } t < t\_A, \end{cases} \tag{12}$$

#### *2.2.2. Empirical firing correlations*

The Fano-factor time curve is a measure of correlation over time. Fano-factor is dispersion in a variable, as a function of an increasing time-window for obtaining data on which to estimate the dispersion. For a spike train, the Fano-factor is the variance of the number of spikes in a time window divided by the mean of number of spikes from a single spike train in that time window. We denote:


Teich and Lowen (1994), Kelly (1994), Teich (1989), Lowen and Teich (1992) plotted empirical Fano-factor time curves for neural activity in mammalian auditory nerve fibers as seen in **Figures 1A,B**. The Fano-factor is 1 for sufficiently small time windows. It slightly decreases to below 1 over time scales on the order of tens of ms after which it increases monotonically and reaches more than 10 for time windows of a few tens of seconds. It has been shown that negative short term correlation observed in the Fano factor curve of spontaneous activity of a simulated AN fiber model with second order refractory behavior matches the data of Lowen and Teich (1992) for time windows between 15 ms and 100 ms (Gaumond, 2002).

#### **2.3. NEW MODELS**

## *2.3.1. Short-term depression in vesicle release probability* **(STDv)**

In AN spontaneous spike trains, the shortest ISIs occur much less frequently than the most likely ISIs (Heil et al., 2007). In Meddis (2006), this feature of ISI statistics is accounted for by ARP and RRP. Given this model includes variable relative refractory times in AN fibers, during which pre-synaptic vesicle release is unaffected, this would mean many vesicles are released that do not give rise to spikes. We therefore seek an alternative model in which what has been attributed to refractoriness is actually mainly due to pre-synaptic effects, due to vesicles not being released at all for durations longer than the absolute refractory period of the ANs. We return to this in Discussion.

Our hypothesis is that all vesicle releases, apart from any that occur during the absolute refractory period, cause action potentials, but that vesicle release is subject to short term depression. We introduce short term depression to pre-synaptic release probability in a manner analogous to standard stochastic models of cortical short term depression (Tsodyks and Markram, 1997; Wang, 1999; Hennig, 2013; McDonnell et al., 2013). In this model, immediately following release, the probability of release drops dramatically and then, increases back to a baseline level over a time frame that matches the spike data.

There are two additional parameters introduced in this model:


The model for the change of *k*(*t*) over time is

$$\frac{dk(t)}{dt} = \frac{\max(0, bc^3(t)) - k(t)}{\tau\_t} + ak(t) \sum\_{i} \delta(t - t\_{\mathbb{V}\_i}), \quad (13)$$

where *tvi* is the time of *i*th release.

#### *2.3.2. Channel noise in inner hair cell calcium channels*

Auditory nerve spike trains show positive long term correlation and usually negative short term correlation (Teich, 1989; Lowen and Teich, 1992; Kelly, 1994; Teich and Lowen, 1994). We hypothesize that a possible origin of the correlation is stochastic variability in the inner hair cell calcium channels. A biophysical model and a phenomenological model of calcium channel noise in the inner hair cell are built.

*2.3.2.1. Biophysical model.* In the Meddis (2006) model, long term correlation observed in AN fibers can be partially explained by depletion of readily available vesicles, as explained in the Results section. In the Results section, we show that the Meddis (2006) depleted model with readily available vesicles depleted by decreasing the maximum number of vesicles in the immediate store, or by increasing the spontaneous rate, both require much higher vesicle release rate than the non-depleted Meddis (2006) model.

Other possible origins of the observed long term correlation have been suggested, including fractal ion channel gating (Teich, 1989; Liebovitch and Toth, 1990), fractal behavior of the specialized proteins with direct role in exocytosis (Lowen et al., 1997), self-organized criticality in ion channel gating for example due to ion-conformational interaction (Kharkyanen et al., 1993; Brazhe and Maksimov, 2006), and fractal dynamics of transmitter diffusion in the synaptic junction (Teich, 1989).

An integrate and fire model with renewal point process input has been suggested to be capable of producing long term correlation that matches empirical data from spike trains of cortical neurons (Jackson, 2004). Unlike cortical neurons, inner hair cells encode graded input with a graded membrane potential (Van Steveninck and Laughlin, 1996). We aim to cast light on the possible biophysical mechanisms in the IHC-AN complex that can produce renewal point processes and hence long term correlation in the spike trains of auditory nerves.

Meddis (2006) assumes the calcium concentration dependence of the release probability to be due to voltage dependent calcium channels. We hypothesize that a possible biophysical mechanism of the fractional Gaussian noise in Jackson and Carney (2005); Zilany et al. (2009); Zilany and Carney (2010); Zilany et al. (2014) is the random fluctuations in the number of open and closed calcium ion channels as they are expected to cause variability in vesicle release probabilities.

We introduce to the Meddis (2006) model a four-state model of channel gating with standard transition rate formulae (Goldwyn and Shea-Brown, 2011; Schmerl and McDonnell, 2013),

$$\alpha(\nu) = \frac{1}{\mathfrak{r}\_m \left(1 + \frac{\varepsilon^{-\gamma \cdot \nu(t)}}{\beta}\right)},\tag{14}$$

$$
\beta(\nu) = \frac{1}{\tau\_m} - \alpha(\nu). \tag{15}
$$

Equation (2) therefore changes to

$$\frac{dc(t)}{dt} = -\frac{c(t)}{\mathbf{r}\_c} + \nu \mathbf{g}\_c n(t)(E\_c - \nu(t)),\tag{16}$$

where *n*(*t*) is the number of open calcium channels out of total of *N* calcium channels and *gc* is the single calcium channel conductance.

*2.3.2.2. Phenomenological model.* We consider a phenomenological model of calcium channel noise that we add to the Meddis (2006) model. Instead of modeling discrete channel noise, we add an Ornstein Uhlenbeck process to the mean fraction of open calcium channels, *m*3(*t*). Equation (2) changes to:

$$\frac{dc(t)}{dt} = -\frac{c(t)}{\tau\_c} + \nu G\_c(f(m^3(t) + X(t)))(E\_c - \nu(t)), \quad (17)$$

where *f*( · ) := max (0, min (1,·)) ensures the fraction of open channels is restricted to the interval [0, 1] and *X*(*t*) is a noise driven from Ornstein Uhlenbeck process; i.e.,

$$dX(t) = \frac{dt}{\mathfrak{r}\_o}(\mu\_o - X(t)) + \sigma\_o dW\_t,\tag{18}$$

where *Wt* denotes the Wiener process and the mean (μ*o*), time constant (τ*o*) and variance (σ*o*) of the noise are positive constants.

#### *2.3.3. Noise in inner hair cell membrane potential*

We also consider an alternative phenomenological model of noise where the IHC membrane potential is subject to an additive Ornstein Uhlenbeck process. Equation (2) changes to:

$$\frac{dc(t)}{dt} = -\frac{c(t)}{\mathfrak{r}\_{\varepsilon}} + \nu G\_{\varepsilon} m^3(t) (E\_{\varepsilon} - \nu(t) - X(t)),\qquad(19)$$

#### *2.3.4. Combination of short term depression in vesicle release model and phenomenological calcium channel noise model*

A possible origin of short term correlation in AN spike trains is a form of refractoriness (Teich and Lowen, 1994). We introduce a model that combines short term depression in vesicle release and phenomenological calcium channel noise as follows

$$\frac{dk(t)}{dt} = \frac{k\_1(t) - k(t)}{\mathbf{r}\_s} + ak(t) \sum\_i \delta(t - \mathbf{t}\_{\mathbf{v}\_i}),\tag{20}$$

where *k*1(*t*) is the vesicle release rate when Ornstein Uhlenbeck noise is added as calcium channel noise in the Meddis (2006) model.

#### **2.4. PARAMETERS**

The parameters in **Table 1** (except *gc*), in **Table 3**, and for*tA* and *tR* (except in **Table 5**) were obtained from publicly available Matlab source code MAP BS at http://www.essexpsychology.macmate. me/HearingLab/modelling.html. The parameters in **Table 2** and for *gc* were obtained from the literature (Hudspeth and Lewis, 1988; Zampini et al., 2013). The values of *b* and *bc*(*t*)<sup>3</sup> were chosen in order to produce the desired spontaneous rates in AN fibers. In **Table 5**, the parameters τ*s*, *tR* and *a* were obtained through parameter searches, in order to obtain a close quantitative fit to the data of Heil et al. (2007), while keeping estimated values of θ and ρ close to the results of Heil et al. (2007). The parameters τ*o*, σ*<sup>o</sup>* and μ*<sup>o</sup>* were chosen to produce spontaneous activity in the auditory nerve that qualitatively matches the empirical Fano factor data of Teich and Lowen (1994).

The maximum number of readily releasable vesicles in the immediate store, *M*, in the Meddis (2006) model is considered to be 10. Moser and Beutner (2000) suggested the average number of vesicles in the immediate store to be about 14 vesicles per active zone. Khimich et al. (2005) suggested a readily release pool of about 22 docked vesicles in the IHC of mouse. Pangršic et al. ˇ (2010) estimated a readily releasable pool of 12 vesicles per active zone in the pachanga mouse. We assumed the maximum readily available pool size, *M*, to be 20 vesicles per active zone.

### **3. RESULTS**

#### **3.1. PREVIOUS MODELS**

**Figures 2A** (Gray) and **2B** (Gray) show the Fano factor time curve of a spike train generated by the Zilany et al. (2014) synapse model. These figures are obtained by running the model code available at http://www.urmc.rochester.edu/labs/ Carney-Lab/publications/auditory-models.cfm, with a relative membrane voltage input of 0 V. Like the empirical Fano factor of **Figure 2A** (Light blue), the Fano factor increases to about 10 for large counting time windows. The Fano factor in **Figure 2A** (Gray) does not decrease below one for shorter time windows as much as the empirical Fano factor shown in **Figure 2A** (Light blue) does. The time scales of the correlation do not match empirical data of **Figure 2A** (Light blue).

In the Meddis (2006) model, long term correlation observed in the auditory nerves can be partially explained by depletion of readily available vesicles in the immediate store. In **Figures 2A,B**, the blue trace is the Fano factor time curve for the original Meddis (2006) model with a maximum number of readily available

**FIGURE 2 | Time window dependent Fano factor for spontaneous activity in the auditory nerve obtained from previous models. (A)** Gray: Fano factor time curve for a spike train generated by the Zilany et al. (2014) model with SR∼100 spikes per second. Blue, green, red: in the original Meddis (2006) model, by decreasing the maximum number of available vesicles from M = 20 with SR∼65 spikes per second (Blue) to M = 6 with SR∼65 spikes per second (Red) or increasing the spontaneous firing

rate from SR∼65 spikes per second with M = 20 to SR∼160 spikes per second with M = 20 (Green), short term negative correlation starts at smaller time windows and long term positive correlation is increased. Light blue: Empirical data from spontaneous activity in auditory nerve with SR∼ 65 spikes per second created from data in Teich and Lowen (1994). **(B)** Time-window dependent Fano factor of **(A)** on linear axes for time windows shorter than 0.1 s. Parameters are summarized in **Table 4**.

vesicles of *M* = 20 and a spontaneous rate of around 65 spikes per second. The red trace is the Fano factor time curve for the original Meddis (2006) model with a maximum number of readily available vesicles of *M* = 6 and a spontaneous rate of around 65 spikes per second. The green trace is the Fano factor time curve for the original Meddis (2006) model with a maximum number of readily available vesicles of *M* = 20 and a spontaneous rate of about 160 spikes per second. The Fano factor curves in red and green increase to higher values than the Fano factor curve in blue does for large time windows and are a better qualitative match to the empirical data of **Figure 2A** (Light blue). The Fano factor does not reach 10 for sufficiently large time windows. The magnitude of the decrease in Fano factor below one for shorter time windows is comparable to the empirical data of **Figure 2A** (Light blue.)

As shown in **Figure 2A**, long term correlation in the Meddis (2006) model can be partially produced if either the maximum number of releasable vesicles is decreased or the firing rate is increased, both of which cause depletion of available vesicles in the immediate store. In this model, low spontaneous rate fibers are associated with smaller pools of vesicles, and high spontaneous rate fibers are associated with larger pools of vesicles. In the depleted model, the time scales of the correlation do not match the empirical data of **Figure 2A** (Light blue). Depletion of vesicles moves the onset of short and long term correlations to smaller time windows.

Depletion of available vesicles in the Meddis (2006) model (by decreasing the maximum number of available vesicles from 20 with SR of about 65 spikes per second to 6 with SR of about 65 spikes per second, or by increasing the spontaneous rate to around 160 spikes per second with maximum number of available vesicles of 20) produces an average vesicle release rate, *k*, of 107 and 55 (s<sup>−</sup>1) respectively that are both much larger than 5 (s<sup>−</sup>1), which is the *k* of the Meddis (2006) model with a maximum number of available vesicles of 20 and the spontaneous rate of about 65 spikes per second.

#### **3.2. SHORT-TERM DEPRESSION MODEL**

Here we consider the case where the relative refractoriness component of the Meddis (2006) model is removed and we use our alternative model of short term depression in vesicle release probability. That is, the release rate in the Meddis (2006) model , *k*(*t*), as given by Equation (1), was replaced by *k*(*t*) of Equation (13), and relative refractoriness in the auditory nerve was omitted. Using this model, ISI data for spontaneous activity in an AN fiber was simulated.

The effect of substituting relative refractoriness in the auditory nerve with short term depression in vesicle release in the Meddis (2006) model is more clearly observed in the simulated data when the absolute refractory period is (unrealistically) assumed to be zero. Under this assumption, **Figure 3A** shows that in the Meddis (2006) model, similar to including relative refractoriness in the auditory nerve, the alternative model of short term depression in vesicle release leads to the least probable ISIs being larger than they would otherwise be.

A distribution fitting application which returns maximum likelihood estimations of the model parameters was used to estimate the parameters that produce the best fit of the simulated ISIs to the empirical results. **Figure 3B** shows that the PDF of the simulated data for the Meddis (2006) model with AN relative refractoriness replaced by short term depression in vesicle release in blue and the best fit to Equation (12) in red. The refractory time constants, *tA* and *tR*, were kept at fixed values. The free parameters, θ and ρ, were estimated.

The models in **Figures 3A,B** were fitted to Equation (12), and the corresponding values of θ and ρ were estimated and summarized in **Table 5**. Parameters τ*<sup>s</sup>* and *a* were obtained through parameter search in order to obtain a good fit to data while keeping θ and ρ close to the result of Heil et al. (2007).

In two different neurons, Heil et al. (2007) obtained θ = 0.0988 (ms<sup>−</sup>1) and ρ = 0.39 for *tA* = 0.69 ms, and *tR* = 0.58 ms when SR = 65 spikes per second and θ = 0.0862 (ms<sup>−</sup>1) and ρ = 0.43 for *tA* = 0.73 ms, and *tR* = 0.41 ms when SR = 57.1 spikes per second. Using the short term depression in vesicle release model, we estimated θ and ρ to be 0.05 (ms<sup>−</sup>1) and 0.37, respectively. Thus, Heil et al. (2007) scaling factors, θ, and fraction of gamma distribution in the mix, ρ, are comparable to what we obtained with our model with comparable spontaneous rate.

However, while (Heil et al., 2007) assumed the post-synaptic refractory period to be less than 1 ms, we obtain our result with a post-synaptic refractory period of a few milliseconds. Despite

Fitting PDF of ISI data for Meddis (2006) model with relative refractoriness in the auditory nerve substituted by short term depression in vesicle release where *tA* = 0.75 ms, τ*<sup>s</sup>* = 2.5 ms and *bc*<sup>3</sup> = 5 s−<sup>1</sup> to Equation (12). In all traces, SR∼65 spikes per second. Parameters are summarized in **Table 5**.

this difference, our model has introduced three features to the model's ISI distribution that are common with the data of Heil et al. (2007): an ISI PDF with a single maxima such that the PDF increases from zero to its peaks for small ISIs just above the absolute refractory period, a comparable scale factor and a comparable fraction of gamma distribution in the mix of exponential and gamma distributions.

#### **3.3. CALCIUM CHANNEL NOISE**

#### *3.3.1. Biophysical model*

Here we consider the case where the biophysical model of calcium channel noise is added to the Meddis (2006) model. That is, in the Meddis (2006) model the state of each of *N* calcium channels is simulated, stochastic changes of states based on the state diagram of **Figure 4** are permitted, and *c*(*t*) given in Equation (2) is replaced by *c*(*t*) given by Equation (16).

Using this model, the time-window dependent Fano factor of spike counts in the auditory nerve model for different numbers of calcium channels were obtained and shown in **Figures 5A,B**. Unlike the empirical data of **Figure 5A** (Light blue), the Fano factor does not increase steadily to a value around 10 for long time windows. A slight decrease in Fano factor for shorter time windows is observed.

In the hair cells of a chick's cochlea, for each hair cell, around 100 calcium channels for short hair cells and 341 for tall hair cells are suggested (Martinez-Dunst et al., 1997), which in turn suggest quite small numbers of channels per synapse. In our model, no improvement was seen in long term correlation by decreasing the number of calcium channels from 200 (Black) to 50 (Green), 10 (Red) and 5 (Blue). We conclude that this calcium channel model fails to add a long term correlation to the spike trains of the

auditory nerve in the Meddis (2006) model in a way that matches experimental observations shown in **Figure 5A** (Light blue).

Adding the biophysical calcium channel model with parameters summarized in **Table 6** to the Meddis (2006) model produces *k* of 4, 6, 5 and 4 (s<sup>−</sup>1) respectively for 5, 10, 50, and 200 calcium channels which are all close to 5 (s<sup>−</sup>1), the *k* of the original Meddis (2006) model with a spontaneous rate around 65 spikes per second and a maximum number of available vesicles of *M* = 20.

#### *3.3.2. Phenomenological model*

Here we consider the case where the phenomenological model of calcium channel noise is added to the Meddis (2006) model. That is, in the Meddis (2006) model, Equation (2) is replaced by Equation (17).

Using this model, the time-window dependent Fano factor of spike counts in the auditory nerve model were obtained and shown in **Figures 6A,B** (Blue) . It can be seen in **Figure 6A** (Blue) that, like empirical Fano factor of **Figure 6A** (Light blue), the Fano factor increases to about 10 for large counting time windows. But, the Fano factor in **Figure 6A** (Blue) does not decrease below one for shorter time windows as much as the empirical Fano factor shown in **Figure 6A** (Light blue) does.

Adding the phenomenological channel noise with parameters summarized in **Table 7** to the Meddis (2006) model produces *k* of 7 (s<sup>−</sup>1) which is close to 5 (s<sup>−</sup>1), the *k* of the original Meddis (2006) model with a spontaneous rate around 65 spikes per second and maximum number of available vesicles of *M* = 20.

#### **3.4. COMBINING SHORT-TERM DEPRESSION AND CALCIUM CHANNEL NOISE**

Here we consider a combination of short term depression in vesicle release with the phenomenological model of channel noise within the Meddis (2006) model. That is, in the Meddis (2006) model, *k*(*t*) from Equation (1) was replaced by *k*(*t*) from Equation (20) and relative refractoriness in the auditory nerve in the AN fiber was omitted.

**Figures 6A,B** (Green) show the time-window dependent Fano factor for auditory nerve fiber spike counts for this model. The Fano factor for this model increases steadily to about 10 for large

**FIGURE 5 | (A)** Time-window dependent Fano factor for spontaneous activity in an auditory nerve fiber model using the biophysical model of calcium channel noise in the IHC-AN complex applied to the Meddis (2006) model for different numbers of calcium channels, *N*. Light blue: Empirical data from

spontaneous activity in auditory nerve with SR∼65 spikes per second created from data in Teich and Lowen (1994). **(B)** Time-window dependent Fano factor of **(A)** on linear axes for time windows shorter than 0.1 s. Parameters are summarized in **Table 6**.

model. Green: adding a combination model of the phenomenological channel noise and short term depression in vesicle release to Meddis (2006) model. Light blue: Empirical data from spontaneous activity in an auditory nerve fiber with SR∼65 spikes per second created from data in Teich and Lowen (1994). **(B)** Time-window dependent Fano-factor of **(A)** on linear axes for time windows shorter than 0.1 s. **(C)** Fano factor for the Meddis (2006) model with

counting time windows. It can be seen in **Figure 6B** (Green) that for counting time windows of a few milliseconds, Fano factor decrease is slightly more than that of **Figure 6A** (Blue) and hence a better match to the empirical data of **Figure 6A** (Light blue).

Adding the combination model of phenomenological channel noise and short term depression in vesicle release with parameters summarized in **Table 7** to the Meddis (2006) model produces *k* of 5 (s<sup>−</sup>1) which is the same as the *k* of the original Meddis (2006) model with a spontaneous rate around 65 spikes per second and maximum number of available vesicles of *M* = 20.

As the maximum number of available vesicles in the immediate store decreases, as shown in **Figure 6C**, the corresponding minima in the Fano factor curve for shorter time windows increases and the short and long term correlations compare quantitatively to the results from the Zilany et al. (2014) model. *k* = 10 remains close to the *k* = 5 from the original Meddis (2006) model with a spontaneous rate around 65 spikes per second and a maximum number of available vesicles of *M* = 20.

This combination model produces a release rate for which the baseline level is mainly controlled by Ornstein Uhlenbeck noise and the post release behavior is mainly controlled by short term depression in vesicle release as shown in **Figure 6D** (Green).

depression in vesicle release for different maximum numbers of vesicles in the immediate store on linear axes for time windows shorter than 0.1 s. **(D)** Vesicle release rate for **(A)**. The color representations are the same as in **(A)**. subfigure is a zoom into the top subfigure. The rapid decreases in the green trace in the **(D)** bottom plot for the combination model of phenomenological channel noise and short term depression in vesicle release are due to actual vesicle release while remaining fluctuations are due to channel noise. Parameter are summarized in **Tables 7**, **8**.

#### **3.5. COMPARISON OF CALCIUM CHANNEL NOISE WITH MEMBRANE POTENTIAL NOISE**

Here we consider the inclusion of the phenomenological model of noise in the inner hair cell membrane potential model in (Meddis, 2006) model. That is, in the Meddis (2006) model, Equation (2) is replaced by Equation (19).

The time-window dependent Fano factor for AN spike counts in this model is shown in **Figures 6A,B** (Red). Like the situation of **Figure 6A** (Blue) where the Ornstein-Uhlenbeck noise is instead included as calcium channel noise, the Fano factor increases steadily to 10 for larger counting time windows, but it decreases below unity less than the empirical Fano factor of **Figure 6A** (Light blue) for smaller counting time windows.

Adding the phenomenological membrane potential noise with parameters summarized in **Table 7** to the Meddis (2006) model produces *k* of 5 (s<sup>−</sup>1) which is the same as the *k* of the original Meddis (2006) model with a spontaneous rate around 65 spikes per second and a maximum number of available vesicles of *M* = 20.

## **4. DISCUSSION**

We have shown that adding a combination of short term depression in vesicle release, and time-correlated channel noise, to the existing model of Meddis (2006) results in qualitatively similar results for spontaneous inter-spike interval correlations observed in empirical data. We make the case that it is the qualitative features of the Fano factor curve (namely the occurrence of positive and negative correlations, and the order of magnitude of the positive correlation) that are of most interest. Our model generates auditory nerve spontaneous spike trains for which the spike-count Fano-factor matches empirical data at short and long time scales qualitatively. The qualitative features of the Fano factor curve obtained from the proposed models are summarized in the last columns of **Tables 4**, **6**–**8**. However, the time scales of maximum negative correlation, and the onset of positive correlation do not exactly match the data. Moreover, the long term correlation in the biophysical model of IHC calcium channel noise does not match empirical data. There are several reasons for these discrepancies. First, the simulation data is only as good as the overall model, which omits many details of the complex calcium channel dynamics of ribbon synapses. We have seen, for example, that a standard biophysical model of channel noise does not induce long-term correlations, while replacing that model with a phenomenological model based on Ornstein-Uhlenbeck noise does so. We suggest that a more biophysically detailed model of calcium channel noise can improve the long term correlation to match empirical data. For example, a model where a single calcium channel controls vesicle release at each docking site (Weber et al., 2010) could potentially lead to a more complicated release dynamics and might produce long term correlation in the auditory nerve spontaneous spiking activity. A second reason might be that the parameters we used (including parameters in the Meddis (2006) model) need to be better tuned to match the empirical data. We left this for future work.

There are several justifications for replacing auditory nerve relative refractoriness with short term depression in vesicle release probability in the model. First, extensive neurotransmitter release can be toxic to neural tissues and cleaning up the excessive transmitters by glia cells requires a large amount of energy (Glowatzki et al., 2006). Short term depression in vesicle release will reduce the number of vesicles released, which in turn will reduce potential for toxicity and energy usage. Moreover, since it is thought that single vesicle produces spikes in AN fibers, for energetics reasons it is wasteful to release vesicles during the refractory period when spikes cannot occur.

A possible mechanism for short term depression in vesicle release could be the presence of auto-inhibitory metabotropic receptors called auto-receptors (Billups et al., 2005). To our knowledge, however, there is no evidence either for or against the presence of such auto-receptors in inner hair cells. Alternatively, it is possible that complex intra-cellular calcium dynamics and its relationship to vesicle exocytosis could cause such effects.

We hypothesize that observations of variable minimum time between spikes attributed to "relative refractoriness" above) in the IHC-AN complex is mainly due to pre-synaptic effects, namely that vesicle release sometimes doe not occur for a period longer than are the absolute refractory period. However, it is also possible that actual relative refractoriness in auditory nerve recovery following a spike (Cartee et al., 2000), and short term depression in vesicle release probability in the ribbon synapse could co-exist.

To obtain a fit close to the data of Heil et al. (2007), we have chosen the time constant of short term depression in the vesicle release to be 2.5 ms. Short term depression in vesicle release has been observed in synapses other than the ribbon synapse of inner hair cells (e.g., Stevens and Wang, 1995; Hjelmstad et al., 1997). Whole cell recordings from hippocampal pyramidal neurons showed that a 20 ms refractory period was required between

**Table 6 | Parameters of Meddis (2006) model with biophysical calcium channel noise.**



*M* = *20 and SR*∼*65 spikes per second.*

**Table 5 | Comparison of the original Meddis (2006) model and Meddis (2006) model with relative refractoriness in the auditoy nerve substituted by short term depression in vesicle release.**


*M* = *20 and SR*∼*65 spikes per second. For fitting to Equation (12), in the Equation (12) tA* = *0.75 ms and tR* = *3.5 ms were used. Log likelihood was used as a measure of goodness of fit.*


**Table 7 | Parameters for the phenomenological models of stochastic variability in the IHC-AN complex.**

*M* = *20, and SR*∼*65 spikes per second.*

**Table 8 | Parameters of the combination model of phenomenological channel noise and short term depression in vesicle release probability with various maximum numbers of vesicles in the available store.**


τ*<sup>o</sup>* = *1.2 ms,* σ*<sup>o</sup>* = *0.3,* μ*<sup>o</sup>* = *0.38,* τ*<sup>s</sup>* = *2.5 ms, a* = *0.001 and SR*∼*65 spikes per second.*

vesicle releases (Stevens and Wang, 1995). In a different experiment, Hjelmstad et al. (1997) observed a 6–7 ms period following release during which the synapse was incapable of transmission. Consequently, the time-scale of 2.5 ms is potentially biologically plausible.

In this paper we aimed to simulate auditory nerve spontaneous spiking patterns that provided an improved statistical match to empirical data. We modified a revised version of the Meddis (2006) model to develop a more biophysically detailed description of stochastic variability in the IHC-AN complex. It has been suggested (Morse and Evans, 1996; McDonnell et al., 2008) that significantly decreased stochastic variability in AN spiking generated by cochlear implants is a contributing factor to imperfect performance of these implants. A potential application of our model, therefore, is as a component in a larger model of the auditory system designed to predict differences in neural activity in higher brain regions, such as the cochlear nucleus, due to electrical stimulation by cochlear implants, in comparison with natural acoustic stimulation.

Based on our findings it will be interesting for future work to build on our study with a more detailed model of the calcium dynamics of the ribbon synapse in inner hair cells. Such a model might be capable of explaining both pre-synaptic shortterm depression in vesicle release, and long-term correlations due to calcium fluctuations.

#### **FUNDING**

Mark D. McDonnell's contribution was supported by the Australian Research Council under ARC grant DP1093425 (including an Australian Research Fellowship).

#### **ACKNOWLEDGMENTS**

We would like to thank Ray Meddis of the University of Essex, Bruce Graham of the University of Stirling, Nigel G. Stocks of The University of Warwick, Anthony N. Burkitt and David B. Grayden of The University of Melbourne, and Christian Stricker of Australian National University for helpful discussions.

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 05 June 2014; accepted: 26 November 2014; published online: 23 December 2014.*

*Citation: Moezzi B, Iannella N and McDonnell MD (2014) Modeling the influence of short term depression in vesicle release and stochastic calcium channel gating on auditory nerve spontaneous firing statistics. Front. Comput. Neurosci. 8:163. doi: 10.3389/ fncom.2014.00163*

*This article was submitted to the journal Frontiers in Computational Neuroscience.*

*Copyright © 2014 Moezzi, Iannella and McDonnell. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## On how correlations between excitatory and inhibitory synaptic inputs maximize the information rate of neuronal firing

## *Pavel A. Puzerey and Roberto F. Galán\**

*Department of Neurosciences, School of Medicine, Case Western Reserve University, Cleveland, OH, USA*

#### *Edited by:*

*Joshua H. Goldwyn, New York University, USA*

#### *Reviewed by:*

*Robert Rosenbaum, University of Pittsburgh, USA Woodrow Shew, University of Arkansas, USA*

#### *\*Correspondence:*

*Roberto F. Galán, Department of Neurosciences, School of Medicine, Case Western Reserve University, 10900 Euclid Avenue, Cleveland, OH 44106-4975, USA e-mail: rfgalan@case.edu*

Cortical neurons receive barrages of excitatory and inhibitory inputs which are not independent, as network structure and synaptic kinetics impose statistical correlations. Experiments *in vitro* and *in vivo* have demonstrated correlations between inhibitory and excitatory synaptic inputs in which inhibition lags behind excitation in cortical neurons. This delay arises in feed-forward inhibition (FFI) circuits and ensures that coincident excitation and inhibition do not preclude neuronal firing. Conversely, inhibition that is too delayed broadens neuronal integration times, thereby diminishing spike-time precision and increasing the firing frequency. This led us to hypothesize that the correlation between excitatory and inhibitory synaptic inputs modulates the encoding of information of neural spike trains. We tested this hypothesis by investigating the effect of such correlations on the information rate (IR) of spike trains using the Hodgkin-Huxley model in which both synaptic and membrane conductances are stochastic. We investigated two different synaptic input regimes: balanced synaptic conductances and balanced currents. Our results show that correlations arising from the synaptic kinetics, τ , and millisecond lags, δ, of inhibition relative to excitation strongly affect the IR of spike trains. In the regime of balanced synaptic currents, for short time lags (δ ∼ 1 ms) there is an optimal τ that maximizes the IR of the postsynaptic spike train. Given the short time scales for monosynaptic inhibitory lags and synaptic decay kinetics reported in cortical neurons under physiological contexts, we propose that FFI in cortical circuits is poised to maximize the rate of information transfer between cortical neurons. Our results also provide a possible explanation for how certain drugs and genetic mutations affecting the synaptic kinetics can deteriorate information processing in the brain.

**Keywords: stochastic Hodgkin-Huxley model, synaptic kinetics, input correlation, information, feed-forward inhibition**

## **INTRODUCTION**

The rate and timing of firing in cortical neurons is strongly affected by the interaction between synaptic excitation and inhibition (Salinas and Sejnowski, 2000). The architecture of cortical circuits ensures that the magnitude of excitatory and inhibitory synaptic inputs is approximately balanced on average and temporally correlated (Shu et al., 2003b; Haider et al., 2006), albeit with a small time delay for inhibition of ∼1–10 ms (Wehr and Zador, 2003; Okun and Lampl, 2008; Wu et al., 2008). This correlation in amplitude and timing presumably arises in feedforward inhibition (FFI) circuits, an anatomical motif present ubiquitously throughout the cortex which drives monosynaptic excitation and disynaptic inhibition onto target neurons (Porter et al., 2001; Sun et al., 2006; Cruikshank et al., 2007). The functional consequences of the correlations imposed by such a layout are far-reaching, encompassing a range of functions such as gain modulation for rapidly fluctuating synaptic inputs (Salinas and Sejnowski, 2000; Chance et al., 2002; Shu et al., 2003a; Pouille et al., 2009), shaping of neuronal tuning properties and stimulus selectivity (Wehr and Zador, 2003; Marino et al., 2005; Wu et al., 2006), directing the propagation of activity by selectively gating firing in neuronal ensembles (Kremkow et al., 2010a,b), and creating "windows of integration" during which excitatory inputs can temporally summate to promote spike generation before being rapidly suppressed by inhibition (Pouille and Scanziani, 2001; Pouille et al., 2009). Furthermore, Marsalek and colleagues demonstrated that small differences in the timing between presynaptic excitatory and inhibitory inputs (i.e., input correlation) is directly correlated with temporal jitter in postsynaptic spikes, i.e., output precision (Marsalek et al., 1997). Since neurons may represent information through the precise timing of spikes (decharms and Merzenich, 1996; Dan et al., 1998; Strong et al., 1998; Liu et al., 2001; Nemenman et al., 2008), it stands to reason that the control of spike timing by correlated excitation and inhibition is likely to govern the transfer of information between cortical neurons. Previous investigations using realistic simulations of cortical neurons have shown that, indeed, balanced excitatory and inhibitory synaptic currents maximize both coding and metabolic efficiency of neuronal spikes (Sengupta et al., 2013). In that study, however, the excitatory and inhibitory synaptic conductances were *uncorrelated* and as a result did not exhibit the correlation characteristic of cortical dynamics under experimental contexts (Wehr and Zador, 2003; Okun and Lampl, 2008; Wu et al., 2008). Furthermore, Kawaguchi and colleagues have shown that the relative balance between excitation and inhibition of a random synaptic input to simulated pyramidal neurons controls the maximal information content of spike trains in the presence of background synaptic noise (Kawaguchi et al., 2011), yet the timing of excitation and inhibition with respect to each other were not considered. To our best knowledge, the relevance of statistical correlations between balanced excitatory and inhibitory synaptic inputs for the information rates of neural spike trains have not been investigated. This may in part be due to the high computational cost of realistic simulations of stochastic neuronal dynamics. Here, we have overcome this limitation by using the stochastic-shielding approximation, which was recently introduced by our lab, accelerating stochastic simulations by up to two orders of magnitude while preserving accuracy (Schmandt and Galán, 2012).

A critical factor that influences the correlation between synaptic conductances and their effect on firing of cortical neurons is the time-course of the conductance change (Svirskis and Rinzel, 2000). This time-varying conductance shapes the trajectory of the membrane potential toward spike threshold and, consequently, alters the probability of firing an action potential. In support of this notion, previous findings have shown that the precision of spike-timing in pyramidal neurons has an inverse relationship with the decay kinetics of excitatory postsynaptic currents (Rodriguez-Molina et al., 2007), that is, spike time precision decrements as the postsynaptic currents slow down. The kinetics of postsynaptic synaptic responses may be modified by electrotonic filtering of the inputs across the dendritic arbor (Kleppe and Robinson, 1999), activation of distinct afferents (Walker et al., 2002), changes in the driving force (Salin and Prince, 1996), developmental changes in postsynaptic receptor (or receptor subunit) expression (Kirson and Yaari, 1996; Cohen et al., 2000; Bannister et al., 2005), interaction with intrinsic conductances (Miller et al., 1985; Wilson, 1995), and the presence of receptorspecific drugs (Orser et al., 1994; Poncer et al., 1996; Cohen et al., 2000). To our knowledge, the relationship between the kinetics of synaptic conductances and the information rates of neural spike trains has not been investigated.

In this study, we test the hypothesis that the rate of information transfer in cortical neurons depends on the correlation between concurrent excitatory and inhibitory synaptic inputs. We predict that an optimal time lag (δ) between excitation and inhibition would maximize information transfer between cortical neurons, since lags that are too short would preclude neuronal firing while long lags will likely decrease the precision of neuronal firing by prolonging the window of integration of presynaptic inputs. This prediction is consistent with a previous study showing an optimal time scale of rapidly fluctuating inputs for spike time reliability (Galán et al., 2008). Extending this hypothesis further, we predict that the rate of information transfer will depend on the kinetics of the synaptic conductance change, which inarguably affects the temporal correlation between synaptic excitation and inhibition (Svirskis and Rinzel, 2000). To test this hypothesis, we employ a biologically inspired Hodgkin-Huxley-type simulated neuron with stochastic ion channel gating (Schmandt and Galán, 2012) and drive it with Poisson trains of matched excitatory and inhibitory synaptic inputs. We test the impact of relative lag times between excitation and inhibition on the information rates of our model neuron across a range of lags and decay kinetics of synaptic conductances. Moreover, we compare the dependency of the information rates on the lags and kinetics in two synaptic regimes of (1) balanced conductances; and (2) balanced currents; this distinction is functionally important since the driving force can directly control the ratio between excitatory and inhibitory synaptic currents. Our findings reveal that the information rate (IR) of the neural spike train is indeed dependent on the synaptic kinetics as well as the relative delay times between excitation and inhibition. We show that the dependence of the IR on the synaptic kinetics shows an optimum at short and physiologically relevant monosynaptic delay times and that this dependence is present in the balanced currents, but not in the balanced conductances regime.

## **RESULTS**

To investigate the role of temporal and cross-correlations in synaptic inputs on information transmission in cortical neurons we modeled excitatory and inhibitory synaptic conductances as two separate input channels injected into a single-compartment conductance-based Hodgkin-Huxley model neuron with stochastic biophysics. Ion channel stochasticity is essential for this model to recreate biologically faithful spike behavior (Fitzhugh, 1965; Skaugen and Walloe, 1979; Strassberg and Defelice, 1993; Schneidman et al., 1998). To carry out this computationally expensive task, we applied the stochastic shielding approximation (SSA) to ion channel gating, which has been shown to recreate the behavior of stochastic Hodgkin-Huxley models using substantially less computational power than other approaches while preserving accuracy (Schmandt and Galán, 2012). Central to our method was our ability to generate trains of synaptic inputs with Poisson statistics and precisely controlled temporal and crosscorrelations. The temporal correlations of the barrages arise from the synaptic kinetics whereas cross-correlations are created by shifting two identical barrages relative to each other.

### **MAGNITUDE, KINETICS, AND CORRELATION OF SYNAPTIC EXCITATION AND INHIBITION**

We modeled excitatory and inhibitory conductances as two separate channels of Poisson-distributed events whose rate was set by the fixed parameter λ (5 ms<sup>−</sup>1), which is inversely proportional to the average inter-event interval, and whose kinetics were varied across a range of τ values (1–10 ms), representing the time-constant of the synaptic conductance decay. This time constant introduces a temporal correlation (auto-correlation time) in the synaptic barrage (see Materials and Methods). Excitatory and inhibitory conductance amplitudes were either matched (*gexc* = 3 pS/µm2, *ginh* = 3 pS/µm2) to simulate the balanced conductances regime or the inhibitory conductance was multiplied by a factor of 8, which in our model generated approximately balanced excitatory and inhibitory synaptic currents (*gexc* = 3 pS/µm2, *ginh* = 24 pS/µm2) on average. **Figure 1A**

represents the raw conductance traces in both the balanced conductance (**Figure 1A**, left) and balanced currents regimes (**Figure 1A**, right). For determining the effect of synaptic kinetics on information rates in cortical neurons, we generated synaptic input trains with different decay kinetics. The τ value was identical for a given pair of excitatory and inhibitory conductances, but was varied across different simulations. **Figure 1B** depicts a unitary synaptic conductance across a range of τ values. This value visibly sets the width of the time-varying conductance without affecting the rise time or the time at peak amplitude (**Figure 1B**). To understand how the kinetics shape the correlation structure between synaptic inputs, we analyzed the cross-correlogram between excitatory and inhibitory synaptic conductances as a function of the synaptic kinetics (**Figure 1C**). Clearly, the cross-correlation between excitation and inhibition is affected by the kinetics of the synaptic conductances, which are identical for both channels. As expected, the cross-correlation decays proportionally with the decay of the conductance waveform itself. Indeed, the numerically determined cross-correlation of the synaptic input trains (circles) accurately fits the analytically derived cross-correlation values (lines), as calculated in Materials and Methods. Central to our goal is the ability to also manipulate the cross-correlation between synaptic excitation and inhibition. To this end, we introduce a relative lag, δ, which delays the inhibitory conductance with respect to the excitatory. **Figure 1D** shows the raw conductance traces with inhibition lagging excitation by δ = 5 ms. In the context of cortical networks, such lagged correlations can arise through the FFI circuit (**Figure 1D**; inset schematic). This motif enables disynaptic inhibition generated by local interneurons (black circle) to lag behind monosynaptic excitation (black triangle) with delays ranging from 1 to 10 ms (Wehr and Zador, 2003; Okun and Lampl, 2008; Wu et al., 2008) thus, providing cortical neurons with windows of integration whose width is determined by the relative lag between excitatory and inhibitory conductances and their decay kinetics.

#### **SPIKING BEHAVIOR OF A STOCHASTIC HODGKIN-HUXLEY NEURON IN RESPONSE TO KINETICALLY VARIANT SYNAPTIC INPUTS**

The synaptic conductances described in the section above were injected into a single compartment Hodgkin-Huxley model of a neuronal membrane (100µm2) with stochastic voltage-gated Na<sup>+</sup> and K<sup>+</sup> conductances, and a deterministic leak conductance. When the magnitude of the synaptic conductances was set to zero, the neuron fired spontaneously at ∼30 Hz. This spontaneous firing resulted from the stochastic flickering of voltage-gated ion channels, as did the subthreshold oscillations of the membrane voltage seen during periods of quiescence (**Figure 2A**; top row). Depicted in **Figure 2A** are also the spike traces of the model neuron injected with matched excitatory and inhibitory conductances (middle row) and matched currents (bottom row). In both regimes, neurons were presented with trains of synaptic events with either fast (τ = 1 ms; left column) or slow kinetics (τ = 10 ms; right column). In the presence of excitatory and inhibitory *balanced conductances* with fast kinetics, the firing rate increased to ∼60 Hz (**Figure 2A**; left column, middle row) and further increased to ∼80 Hz when the kinetics were slow (**Figure 2A**; right column, middle row). In the synaptic input regime of balanced currents, the spike rate increased to ∼60 Hz when the synaptic kinetics were fast, but dropped to ∼30 Hz when the conductance decay was slow (**Figure 2A**; right column, bottom row). Thus, the decay time constant of the synaptic conductance impacts the firing rate differentially in the presence of balanced conductances vs. balanced currents. This becomes apparent when the firing rate is determined across the full range of synaptic kinetics and lags in both the balanced conductance (**Figure 2B**) and balanced currents (**Figure 2C**) regimes. Longer decay kinetics effectively increase the firing rate when the synaptic conductances are balanced and reduce the firing rate when the currents are balanced. Moreover, when the synaptic conductances are balanced, the firing rate shows no dependence on the lag between excitation and inhibition, but when the currents are balanced the firing rate is highly sensitive to short lags (0 >δ> 3 ms; **Figure 2C**).

**different input regimes. (A)** Raw traces of membrane potential dynamics in three different input regimes: top row corresponds to spontaneous firing in the absence of synaptic inputs; middle row corresponds to firing in response to balanced synaptic conductances; bottom row corresponds to firing in response to balanced synaptic currents. The left column represents neuronal firing in response to synaptic events with very fast kinetics (τ = 1 ms) and the

kinetics (τ = 10 ms). Note that the offset between excitation and inhibition in all traces is set at δ = 5 ms. **(B)** Surface plot of firing rates of the model neuron in response to *balanced synaptic conductances* with varying synaptic kinetics (τ ) and relative lags between excitation and inhibition (δ). **(C)** Surface plot of firing rates of the model neuron in response to *balanced synaptic currents* with varying synaptic kinetics and relative lags between excitation and inhibition.

#### **ENTROPY OF NEURAL SPIKE TRAINS**

Information content of a sequence of action potentials is by definition related to the variability of spike timing in response to an input signal. Repeated presentation of the same input conductance to the model neuron, therefore, enables us to measure the reproducibility of the resulting spike pattern. The top panel of **Figure 3A** shows the spike trains as in response to the same input conductance across ten trials (frozen input). Applying entropy measures as a proxy for spike variability, we obtain the "noise entropy" of the response across repeated presentations of the input signal. Noise entropy, however, informs us only about the spike variability to a single input pattern. To account for the full spectrum of potential spike responses of the model neuron, we presented a different set of input conductances across trials (unfrozen input; **Figure 3A**, bottom), this time yielding the "total entropy" of the spike train. Entropy measurements were carried out using the "direct method" (see Materials and Methods), which involved converting the output signal into a binary string of 0's and 1's by binning the spike trace with small time windows (*t* = 5 ms) and counting spikes within each bin. A bin containing no spikes corresponds to a 0, while a bin containing one or more spikes corresponds to a 1 (**Figure 3B**). We then generated sequences of words of various lengths (T = *t*× number of bins) which were then used to calculate the entropy based on the probability of occurrence of each possible word. Entropy, being an extensive property, scales with the length of the signal being measured and is sensitive to the temporal resolution of binning (Strong et al., 1998). Thus, to estimate the maximal entropy of the spike trains we extrapolated the entropy for words of length T→ ∞ (**Figure 3C**). Entropies were normalized by time to give entropy rates per time unit (bits/s).

We calculated the entropy rates for sets of paired excitatory and inhibitory conductances across a range of time lags for inhibition. **Figure 3D** shows the entropy rates as a function of δ for the sample traces shown in **Figure 3A**. These rates are exemplary of only a single value of the synaptic kinetics and are presented strictly heuristically. Our results show that the total and noise entropies are initially very low when excitatory and inhibitory conductances occur simultaneously but rise rapidly across a short range of δ values until they plateau around δ = 2 ms. The following sections will deal with the use of these entropy rates for the determination of spike train information rates.

## **INFORMATION RATE OF SPIKE TRAINS IS INSENSITIVE TO SYNAPTIC KINETICS AND THE RELATIVE DELAY OF SYNAPTIC INHIBITION IN THE BALANCED CONDUCTANCES REGIME**

Measuring the information rates of a neural spike trains requires that we take the difference between the total and noise entropy rates. This difference quantifies the IR without necessitating assumptions about the nature of the signal being represented. We applied this measure to spike trains generated across a range of τ and δ values to assess the dependence of the IR on the temporal correlation of excitatory and inhibitory synaptic inputs. This was first done for the balanced conductances regime. The top panel of **Figure 4A** shows the dependence of the information across a range of τ and δ values. The IR remains high and constant across

were generated and the probability of their occurrence was calculated to yield entropy rates. **(C)** Entropy rates for spike responses in response to frozen input (noise entropy) and in response to unfrozen input (total entropy) calculated across different word durations. The true entropy rates were extrapolated by taking the entropy rate in the limit of T→ ∞ (or 1/*T* → 0). **(D)** True entropy rates of neural spike trains in response to synaptic inputs with different lag times (δ) between excitation and inhibition.

spike trains as a function of the synaptic kinetics (τ ) and delays in inhibition relative to excitation (δ). (**A**, bottom left) Information rate as a function of synaptic kinetics for three different values of δ. This plot corresponds to three different slices taken from (**A**, top). (**A**, bottom right) Plot of firing rate as a

different values of δ when the synaptic kinetics are fast (τ < 4 ms) and increases by no more than 40% with increasing lags when the kinetics are slow (τ > 5 ms). The bottom left panel of **Figure 4A** shows three slices taken from the surface plot corresponding to the IR for three values of δ as a function of the kinetics. The selection of these three points will be clearly explained in the next section. Visible from this panel is that for the three different δ values, the change in IR follows a similar trajectory: for fast kinetics the IR slowly increases until reaching a maximum at ∼3 ms and then decreases for higher values of τ . The relationship between the IR and τ cannot be accounted for by changes in firing rate, which increases monotonically with increasing τ for all values of δ (**Figure 4A**, bottom right; **Figure 2B**). To quantify the range of information rates of the spike train across the full range of synaptic kinetics, we took the difference between the maximal and minimal IR values along the τ dimension for different δ values and saw that the IR range was highest for short lags (35 bits/s) and decreased steadily with increasing lags. This decrease in the IR range of the spike train corresponds to the flattening of the IR curve across the τ dimension with increasing values of δ.

synaptic kinetics is not accounted for by similar changes in firing rate. **(B)** Range of information rate as a function of relative delay between excitation and inhibition. The red dots correspond to the three values of δ shown in (**A**, bottom left) and (**A**, bottom right) (see Results for explanation).

## **INFORMATION RATE OF SPIKE TRAINS EXHIBITS DEPENDENCE ON SYNAPTIC KINETICS AT SHORT DELAYS FOR INHIBITION IN THE BALANCED CURRENTS REGIME**

We next determined how the IR changes with τ and δ in the balanced synaptic currents regime. The surface plot in top panel of **Figure 5A** shows an entirely different dependency of the IR on synaptic kinetics and relative lags times. For δ > 2 ms, the IR remains high and relatively constant across different values of τ ; however, as the synaptic lags decrease below 2 ms, the IR begins to show an optimal dependence to the synaptic kinetics. The bottom left panel of **Figure 5A** shows the IR as a function of τ for three values of δ. These results show that for instantaneous lags (δ = 0 ms) the IR is relatively low (*IR* < 20 bits/s) and decreases slowly across the τ dimension; for δ = 4 ms, the IR does not undergo dramatic changes and remains relatively high (*IR* > 95 bits/s); for δ = 0.8 ms, however, the IR begins at an intermediate value (70 bits/s) and increases until it reaches an optimum at τ = 4 ms, then drops 55% relative to the maximal value (**Figure 5A**, bottom left). Again, this dependency of the IR on the kinetics cannot be explained by changes in firing rate which decrease approximately monotonically with increasing

**FIGURE 5 | Information rates of neural spike trains in the balanced currents regime.** (**A**, top) Surface plot of the information rate of neural spike trains as a function of the synaptic kinetics (τ ) and delays in inhibition relative to excitation (δ). (**A**, bottom left) Information rate as a function of synaptic kinetics for three different values of δ. This plot corresponds to three different slices taken from (**A**, top). (**A**, bottom right) Plot of firing rate as a function of synaptic kinetics for the same three δ values presented in (**A**, bottom left)

shows that the dependency of the information rate on the synaptic kinetics is not accounted for by similar changes in firing rate. **(B)** Range of information rate as a function of relative delay between excitation and inhibition. The red dots correspond to the three values of δ shown in (**A**, bottom left) and (**A**, bottom left). Note the three points circled in red correspond to the peak IR range values and two non-adjacent δ that do not exhibit the optimum in information rate as a function of kinetics.

τ values (**Figure 5A**, bottom right). Applying the same analysis used in the previous section, we compute the IR range across the τ dimension for different values of δ and observe that the IR exhibits the most dramatic dependence of synaptic kinetics at an optimal value of the relative lag (δ = 0.8 ms). Thus, the selection of the three δ values shown in the bottom panels of **Figures 4A**, **5A** and circled in red in **Figures 4B**, **5B** are based on the range of δ values within which the optimum occurs (δ = 0 ms: δ = 4 ms) and the δ value at the optimum (δ = 0.8 ms). This optimal dependence of the IR on synaptic kinetics is only present when the synaptic currents are balanced, but not the synaptic conductances (**Figures 4B**, **5B**). Incidentally, when the synaptic input trains are normalized by the integral of their conductance, the IR decreases slowly and monotonically with τ in the balanced conductance regime and the observed peak of the IR as a function of τ disappears in the balanced currents regime (data not shown). This normalization, however, is not physiologically relevant considering that changes in synaptic kinetics in biological neurons are not compensated for by changes in the amplitude of the synaptic inputs on an event-by-event basis.

## **DISCUSSION**

In this study we set out to investigate how the encoding of information in neurons depends on the temporal and cross-correlation of balanced synaptic inputs. We manipulated the correlation between identical trains of excitatory and inhibitory inputs by directly controlling the decay kinetics (τ ) of the synaptic conductance and/or the relative time delay between excitation and inhibition (δ), with inhibition always lagging behind. Our results show that the encoding of information in neural spike trains exhibits a dependence on the correlation between balanced excitatory and inhibitory synaptic currents and that this dependence is absent in the input regime of balanced synaptic conductances. Specifically, findings reported herein demonstrate that the synaptic kinetics modulate the IR range at which the spike train maximally encodes information, but do so only when synaptic inhibition lags behind excitation with very short monosynaptic delays (δ < 2 ms). Furthermore, our model exhibits an optimal delay (δ = 0.8 ms) for inhibition at which the modulation of the IR by the synaptic kinetics is highest. Such delays between excitation and inhibition are within the physiological range of monosynaptic lags obtained from *in vitro* and *in vivo* recordings of synaptic barrages in cortical neurons (Wehr and Zador, 2003; Okun and Lampl, 2008; Wu et al., 2008). The optimum of the IR as a function of τ emerges as the result of the following: As stated in the Materials and Methods, the IR is determined as a difference between the total and noise entropies which represent the variability of the spike patterns in response to unfrozen and frozen input trains, respectively. Intuitively and empirically, the total entropy is substantially larger across different values of τ and δ and changes most drastically at values of δ < 2 ms (data not shown), at which the neuronal spiking is subject to a dramatic modulation by the inhibitory inputs. It is at this exact range of δ that the noise entropy is the highest across the τ (for τ < 5 ms) and δ dimensions. Why is the noise entropy highest when synaptic kinetics are fast? To answer this question we consider the relationship between synaptic input-driven spiking and spontaneous firing from stochastic fluctuations of intrinsic regenerative conductances. During synaptic bombardment, the synaptic conductance is the dominant driver of neuronal firing as it overwhelms the intrinsic conductances in both magnitude and duration. However, with increasing synaptic kinetics (smaller τ ), the integral of the synaptic conductance decreases and the dominance of the synaptic conductance abates, so that stochastic fluctuations of the intrinsic conductances allow for spontaneous firing. As a result, the spike patterns become more variable, thereby increasing the noise entropy. Thus, the peak of the IR emerges as a result of this increase in the noise entropy at small values of τ and δ and endows the neuronal membrane with the observed dependence of the IR on synaptic kinetics. These results are consistent with a previous report showing that spike-time reliability, an analytic measure related to the information capacity of a spike train, shows an optimal value at specific auto-correlation times of their uncorrelated synaptic inputs (Galán et al., 2008). Therefore, we conclude that synaptic kinetics as well as relative delays between synaptic excitation and inhibition may be tuned to optimize information transfer between neurons.

The time-course of the postsynaptic response may be subject to modulation by various factors including electrotonic distance of inputs from sites of integration (Kleppe and Robinson, 1999), pattern of afferent activation (Walker et al., 2002), driving force (Salin and Prince, 1996), postsynaptic receptor (and subunit) type (Kirson and Yaari, 1996; Cohen et al., 2000; Bannister et al., 2005), intrinsic conductances (Miller et al., 1985; Wilson, 1995), and the presence of receptor-specific drugs (Orser et al., 1994; Poncer et al., 1996; Cohen et al., 2000). The many ways in which the kinetics of the postsynaptic response to incoming inputs can be altered provides cortical neurons with a myriad of mechanisms to tune the correlation structure of incoming synaptic inputs. In particular, drugs, neuromodulators, etc. may change the synaptic kinetics to the point that the IR is outside its range, thereby deteriorating the processing of information in the brain and altering the state of awareness and consciousness.

We have shown here that the IR is also sensitive to the arrival times of inhibition with respect to excitation. Precisely controlling monosynaptic delay times for inhibition may be less trivial than tuning the synaptic kinetics, but is still possible. In the context of a feed-forward inhibitory circuit, one potential mechanism to tune inhibitory lags may be to alter the integration times of the feed-forward interneuron. Experiments in rats have shown that integration time in layer 4 stellate cells of somatosensory cortex is tightly regulated by thalamocortical FFI, thus controlling the precise spike timing of those neurons (Gabernet et al., 2005). Cortical interneurons also receive reciprocal inhibition (Lee et al., 2013; Pfeffer et al., 2013; Pi et al., 2013) and, as a consequence, are likely to have their integration windows regulated by inhibitory circuits. The size of the integration window of the feed-forward interneuron would control its precise spike timing and resultantly the lag of the inhibition in the excitatory neuron. Another way in which delays in inhibition can be modulated in a feed-forward circuit is through recruitment of distinct inhibitory networks (Beierlein et al., 2003). These networks are comprised of molecularly and physiologically distinct interneuron populations that exhibit differential responsiveness to temporally patterned inputs and distinct synaptic dynamics.

Pivotal to the simulations carried out in this study was our ability to efficiently simulate the spike behavior of the model neuron across numerous trials (sampling rate = 10 KHz; 5 s/trial; 56 trials for each δ and τ ; which yields ∼2.2 Gigabytes per data point in **Figures 4A**, **5A**). Stochastic simulations of ion channels are notoriously expensive computationally and often create the bottleneck for generating sufficient data across a large enough parameter range. We used the SSA for simulating stochastic ion channel gating dynamics (Schmandt and Galán, 2012) to avoid this problem. The SSA reduces the number of ion channel states requiring stochastic simulation, and therefore, dramatically reduces the computational load.

Modeling of the synaptic inputs required that several assumptions be made about the nature of cortical excitation and inhibition. First, the model assumes that excitatory and inhibitory synaptic inputs are correlated. This assumption has been validated by *in vitro* (Graupner and Reyes, 2013) and *in vivo* (Wehr and Zador, 2003; Okun and Lampl, 2008; Wu et al., 2008) recordings of synaptic barrages from cortical neurons showing that, indeed, excitation and inhibition are correlated in magnitude and timing, with inhibition tracking excitation by a few milliseconds. Secondly, the rate of synaptic events in time was assumed to be fast (5 ms inter-event interval), corresponding to high levels of correlated activity in presynaptic neurons. Recordings from cortical neurons in awake behaving mice during sensory stimulation (Crochet and Petersen, 2006), in anaesthetized ferrets during spontaneous active states (Haider et al., 2006), and in spontaneous active cortical slices (Compte et al., 2008) have confirmed high rates of synaptic bombardment, therefore, lending validation to the use of high rates of synaptic activity in our model. It is important to note, however, that synaptic inputs onto cortical neurons have also been shown to occur as sparse and synchronous population events (Wehr and Zador, 2003; DeWeese and Zador, 2006). Our study focused exclusively on synaptic regimes with high levels of activity, thus, it will be important to understand how temporal correlations between excitation and inhibition in sparse regimes affect information encoding. Previous findings by Miura et al. suggest that balanced excitation and inhibition in cortical neurons may in fact decouple irregularity of the spike train from rate modulations in firing, which may arise from changes in the synaptic input rate (Miura et al., 2007). Thus, the IR of the spike trains, being dependent on irregularity of the spike times, may be insensitive to changes in synaptic input rate if excitation and inhibition are balanced. On a related note, the magnitude of synaptic inhibition has been shown to have an inverse relationship with the overall rate of synaptic activity (Taub et al., 2013). This dependence shifts the relative balance between excitation and inhibition and may have a profound effect on encoding of information in cortical neurons. Future studies will need to address this problem to better understand the role of synaptic dynamics in neural coding.

The bulk of our study focused on the role of balanced synaptic inputs in encoding of information. A previous study by Sengupta and colleagues demonstrated that uncorrelated and balanced synaptic currents maximize the coding and metabolic efficiency of neuronal spikes by reducing the spike rate without substantially affecting the information rates (Sengupta et al., 2013). Our work applies the information theoretic approach using a similar model of a stochastic Hodgkin-Huxley neuron to address a different question: Are the correlations between synaptic inputs relevant for information processing? Our results indeed show a dependency of information encoding on the correlation between balanced synaptic currents. Moreover, the dependence of the information rates on the synaptic kinetics cannot be accounted for by changes in firing rate. Though balanced synaptic currents effectively decrease the firing rate as the kinetics slow down, this relationship is monotonic and does not exhibit the optimum dependence to the kinetics seen for the IR.

Using a conductance-based single-compartment Hodgkin-Huxley model offers insight into the interaction between synaptic inputs and an active neuronal membrane, but it ignores the complex shape and electrotonic geometry of cortical neurons. These features are important for spatiotemporal integration of synaptic inputs (Bernander et al., 1991) since distal dendritic inputs may be processed differently due to interactions with active dendritic conductances (Miller et al., 1985), differences in electrotonic properties (Kleppe and Robinson, 1999) or longer integration times caused by differences in FFI (Pouille and Scanziani, 2001; Pouille et al., 2009). Future studies should consider the complex geometry of cortical neurons and how it may impact information processing.

An important aspect of the work presented herein is its focus on information encoding at the level of individual neurons. Though single cells certainly have the capacity to encode and represent information (Nemenman et al., 2008), distributed networks of anatomically and functionally connected neurons (i.e., neural ensembles) also carry out this task (Nicolelis et al., 1995; Rothschild et al., 2010; Ince et al., 2013). The role of balanced synaptic inputs on information transfer in cortical networks has been addressed in previous studies. For instance, using multi-site recordings of local field potentials (LFP) in rats and monkeys, Shew et al. showed that cortical networks with balanced excitation and inhibition maximize information capacity and transfer (Shew et al., 2011). Our results are in agreement with these findings, showing optimal information rates of neural spike trains when synaptic currents are balanced. It is important to note, however, that LFP recordings of cortical networks capture coordinated activity of large ensembles of neurons operating at substantially slower time-scales than that of single neurons. Thus, the nature of the computations performed and information encoded at the level of single cells vs. that of neural ensembles is likely to have marginal correspondence.

In conclusion, we provide a biologically realistic model of neurons with stochastic ion channel biophysics and synaptic inputs and apply information theoretic approaches to show that information rates of neural spike trains is dependent on the temporal correlation of balanced synaptic currents. Our findings emphasize the importance of these correlations for information encoding and suggest that cortical neurons may optimize this process through precise tuning of synaptic kinetics and timing of excitatory and inhibitory inputs.

## **MATERIALS AND METHODS**

#### **SYNAPTIC INPUTS**

All simulations were carried out using the Matlab R2013b software package (Mathworks). We modeled the synaptic events as Poisson trains with a rate of λ = 5 ms<sup>−</sup>1, which was fixed across all simulations. The synaptic train was then convolved with an "alpha function" to yield the time-dependent conductance, g(*t*), of the following form,

$$g(t) = G\left(e^{-t/\mathfrak{r}\_r} - e^{-t/\mathfrak{r}}\right) \tag{1}$$

where the constant *G* is set at 300 pS for all simulations trials in the balanced conductances regime, *e* is the base of the natural logarithm, and τ*<sup>r</sup>* and τ are the rise and decay timeconstants, respectively. τ*<sup>r</sup>* is set at a fixed value of 0.2 ms while τ is varied across the range of 1 ≤ τ ≤ 10 ms. The excitatory and inhibitory synaptic conductances (*gexc* and *ginh*, respectively) were created as identical realizations of a Poisson process in the balanced conductances input regime. For the balanced currents input regime, the inhibitory conductance was multiplied by a factor of 8, yielding a maximal conductance amplitude of 2400 pS. To generate conductance traces in which inhibition lagged behind excitation, we offset the two waveforms by a lag time, δ, which for a given simulation was taken from a range of lags, (1 ≤ δ ≤ 10 ms).

#### **ANALYTICAL EXPRESSION FOR THE CROSS-CORRELOGRAM OF THE SYNAPTIC INPUTS**

To calculate an analytical expression for the cross-correlogram of the excitatory and inhibitory synaptic inputs, we first note that the inhibitory input is identical with the excitatory input but delayed with a lag, δ, so that the cross-correlogram *C*(δ) is actually equivalent to the auto-correlogram of the excitatory input. We also note that the excitatory input is the convolution of a Poisson process with the kinetics of a single synaptic event given by (1) and recall the following two theorems of time-series analysis: 1) The Wiener-Khinchin theorem, stating that for a given signal the auto-correlogram is the Fourier transform of its power spectrum; and 2) the convolution theorem, stating that the power spectrum of the convolution of two signals is the product of their power spectra. Therefore, since the power spectrum of a Poisson process is a constant, the cross-correlogram is determined by the Fourier transform of the power spectrum of a single synaptic event. Defining α = 1/τ*<sup>r</sup>* and β = 1/τ , the power spectrum of a single synaptic event is given by:

$$k \left| \int\_0^\infty (e^{-\alpha t} - e^{-\beta t}) \, e^{i\alpha dt} \right|^2 = k \frac{(\alpha - \beta)^2}{(\alpha^2 + \omega^2) \left(\beta^2 + \omega^2\right)}, \quad \text{(2)}$$

where *k* is a constant and |...| <sup>2</sup> denotes the square of the modulus of a complex number. The un-normalized cross-correlogram of the excitatory and inhibitory inputs is then given by the Fourier transform of (2)

$$C(\delta) = \frac{k}{2\pi} \int\_{-\infty}^{\infty} \frac{(\alpha - \beta)^2}{\left(\alpha^2 + \omega^2\right)\left(\beta^2 + \omega^2\right)} e^{i\alpha\delta} d\omega. \tag{3}$$

To solve the integral in (3) we apply the residues theorem to a closed integration path containing two poles on the upper-half of the complex plane, ω = *i*α, *i*β, whose respective residues are

$$i\frac{(\alpha-\beta)\,e^{-\alpha\delta}}{2\alpha\,(\alpha+\beta)}\quad\text{and}\quad-i\frac{(\alpha-\beta)\,e^{-\beta\delta}}{2\beta\,(\alpha+\beta)}.$$

Thus, expression (3) yields

$$C(\delta) = \frac{k \left(\alpha - \beta\right)}{2 \left(\alpha + \beta\right)} \left(\frac{e^{-\beta\delta}}{\beta} - \frac{e^{-\alpha\delta}}{\alpha}\right).$$

Finally, the cross-correlogram, normalized so that *C* (0) = 1 reads

$$C(\delta) = \frac{\mathfrak{r}\_r e^{-\delta/\mathfrak{r}\_r} - \mathfrak{r} e^{-\delta/\mathfrak{r}}}{\mathfrak{r}\_r - \mathfrak{r}}.$$

This analytical expression accurately describes the crosscorrelogram obtained from the numerical simulations, as shown in **Figure 1C**.

#### **SINGLE COMPARTMENT MODEL**

Neuronal dynamics were simulated in a single compartment model of a Hodgkin-Huxley (Hodgkin and Huxley, 1952) neuron with stochastic voltage-gated fast Na+, delayed rectifier voltagegated K<sup>+</sup> channels, and a deterministic leak conductance as detailed in (Schmandt and Galán, 2012). Modeling of stochastic ion channel gating was made more computationally efficient by applying the SSA of Markov chains, which reduces observable states (Schmandt and Galán, 2012). The fluctuations in the membrane voltage were described by the following current balance equation:

$$\mathbf{C}\_{m}\frac{dV}{dt} = \overbrace{\mathbf{g}\_{\text{Na}}(t)\left(\mathbf{E}\_{\text{Na}} - V(t)\right) + \mathbf{g}\_{\text{K}}(t)\left(\mathbf{E}\_{\text{K}} - V(t)\right) + \mathbf{g}\_{\text{calc}}(\mathbf{E}\_{\text{lack}} - V)}^{\text{membrane currents}}}^{\text{membrane currents}} + \dots$$

$$+ \underbrace{\mathbf{g}\_{\text{xxx}}(t)\left(\mathbf{E}\_{\text{xxx}} - V(t)\right) + \mathbf{g}\_{\text{imh}}(t)\left(\mathbf{E}\_{\text{inh}} - V(t)\right)}\_{\text{synantic currents}}$$

where *Cm* corresponds to the membrane capacitance, *gNa*, *gK*, and *gleak* are the Na+, K+, and leak conductances with their respective reversal potentials, *ENa*, *EK*, and *Eleak*. The excitatory and inhibitory synaptic conductances (*gexc* and *ginh*) along with their respective reversal potentials, *Eexc* (0 mV) and *Einh* (−80 mV), dictate the extent to which synaptic currents affect the membrane potential fluctuations. The membrane potential was simulated with a time resolution of *dt* = 0.01 ms.

#### **DETERMINATION OF INFORMATION RATES**

Spike train entropy was determined using the "direct method" (Nemenman et al., 2008). This approach quantifies the entropy of the spike trains without making assumptions about the nature of the stimulus. Spike trains were binned in small time windows (*t* = 5 ms) and spikes were counted for each bin. A value of 0 was assigned to each bin containing no spikes and a value of 1 for those containing one spike. With the maximal firing rate of the model peaking at 80 Hz (inter-spike interval = 12.5 ms), our choice of *t* ensures that at most one spike can occur within a given time bin, therefore, providing information rates for timing of action potentials with millisecond precision. The resultant binary strings of 0's and 1's were used to generate words of length *n* where *n* = 2, 4, 6, 8, 10, yielding words that spanned time windows of *T* = *nt*. Probability distributions were then generated to quantify the occurrence probability of a given word, *P(W)*, within a response pattern. Noise entropy, which measures the reproducibility of spike trains in response a fixed input stimulus across trials (56 trials), was measured as with respect to the conditional probability of a word occurring at time *t* and calculated with the following equation:

$$H\_{noise} = \left\langle -\sum\_{W}^{p} \left( W|t\rangle \log\_2 P\left(W|t\right) \right)\_{t} \right\rangle$$

where the operator ...*<sup>t</sup>* denotes averaging over time. The total entropy, which quantifies the possible permutations of output patterns with respect to a broad set of inputs, was determined by presenting the model neuron with a different input pattern across 56 trials and measuring the occurrence probability of a given word. The total entropy was calculated as:

$$H\_{\text{total}} = -\sum\_{W} P(W) \log\_2 P(W).$$

By definition, the information encoded by the spike train is the difference between the total and noise entropies. We thus computed the information as:

$$I = H\_{total} - H\_{noise}.\tag{4}$$

Both the noise and the total entropies were normalized by *T* to yield entropy rates (bits/s). Since entropy is sensitive to the word length, we extrapolated the entropy rates in the limit of *T* → ∞, yielding the true rates, and using (4) the IR.

## **AUTHOR CONTRIBUTIONS**

Pavel A. Puzerey and Roberto F. Galán conceived and designed the experiments. Pavel A. Puzerey performed the experiments. Pavel A. Puzerey and Roberto F. Galán carried out data analysis. Pavel A. Puzerey generated the figures. Pavel A. Puzerey and Roberto F. Galán wrote the manuscript.

## **ACKNOWLEDGMENT**

We would like to acknowledge the High Performance Computing Cluster at Case Western Reserve University, which we extensively used in our study to run simulations in parallel.

## **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 14 March 2014; accepted: 15 May 2014; published online: 06 June 2014. Citation: Puzerey PA and Galán RF (2014) On how correlations between excitatory and inhibitory synaptic inputs maximize the information rate of neuronal firing. Front. Comput. Neurosci. 8:59. doi: 10.3389/fncom.2014.00059*

*This article was submitted to the journal Frontiers in Computational Neuroscience. Copyright © 2014 Puzerey and Galán. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Volterra dendritic stimulus processors and biophysical spike generators with intrinsic noise sources

## *Aurel A. Lazar\*† and Yiyin Zhou†*

*Department of Electrical Engineering, Columbia University, New York, NY, USA*

#### *Edited by:*

*Mark D. McDonnell, University of South Australia, Australia*

*Reviewed by: Ron Meir, Technion, Israel Zoran Tiganj, Boston University, USA*

#### *\*Correspondence:*

*Aurel A. Lazar, Department of Electrical Engineering, Columbia University, 500 W. 120th Street, New York, NY 10027, USA e-mail: aurel@ee.columbia.edu*

*†The authors' names are listed in alphabetical order.*

We consider a class of neural circuit models with internal noise sources arising in sensory systems. The basic neuron model in these circuits consists of a dendritic stimulus processor (DSP) cascaded with a biophysical spike generator (BSG). The dendritic stimulus processor is modeled as a set of nonlinear operators that are assumed to have a Volterra series representation. Biophysical point neuron models, such as the Hodgkin-Huxley neuron, are used to model the spike generator. We address the question of how intrinsic noise sources affect the precision in encoding and decoding of sensory stimuli and the functional identification of its sensory circuits. We investigate two intrinsic noise sources arising (i) in the active dendritic trees underlying the DSPs, and (ii) in the ion channels of the BSGs. Noise in dendritic stimulus processing arises from a combined effect of variability in synaptic transmission and dendritic interactions. Channel noise arises in the BSGs due to the fluctuation of the number of the active ion channels. Using a stochastic differential equations formalism we show that encoding with a neuron model consisting of a nonlinear DSP cascaded with a BSG with intrinsic noise sources can be treated as generalized sampling with noisy measurements. For single-input multi-output neural circuit models with feedforward, feedback and cross-feedback DSPs cascaded with BSGs we theoretically analyze the effect of noise sources on stimulus decoding. Building on a key duality property, the effect of noise parameters on the precision of the functional identification of the complete neural circuit with DSP/BSG neuron models is given. We demonstrate through extensive simulations the effects of noise on encoding stimuli with circuits that include neuron models that are akin to those commonly seen in sensory systems, e.g., complex cells in V1.

**Keywords: Volterra dendritic stimulus processors, biophysical spike generators, noise, neural encoding, neural decoding, functional identification, Hodgkin-Huxley neuron, phase response curve**

## **1. INTRODUCTION**

Intrinsic noise sources are diverse and appear on many levels of a neuronal system ranging from electrical to chemical noise sources (Faisal et al., 2008; Destexhe and Rudolph-Lilith, 2012) and from single cells to networks of neurons. At the cellular and subcellular level, variability in biochemical reactions leads to stochastic transduction processes (Song et al., 2012), and ion channel fluctuations (Neher and Sakmann, 1976; White et al., 1998) result in variability in spike generation and propagation (Faisal and Laughlin, 2007). At the network level, probabilistic quantal release of neurotransmitters (Katz, 1962), background synaptic activity (Destexhe et al., 2003; Jocobson et al., 2005) and variability in timing of spikes from presynaptic neurons (Faisal and Neishabouri, 2014) are sources of stochastic fluctuation of synaptic conductances (Destexhe et al., 2001) that are believed to have a major impact on spike time variability (Yarom and Hounsgaard, 2011).

The existence of sources of noise also leads to variability in the spike times even when neurons are subject to the same, repeated inputs (Calvin and Stevens, 1968; Berry et al., 1997; de Ruyter van Steveninck et al., 1997). Spikes are the primary form of carriers of information in the nervous system and their timing is thought to be relevant to the message neurons need to convey (Rieke et al., 1999). Therefore, the variability of spike timing may reduce or damage the information being transmitted. It is quite remarkable, however, that sensory systems manage to be very robust even if they are subject to interference due to noise. Visual and auditory systems are two examples in which the stimuli are highly time varying. These systems have been reported to convey information with high spike timing precision (Butts et al., 2007; Kayser et al., 2010).

Noise may be useful in facilitating signal detection (McDonnell and Ward, 2011). Still, interference due to noise poses an important limit on how well sensory systems can represent input stimuli. It is not clear how intrinsic noise sources affect the representation of sensory inputs based on spike times, and how they impact the functional identification of sensory neurons.

We study the representation of sensory stimuli using a novel neural circuit model, that extends previously proposed models (Lazar et al., 2010; Lazar and Slutskiy, 2014, in press) in terms of architectural complexity and the existence of intrinsic noise sources. Our base level circuit architecture consists of two interconnected neurons, each with two cascaded stages. The first stage comprises two types of dendritic stimulus processors. The first dendritic stimulus processor performs *nonlinear* processing of input stimuli in the feedforward path leading to the spike generator. The second dendritic stimulus processor performs *nonlinear* processing in the feedback loop whose inputs are spike trains generated by biophysical spike generators (BSGs). The BSGs constitute the second stage of the base level circuit.

Our nonlinear dendritic stimulus processors describe functional I/O relationships between the dendritic outputs in the first stage and inputs that are either sensory stimuli or spikes generated by BSGs. DSPs are modeled using Volterra series. Volterra series have been used for analyzing nonlinear neuronal responses in many contexts (Lu et al., 2011; Eikenberry and Marmarelis, 2012), and have been applied to the identification of single neurons in many of sensory areas (Benardete and Kaplan, 1997; Theunissen et al., 2000; Clark et al., 2011). Volterra dendritic processors can model a wide range of nonlinear effects commonly seen in sensory systems (Lazar and Slutskiy, in press). Here, in addition, we introduce nonlinear interactions between neurons in the feedback and cross-feedback paths. This gives rise to interesting neural processing capabilities directly in the spike domain, e.g., coincidence detection (Agmon-Snir et al., 1998; Stuart and Häusser, 2001). The relationships described here by the Volterra model are functional and do not address the underlying circuit/dendritic tree level interactions. However, the latter have recently been subject to intense investigations (London and Häusser, 2005; Wohrer and Kornprobst, 2009; Werblin, 2011; Xu et al., 2012; Yonehara et al., 2013; Zhang et al., 2013). Conductance-based, biophysical spike generators are well established models that have been extensively used in studies of neuronal excitability and in large simulations of spiking neural networks (Izhikevich, 2007). Following Lazar (2010), we use formal BSG models to represent sensory stimuli under noisy conditions.

We formulate the encoding, decoding and functional identification problems under the neural encoding framework of Time Encoding Machines (TEMs). In this modeling framework the exact timing of spikes is considered to carry information about input stimuli (Lazar and Tóth, 2004). The separation into dendritic stimulus processors and spike mechanisms mentioned above allows us to study synaptic inputs and spike generation mechanisms separately, and hence independently model the intrinsic noise sources of each component. We incorporate two important noise sources into a general single-input multi-output neural circuit model. The first is a channel noise source that arises in spike generation (White et al., 2000). The second is a synaptic noise source due to a variety of fluctuating synaptic currents (Manwani and Koch, 1999).

Based on the rigorous formalism of TEMs, we show how noise arising in dendritic stimulus processors and in biophysical spike generators is related to the measurement error in generalized sampling. Dendritic stimulus processing and spike generation can then be viewed as a generalized sampling scheme that neurons utilize to represent sensory inputs (Lazar et al., 2010). Contrary to traditional sampling where the signal amplitude is sampled at clock times, neurons asynchronously sample all stimuli.

We systematically investigate how the strength of noise sources degrades the faithfulness of stimulus representation and the quality of functional identification of our proposed class of neural circuits. Furthermore, since the representation is based on spike timing, it is natural to investigate how spike timing variability affects the precision in representing the amplitude information of sensory stimuli.

The work presented here requires a substantial amount of investment in the mathematical formalism employed throughout. There are a number of benefits in doing so, however. Formulating the problem of stimulus encoding with a neural circuit with intrinsic noise sources as one of generalized sampling, i.e., of taking noisy measurements is of interest to both experimentalists and theoreticians alike. Understanding that the problem of neural decoding and functional identification are dual to each other is key to building on either or both. Finding how many repeat experiments need to be performed for a precise quantitative identification of Volterra kernels is of great value in neurophysiology. A further qualitative insight of our work is that for neural circuits with arbitrary connectivity, feedforward kernels are typically easier to estimate than feedback kernels. Finally, our finding that some key nonlinear neural circuits are tractable for detailed noise analysis suggests a wide reaching analytical methodology.

## **2. MODELING NONLINEAR NEURAL CIRCUITS, STIMULI, AND NOISE**

We present in Section 2.1 the general architecture of the neural circuits considered in this paper. In Section 2.2 we discuss the modeling of the space of stimuli. Volterra DSPs are the object of Section 2.3. Finally, in Section 2.4 we provide models of BSGs with intrinsic noise sources.

## **2.1. NEURAL CIRCUIT ARCHITECTURE**

The general architecture of the neural circuit considered here is shown in simplified form in **Figure 1**. It consists of two neurons with a common time-varying input stimulus. With added notational complexity the neural circuit in **Figure 1** can easily be extended in two ways. First, multiples of such circuits can encode a stimulus in parallel (see Section 2.1 in the Supplementary Material). In this case only pairs of neurons are interconnected through the feedback kernels. Second, more neurons can be considered in the neural circuit of **Figure 1**; all these neurons can be fully interconnected through feedback loops.

Each neuron *i*, *i* = 1, 2, receives a single time-varying input stimulus *u*1(*t*). The modeling of the input stimulus is discussed in Section 2.2. The output of each of the biophysical spike generators (BSGs) is a spike sequence denoted by (*t* 1 *<sup>k</sup>* ) and (*t* 2 *<sup>l</sup>* ), *<sup>k</sup>*, *<sup>l</sup>* <sup>∈</sup> <sup>Z</sup>.

The input stimulus *u*1(*t*) is first processed by a feedforward Dendritic Stimulus Processor (feedforward DSP) (Lazar and Slutskiy, in press). The feedforward DSP models the aggregated effect of processing in the neural circuits in the prior stages and in the dendritic tree of neuron *i* = 1, 2. For example, if the neurons in the model circuit are considered to be Retinal Ganglion Cells (RGCs), then the feedforward Volterra DSP models the processing that takes place in the outer- and inner-plexiform layers of the retina as well as in the dendritic trees of an RGC (Werblin, 2011; Masland, 2012). The feedforward

DSPs are modeled here as second order Volterra expansion terms (Volterra, 1930). The first order terms *h*11*<sup>i</sup>* <sup>1</sup> (*t*) in the feedforward DSPs are linear filters typically used in modeling receptive fields. The second order terms *h*11*<sup>i</sup>* <sup>2</sup> (*t*1, *t*2) model nonlinear operations on the stimulus *u*1(*t*).

A second group of Volterra DSPs models the cross-feedback interactions between the two neurons. Instead of time-varying stimuli, the output spikes generated by the BSGs are the inputs to these DSPs. We therefore refer to these as feedback Dendritic Stimulus Processors (feedback DSPs). The output spikes of each individual neuron *i* are processed by the first order term *h* 2*ji* <sup>1</sup> (*t*), *i*, *j* = 1, 2, *i* = *j*. In addition, output spikes from both neurons interact nonlinearly through the second order terms *h* 2*ji* <sup>2</sup> (*t*1, *t*2), *i*, *j* = 1, 2, *i* = *j*. The summed responses from the first order feedback DSP *h* 2*ji* <sup>1</sup> and the second order feedback DSP *h* 2*ji* <sup>2</sup> are fed back to neuron *i* as additional dendritic currents.

The dendritic currents consisting of the output of the DSPs with added noise are subsequently encoded by biophysical spike generators. BSGs are biophysically realistic axon hillock spike generator models that are governed by a set of differential equations with multiple types of ion channels (Hodgkin and Huxley, 1952; Izhikevich, 2007). The detailed BSG models are introduced in Section 2.4. The spike times of output spikes generated by the BSGs are assumed to be observable.

We identify two intrinsic noise sources of the proposed neural circuit. First, the feedforward DSPs and the feedback DSPs are affected by additive Gaussian white noise. This noise arises from the combined effect along the path from sensory transduction to synaptic integration and includes synaptic background noise and stochasticity in the dendritic tree (Manwani and Koch, 1999; Fellous et al., 2003; Destexhe and Rudolph-Lilith, 2012). Since the outputs of the feedforward and feedback DSPs are additively combined, we consider, for simplicity, a single source of additive Gaussian white noise. Second, the ion channels of the BSGs are intrinsically stochastic and introduce noise in the spike generators (White et al., 2000; Hille, 2001).

### **2.2. MODELING SIGNAL SPACES**

Two signal spaces will be considered here. The first, models the space of input signals to feedforward DSPs. The second models the space of input spikes to feedback DSPs. These spaces will be formally described below.

#### *2.2.1. Modeling the space of input stimuli*

We model the space of input stimuli as a Reproducing Kernel Hilbert Space (RKHS) (Berlinet and Thomas-Agnan, 2004). RKHSs are versatile vector spaces for modeling signals arising in computational neuroscience, signal processing and machine learning. For example, auditory signals, olfactory signals and visual signals can readily be modeled as band-limited functions of an RKHS with a sinc or Dirichlet kernel (Lazar et al., 2010; Lazar and Slutskiy, 2013). A particular choice of RKHSs in this article is the space of trigonometric polynomials. The computational advantage of working on the space of trignometric polynomials has been discussed (Lazar et al., 2010) and is closely related to the algorithmic tractability of the Fourier series in the digital domain. If the biological signals have unknown bandwidth with a spectrum that falls off fast enough, many Sobolev spaces might be a suitable choice of RKHS (Berlinet and Thomas-Agnan, 2004; Lazar and Pnevmatikakis, 2009). In such spaces the norm may include the derivative of the signal, i.e., the rate of change of the signal that many neurons are sensitive to Kim et al. (2011).

The space of trigonometric polynomials is defined as below.

**Definition 2.1.** *The space of trigonometric polynomials <sup>H</sup>*<sup>1</sup> <sup>1</sup> *is a function space whose elements are functions defined on the domain* D<sup>1</sup> = [0, *S*1], *S*<sup>1</sup> ∈ R+*, of the form*

$$\mu\_1(t) = \sum\_{l=-L^1}^{L^1} \mu\_l e\_l(t),\tag{1}$$

*where*

$$e\_l(t) = \frac{1}{\sqrt{S^1}} e^{jl\frac{\Omega^1}{L^1}t}, l = -L^1, \dots, L^1,\tag{2}$$

*are a set of orthonormal basis functions.* -<sup>1</sup> *denotes the bandwidth and L*<sup>1</sup> *is the order of the space.*

*H*1 <sup>1</sup> endowed with the inner product:

$$
\langle u\_1, \nu\_1 \rangle = \int\_{\mathbb{D}\_1} u\_1(t) \overline{\nu\_1(t)} dt \tag{3}
$$

is a Hilbert Space. Intuitively, the basis functions *el*(*t*), *l* = −*L*1,..., *L*1, can be interpreted as a set of discrete spectral lines uniformly spaced in the frequency domain between −-1 and -1. For a given signal *u*1(*t*), the amplitude of its spectral lines is determined by the coefficients *ul*, *l* = −*L*1,..., *L*1.

**Remark 2.2.** *Functions in <sup>H</sup>*<sup>1</sup> <sup>1</sup> *are periodic over* <sup>R</sup> *with period S*<sup>1</sup> = <sup>2</sup>π*L*<sup>1</sup> -<sup>1</sup> *. Therefore, the domain* D<sup>1</sup> *covers exactly one period of the function. Note that the ul's are closely related to the Fourier coefficients of the periodic signal u*1(*t*)*, and can thereby be very efficiently computed via the Fast Fourier Transform.*

*H*1 <sup>1</sup> is an RKHS with reproducing kernel (RK)

$$K\_1^1(t;s) = \sum\_{l=-L^1}^{L^1} e\_l(t-s). \tag{4}$$

It can be easily verified that the RK satisfies the reproducing property

$$
\langle \mu\_1(\cdot \cdot), K\_1^1(t; \cdot) \rangle = \mu\_1(t), \forall \mu\_1 \in \mathcal{H}\_1^1, t \in \mathbb{D}\_1. \tag{5}
$$

**Definition 2.3.** *We shall also consider the tensor product space <sup>H</sup>*<sup>1</sup> 2 *on the domain* D<sup>2</sup> = [0, *S*1]×[0, *S*1]*, whose elements are of the form*

$$u\_2(t\_1, t\_2) = \sum\_{l\_1 = -L^1}^{L^1} \sum\_{l\_2 = -L^1}^{L^1} u\_{l\_1 l\_2} e\_{l\_1 l\_2}(t\_1, t\_2), \tag{6}$$

*where*

$$e\_{l\_1 l\_2}(t\_1, t\_2) = \frac{1}{S^1} e^{j \int\_{\underline{l}}^{\underline{l}\_1} \frac{\Omega^1}{L^1} t\_1} e^{j \int\_{\underline{l}} \frac{\Omega^1}{L^1} t\_2},\tag{7}$$

*are a set of functions forming an orthonormal basis.*

*H*1 <sup>2</sup> is again an RKHS with RK

$$K\_2^1(t\_1, t\_2; s\_1, s\_2) = \sum\_{l\_1 = -L^1}^{L^1} \sum\_{l\_2 = -L^1}^{L^1} e\_{l\_1 l\_2}(t\_1 - s\_1, t\_2 - s\_2). \tag{8}$$

Note that we use the subscript to indicate the dimension of the domain of functions, i.e., the number of variables the functions in the RKHS have, and use the superscript 1 to indicate the input space.

Projections of functions onto the RKHSs introduced here can be defined as follows:

**Definition 2.4.** *Let h*<sup>1</sup> ∈ L1(D1)*, where* L<sup>1</sup> *denotes the space of Lebesgue integrable functions. The operator <sup>P</sup>*<sup>1</sup> : <sup>L</sup>1(D1) <sup>→</sup> *<sup>H</sup>*<sup>1</sup> 1 *given by*

$$(\mathcal{P}^1 h\_1)(t) = \int\_{\mathbb{D}\_1} h\_1(s) K\_1^1(t;s) ds,$$

*is called the projection operator from* <sup>L</sup>1(D1) *to <sup>H</sup>*<sup>1</sup> <sup>1</sup>*. Similarly, let <sup>h</sup>*2(*t*1, *<sup>t</sup>*2) <sup>∈</sup> <sup>L</sup>1(D2)*, the operator <sup>P</sup>*<sup>1</sup> : <sup>L</sup>1(D2) <sup>→</sup> *<sup>H</sup>*<sup>1</sup> <sup>2</sup> *(by abuse of notation) given by*

$$(\mathcal{P}^1 h\_2)(t\_1, t\_2) = \int\_{\mathbb{D}\_2} h\_2(s\_1, s\_2) K\_2^1(t\_1, t\_2; s\_1, s\_2) ds\_1 ds\_2,$$

*is called the projection operator from* <sup>L</sup><sup>1</sup> (D2) *to <sup>H</sup>*<sup>1</sup> 2*.*

### *2.2.2. Modeling the space of spikes*

The feedback kernels of the neural circuit in **Figure 1** receive as inputs spike trains generated by the BSGs. Spike trains are often modeled as sequences of Dirac delta pulses and, consequently, the outputs of linear feedback kernels are the result of superposition of their impulse responses (Keat et al., 2001; Pillow et al., 2008; Lazar et al., 2010).

Dirac delta pulses have infinite bandwidth. Spikes generated by the BSGs, however, have limited effective bandwidth. Following (Lazar and Slutskiy, 2014) spikes are modeled to be the RK of an one-dimensional Hilbert space *<sup>H</sup>*<sup>2</sup> <sup>1</sup> at spike time occurrence. Here *<sup>H</sup>*<sup>2</sup> <sup>1</sup> is a space of trigonometric polynomials whose order *<sup>L</sup>*2, period *S*<sup>2</sup> and bandwidth -<sup>2</sup> may differ from the input stimulus space *<sup>H</sup>*<sup>1</sup> <sup>1</sup>, where -<sup>2</sup> shall be larger than the bandwidth assumed for the feedback kernel, and *S*<sup>2</sup> is much larger than the support of the feedback kernel (Lazar and Slutskiy, 2014). A spike at time *t i k* of neuron *i* can then be expressed in functional form as *K*<sup>2</sup> 1 (*t i <sup>k</sup>*;*t*), where the superscript indicates that the RK belongs to the spike input space.

Due to the reproducing property, single or pairs of input spikes have the property

$$h\_1(t) \* K\_1^2 \left( t\_k^i; t \right) = \int\_{\mathbb{D}\_1} h\_1(t - s) K\_1^2 \left( t\_k^i; s \right) ds = \left( \mathcal{P}^2 h\_1 \right) \left( t - t\_k^i \right)$$

and

$$\int\_{\mathbb{D}\_2} h\_2\left(t - s\_1, t - s\_2\right) K\_2^2\left(t\_k^i, t\_l^j; s\_1, s\_2\right) ds\_1 ds\_2 = \left(\mathcal{P}^2 h\_2\right)\left(t - t\_k^i, t - t\_l^j\right)$$

for *<sup>i</sup>*, *<sup>j</sup>* <sup>=</sup> <sup>1</sup>, <sup>2</sup>, *<sup>i</sup>* = *<sup>j</sup>*. The operator *<sup>P</sup>*<sup>2</sup> is similarly defined to *<sup>P</sup>*<sup>1</sup> above; it denotes, however, the projection onto the space of spikes. Thus, not surprisingly, incoming spikes directly readout the projection of the feedback kernels. By letting *L*<sup>2</sup> → ∞, *<sup>P</sup>*2*h*<sup>1</sup> (*t* − *tk*) shall converge to *h*<sup>1</sup> (*t* − *tk*) in L<sup>2</sup> norm as the RK converges to the *sinc* function and the RKHS becomes the space of band-limited signals (Lazar et al., 2010). A more detailed analysis is available in Lazar and Slutskiy (2014). This formalism will be employed for solving the functional identification problem formulated in Section 4.1.

#### **2.3. VOLTERRA DENDRITIC STIMULUS PROCESSORS**

As mentioned in Section 2.1, two forms of dendritic stimulus processing appear in our model.

#### *2.3.1. Feedforward Volterra dendritic stimulus processors*

The feedforward DSPs are modeled as up to second order terms in the Volterra series. The feedforward DSPs take continuous signals in the stimulus space as inputs, while the output can be expressed as (see also **Figure 1**)

$$\int\_{\mathbb{D}\_1} h\_1^{11i}(t-s) u\_1(s) ds + \int\_{\mathbb{D}\_2} h\_2^{11i}(t-s\_1, t-s\_2) u\_1(s\_1) u\_1(s\_2) ds\_1 ds\_2,\tag{9}$$

where *h*11*<sup>i</sup>* <sup>1</sup> <sup>∈</sup> <sup>L</sup>1(D1) and *<sup>h</sup>*11*<sup>i</sup>* <sup>2</sup> <sup>∈</sup> <sup>L</sup>1(D2) denote, respectively, the first and second order Volterra kernels, *i* = 1, 2. They are assumed to be real, causal and bounded-input bounded-output (BIBO)-stable. It is also assumed that both *h*11*<sup>i</sup>* <sup>1</sup> and *<sup>h</sup>*11*<sup>i</sup>* <sup>2</sup> have finite memory. In addition, *h*11*<sup>i</sup>* <sup>2</sup> is assumed, without loss of generality, to be symmetric, i.e., *h*11*<sup>i</sup>* <sup>2</sup> (*t*1, *<sup>t</sup>*2) <sup>=</sup> *<sup>h</sup>*11*<sup>i</sup>* <sup>2</sup> (*t*2, *t*1).

**Example 2.5.** *We present here a Volterra DSP that is akin to a model of dendritic stimulus processing of complex cells in the primary visual cortex (V1). The difference is that the complex cells operate spatio-temporally, whereas in the example given below they operate temporally. We first consider two first order kernels based on Gabor functions,*

$$g\_{\ell}(t) = \exp\left(-\frac{(t - 0.13)^2}{2 \cdot 0.0005}\right) \cos\left(2\pi \cdot 10 \cdot (t - 0.13)\right),$$

$$g\_{\ell}(t) = \exp\left(-\frac{(t - 0.13)^2}{2 \cdot 0.0005}\right) \sin\left(2\pi \cdot 10 \cdot (t - 0.13)\right).$$

*The two filters are Gaussian modulated sinusoids, that are typically used to model receptive fields of simple cells in the primary visual cortex (V1) where the variables denote space instead of time (Lee,* *1996; Dayan and Abbott, 2001). In addition, the two filters are quadrature pair in phase. Both filters are illustrated in Figure 2A. The response of applying the input stimulus u*<sup>1</sup> *on the temporal filters with impulse response gc and gs is given by* <sup>D</sup><sup>1</sup> *gc*(*<sup>t</sup>* <sup>−</sup> *<sup>s</sup>*)*u*1(*s*)*ds and* <sup>D</sup><sup>1</sup> *gs*(*<sup>t</sup>* <sup>−</sup> *<sup>s</sup>*)*u*1(*s*)*ds, respectively.*

*The responses of the two linear filters of the complex cell model are squared and summed to produce the phase invariant measure v<sup>i</sup> (Carandini et al., 2005), where*

$$\begin{split} \boldsymbol{v}^{i}(t) &= \left[ \int\_{\mathbb{D}\_{1}} \boldsymbol{g}\_{\boldsymbol{\varepsilon}}(t-s) \boldsymbol{u}\_{1}(s) ds \right]^{2} + \left[ \int\_{\mathbb{D}\_{1}} \boldsymbol{g}\_{\boldsymbol{\varepsilon}}(t-s) \boldsymbol{u}\_{1}(s) ds \right]^{2} \\ &= \int\_{\mathbb{D}\_{2}} \boldsymbol{g}\_{\boldsymbol{\varepsilon}}(t-s\_{1}) \boldsymbol{h}\_{1}(t-s\_{2}) \boldsymbol{u}\_{1}(s\_{1}) \boldsymbol{u}\_{1}(s\_{2}) ds\_{1} ds\_{2} \\ &+ \int\_{\mathbb{D}\_{2}} \boldsymbol{g}\_{\boldsymbol{\varepsilon}}(t-s\_{1}) \boldsymbol{g}\_{\boldsymbol{\varepsilon}}(t-s\_{2}) \boldsymbol{u}\_{1}(s\_{1}) \boldsymbol{u}\_{1}(s\_{2}) ds\_{1} ds\_{2} \\ &= \int\_{\mathbb{D}\_{2}} \left[ \boldsymbol{g}\_{\boldsymbol{\varepsilon}}(t-s\_{1}) \boldsymbol{g}\_{\boldsymbol{\varepsilon}}(t-s\_{2}) + \boldsymbol{g}\_{\boldsymbol{\varepsilon}}(t-s\_{1}) \boldsymbol{g}\_{\boldsymbol{\varepsilon}}(t-s\_{2}) \right] \\ & \qquad \quad \boldsymbol{u}\_{1}(s\_{1}) \boldsymbol{u}\_{1}(s\_{2}) ds\_{1} ds\_{2} \\ &= \int\_{\mathbb{D}\_{2}} \boldsymbol{h}\_{2}^{11i}(t-s\_{1}, t-s\_{2}) \boldsymbol{u}\_{1}(s\_{1}) \boldsymbol{u}\_{1}(s\_{2}) ds\_{1} ds\_{2}, \end{split} \tag{10}$$

*where h*11*<sup>i</sup>* <sup>2</sup> (*t*1, *t*2) = *gc*(*t*1)*gc*(*t*2) + *gs*(*t*1)*gs*(*t*2)*. Therefore, the operation performed by a complex cell can be modeled with a second order Volterra kernel. h*11*<sup>i</sup>* <sup>2</sup> *is shown in Figure 2B.*

*We now take a closer look at the operation of the second order kernel. The two dimensional convolution of the second order kernel with u*2(*t*1, *t*2) *is shown in Figure 2C.*

*It is important to note that, since the second order kernel has finite memory, it may not have enough support to cover the entire domain* D<sup>2</sup> *for u*2(*t*1, *t*2)*. For example, as illustrated in Figure 2C, the output of the second order feedforward DSP at time t is given by the integral of the product of u*2(*t*1, *t*2) *and a rotated h*11*<sup>i</sup>* <sup>2</sup> *with the origin shifted to* (*t*,*t*) *[see also* (10)*]. Since the shift is along the diagonal, only u*2(*t*1, *t*2) *in the domain that is contained within the black lines is multipled by nonzero values of h*11*<sup>i</sup>* <sup>2</sup> *. u*2(*t*1, *t*2)*elsewhere in the domain is always multiplied by zero in evaluating the output. Therefore, the output of the second order filter only contains information about u*<sup>2</sup> *within the domain located in between the black lines in Figure 2C. This has implications on decoding the signal (see also Remark 3.11 in Section 3.2)*

#### *2.3.2. Feedback Volterra dendritic stimulus processors*

As already mentioned, the feedback DSPs do not operate on stimuli directly but rather on spikes generated by BSGs. We assume that *h* 2*ji* <sup>1</sup> <sup>∈</sup> <sup>L</sup>1(D1), *<sup>h</sup>* 2*ji* <sup>2</sup> <sup>∈</sup> <sup>L</sup>1(D2), *<sup>i</sup>* = *<sup>j</sup>*, are real, causal, BIBOstable and have finite memory. In addition, we assume that these kernels are effectively band-limited (see also Section 2.2.2). In functional form we denote a train of spikes as *<sup>k</sup> K*<sup>2</sup> 1 (*t i <sup>k</sup>*;*t*). The output of the feedback DSP *i* amounts to

$$\sum\_{l \in \mathbb{Z}} \left( \mathcal{P}^2 h\_1^{2jl} \right) \left( t - t\_l^j \right) + \sum\_{k \in \mathbb{Z}} \sum\_{l \in \mathbb{Z}} \left( \mathcal{P}^2 h\_2^{2jl} \right) \left( t - t\_l^j, t - t\_k^i \right) \tag{11}$$

with *j* = *i*.

**FIGURE 2 | Examples of Volterra kernels. (A)** First order kernels of quadrature pair of Gabor functions modeling the receptive fields of simple cells. **(B)** Second order kernel modeling receptive fields of complex cells. **(C)** The mechanics of the two dimensional convolution operation between the *u*<sup>2</sup> (*S*<sup>1</sup> <sup>=</sup> <sup>0</sup>.8, <sup>D</sup><sup>2</sup> = [0, <sup>0</sup>.8]×[0, <sup>0</sup>.8]) and *<sup>h</sup>*11*<sup>i</sup>* <sup>2</sup> . *<sup>u</sup>*2(*t*1, *<sup>t</sup>*2) <sup>=</sup> *<sup>u</sup>*1(*t*1)*u*1(*t*2) is shown in the background. The inset shows the second order Volterra kernel *<sup>h</sup>*11*<sup>i</sup>* <sup>2</sup>

In particular, the inputs to the second order term of the feedback DSPs are generated by two neurons. This allows for modeling nonlinear interactions between the two neurons in the spike domain.

#### *2.3.3. Overall output from DSPs*

The overall inputs (without noise) to the two BSGs in **Figure 1** are

$$\begin{split} \nu^{i}(t) &= \int\_{\mathbb{D}} h\_{1}^{11i}(t-s) u\_{1}(s) ds + \int\_{\mathbb{D}^{2}} h\_{2}^{11i} \left(t - s\_{1}, t - s\_{2}\right) u\_{1}(s\_{1}) \\ &\quad u\_{1}(s\_{2}) ds\_{1} ds\_{2} + \sum\_{l \in \mathbb{Z}} \left(\mathcal{P}^{2} h\_{1}^{2\bar{j}i}\right) \left(t - t\_{l}^{j}\right) + \sum\_{l \in \mathbb{Z}} \sum\_{k \in \mathbb{Z}} \left(\mathcal{P}^{2} h\_{2}^{2\bar{j}i}\right), \\ &\quad \left(t - t\_{l}^{j}, t - t\_{k}^{i}\right), \quad \text{and} \quad i, j = 1, 2, \ i \neq j. \end{split} \tag{12}$$

The system of Equations (12) above functionally describe the post-synaptic aggregate currents that are injected into the BSG *i*.

There are a variety of noise sources to be considered. Synaptic variability of feedforward DSPs adds noise sources to the current input to the BSGs. These include thermal noise, synaptic background noise, etc. (Jonston, 1927; Calvin and Stevens, 1968; Manwani and Koch, 1999; Fellous et al., 2003; Destexhe and Rudolph-Lilith, 2012). Feedback DSP kernels may themselves be subject to intrinsic noise sources that may lead to variability in the spike generation process. Intrinsic variability of BSG spike times can, e.g., contribute to the variability of the aggregate current driving the axon hillock in feedback loops.

Overall, the combined effect of DSP noise sources is modeled as Gaussian white noise processes that are added to the feedforward and feedback DSP outputs. The sum total of signal and noise represents the aggregate current input to the BSGs (see **Figure 1**). Formal DSP noise models will be incorporated directly into the BSG model presented in the next section.

#### rotated 180◦ around origin [see also **(B)**]. (*h*11*<sup>i</sup>* <sup>2</sup> is only shown in a restricted domain and is zero elsewhere). For *t* = 0.3, the output of the convolution is the integral of the product of the rotated Volterra kernel and the signal underneath. Since the convolution is evaluated on the diagonal *t* = *t*<sup>1</sup> = *t*2, the second order kernel shifts, as *t* increases, along the arrow on the diagonal. See also Supplementary Figure 5E.

## **2.4. BIOPHYSICAL SPIKE GENERATORS**

#### *2.4.1. BSGs and phase response curves*

We consider biophysically realistic spike generators such as the Hodgkin-Huxley, Morris-Lecar, Connor-Stevens neurons (Hodgkin and Huxley, 1952; Connor and Stevens, 1971; Morris and Lecar, 1981). The class of BSGs can be expressed in vector notation as

$$\frac{d\mathbf{x}^i}{dt} = \mathbf{f}^i\left(\mathbf{x}^i, I^i\right), i = 1, 2,\tag{13}$$

where **x***<sup>i</sup>* are the state variables, **f** *<sup>i</sup>* are vector functions of the same dimension, and *I<sup>i</sup>* are the constant bias currents in the voltage equation of each BSG.

Each input current *v<sup>i</sup>* (*t*) is applied to the neuron *i* by additive coupling to the voltage equation, typically the first of the set of ordinary differential equations, i.e.,

$$\frac{d\mathbf{x}^i}{dt} = \mathbf{f}^i\left(\mathbf{x}^i, I^i\right) + \left[\boldsymbol{\nu}^i(t), \mathbf{0}\right]^T, i = 1, 2,\tag{14}$$

where **0** is a row vector of appropriate size.

We assume that the neuron is periodically spiking when no external input is applied. This can be satisfied by a constant bias current *I<sup>i</sup>* additively coupled onto the voltage equation. The use of *I<sup>i</sup>* is necessary to formulate the encoding for the single neuron case, and this assumption will be relaxed later in this article.

A large enough bias current induces a periodic oscillation of the biophysical spike generator. Therefore, the phase response curve (PRC) is well defined for this limit cycle (Izhikevich, 2007). We denote the PRC of the limit cycle induced by the bias current *I<sup>i</sup>* as *ψ<sup>i</sup> t*, *I<sup>i</sup>* = ψ*i* 1 *t*, *I<sup>i</sup>* , ψ*<sup>i</sup>* 2 *t*, *I<sup>i</sup>* , ··· , ψ*<sup>i</sup> Ni t*, *I<sup>i</sup> T* with appropriate dimension *N<sup>i</sup>* , where ψ*<sup>i</sup> n t*, *I<sup>i</sup>* , *n* = 1, 2, ··· , *N<sup>i</sup>* , are the PRCs associated with the *n*th state variable. Without loss of generality, we assume that ψ*<sup>i</sup>* 1 *t*, *I<sup>i</sup>* is always the PRC associated with the voltage variable.

An example of a Hodgkin-Huxley neuron model of a BSG can be found in Section 2.2 in the Supplementary Material.

#### *2.4.2. Channel noise in BSGs*

As shown in **Figure 1**, we consider BSGs with noise sources in the ion channels. The noise arises due to thermal fluctuations (White et al., 2000; Hille, 2001) as the finite number of ion channels in the BSGs open and close stochastically.

The differential equations that govern the dynamics of the BSGs in (14) are deterministic. The set of stochastic differential equations (SDEs) below represent their stochastic counterpart (Lazar, 2010):

$$d\mathbf{Y}^{i} = \mathbf{f}^{i}\left(\mathbf{Y}^{i}, I^{i}\right)dt + \mathbf{B}^{i}\left(\mathbf{Y}^{i}\right)d\mathbf{Z}^{i}(t), i = 1, 2,\tag{15}$$

where **B***<sup>i</sup>* is a matrix with state dependent values, *d***Z***<sup>i</sup>* = *vi dt*, *dW<sup>i</sup>* <sup>2</sup>, *dW<sup>i</sup>* <sup>3</sup>, ··· , *dW<sup>i</sup> Pi T* , and *W<sup>i</sup> <sup>p</sup>*(*t*), *p* = 2, ··· , *P<sup>i</sup>* , are independent Brownian motion processes. Note that *P<sup>i</sup>* does not necessarily have to be equal to *N<sup>i</sup>* , the number of state variables. The first element in the stochastic differential *d***Z***<sup>i</sup>* is the aggregate dendritic input *v<sup>i</sup> dt* driving the voltage equation. The other entries in *d***Z***<sup>i</sup>* are noise terms that reflect the stochastic fluctuation in the ion channels / gating variables.

Randomness is often added to BSGs by setting **B***<sup>i</sup>* = **I**, where **I** is a *N<sup>i</sup>* × *N<sup>i</sup>* identity matrix. The later setting can be viewed as adding subunit noise (Goldwyn and Shea-Brown, 2011). Recently, it has been suggested that a different way of adding channel noise into the BSGs may result in more accurate stochastic behavior (Goldwyn and Shea-Brown, 2011; Goldwyn et al., 2011; Linaro et al., 2011; Orio and Soudry, 2012). The SDEs in (15) are of general form and do not preclude them. In fact, by setting **B***<sup>i</sup>* to be a block matrix with blocks equal to be the square root of the diffusion matrix for each ion channel, the channel SDE model (Goldwyn et al., 2011; Orio and Soudry, 2012) can easily be incorporated into (15).

Finally, we note that, under appropriate technical conditions the SDE formulation applies to BSGs with voltage-gated ion channels as well as other types of ion channels. The conditions require that the BSG model can be treated mathematically as a system of SDEs of the form (15) and that the latter satisfies the assumptions of Section 2.4.1.

#### *2.4.3. Overall encoding of the neural circuit model*

Taking into account the dendritic input from the feedforward DSPs and feedback DSPs, the encoding by the neural circuit model under the two noise sources is given by two systems of SDEs. With the Brownian motion *W<sup>i</sup>* <sup>1</sup> modeling the DSP white noise, the encoding of neuron *i*, *i* = 1, 2, can be expressed as

$$d\mathbf{Y}^{i} = \mathbf{f}^{i}\left(\mathbf{Y}^{i}, I^{i}\right)dt + \mathbf{B}^{i}\left(\mathbf{Y}^{i}\right)d\mathbf{Z}^{i}(t),\tag{16}$$

where

$$d\mathbf{Z}^i = \begin{bmatrix} \nu^i dt + dW\_1^i \\ dW\_2^i \\ \vdots \\ dW\_{pi}^i \end{bmatrix},$$

with *v<sup>i</sup>* (*t*) given by Equation (12).

Note that in the system of Equations (16) the two output spikes trains *t i k* , *i* = 1, 2, *k* ∈ Z, are the observables. Due to the intrinsic noise sources in the DSPs and in the BSGs, spike timing jitter may be observed from trial to trial by repeatedly applying the same stimulus to the neural circuit (see Section 2.3 in the Supplementary Material).

### **3. ENCODING, DECODING, AND NOISE**

In Section 3.1 we present the mathematical encoding formalism underlying the neural circuit in **Figure 1**. We formulate stimulus decoding as a smoothing spline optimization problem and derive an algorithm that reconstructs the encoded signal in Section 3.2. Finally, we analyze the effect of noise on stimulus decoding in Section 3.3.

#### **3.1. ENCODING**

In this section, we formulate a rigorous stimulus encoding model based on the neural circuit shown in **Figure 1**. The input of the circuit is a signal *u*<sup>1</sup> modeling a typical sensory stimulus as described in Section 2.2.1. The neural circuit generates a multidimensional spike train that is assumed to be observable. We establish model equations by first describing the I/O relationship (i.e., the t-transform) of a single BSG. We then provide the *t*-transform of the entire neural circuit model that maps the input stimulus amplitude into a multidimensional spike timing sequence.

### *3.1.1. The I/O of the BSG*

In the presence of a bias current *I<sup>i</sup>* and absence of external inputs, each BSG in **Figure 1** is assumed to be periodically spiking. Provided that the inputs are small enough, and by using the PRC, the BSG dynamics of spike generation can be described in an one-dimensional phase space (Lazar, 2010).

**Definition 3.1.** *A neuron whose spike times* (*t i <sup>k</sup>*), *<sup>k</sup>* <sup>∈</sup> <sup>Z</sup>*, i* <sup>=</sup> <sup>1</sup>, <sup>2</sup>*, verify the system of equations*

$$\int\_{t\_k^i}^{t\_{k+1}^i} \left[ \Psi^i \left( s - t\_k^i + \tau^i \left( s - t\_k^i, I^i \right), I^i \right) \right]^T$$

$$\mathbf{B}^i \left( \mathbf{x}^i \left( s - t\_k^i + \tau^i \left( s - t\_k^i, I^i \right), I^i \right) \right) d\mathbf{Z}^i(s)$$

$$= T^i \left( I^i \right) - \left( t\_{k+1}^i - t\_k^i \right), \tag{17}$$

*where*

$$d\pi^i \left( t - t\_k^i, I^i \right) = \left[ \Psi^i \left( t - t\_k^i + \pi^i \left( t - t\_k^i, I^i \right), I^i \right) \right]^T \qquad \text{(18)}$$

$$\mathbf{B}^i \left( \mathbf{x}^i \left( t - t\_k^i + \pi^i \left( t - t\_k^i, I^i \right), I^i \right) \right) d\mathbf{Z}^i(t),$$

*with* τ *<sup>i</sup>* 0, *I<sup>i</sup>* = 0 *and* **x***<sup>i</sup> t*, *I<sup>i</sup> the periodic solution to* (13) *with bias current I<sup>i</sup> , is called a Project-Integrate-and-Fire (PIF) neuron with random thresholds. In* (17)*,* [·]*<sup>T</sup> denotes transpose and T<sup>i</sup>* (*Ii* ) *is the period of limit cycle with bias current I<sup>i</sup> .*

As its name suggests, the PIF projects a weighted version of the input embedded in noise and the ion channel noise associated with the gating variables (**B***<sup>i</sup> d***Z***<sup>i</sup>* ) onto the PRCs of the corresponding gating variables on a time interval between two consecutive spikes. Note that the integrand in (17) is identical to the RHS of (19). τ *<sup>i</sup>* (*t*,*I<sup>i</sup>* ) on the LHS of (19) denotes the phase deviation and is driven by the perturbation on the RHS. The LHS of (17) represents the phase deviation measurement performed by the PIF neuron. The RHS of (17) provides the value of the measurement and is equal to the difference between the inter-spike interval and the period of the limit cycle.

The BSG and the PIF neuron with random thresholds are, to the first order, I/O equivalent (Lazar, 2010). In Lazar (2010) it was also shown that a good approximation to the PIF neuron is the reduced PIF with random threshold. The functional description of the reduced PIF is obtained by setting the phase deviation in (17) to zero.

**Definition 3.2.** *The reduced PIF neuron with random threshold is given by the equations*

$$\sum\_{n=1}^{N} \int\_{t\_k^i}^{t\_{k+1}^i} \psi\_n^i \left(s - t\_k^i, I^i\right) b\_{n1}^i \left(\mathbf{x}^i \left(s - t\_k^i, I^i\right)\right) \nu^i(s) ds$$

$$= T^i(I^i) - \left(t\_{k+1}^i - t\_k^i\right) + \varepsilon\_k^i,\tag{19}$$

*where* (ε*<sup>i</sup> <sup>k</sup>*), *<sup>k</sup>* <sup>∈</sup> <sup>Z</sup>*, is a sequence of independent Gaussian random variables with zero mean and variance*

$$\begin{aligned} & \left( \mathbb{E} \left[ \varepsilon\_k^i \right]^2 \right) \left( I^i \right) \\ &= \sum\_{p=1}^{pi} \int\_{t\_k^i}^{t\_{k+1}^i} \left[ \sum\_{n=1}^{N^i} \psi\_n^i \left( s - t\_k^i, I^i \right) b\_{np}^i \left( \mathbf{x}^i \left( s - t\_k^i, I^i \right) \right) \right]^2 ds. \end{aligned} (20)$$

For reasons of notational simplicity and without loss of generality, and unless otherwise stated, we shall assume here that **B** = **I** (*N<sup>i</sup>* = *P<sup>i</sup>* ). The reduced PIF (rPIF) with random threshold can now be written as

$$\int\_{t\_k^i}^{t\_{k+1}^i} \psi\_1^i \left(s - t\_k^i, I^i\right) \nu^i(s) ds = T^i \left(I^i\right) - \left(t\_{k+1}^i - t\_k^i\right) + \varepsilon\_k^i,\tag{21}$$

where (ε*<sup>i</sup> <sup>k</sup>*), *<sup>k</sup>* <sup>∈</sup> <sup>Z</sup>, *<sup>i</sup>* <sup>=</sup> <sup>1</sup>, 2, is a sequence of independent Gaussian random variables with zero mean and variance

$$\mathbb{E}\left(\mathbb{E}\left[\boldsymbol{\varepsilon}\_{k}^{i}\right]^{2}\right)(I^{i}) = \sum\_{n=1}^{N^{i}} \int\_{t\_{k}^{i}}^{t\_{k+1}^{i}} \left[\psi\_{n}^{i}(s - t\_{k}^{i}, I^{i})\right]^{2} ds.\tag{22}$$

The above analysis assumes that the inputs are weak and therefore the BSGs operate on a limit cycle. Stronger signals can be taken into account by considering a manifold of PRCs associated with a wide range of limit cycles (Kim and Lazar, 2012). By estimating the limit cycle and hence its PRC using spike times, we have the following I/O relationship for each of the BSGs.

**Definition 3.3.** *The reduced PIF neuron with conditional PRC and random threshold is given by the system of equations*

$$\int\_{t\_k^i}^{t\_{k+1}^i} \psi\_1^i \left(s - t\_k^i, b\_k^i\right) \left(\nu^i(s) - b\_k^i + I\_0^i\right) ds = \varepsilon\_k^i,\qquad(23)$$

*where bi <sup>k</sup>* = [*T<sup>i</sup>* ] <sup>−</sup><sup>1</sup> *t i <sup>k</sup>* <sup>+</sup> <sup>1</sup> − *t i k* , *k* ∈ Z*, is the total estimated bias current on the inter-spike interval* [*t i k*, *t i <sup>k</sup>* <sup>+</sup> <sup>1</sup>]*, I<sup>i</sup>* <sup>0</sup> *is an initial bias that brings the neuron close to the spiking region in the absence of input and (by abuse of notation)* ε*<sup>i</sup> <sup>k</sup>*, *<sup>k</sup>* <sup>∈</sup> <sup>Z</sup>, *<sup>i</sup>* <sup>=</sup> <sup>1</sup>, <sup>2</sup>*, is a sequence of independent Gaussian random variables with zero mean and variance*

$$\mathbb{E}\left(\mathbb{E}\left[\varepsilon\_k^i\right]^2\right)\left(b\_k^i\right) = \sum\_{n=1}^{N^i} \int\_{t\_k^i}^{t\_{k+1}^i} \left[\psi\_n^i\left(s - t\_k^i, b\_k^i\right)\right]^2 ds,\quad(24)$$

*and* ψ*<sup>i</sup>* <sup>1</sup>(*s*, *bi <sup>k</sup>*) *is the conditional PRC (Kim and Lazar, 2012).*

The conditional PRC formulation above allows us to separate BSG inputs into a constant bias current and fluctuations around it on short inter-spike time intervals. The bias current can be estimated between consecutive spikes, making the deviation from the limit cycle small in each inter-spike interval even for strong inputs. Moreover, by considering the conditional PRCs, the assumption that BSGs oscillate in the absence of input can be nearly dropped. Thus, it is not required for BSGs to always be on a limit cycle. Only when the neuron enters the limit cycle do we consider formulating the encoding using the rPIF model with conditional PRCs.

**Remark 3.4.** *Note that by parametrizing each of the PRCs with b<sup>i</sup> k, the variance of the error in* (24) *depends on the estimated PRC on each inter-spike interval. In conjunction with* (23)*, we see that the variability of spike times depends on the strength of the input to the BSGs.*

#### *3.1.2. The t-transform of the neural circuit*

The overall encoding by the neural circuit model can be expressed as

$$\begin{aligned} &\int\_{t\_k^i}^{t\_{k+1}^i} \psi\_1^i \left(s - t\_k^i, b\_k^i\right) \nu^i(s) ds \\ &= \left(b\_k^i - I^i\right) \int\_{t\_k^i}^{t\_{k+1}^i} \psi\_1^i \left(s - t\_k^i, b\_k^i\right) ds + \varepsilon\_k^i, i = 1, 2, k \in \mathbb{Z}. \end{aligned}$$

Substituting (12) into the above, we have

$$\begin{aligned} &\int\_{t\_k^i}^{t\_{k+1}^i} \psi\_1^i \left(s - t\_k^i, b\_k^i\right) \int\_{\mathbb{D}\_1} h\_1^{11i}(s - r) u\_1(r) dr ds \\ &+ \int\_{t\_k^i}^{t\_{k+1}^i} \psi\_1^i \left(s - t\_k^i, b\_k^i\right) \int\_{\mathbb{D}\_2} h\_2^{11i}(s - r\_1, s - r\_2) u\_1(r\_1) ds \\ &\int\_{\mathbb{D}} u\_1(r\_2) dr\_1 dr\_2 ds \end{aligned}$$

= *bi <sup>k</sup>* − *I i <sup>t</sup> i k* + 1 *t i k* ψ*i* 1 *s* − *t i <sup>k</sup>*, *<sup>b</sup><sup>i</sup> k ds* −- *l* ∈ Z *t i k* + 1 *t i k* ψ*i* 1 *s* − *t i <sup>k</sup>*, *<sup>b</sup><sup>i</sup> k P*2*<sup>h</sup>* 2*ji* 1 (*s* − *t j l* )*ds* (25) *i*

$$\begin{split} & -\sum\_{l \in \mathbb{Z}} \sum\_{m \in \mathbb{Z}} \int\_{t\_k^i}^{t\_{k+1}^i} \psi\_1^i \left(s - t\_k^i, b\_k^i\right) \left(\mathcal{P}^2 h\_2^{2\bar{j}i}\right) \left(s - t\_l^j, s - t\_m^i\right) ds \\ & + s\_k^i, i, j = 1, 2, i \neq j. \end{split}$$

We arrived at the following.

**Lemma 3.5.** *The model of encoding in Figure 1 is given in operator form by*

$$T\_{1k}^i u\_1 + T\_{2k}^i u\_2 = q\_k^i + \epsilon\_k^i, i = 1, 2, k \in \mathbb{Z},\tag{26}$$

*where u*<sup>1</sup> <sup>∈</sup> *<sup>H</sup>*<sup>1</sup> <sup>1</sup>*, u*<sup>2</sup> <sup>∈</sup> *<sup>H</sup>*<sup>1</sup> <sup>2</sup>, *<sup>u</sup>*2(*t*1, *<sup>t</sup>*2) <sup>=</sup> *<sup>u</sup>*1(*t*1)*u*1(*t*2)*, and, <sup>T</sup> <sup>i</sup>* 1*k* : *H*1 <sup>1</sup> <sup>→</sup> <sup>R</sup> *and <sup>T</sup> <sup>i</sup>* <sup>2</sup>*<sup>k</sup>* : *<sup>H</sup>*<sup>1</sup> <sup>2</sup> <sup>→</sup> <sup>R</sup> *are bounded linear functionals given by*

$$\begin{split} T^{i}\_{1k}u\_{1} &= \int\_{t^{i}\_{k}}^{t^{i}\_{k+1}} \varphi^{i}\_{k}(s) \int\_{\mathbb{D}\_{1}} h^{11i}\_{1}(s-r)u\_{1}(r) dr ds, \\ T^{j}\_{2k}u\_{2} &= \int\_{t^{i}\_{k}}^{t^{j}\_{k+1}} \varphi^{i}\_{k}(s) \int\_{\mathbb{D}\_{2}} h^{11i}\_{2}(s-r\_{1}, s-r\_{2})u\_{2}(r\_{1}, r\_{2}) dr\_{1} dr\_{2} ds, \\ q^{i}\_{k} &= (b^{i}\_{k} - I^{i}) \int\_{t^{i}\_{k}}^{t^{i}\_{k+1}} \varphi^{i}\_{k}(s) ds - \sum\_{l \in \mathcal{Z}} \int\_{t^{i}\_{k}}^{t^{i}\_{k+1}} \varphi^{i}\_{k}(s) \left(\mathcal{P}^{2}h^{2\bar{j}}\_{1}\right)(s - t^{j}\_{l}) ds, \\ & - \sum\_{l \in \mathcal{Z}} \sum\_{m \in \mathcal{Z}} \int\_{t^{i}\_{k}}^{t^{i}\_{k+1}} \varphi^{i}\_{k}(s) \left(\mathcal{P}^{2}h^{2\bar{j}}\_{2}\right)\left(s - t^{j}\_{l}, s - t^{i}\_{m}\right) ds, \\ \varphi^{i}\_{k}(t) &= \frac{\psi^{i}\_{1}(t - t^{i}\_{k}, b^{i}\_{k})}{\left(\mathbb{E}\left[\varepsilon^{j}\_{k}\right]^{2}\right)^{\frac{1}{2}}} \end{split}$$

*and <sup>i</sup> <sup>k</sup>*, *<sup>k</sup>* <sup>∈</sup> <sup>Z</sup>*, are independent random variables with normal distribution N* (0, 1) *and j* = 1, 2, *j* = *i. Equation* (26) *is called the t-transform (Lazar and Tóth, 2004) of the neural circuit in Figure 1.*

**Remark 3.6.** *The t-transform describes the mapping of the input stimulus u*<sup>1</sup> *into the spike timing sequence* (*t i <sup>k</sup>*), *<sup>i</sup>* <sup>=</sup> <sup>1</sup>, <sup>2</sup>, *<sup>k</sup>* <sup>∈</sup> <sup>Z</sup>*. Thus, the t-transform shows how the amplitude information of the input signal is related to or transformed into the time information contained in the sequence of output spikes generated by the neural circuit.*

We provide here further intuition behind the Equations (26). By the Riesz representation theorem (Berlinet and Thomas-Agnan, 2004), there exists functions φ*<sup>i</sup>* <sup>1</sup>*<sup>k</sup>* <sup>∈</sup> *<sup>H</sup>*<sup>1</sup> <sup>1</sup> such that

$$T^i\_{1k}u\_1 = \langle u\_1, \phi^i\_{1k} \rangle\_{\mathcal{H}^1\_1}, \text{ for all } u\_1 \in \mathcal{H}^1\_1, \mathcal{H}$$

and φ*<sup>i</sup>* <sup>2</sup>*<sup>k</sup>* <sup>∈</sup> *<sup>H</sup>*<sup>1</sup> <sup>2</sup> such that

$$T\_{2k}^i \mu\_2 = \langle \mu\_2, \phi\_{2k}^i \rangle\_{\mathcal{H}\_2^1}, \text{ for all } \mu\_2 \in \mathcal{H}\_2^1.$$

Therefore, (26) can be rewritten in inner product form:

$$
\langle \langle u\_1, \phi^i\_{1k} \rangle\_{\mathcal{H}^1\_1} + \langle u\_2, \phi^i\_{2k} \rangle\_{\mathcal{H}^1\_2} = q^i\_k + \varepsilon^i\_k. \tag{27}
$$

Recall that inner products are projections that are typically interpreted as measurements. In the Equation (27) above, the signals *u*<sup>1</sup> and *u*<sup>2</sup> are projected onto the sampling functions φ*<sup>i</sup>* <sup>1</sup>*<sup>k</sup>* and <sup>φ</sup>*<sup>i</sup>* 2*k*, respectively. We also note that traditional amplitude sampling of a bandlimited signal *u*<sup>1</sup> at times (*tn*), *n* ∈ Z, can be expressed as

$$\langle \mu\_1(\cdot \cdot), \text{sinc}\,(t\_n - \cdot) \rangle\_{\mathbb{L}^2(\mathbb{R})} = \mu\_1(t\_n),$$

where sinc (*t*) = sin (-1*t*) <sup>π</sup>*<sup>t</sup>* is the impulse response of the ideal low pass filter bandlimited to -<sup>1</sup> or in other words, the kernel of the RKHS of finite-energy band-limited functions (Lazar and Pnevmatikakis, 2009). Thus, the neural encoding model described by the Equation (27) can be interpreted as generalized sampling with noisy measurements with sampling functions φ*<sup>i</sup>* 1*k* and φ*<sup>i</sup>* 2*k*.

The formulation of the encoding model can easily be extended to the case when *M* neural circuits encode a stimulus in parallel. This is shown schematically in Supplementary Figure 1. A left superscript was added in the figure to each of the components to indicate the circuit number.

#### **3.2. DECODING**

In the previous section, we showed that the encoding of a signal *u*<sup>1</sup> by the neural circuit model with feedforward and feedback DSPs and BSGs can be characterized by the set of t-transform Equations (26). We noticed that the Equations (26) are nonlinear in *u*<sup>1</sup> due to the second order Volterra term. However, by reinterpreting the second order term as linear functionals *<sup>T</sup> <sup>i</sup>* <sup>2</sup>*<sup>k</sup>* on the higher dimensional tensor space *<sup>H</sup>*<sup>1</sup> <sup>2</sup>, (26) implies that the measurements taken by each of the neurons are the sum of linear measurements in two different vector spaces [see also Equations (27)].

In this section we investigate the decoding of signals encoded with the neural circuit in **Figure 1**. The purpose of decoding is to recover from the set of spike times the original signals, *u*1(*t*) and *u*2(*t*1, *t*2), that respectively belong to the two different vector spaces *<sup>H</sup>*<sup>1</sup> <sup>1</sup> and *<sup>H</sup>*<sup>1</sup> <sup>2</sup>. We formulate the decoding problem as the joint smoothing spline problem

$$\left(\hat{\boldsymbol{u}}\_{1},\hat{\boldsymbol{u}}\_{2}\right) = \underset{\boldsymbol{u}\_{1}\in\mathcal{H}\_{1}^{1},\boldsymbol{u}\_{2}\in\mathcal{H}\_{2}^{1}}{\operatorname{argmin}} \left\{ \lambda\_{1} \left\|\boldsymbol{u}\_{1}\right\|\_{\mathcal{H}\_{1}^{1}}^{2} + \lambda\_{2} \left\|\boldsymbol{u}\_{2}\right\|\_{\mathcal{H}\_{2}^{1}}^{2} \right. $$

$$+ \sum\_{i=1}^{2} \sum\_{k=1}^{n^{i}} \left( T\_{1k}^{i} \boldsymbol{u}\_{1} + T\_{2k}^{i} \boldsymbol{u}\_{2} - \boldsymbol{q}\_{k}^{i} \right)^{2} \right\}, \quad \text{(28)}$$

where *ni* + 1 is the number of spikes generated by BSG *i* = 1, 2.

**Theorem 3.7.** *The solution to* (28) *is of the form*

$$\begin{aligned} \hat{u}\_1(t) &= \sum\_{i=1}^2 \sum\_{k=1}^{n^i} \boldsymbol{c}\_k^i \boldsymbol{\phi}\_{1k}^i(t) \\ \hat{u}\_2(t\_1, t\_2) &= \sum\_{i=1}^2 \sum\_{k=1}^{n^i} \boldsymbol{c}\_k^i \boldsymbol{\phi}\_{2k}^i(t\_1, t\_2), \end{aligned} \tag{29}$$

*where* φ*<sup>i</sup>* <sup>1</sup>*k*(*t*) <sup>=</sup> *<sup>T</sup> <sup>i</sup>* 1*kK*<sup>1</sup> <sup>1</sup>|*<sup>t</sup> and* <sup>φ</sup>*<sup>i</sup>* <sup>2</sup>*k*(*t*1, *<sup>t</sup>*2) <sup>=</sup> *<sup>T</sup> <sup>i</sup>* 2*kK*<sup>1</sup> 2|*t*1, *t*<sup>2</sup> , *i* = 1, 2*, k* = 1, ··· , *n<sup>i</sup> ,* **c** = *c*1 <sup>1</sup>, ··· ,*c*<sup>1</sup> <sup>1</sup>, ··· ,*c*<sup>2</sup> *T is the solution of the system of*

*<sup>n</sup>*<sup>1</sup> ,*c*<sup>2</sup> *n*2 *linear equations*

$$\left(\left(\Phi\_1 + \Phi\_2\right)^2 + \lambda\_1\Phi\_1 + \lambda\_2\Phi\_2\right)\mathbf{c} = \left(\Phi\_1 + \Phi\_2\right)\mathbf{q},\qquad(\text{30})$$

$$where \; \mathbf{q} = \begin{bmatrix} q\_1^1, \cdots, \ q\_{n^1}^1, q\_1^2, \cdots, q\_{n^2}^2 \end{bmatrix}^T, and$$

$$\Phi\_i = \begin{bmatrix} \Phi\_i^{11} \Phi\_i^{12} \\ \Phi\_i^{21} \Phi\_i^{22} \end{bmatrix}, i = 1, 2,$$

*and*

$$[\Phi\_i^{mn}]\_{kl} = \langle \phi\_{ik}^m, \phi\_{il}^n \rangle.$$

**Proof:** Proof of the theorem follows the Representer Theorem (Berlinet and Thomas-Agnan, 2004) and is given in detail in Appendix.

**Remark 3.8.** *When* λ<sup>1</sup> = λ2*, the solution* **c** *amounts to*

$$\mathbf{c} = (\Phi\_1 + \Phi\_2 + \lambda\_1 \mathbf{I})^{-1} \mathbf{q},$$

*where* **I** *is an identity matrix of appropriate dimensions.*

**Remark 3.9.** *Although* (29) *solves* (28)*, in practice a minimum number of spikes is needed to obtain a meaningful estimate of the original signal. A minimum bound for the number of measurements/spikes can be derived in the noiseless case. Clearly, the bound has to be larger than the dimension of the space. This may require the signal to be encoded by a circuit with a larger number of neurons than the two shown in Figure 1 (Lazar and Slutskiy, in press). A number of such neural circuits in parallel can be used to encode input stimuli as shown in the Supplementary Figure 1. Theorem 3.7 can be easily extended to solving the smoothing spline problem*

$$\begin{split} \left(\hat{\boldsymbol{u}}\_{1}, \hat{\boldsymbol{u}}\_{2}\right) &= \operatorname\*{argmin}\_{\boldsymbol{u}\_{1} \in \mathcal{H}^{1}\_{1}, \boldsymbol{u}\_{2} \in \mathcal{H}^{1}\_{2}} \left\{ \lambda\_{1} \left\|\boldsymbol{u}\_{1}\right\|\_{\mathcal{H}^{1}\_{1}}^{2} + \lambda\_{2} \left\|\boldsymbol{u}\_{2}\right\|\_{\mathcal{H}^{1}\_{2}}^{2} \\ &+ \sum\_{m=1}^{M} \sum\_{i=1}^{2} \sum\_{k=1}^{m\_{m}^{i}} \left( {}^{m}\boldsymbol{T}\_{1k}^{i}\boldsymbol{u}\_{1} + {}^{m}\boldsymbol{T}\_{2k}^{i}\boldsymbol{u}\_{2} - {}^{m}\boldsymbol{q}\_{k}^{i} \right)^{2} \right\}, \end{split}$$

*where m* = 1, 2,..., *M, denotes the circuits number in Supplementary Figure 1. In addition, if the circuits consist of only first order feedforward kernels, then only u*1(*t*) *can be reconstructed. Similarly, if the circuits are comprised of only the second order feedforward kernels, then u*2(*t*1, *t*2) *can be reconstructed but not u*1(*t*)*.*

**Remark 3.10.** *Since u*2(*t*1, *t*2) = *u*1(*t*1)*u*1(*t*2) = *u*2(*t*2, *t*1)*, u*<sup>2</sup> *belongs to a subspace of <sup>H</sup>*<sup>1</sup> <sup>2</sup> *whose elements are symmetric functions. We also note that since the second order feedforward kernels are symmetric, the sampling functions* φ*i* <sup>2</sup>*k*(*t*1, *t*2) , *i* = 1, 2*, k* = 1, ··· , *n<sup>i</sup> , also belong to the same subspace. Therefore, if the sampling functions span the subspace of symmetric functions in <sup>H</sup>*<sup>1</sup> 2*, u*<sup>2</sup> *can readily be reconstructed with only L*<sup>1</sup> + 1 2*L*<sup>1</sup> + 1 *measurements/spikes, rather than* 2*L*<sup>1</sup> + 1 2 *, the dimension of <sup>H</sup>*<sup>1</sup> 2*.*

**Remark 3.11.** *The reconstruction of u*2(*t*1, *t*2) *on* D<sup>2</sup> *strongly depends on the support (in practice the finite memory) of the kernels h*11*<sup>i</sup>* <sup>2</sup> , *i* = 1, 2 *(see also Figure 2C). In the reconstruction example of the Supplementary Figure 5, we show that u*ˆ <sup>2</sup> *approximates u*<sup>2</sup> *well in the restricted domain where h*11*<sup>i</sup>* <sup>2</sup> *is nonzero. Outside this restricted domain, h*11*<sup>i</sup>* <sup>2</sup> *vanishes and u*<sup>2</sup> *is not well recovered as suggested by the large error in the Supplementary Figure 5E.*

#### **3.3. EFFECT OF NOISE ON STIMULUS DECODING**

In this section, we investigate the effect of noise sources (i) on spike timing of the reduced PIF neuron, and (ii) on the decoding of stimuli encoded with a neural circuit. We will also present the effect of an alternative noise source model on both spike timing and stimulus decoding.

### *3.3.1. Effect of noise on measurement and spike timing errors of the reduced PIF neuron*

As suggested by (22), the variance of the measurement error of the reduced PIF neuron is directly related to the PRC of the associated limit cycle. We first characterize the variance of the measurement error due to each individual noise source parametrized by the bias current *I<sup>i</sup>* . We then evaluate the spike timing variance between the spike trains generated by the Hodgkin-Huxley neuron and the reduced PIF neuron again as a function of the bias current *I<sup>i</sup>* . We start with a brief description of the key elements of Hodgkin-Huxley neuron and the PIF neuron.

We consider the stochastic Hodgkin-Huxley equations

$$d\mathbf{Y}^i = \mathbf{f}^i\left(\mathbf{Y}^i, I^i\right)dt + d\mathbf{Z}^i(t),\tag{31}$$

where **f** *<sup>i</sup>* is defined as in Section 2.2 of the Supplementary Material with additional normalization such that the unit of time is in seconds instead of milliseconds and the unit of voltage is in Volts instead of milivolts as conventionally used. **Z***<sup>i</sup>* (*t*) takes the form

$$d\mathbf{Z}^i(t) = \begin{bmatrix} \nu^i dt + \sigma\_1^i dW\_1^i\\ \sigma\_2^i dW\_2^i\\ \sigma\_3^i dW\_3^i\\ \sigma\_4^i dW\_4^i \end{bmatrix}.$$

Here *W<sup>i</sup> n*(*t*) are independent standard Brownian motion processes and σ*<sup>i</sup> <sup>n</sup>*, *n* = 1, 2, 3, 4, are associated scaling factors.

The variance of the measurement error of the reduced PIF neuron due to each Brownian motion process *W<sup>i</sup> <sup>n</sup>*, *n* = 1, ··· , 4, is given by [see also Equation (22)]

$$\mathbb{E}\left(\mathbb{E}\left[\varepsilon^{i}\_{kn}\right]^2\right)\left(I^i\right) = \left(\sigma^i\_n\right)^2 \int\_{t^i\_k}^{t^i\_{k+1}} \left[\psi^i\_n\left(s - t^i\_k, I^i\right)\right]^2 ds. \tag{32}$$

We show in **Figure 3A** the variance of the measurement error in (32) associated with each source of noise of the reduced PIF neuron for the unitary noise levels σ*<sup>i</sup> <sup>n</sup>* = 1, *n* = 1, 2, 3, 4. The variances given by (32) are plotted as a function of the bias current *Ii* . Clearly, the noise arising in dendritic stimulus processing (*W<sup>i</sup>* 1) induces the largest error, and together with noise in the potassium channels (*W<sup>i</sup>* <sup>2</sup>), these errors are about two magnitudes larger in variance than those induced by the noise sources in the sodium channels (*W<sup>i</sup>* <sup>3</sup>, *<sup>W</sup><sup>i</sup>* 4).

The above analysis is based on the analytical derivation of the measurement error in (32) for the rPIF neurons. The measurement error is closely related, however, to the spike timing variation of the BSGs subject to noise sources. A variance of 10−<sup>6</sup> in **Figure 3A** corresponds to a standard deviation of 1 ms in spike timing. In practice the error between the spike times of the Hodgkin-Huxley neuron and the reduced PIF neuron can be directly evaluated.

In order to do so, we randomly generated a weak bandlimited dendritic input. All evaluations were based on encoding a signal with the Hodgkin-Huxley neuron model described above with internal noise sources and bias current *I<sup>i</sup>* . The spike times (*t i <sup>k</sup>*) of the Hodgkin-Huxley neuron were recorded. Starting from each spike time *t i k*, we encoded the appropriate portion of the signal by the reduced PIF neuron until a spike *<sup>r</sup> t i <sup>k</sup>* <sup>+</sup> <sup>1</sup> was generated. The difference between *<sup>r</sup> t i <sup>k</sup>* <sup>+</sup> <sup>1</sup> and *t i <sup>k</sup>* <sup>+</sup> <sup>1</sup> is the error in approximating the encoding using the reduced PIF formulation. This process was repeated for each *I<sup>i</sup>* . We computed the variance of the errors based on some 3000–5000 spikes generated in encoding the input.

In **Figure 3B**, the variance of the spike timing error *<sup>r</sup> t i <sup>k</sup>* <sup>+</sup> <sup>1</sup> − *t i <sup>k</sup>* <sup>+</sup> <sup>1</sup> for σ*<sup>n</sup>* = 0, *n* = 1, 2, 3, 4, is shown. Since the reduced PIF is an approximation (even under noiseless conditions) and, although small, the error is nonzero. From **Figure 3B**, the variance of the spike timing error is on the order of 10<sup>−</sup>9. We shall evaluate the spike timing error variance of the intrinsic noise sources in a range much larger than 10<sup>−</sup>9.

We also tested to what extent each individual source of noise contributes to the variance of spike timing as suggested by the theoretical analysis depicted in **Figure 3A**. Indeed, the error variance obtained through simulations in **Figure 3C** follows the basic pattern shown in **Figure 3A**. **Figure 3C** was obtained by setting one of the σ*n*'s to a nonzero value and the rest to 0 (the nonzero values were σ<sup>1</sup> = σ<sup>2</sup> = 0.01, σ<sup>3</sup> = σ<sup>4</sup> = 0.1). Each nonzero value was picked to be large enough so that the error variance in the

**FIGURE 3 | Variance of the measurement and spike timing errors. (A)** Error measurement variances computed from the PRCs of the Hodgkin-Huxley neuron [Equation (32)]. Each individual variance is parametrized by the bias current *I i* . **(B)** Error variance between spike times generated by the noiseless Hodgkin-Huxley neuron and its reduced PIF counterpart. (**C)** The spike timing error variance due to each source of noise, obtained from simulations of the Hodgkin-Huxley neuron follow the pattern of the theoretically derived measurement error shown in **(A)**. The spike timing error variances are obtained by setting, at each time, one of the σ*n*'s to a nonzero value and the rest

to zero. The spikes generated by the Hodgkin-Huxley neuron are compared with the spikes generated by its reduced PIF counterpart. The variance of the differences between two spike times are normalized by the nonzero σ*<sup>n</sup>* mentioned before. **(D)** The spike timing variance due to the simultaneous presence of multiple noise sources approximates the sum of spike timing variances due to individual noise sources. Blue curve shows the spike timing variance obtained by simulating Hodgkin-Huxley equations using nonzero values for all σ*n*, *n* = 1, 2, 3, 4. Red curve shows the sum of spike timing variances obtained in **(C)** with proper scaling.

absence of noise (**Figure 3B**) becomes negligible, and at the same time, it was small enough such that the states of the neurons did not substantially deviate from the limit cycle. To compare the with the ones in **Figure 3A** we normalized the error variance obtained in simulations by σ*n*.

Next, we tested whether the variance of spike timing due to presence of multiple noise sources is truly the summation of error variances due to individual noise sources. We simulated the Hodgkin-Huxley equations with σ<sup>1</sup> = σ<sup>2</sup> = 0.005, σ<sup>3</sup> = σ<sup>4</sup> = 0.05. The total spike timing error variance shown in **Figure 3D** (blue curve) is very close to the sum of error variances in **Figure 3C** with proper scaling (red curve in **Figure 3D**).

As suggested by the above analysis, the reduced PIF neuron with random thresholds largely captures the encoding of stimuli by BSGs subject to intrinsic noise sources.

#### *3.3.2. Effect of noise on stimulus decoding*

In order to quantitatively explore how noise impacts signal decoding, we recovered from spikes the signal encoded by the noisy neural circuit of Supplementary Figure 1. We started with the base-level noise-less case described in Section 3.2 of the Supplementary Material (*M* = 4) and proceeded to introduce individual noise terms with a range of scaling factors. For example, we set σ*<sup>i</sup>* <sup>2</sup> <sup>=</sup> <sup>σ</sup>*<sup>i</sup>* <sup>3</sup> <sup>=</sup> <sup>σ</sup>*<sup>i</sup>* <sup>4</sup> <sup>=</sup> 0 and varied <sup>σ</sup>*<sup>i</sup>* <sup>1</sup>. We also tested the case when 10σ*<sup>i</sup>* <sup>1</sup> <sup>=</sup> <sup>10</sup>σ*<sup>i</sup>* <sup>2</sup> <sup>=</sup> <sup>σ</sup>*<sup>i</sup>* <sup>3</sup> <sup>=</sup> <sup>σ</sup>*<sup>i</sup>* <sup>4</sup> for the aggregated effect on stimulus recovery. We choose to use σ*<sup>i</sup>* <sup>3</sup> and <sup>σ</sup>*<sup>i</sup>* <sup>4</sup> 10 times larger than σ*<sup>i</sup>* <sup>1</sup> and <sup>σ</sup>*<sup>i</sup>* <sup>2</sup> so that each noise source introduced a similar error.

In all simulations, the Euler-Maruyama scheme (Kloeden and Platen, 1992) was used for the numerical integration of the SDEs. We performed 20 encoding and decoding experiments. Each time, the input stimulus was generated by randomly picking from a Gaussian distribution the real and imaginary parts of the coefficients *ul* in (1). We further constrained the stimuli to be real-valued. (An example is given in Supplementary Figure 5.) For each noise level, the input signal was encoded/decoded. The mean Signal-to-Noise Ratio (SNR) across 20 experiments is reported for each noise level. The SNR for the reconstruction of *u*<sup>1</sup> was computed as

$$\text{SNR} = 10 \log\_{10} \left[ \frac{\|u\_1\|^2}{\|u\_1 - \hat{u}\_1\|^2} \right],\tag{33}$$

where *u*<sup>1</sup> is the original signal and *u*ˆ <sup>1</sup> is its reconstruction. Note that the spike time occurrences generated for the same signal are different for each noise level. Since the sampling functions are spike time dependent, the number of measurements/spikes may not be the same for each noise level. Moreover, at times, the sampling functions may not fully span the stimulus space. To reduce the uncertainty caused by the stimulus dependent sampling we averaged our SNR data over 20 different signals.

**Figure 4A** shows the SNR of the reconstruction of signal *u*1(*t*) against different noise strength. **Figure 4B** shows the SNR of the reconstruction of signal *u*<sup>2</sup> <sup>1</sup>(*t*) = *u*2(*t*,*t*). The reconstruction SNR in **Figure 4A** largely matches the inverse ordering of noise strength of each of the individual noise sources shown in **Figure 3A**. DSP noise sources degrade the reconstruction performance most strongly while noise sources associated with gating variables *m* and *h* have a much smaller effect for the same variance level. Since the variance of measurement error is the sum of error variance in each variable, the case when 10σ<sup>1</sup> = 10σ<sup>2</sup> = σ<sup>3</sup> = σ<sup>4</sup> = σ exhibits the lowest performance.

## *3.3.3. Effect of an alternative noise model on spike timing and stimulus decoding*

Biologically, the effect of channel noise on the operation of the BSGs is due to the ON-OFF activity of a finite number of ion channels. The Hodgkin-Huxley equations and the noise terms used in Section 3.3.2 correctly capture the dynamics in the limit of infinitely many channels. Recent research, however, suggests that the model equations may not correctly model the ion current fluctuations for a finite number of channels (Goldwyn and Shea-Brown, 2011).

We consider here an alternative stochastic formulation of the Hodgkin-Huxley model that more precisely captures the ion channel kinetics. By using a finite number of ion channels the strength of noise amplitude becomes directly related to the actual number of ion channels. Therefore, the free variables are only the number of potassium and sodium channels that are both biologically meaningful. The successful use of an alternative noise model as described below also suggests that our analysis can be applied to a wide range of stochastic formulations of BSGs based on SDEs.

We shall construct here stochastic ion channels using conductance noise rather than subunit noise as in the previous Sections (Goldwyn and Shea-Brown, 2011; Goldwyn et al., 2011). This stochastic Hodgkin-Huxley system is simulated using a diffusion approximation following (Orio and Soudry, 2012). The system of SDEs can be expressed as

$$d\mathbf{Y}^i = \mathbf{f}^i\left(\mathbf{Y}^i, I^i\right)dt + \mathbf{B}^i\left(\mathbf{Y}^i\right)d\mathbf{Z}^i(t),$$

where **Y***<sup>i</sup>* has 14 state variables and the full system can be found in Section 3.3 of the Supplementary Material. Here *i* = 1 for simplicity.

The variance of the measurement error is now given by (20). We can decompose the variance into three terms as

$$\mathbb{E}\left[\varepsilon\_k^i\right]^2 = \mathbb{E}\left[\varepsilon\_{kV}^i\right]^2 + \mathbb{E}\left[\varepsilon\_{kK}^i\right]^2 + \mathbb{E}\left[\varepsilon\_{kNa}^i\right]^2,$$

where ε*<sup>i</sup> kV*, ε*<sup>i</sup> kK*, ε*<sup>i</sup> kNa* are measurement errors associated with the noise in the DSP, in potassium channels and in sodium channels, respectively.

As ε*<sup>i</sup> kV* is quantitatively the same as that in Section 3.3.2, we focus here on ε*<sup>i</sup> kK* and <sup>ε</sup>*<sup>i</sup> kNa*. The variance of the errors can be respectively expressed as

$$\begin{aligned} & \left( \mathbb{E} \left[ \boldsymbol{\varepsilon}\_{\boldsymbol{k} \boldsymbol{K}}^{\boldsymbol{i}} \right]^2 \right) (\boldsymbol{I}^{\boldsymbol{i}}) \\ &= \sum\_{p=2}^5 \int\_{t\_k^{\boldsymbol{i}}}^{t\_{\boldsymbol{k}+1}^{\boldsymbol{i}}} \left[ \sum\_{n=2}^6 \psi\_n^{\boldsymbol{i}} (\boldsymbol{s} - t\_k^{\boldsymbol{i}}, \boldsymbol{I}^{\boldsymbol{i}}) b\_{np}^{\boldsymbol{i}} \left( \mathbf{x}^{\boldsymbol{i}} (\boldsymbol{s} - t\_k^{\boldsymbol{i}}, \boldsymbol{I}^{\boldsymbol{i}}) \right) \right]^2 ds, \end{aligned}$$

and

σ*i* <sup>4</sup> <sup>=</sup> <sup>σ</sup>, <sup>σ</sup>*<sup>i</sup>*

<sup>1</sup> <sup>=</sup> <sup>σ</sup>*<sup>i</sup>* <sup>2</sup> <sup>=</sup> <sup>σ</sup>*<sup>i</sup>*

$$\begin{aligned} & \left( \mathbb{E} \left[ \boldsymbol{\varepsilon}\_{k \text{Na}}^{i} \right]^2 \right) (I^i) \\ &= \sum\_{p=6}^{15} \int\_{t\_k^i}^{t\_{k+1}^i} \left[ \sum\_{n=7}^{14} \psi\_n^i \left( s - t\_k^i, I^i \right) b\_{np}^i \left( \mathbf{x}^i (s - t\_k^i, I^i) \right) \right]^2 ds. \end{aligned}$$

<sup>1</sup> <sup>=</sup> <sup>10</sup>σ*<sup>i</sup>*

<sup>2</sup> <sup>=</sup> <sup>σ</sup>*<sup>i</sup>* <sup>3</sup> <sup>=</sup> <sup>σ</sup>*<sup>i</sup>*

<sup>4</sup> = σ. In-sets (on

<sup>3</sup> <sup>=</sup> 0. (Magenta) 10σ*<sup>i</sup>*

Note that *bnp*, *n* = 1, ··· , 14, *p* = 2, 3, ··· , 15, are functions that dependent on either the number of potassium channels *NNa* or the number of sodium channels *NK*, and the states of the neuron.

We first evaluate (E ε*i kNa*<sup>2</sup> )(*I<sup>i</sup>* ) using the PRCs. The PRCs are obtained by letting *NNa* = *NK* = ∞ and thereby making the system deterministic. Since the measurement error variance for fixed *I<sup>i</sup>* is proportional to (*NNa*)<sup>−</sup>1, it is shown in **Figure 5A** as a function of the bias current *I<sup>i</sup>* for *NNa* = 1. Similarly, the variance of the measurement error E ε*i kK*2 (*Ii* ) for *NK* = 1 is shown in **Figure 5A**, and it is proportional to (*NK*)−<sup>1</sup> for a fixed *Ii* . We notice that, when the number of channels is the same, the measurement error due to the sodium channels is of the same order of magnitude with the measurement error due to the potassium channels. However, the number of sodium channels is

reconstruction of *u*1(*t*). **(B)** SNR of reconstruction of *u*<sup>2</sup>

<sup>1</sup>(*t*) = *u*2(*t*, *t*).

typically 3–4 times larger than the number of potassium channels. Therefore, in contrast to the previous case, the error induced by sodium channels shall be larger than that induced by potassium channels.

We also analyzed in simulations the difference between spike times generated by the alternative stochastic formulation of the Hodgkin-Huxley equations and those generated by the corresponding reduced PIF neuron. We used in simulation *NNa* = <sup>5</sup> <sup>×</sup> <sup>10</sup>4, *NK* = ∞, to obtain the variance E ε*i kNa*<sup>2</sup> (*Ii* ) and scaled it by *NNa* to compare it with **Figure 5A**. Similarly, we used *NK* = 5 × 104, *NNa* = ∞, to obtain the variance E ε*i kK*2 (*Ii* ). The spike timing variances of error across different *I<sup>i</sup>* are shown in **Figure 5B** The pattern of similarity between variances in **Figures 5A,B** suggest that the reduced PIF with random threshold associated with this formulation of Hodgkin-Huxley equations is highly effective in capturing the encoding under internal noise sources.

We now show how ion channel noise sources affect the decoding of the input signal. We varied the number of sodium channels *NNa* and fixed the number of potassium channels to be *NK* = 0.3*NNa*, a ratio typically used for Hodgkin-Huxley neurons with the alternative noise source model. By decoding the input

stimulus we show how increasing the number of ion channels improves the faithfulness of signal representation. The SNR of the reconstruction of *u*1(*t*) and *u*<sup>2</sup> <sup>1</sup>(*t*) are depicted in **Figure 6**. SNR goes down to about 4 dB when 600 sodium channels and 180 potassium channels are used. This corresponds to a membrane area of about 10µm<sup>2</sup> with a density of 60µm<sup>2</sup> in sodium channels and 18µm<sup>2</sup> in potassium channels (Goldwyn et al., 2011). We also tested the reconstruction for the case when one type of ion channels is infinitely large, i.e., deterministic, while varying the number of ion channels of the other type. The result is also shown in **Figure 6**. The noise from the dendritic tree shall have similar effect on the representation since the voltage equation is the same as in Section 3.3.2.

### **4. FUNCTIONAL IDENTIFICATION AND NOISE**

In Section 4.1 we show how to functionally identify the feedforward and feedback DSPs of the circuit in **Figure 1** under noisy conditions. We assume here that the BSGs have already been identified using a methodology such as the one developed in Lazar and Slutskiy (2014). In Section 4.2 we discuss the effect of noise parameters on the quality of DSP identification.

#### **4.1. FUNCTIONAL IDENTIFICATION**

In the decoding problem analyzed in Section 3.2, we reconstructed unknown input stimuli by assuming that the neural circuit in **Figure 1** is known and the spike trains are observable. In contrast, the objective of the functional identification problem investigated in this section is to estimate the unknown circuit parameters of the feedforward and feedback DSPs from I/O data. The I/O data is obtained by stimulating the circuit with controllable inputs and by measuring the time occurrences of the output spikes. This basic methodology has been a standard practice in neurophysiology for inferring the function of sensory systems (Hubel and Wiesel, 1962). We assume here that either the BSGs are known in functional form or the family of PRCs associated with the BSGs have already been identified (Lazar and Slutskiy, 2014).

Although decoding and functional identification are seemingly two different problems, they are closely related. By exploiting the commutative property of linear operators, we can rearrange the diagram of the neural circuit model of **Figure 1** into the form shown in **Figure 7**. We notice that the outputs of **Figure 7** and those of **Figure 1** are spike time equivalent, as long as the RKs *K*2 <sup>1</sup> and *<sup>K</sup>*<sup>2</sup> <sup>2</sup> have large enough bandwidth. In what follows we will evaluate the four Volterra terms, i.e., the four dendritic currents feeding the BSG of Neuron 1 in **Figure 7**.

Formally, the first order (feedforward) Volterra term can be written as (Lazar and Slutskiy, in press)

$$\int\_{\mathbb{D}\_1} h\_1^{11i}(t-s) u\_1(s) ds = \int\_{\mathbb{D}\_1} u\_1(t-s) (\mathcal{P}^1 h\_1^{11i})(s) ds. \tag{34}$$

Similarly, the second order (feedforward) Volterra term amounts to

$$\int\_{\mathbb{D}\_2} h\_2^{11\dot{i}}\left(t - s\_1, \left.t - s\_2\right)\right) u\_2\left(s\_1, s\_2\right) \, ds\_1 ds\_2\tag{35}$$

$$=\int\_{\mathbb{D}\_2} \mu\_2\left(t-s\_1,\,t-s\_2\right) \left(\mathcal{P}^1 h\_2^{11i}\right)(s\_1,s\_2) \,ds\_1 ds\_2\ldots$$

The above equations suggest that the projections of the feedforward kernels can be re-interpreted as inputs, whereas the signals *u*<sup>1</sup> and *u*<sup>2</sup> can be treated as feedforward kernels.

In Section 2.2.2 we introduced two RKHSs, *<sup>H</sup>*<sup>2</sup> <sup>1</sup> and *<sup>H</sup>*<sup>2</sup> <sup>2</sup>, for modeling two different spaces of spikes. The elements of *<sup>H</sup>*<sup>2</sup> <sup>1</sup> are functions defined over the domain [0, *S*2] with

$$S^2 \ge \text{supp}\left\{h\_1^{2\bar{j}i}\right\} + \max\left\{\left(t\_{k+1}^i - t\_k^i\right)\right\}\_{i=1,2, k \in \mathbb{Z}}.$$

The period *S*<sup>2</sup> is large enough to ensure that any spike that arrives supp{*h* 2*ji* <sup>1</sup> } seconds prior to the arrival of *t i <sup>k</sup>*, or earlier, will not affect the output of the feedback kernel on the inter-spike time interval [*t i k*, *t i <sup>k</sup>* <sup>+</sup> <sup>1</sup>]. Thus, such spikes will not introduce additional error terms in the *t*-transform evaluated on the inter-spike time interval [*t i k*, *t i <sup>k</sup>* <sup>+</sup> <sup>1</sup>]. Note that the domain [0, *<sup>S</sup>*2] of the functions in *<sup>H</sup>*<sup>2</sup> <sup>1</sup> may not be the same as the domain of the input space *<sup>H</sup>*<sup>1</sup> 1. However, such a domain can be shifted on a spike by spike basis to [*t i <sup>k</sup>* <sup>+</sup> <sup>1</sup> <sup>−</sup> *<sup>S</sup>*2, *<sup>t</sup> i <sup>k</sup>* <sup>+</sup> <sup>1</sup>] for the inter-spike time interval [*t i k*, *t i <sup>k</sup>* <sup>+</sup> <sup>1</sup>]. This is important for mitigating the practical limitation of modeling the stimuli as periodic functions in *<sup>H</sup>*<sup>1</sup> 1.

The response of the first-order feedback term to its spiking input on the inter-spike time interval [*t i k*, *t i <sup>k</sup>* <sup>+</sup> <sup>1</sup>] in **Figure 7** amounts to (*i* = *j*)

$$\left(\mathcal{P}^2 h\_1^{2\bar{i}\bar{i}}\right)(t) \ast \sum\_{\substack{l:\mathbf{i}\_{k+1}^i - S^2 \le \mathbf{i}\_l^j \prec \mathbf{i}\_{k+1}^i}} K\_1^2(t\_l^{\bar{i}}, t) = \sum\_{\substack{l:\mathbf{i}\_{k+1}^i - S^2 \le \mathbf{i}\_l^j \prec \mathbf{i}\_{k+1}^i}} (\mathcal{P}^2 h\_1^{2\bar{i}\bar{i}})(t - t\_l^{\bar{i}}). \tag{36}$$

It is clear from Section 2.2.2 that

$$\sum\_{\substack{l:\,l\_{k+1}^{i}-S^{2}\leq\mathfrak{r}\_{l}^{i}<\mathfrak{r}\_{k+1}^{i}}} \left(\mathcal{P}^{2}h\_{1}^{2\bar{j}\bar{l}}\right)(t-\mathfrak{r}\_{l}^{\bar{j}}) \xrightarrow{\mathbb{L}^{2}} \sum\_{\substack{l:\,l\_{k+1}^{i}-S^{2}\leq\mathfrak{r}\_{l}^{i}<\mathfrak{r}\_{k+1}^{i}}} h\_{1}^{2\bar{j}\bar{l}}(t-\mathfrak{r}\_{l}^{\bar{j}})$$

if -<sup>2</sup> is at least larger than the effective bandwidth of *h* 2*ji* <sup>1</sup> and *L*<sup>2</sup> → ∞.

Similarly, the response of the second-order feedback kernel to its spiking input on the inter-spike time interval [*t i k*, *t i <sup>k</sup>* <sup>+</sup> <sup>1</sup>] amounts to

$$\sum\_{\substack{\boldsymbol{k}:\boldsymbol{t}\_{k+1}^{i}-S^{2}\leq\boldsymbol{t}\_{l}^{i}<\boldsymbol{t}\_{k+1}^{i}\ \boldsymbol{n};\boldsymbol{t}\_{k+1}^{i}-S^{2}\leq\boldsymbol{t}\_{n}^{i}<\boldsymbol{t}\_{k+1}^{i}}} \sum\_{\substack{\boldsymbol{n}:\boldsymbol{t}\_{k+1}^{i}-S^{2}\leq\boldsymbol{t}\_{k}^{i}<\boldsymbol{t}\_{k+1}^{i}}} \left(\mathcal{P}^{2}h\_{2}^{2\boldsymbol{i}\boldsymbol{t}}\right) \left(\boldsymbol{t}-\boldsymbol{t}\_{l}^{i},\boldsymbol{t}-\boldsymbol{t}\_{n}^{i}\right)$$
 
$$\xleftarrow{\mathcal{L}^{2}}\sum\_{\substack{\boldsymbol{k}:\boldsymbol{t}\_{k+1}^{i}-S^{2}\leq\boldsymbol{t}\_{l}^{i}<\boldsymbol{t}\_{k+1}^{i}\ \boldsymbol{n};\boldsymbol{t}\_{k+1}^{i}-S^{2}\leq\boldsymbol{t}\_{n}^{i}<\boldsymbol{t}\_{k+1}^{i}}} \sum\_{\substack{\boldsymbol{n}:\boldsymbol{t}\_{k}^{i}\leq\boldsymbol{t}\_{k}^{i}<\boldsymbol{t}\_{k}^{i}\ \boldsymbol{n}}}h\_{2}^{2\boldsymbol{i}\boldsymbol{t}} \left(\boldsymbol{t}-\boldsymbol{t}\_{l}^{i},\boldsymbol{t}-\boldsymbol{t}\_{n}^{i}\right)$$

if -<sup>2</sup> is large enough and *L*<sup>2</sup> → ∞.

Combining (34), (36), (36), and (37), for each spike interval [*t i k*, *t i <sup>k</sup>* <sup>+</sup> <sup>1</sup>], the aggregated output current of the DSPs of Neuron *i* in **Figure 7**, shall converge to the DSP aggregated output current of Neuron *i* in **Figure 1** for large enough -2. A direct consequence of this equivalence is that, under the same additive Gaussian white noise and channel noise in the BSGs, the t-transform of the circuit in **Figure 7** and in **Figure 1** are identical.

Note that the outputs of the feedforward kernels are always equivalent; the equivalence of the outputs of the feedback kernels requires, however, the use of large enough bandwidth -2. Otherwise, the equivalence in the t-transform is invalid and an additional noise term appears in the *t*-transform of the Neuron 1 in **Figure 7**.

The projections of the Volterra DSP kernels of **Figure 7** are interpreted as inputs, while the input stimuli and the train of RKs at spike times replace the impulse response of the corresponding filters. Therefore, the functional identification problem has been transformed into a dual decoding problem, where the objects to decode are the set of projections of Volterra DSP kernels and the neural circuit is comprised of "stimulus DSP kernels" and "spike DSP kernels" with the same BSGs and noise sources. The only difference is that, instead of a Single-Input Multi-Output decoding problem, the identification was transformed into a Multi-Input Multi-Output decoding problem. In addition, multiple trials using different stimuli are needed; this procedure is illustrated in block diagram form in **Figure 8**. By stimulating the neural circuit with multiple stimuli in the functional identification setting, multiple neural circuits effectively encode the projections of the DSP kernels.

We are now in the position to derive the t-transform of Neuron 1 in **Figure 7**. Assuming that *m* = 1, ··· , *M*, trials are performed for identification, the *t*-transform (26) can be written as

$$\begin{split} & \, ^m \mathcal{L}\_{1k}^{1i} [\mathcal{P}^1 h\_1^{11i}] + ^m \mathcal{L}\_{2k}^{1i} [\mathcal{P}^1 h\_2^{11i}] + ^m \mathcal{L}\_{1k}^{2i} [\mathcal{P}^2 h\_1^{2\ddot{\eta}}] + ^m \mathcal{L}\_{2k}^{2i} [\mathcal{P}^2 h\_2^{2\ddot{\eta}}] \\ & = \, ^m q\_k^i + ^m \epsilon\_k^i, \end{split} \tag{38}$$

for *<sup>i</sup>*, *<sup>j</sup>* <sup>=</sup> <sup>1</sup>, <sup>2</sup>, *<sup>i</sup>* = *<sup>j</sup>*, *<sup>k</sup>* <sup>∈</sup> <sup>Z</sup>. Here *<sup>m</sup>L*1*<sup>i</sup>* <sup>1</sup>*<sup>k</sup>* : *<sup>H</sup>*<sup>1</sup> <sup>1</sup> <sup>→</sup> <sup>R</sup>, *<sup>m</sup>L*1*<sup>i</sup>* 2*k* : *H*1 <sup>2</sup> <sup>→</sup> <sup>R</sup> are bounded linear functionals associated with the feedforward DSP kernels, and *<sup>m</sup>L*2*<sup>i</sup>* <sup>1</sup>*<sup>k</sup>* : *<sup>H</sup>*<sup>2</sup> <sup>1</sup> <sup>→</sup> <sup>R</sup>, *<sup>m</sup>L*2*<sup>i</sup>* <sup>2</sup>*<sup>k</sup>* : *<sup>H</sup>*<sup>2</sup> <sup>2</sup> <sup>→</sup> <sup>R</sup> are bounded linear functionals associated with the feedback DSP kernels for each trial *m*. The above functionals are defined as

$$\begin{split} \, ^m \mathcal{L}\_{1k}^{1i} [\mathcal{P}^1 h\_1^{11i}] &= \int\_{\, ^m t\_k^i}^{\, ^m t\_{k+1}^i} \, ^m \varphi\_k^i(s) \int\_{\mathbb{D}\_1} u\_1^m(s-r) (\mathcal{P}^1 h\_1^{11i})(r) dr ds, \\ \, ^m \mathcal{L}\_{2k}^{1i} [\mathcal{P}^1 h\_2^{11i}] &= \int\_{\, ^m t\_k^i}^{\, ^m t\_{k+1}^i} \, ^m \varphi\_k^i(s) \int\_{\mathbb{D}\_2} u\_2^m(s-r\_1, s-r\_2) (\mathcal{P}^1 h\_2^{11i}) \\ &\qquad \qquad (r\_1, r\_2) dr\_1 dr\_2 ds, \end{split}$$

$$\prescript{m}{}{\mathcal{L}}\_{1k}^{2i} \| \mathcal{P}^2 h\_1^{2\ddot{\imath}t} \| = \sum\_{\substack{l \asymp \,^i t\_{k+1}^i - S^2 \subseteq \,^m t\_l^i \prec^m t\_{k+1}^i}} \int\_{\rightsquigarrow\_{k+1}^i}^{\prime \pi\_{k+1}^i} \,^m \varphi\_k^{\dot{\imath}}(s) (\mathcal{P}^2 h\_1^{2\ddot{\imath}t}) $$
 
$$\left( s - \,^m t\_l^j \right) ds$$

$$\begin{split} ^m\mathcal{L}\_{2k}^{2i}[\mathcal{P}^2h\_2^{2ji}] &= \sum\_{l^m t\_{k+1}^i - S^2 \le ^m t\_l^i < ^m t\_{k+1}^i} \sum\_{m^m t\_{k+1}^i - S^2 \le ^m t\_n^i < ^m t\_{k+1}^i} ^{m^i t\_{k+1}^i} \\ & \left[ ^m\wp\_k^i(s) \cdot (\mathcal{P}^2h\_2^{2ji})(s - ^m t\_l^j, s - ^m t\_n^i) \right] ds \\ ^mq\_k^i &= (^mb\_k^i - ^mI^i) \int\_{s = t\_k^i}^{t\_{k+1}^i} \!^m\wp\_k^i(s) ds, \\ ^m\wp\_k^i(t) &= \frac{\psi\_1^i(t - ^m t\_k^i, ^m b\_k^i)}{\left(\mathbb{E}\left[\!^m\acute{e}\_k^i\!\right]^2\right)^{\frac{1}{2}}} \end{split}$$

and *<sup>m</sup><sup>i</sup> <sup>k</sup>*, *<sup>i</sup>* <sup>=</sup> <sup>1</sup>, <sup>2</sup>, *<sup>k</sup>* <sup>∈</sup> <sup>Z</sup>, *<sup>m</sup>* <sup>=</sup> <sup>1</sup>, ··· , *<sup>M</sup>*, are independent random variables with normal distribution *N* (0, 1).

The functional identification of the neural circuit in **Figure 7** can then be similarly defined to the decoding problem. We formulate the identification of the noisy neural circuit again as two smoothing spline problems, one for each neuron,

$$
\begin{split}
\begin{bmatrix}
(\overline{\mathcal{P}^1 h^{11}\_1}\_1) \\
(\overline{\mathcal{P}^1 h^{21}\_2}\_2) \\
\overline{\mathcal{P}^2 h^{21}\_1}\_2
\end{bmatrix} &= \underset{\left\{\mathcal{P}^p h^{p\dagger}\_r \in \mathcal{H}^p\_r \middle|\, \mathcal{P}^{r=1,2}\_r\right\}}{\arg\min} \left\{ \sum\_{p=1}^2 \sum\_{r=1}^2 \lambda\_r^p \|\mathcal{P}^p h^{p\dagger}\_r\|^2 \\ &+ \sum\_{m=1}^M \sum\_{k=1}^{m\_l} \left( \sum\_{p=1}^2 \sum\_{r=1}^2 \ll\_r^{p1} \left[ \mathcal{P}^p h^{p\dagger}\_r \right] - \,^m q\_k^1 \right)^2 \right\}
\end{split} \tag{39}
$$

**Figure 7** for details of a single trial.

and

$$
\begin{split}
\widehat{\left[\begin{array}{c}
(\widehat{\mathcal{P}^{1}h\_{1}^{112}}) \\
\widehat{(\mathcal{P}^{1}h\_{2}^{12})} \\
\widehat{(\mathcal{P}^{2}h\_{2}^{12})}
\end{array}\right]} &= \underset{\left\{\mathcal{P}^{p}h\_{r}^{p12}\in\mathcal{H}\_{r}^{p}\right\}\_{p=1,2}}{\operatorname{arg\,min}} \left\{\sum\_{p=1}^{2}\sum\_{r=1}^{2}\lambda\_{r}^{p}\|\mathcal{P}^{p}h\_{r}^{p12}\|^{2} \\ &+\sum\_{m=1}^{M}\sum\_{k=1}^{m\_{\mathrm{r}}^{2}}\left(\sum\_{p=1}^{2}\sum\_{r=1}^{2}\,^{m}\mathcal{L}\_{rk}^{p2}\left[\mathcal{P}^{p}h\_{r}^{p12}\right]^{2} - \,^{m}q\_{k}^{2}\right)^{2}\right\},
\end{split} \tag{40}
$$

where *mni* is the number of spikes generated by Neuron *i* in trial *m*.

The solution can be obtained in a similar way as in Theorem 3.7.

**Theorem 4.1.** *The solutions to* (40) *is*

$$\begin{aligned} (\widehat{\mathcal{P}^1 h\_1^{111}})(t) &= \sum\_{m=1}^M \sum\_{k=1}^{m\_1} {}^m c\_k {}^m \phi\_{1k}(t) \\ (\widehat{\mathcal{P}^1 h\_2^{111}})(t\_1, t\_2) &= \sum\_{m=1}^M \sum\_{k=1}^{m\_1} {}^m c\_k {}^m \phi\_{2k}(t\_1, t\_2) \\ (\widehat{\mathcal{P}^2 h\_1^{221}})(t) &= \sum\_{m=1}^M \sum\_{k=1}^{m\_1} {}^m c\_k {}^m \phi\_{3k}(t) \\ (\widehat{\mathcal{P}^2 h\_2^{221}})(t\_1, t\_2) &= \sum\_{m=1}^M \sum\_{k=1}^{m\_1} {}^m c\_k {}^m \phi\_{4k}(t\_1, t\_2), \end{aligned}$$

*where*

$$\mathbf{c} = \begin{bmatrix} ^1c\_1 \cdots \ ^1c\_{1^n} , \ \cdots \ ^1 , \ \cdots \ ^1 , \ \cdots \ ^Mc\_1 \cdots \ ^Mc\_{M^1} \end{bmatrix}^T \ , \ $$

*is the solution to the system of linear equations*

$$\left( (\Phi\_1 + \Phi\_2 + \Phi\_3 + \Phi\_4)^2 + \lambda\_1^1 \Phi\_1 + \lambda\_1^2 \Phi\_2 + \lambda\_2^1 \Phi\_3 + \lambda\_2^2 \Phi\_4 \right) \mathbf{c}$$

$$= \left( \Phi\_1 + \Phi\_2 + \Phi\_3 + \Phi\_4 \right) \mathbf{q},\tag{41}$$

*where*

$$\mathbf{q} = \left[ {}^1q\_1^1 \cdots \,^1q\_{1\_{n^1}}^1, \cdots \,^1, \cdots \,^Mq\_1^1 \cdots \,^Mq\_{M\_{n^1}}^1 \right]^T, \ . $$

*and*

$$
\Phi\_i = \begin{bmatrix}
\Phi\_i^{11} & \cdots & \Phi\_i^{1M} \\
\vdots & \ddots & \vdots \\
\Phi\_i^{M1} & \cdots & \Phi\_i^{MM},
\end{bmatrix}
$$

*and finally*

$$\left[\Phi\_i^{mn}\right]\_{kl} = \left< {}^m\phi\_{ik}, {}^n\phi\_{il}\right>.$$

*In addition, the sampling functions <sup>m</sup>*φ*ik are given by*

$${}^{m}\phi\_{1k}(t) = {}^{m}\mathcal{L}\_{1k}^{11}\mathcal{K}\_{1|t}^{1},$$

$${}^{m}\phi\_{2k}(t\_1, t\_2) = {}^{m}\mathcal{L}\_{1k}^{11}K\_{2|t\_1, t\_2}^{1},$$

$${}^{m}\phi\_{3k}(t) = {}^{m}\mathcal{L}\_{1k}^{21}K\_{1|t}^{2},$$

$${}^{m}\phi\_{4k}(t\_1, t\_2) = {}^{m}\mathcal{L}\_{2k}^{21}K\_{2|t\_1, t\_2|}^{2}.$$

**Proof:** The proof is similar to the one of Theorem 3.7. -

Since each of the kernel projections may be in a different RKHS, and their domain may also be different, the identification of all filters resemble that of the multi-sensory Time Encoding Machines. Recall that multi-sensory TEMs encode within the same circuit time-varying and space-time varying sensory signals while decoding remains tractable (Lazar and Slutskiy, 2013). The solution to (41) can similarly be obtained as the solution to (40) above.

Note that we are only able to identify the projection of the Volterra kernels. This is because, in practice, we can only probe the system with signals in a bandlimited space. By increasing the bandwidth of the elements of the Hilbert space, the projection of the kernels will converge to their original form (Lazar and Slutskiy, 2012).

**Remark 4.2.** *It is important to note that in order to have a good estimate of the kernels, stimuli must fully explore all input spaces. This can be quite easily achieved for the feedforward DSP kernels by using many (randomly generated) signals that cover the entire frequency spectrum. However, to properly identify the feedback DSP kernels, spike trains must be diverse enough to sample its different frequency components. This may not be easy to realize in practice. For first order feedback kernels, spike trains with constant spike intervals are, for example, undesirable. The Fourier transform of regular Diracdelta pulses is a train of Dirac-delta pulses in the Fourier domain. This means that only certain frequency responses of the DSP kernels are, for example the DC component, sampled. The rest of the frequency components are essentially in the null space of the sampling functions <sup>m</sup>*φ*ik*, *i* = 1, 2, *m* = 1, ··· , *M. Similar effect applies to the space of trigonometric polynomials. If the spike intervals exhibit small variations, many of the frequency components may be sampled but the energy at DC may be too dominant. In this case, noise may contaminate more severely the measurement of non-DC components and may yield unsatisfactory identification. This effect may pose even more stringent requirements on the identification of the second order feedback kernels, as it requires the interaction between two spike trains.*

#### **4.2. EFFECT OF NOISE ON IDENTIFICATION**

In order to evaluate the effect of noise on the identification of the neural circuit in **Figure 1** we included intrinsic noise into the example neural circuit discussed under noiseless conditions in Section 4.1 of the Supplementary Material. Randomly generated signals were used in the identification examples given here. Chosen in the same way as in the decoding example in Section 3.3.2 all these signals are used here to identify the neuron in question. Therefore, in this section, multiple signals are used in repeat experiments to identify the parameters of a neural circuit. By contrast in Section 3.3.2, multiple neurons are used to encode a single signal.

First, we evaluated the effect of noise on the quality of identification of DSP kernels of Neuron 1 in **Figure 7** with a BSG modeled by the SDE (31) with 10σ*<sup>i</sup>* <sup>1</sup> <sup>=</sup> <sup>10</sup>σ*<sup>i</sup>* <sup>2</sup> <sup>=</sup> <sup>σ</sup>*<sup>i</sup>* <sup>3</sup> <sup>=</sup> <sup>σ</sup>*<sup>i</sup>* <sup>4</sup> = σ. **Figure 9** shows the SNR of the identified DSP kernels in **Figure 7** across several noise levels σ. As expected, the general trend for all four kernels is decreasing SNR with increasing noise levels. We notice that the identified feedforward DSP kernels have similar shape as the original kernel, even at high noise levels. However, the feedback DSP kernels undergo a change in shape at high noise levels. We can see that the time instance of the peak amplitude in the first order feedback kernel is shifted to an earlier time instance.

Second, we investigated the identification of DSPs for a BSG noise model already described in Section 3.3.3. **Figure 10** shows the SNR of the identified DSP kernels across a different number of sodium channels *NNa* while *NK* = 0.3*NNa*. The SNR plots suggest that the identification quality increases as more ion channels are present in the BSGs.

Additionally, as discussed in Remark 4.2, BSG noise sources may degrade severely the identification of feedback kernels when the spike trains generated in trials are not sufficient for exploring the two spike input spaces. We show an example of the later in **Figure 11**. The two BSGs have higher bias currents and lower input current magnitude. The later was achieved by scaling down the magnitude of the DSP kernels. The combined effect results in regular spiking intervals in both neurons. The identification

10σ*<sup>i</sup>* <sup>1</sup> <sup>=</sup> <sup>10</sup>σ*<sup>i</sup>* <sup>2</sup> <sup>=</sup> <sup>σ</sup>*<sup>i</sup>* <sup>3</sup> <sup>=</sup> <sup>σ</sup>*<sup>i</sup>* <sup>4</sup> <sup>=</sup> <sup>σ</sup>. **(A)** Kernel *<sup>h</sup>*<sup>111</sup> <sup>1</sup> . In-sets provide a comparison between the original and the identified kernel. **(B)** Kernel *h*<sup>111</sup> <sup>2</sup> . In-sets are

provide a comparison between the original and the identified kernel. **(D)** Kernel *h*<sup>221</sup> <sup>2</sup> . In-sets are identified kernels. Original kernel is on the lower left.

result under *noiseless conditions* is shown in **Figure 11**. Note that since the *t*-transform of the Hodgkin-Huxley neuron is not exact, a small error is introduced even if intrinsic noise is not present. We see that the feedforward DSP kernels can be identified quite well, yielding SNRs of around 17 dB. However, the feedback DSP kernels are not well identified. In particular, the identified secondorder feedback kernel has a wide spread, similar to the high noise case in **Figure 9D**. This suggest that the spike pattern is not sufficiently exploring the space of feedback kernels. A large number of frequency components are only weakly sampled and they can be very easily contaminated by noise. The presence of both intrinsic noise sources can exacerbate the condition further. This effect is confirmed with a simulation of the circuit using Integrate-and-Fire (IAF) neurons. Since the *t*-transform for the IAF neuron is exact (Lazar and Tóth, 2004), both feedback kernels can be identified even if the generated spikes only weakly explore certain frequency components. However, by artificially adding a small measurement error to the t-transform of the circuit with IAF neurons, similar results to those in **Figure 11** can be obtained (data not shown).

#### **5. DISCUSSION**

In this paper, we introduced a novel neural circuit architecture based on a neuron model with a biophysical mechanism of spike generation and feedforward as well as feedback dendritic stimulus processors with intrinsic noise sources. Under this architectural framework, we quantitatively studied the effect of intrinsic noise on dendritic stimulus processing and on spike generation. We investigated how intrinsic noise sources affect the stimulus representation by decoding encoded stimuli from spikes, and quantified the effect of noise on the functional identification of neural circuits. We argued that a duality between stimulus decoding and functional identification holds. Therefore, the encoding framework based on the neural circuit architecture studied here can be applied to both the reconstruction of the encoded signal and the identification of the dendritic stimulus processors. We systematically showed how the precision in decoding is affected by different levels of stochastic variability within the circuit. These results apply verbatim to the functional identification of dendritic stimulus processors via the key duality property mentioned above.

Our theoretical framework highlights two indispensable components of modeling signal representation/processing in a neural circuit—dendritic stimulus processing and spike generation. Such a divide and conquer strategy is ubiquitous in engineering circuits and leads to a separation of concerns. Recent experimental studies also showed that interesting nonlinear processing effects take place first in the dendritic trees rather than in the axon hillock (Yonehara et al., 2013).

We presented here two types of nonlinear dendritic stimulus processors. The first type are feedforward DSPs that respond to continuous input sensory stimuli. The second type are feedback DSPs that respond to *spiking* inputs. We quantitatively demonstrated how intrinsic noise sources would affect the identification quality of all these DSPs. The examples in Section 4.2 suggest that in identification feedback kernels are more vulnerable to internal noise sources than feedforward kernels. In particular, the overall shape of the identified feedback kernels differs significantly from that of the underlying kernels when the strength of noise sources becomes large. Meanwhile the identified feedforward kernels are qualitatively preserved, albeit not accurately.

Most of the single neuron models (LIF, QIF) in the literature have focused on the spike generation mechanism. The encoding capability of these models is typically investigated based on rate encoding (Eliasmith and Anderson, 2003; Lundstrom et al., 2008; Ostojic and Brunel, 2011). For both decoding and identification we used here the occurrence times of spikes generated by spiking neuron models. Most importantly, the BSG models discussed here were characterized by a PRC manifold (Kim and Lazar, 2012) in the presence of noise, while many simplified models (such as the LIF) can be effectively described with a single PRC. Other sensory neuron models, e.g., GLM (Pillow et al., 2011), usually rely on a rate-based output or Poisson spike generation that do not take into account key advances in dynamical systems-based spiking neuron models.

As already mentioned before, we investigated how intrinsic noise sources affect the stimulus representation by decoding encoded stimuli from spikes. We are not suggesting, however, that the decoding algorithm considered here is implemented in the brain. Rather, we argue that decoding is effective in measuring how well information is preserved in the spike domain. The decoding formalism allowed us to investigate how noise affects the fidelity of signal representation by a population of neurons by reconstructing stimuli and comparing their information content in the stimulus space.

While decoding can serve as an "oscilloscope" in understanding stimulus representation in sensory systems, functional identification serves as a guide in experiments to functionally identify sensory processing. Based on spike times, the identification algorithm presents a clear bound on the number of spikes that are necessary for perfect identification under noiseless conditions.

Phrased differently, when a certain number of spikes are acquired from a neuron of interest, the identification algorithm places a constraint on the maximum DSP kernel bandwidth that can perfectly be recovered.

In more practical terms, we advanced two important applications of the circuit architecture considered in this paper. The first one considers dendritic stimulus processors that process information akin to complex cells in V1. The second one adapts the widely used Hodgkin-Huxley model known in the context of neural excitability (Izhikevich, 2007) and analysis of neuronal stochastic variability to stimulus encoding in the presence of noise.

Based on the rigorous formalism of TEMs (Lazar and Tóth, 2004), we extended our previous theoretical framework (Lazar et al., 2010) and argued that spike timing is merely a form of generalized sampling of stimuli. By studying sampling (or measurements) in the presence of intrinsic noise sources, we showed to what extent neurons can represent sensory stimuli in noisy environments as well as how much noise the identification process can tolerate while preserving an accurate understanding of circuit dynamics.

The reconstruction and identification quality are certainly not only related to the strength of noise, but also the strength of the signal. In particular, when the signal strength is small, two facts may affect the quality of reconstruction. First, neurons may not produce enough spikes that have valid *t*-transforms. Second, they may be contaminated by even weak noise, i.e., the signal-to-noise ratio is low. It is well known, however, that neural systems use gain control to boost the relevant signal (Shapley and Victor, 1978; Wark et al., 2007; Friederich et al., 2013). Such strategy may be useful for increasing the signal strength relatively to the strength of the noise. Gain control may also suppress large signals to fit into the range of operation of the BSGs. The gain control itself, maybe considered as a type of Volterra feedforward DSP kernel (Lazar and Slutskiy, in press) and the interaction with feedback loops driven by spikes. The lack of spikes may be compensated by adding other neurons that are sensitive to other features in the input stimuli.

A key feature in our neural circuit model is the nonlinear processing in the feedforward and feedback paths. Nonlinear interaction between feedforward DSPs and feedback DSPs have not been considered here. However, they are of interest and could be addressed in the future. Self-feedback was not included in the model for clarity, but can be considered within the framework of our approach. Self-feedback was introduced to add refractoriness to phenomenological neuron models (Keat et al., 2001; Pillow et al., 2008). Our BSG models, on the contrary, are conductance-based models that have a refractory period built in.

Throughout this paper we assumed that the BSGs themselves have been perfectly identified. The intrinsic noise in the BSGs may degrade the identification quality of conditional PRCs. This may result in a lower identification quality as shown in the examples. It is beneficial to investigate in the future a method that can identify the entire circuit at once so that intrinsic noise in the circuit only affects the identification process a single time.

The theoretical results obtained here suggest a number of experiments in the early olfactory system of fruit flies. The glomeruli of the antennal lobe can be modeled using the Volterra DSPs discussed here and the projection neurons in the antennal lobe are accessible by patch clamping (Lazar and Yeh, 2014). Functional identification of DSPs can then be carried out for studying olfactory stimulus processing in an accessible circuit with intrinsic noise sources (Masse et al., 2009).

#### **ACKNOWLEDGMENT**

The research reported here was supported by AFOSR under grant #FA9550-12-1-0232.

#### **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at: http://www.frontiersin.org/journal/10.3389/fncom. 2014.00095/abstract

#### **REFERENCES**


Ostojic, S., and Brunel, N. (2011). From spiking neuron models to linear-nonlinaer models. *PLoS Comput. Biol.* 7:e1001056. doi: 10.1371/journal.pcbi.1001056


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 22 April 2014; accepted: 23 July 2014; published online: 01 September 2014. Citation: Lazar AA and Zhou Y (2014) Volterra dendritic stimulus processors and biophysical spike generators with intrinsic noise sources. Front. Comput. Neurosci. 8:95. doi: 10.3389/fncom.2014.00095*

*This article was submitted to the journal Frontiers in Computational Neuroscience. Copyright © 2014 Lazar and Zhou. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## **APPENDIX**

## **PROOF OF THEOREM 3.7**

**Proof:** By the Riesz representation theorem (Berlinet and Thomas-Agnan, 2004), there exists a function φ*<sup>i</sup>* <sup>1</sup>*<sup>k</sup>* <sup>∈</sup> *<sup>H</sup>*<sup>1</sup> <sup>1</sup> such that *<sup>T</sup> <sup>i</sup>* <sup>1</sup>*ku*<sup>1</sup> = *u*1, φ*<sup>i</sup>* <sup>1</sup>*k*, <sup>∀</sup>*u*<sup>1</sup> <sup>∈</sup> *<sup>H</sup>*<sup>1</sup> <sup>1</sup>. Moreover by the reproducing property

$$\phi\_{1k}^i(t) = \left\langle \phi\_{1k}^i, K\_{1\parallel t}^1 \right\rangle = \mathcal{T}\_{1k}^i \overline{K\_{1\parallel t}^1}.$$

Let *<sup>H</sup>*<sup>1</sup> <sup>10</sup> be a linear subspace of *<sup>H</sup>*<sup>1</sup> <sup>1</sup> spanned by <sup>φ</sup>*<sup>i</sup>* 1*k*

$$\mathcal{H}\_{10}^1 = \text{span}\left(\left\{\phi\_{1k}^i\right\}, i = 1, 2, k = 1, \dots, n^i\right)$$

and let *<sup>H</sup>*1<sup>⊥</sup> <sup>10</sup> be a linear subspace of *<sup>H</sup>*<sup>1</sup> <sup>1</sup> defined by

$$\mathcal{H}\_{10}^{1\perp} = \left\{ u\_1 \in \mathcal{H}\_1^1 \vert \mathcal{T}\_{1k}^i u\_1 = 0, i = 1, 2, k = 1, \dots, n^i \right\}.$$

Then, for any *<sup>u</sup>*<sup>1</sup> <sup>∈</sup> *<sup>H</sup>*1<sup>⊥</sup> <sup>10</sup> and any <sup>2</sup> *i* = 1 *n<sup>i</sup> <sup>k</sup>* <sup>=</sup> <sup>1</sup> *c<sup>i</sup> k*φ*i* <sup>1</sup>*<sup>k</sup>* <sup>∈</sup> *<sup>H</sup>*<sup>1</sup> <sup>10</sup>, we have

$$\left\langle u\_1, \sum\_{i=1}^2 \sum\_{k=1}^{n^i} c\_k^i \phi\_{1k}^i \right\rangle = \sum\_{i=1}^2 \sum\_{k=1}^{n^i} c\_k^i \left\langle u\_1, \phi\_{1k}^i \right\rangle = \sum\_{i=1}^2 \sum\_{k=1}^{n^i} c\_k^i T\_{1k}^i u\_1 = 0.1$$

Since *<sup>H</sup>*<sup>1</sup> <sup>1</sup> <sup>=</sup> *<sup>H</sup>*<sup>1</sup> <sup>10</sup> <sup>⊕</sup> *<sup>H</sup>*1<sup>⊥</sup> <sup>10</sup> , *u*<sup>1</sup> can be represented as *u*<sup>1</sup> = *u*<sup>10</sup> + *u*<sup>⊥</sup> <sup>10</sup> where *<sup>u</sup>*<sup>10</sup> <sup>∈</sup> *<sup>H</sup>*<sup>1</sup> <sup>10</sup> and *<sup>u</sup>*<sup>⊥</sup> <sup>10</sup> <sup>∈</sup> *<sup>H</sup>*1<sup>⊥</sup> <sup>10</sup> are orthogonal. Therefore,

$$\|\mu\_{10} + \mu\_{10}^\perp\|^2 = \|\mu\_{10}\|^2 + \|\mu\_{10}^\perp\|^2.$$

Similarly, there exists a function φ*<sup>i</sup>* <sup>2</sup>*<sup>k</sup>* <sup>∈</sup> *<sup>H</sup>*<sup>1</sup> <sup>2</sup> such that *<sup>T</sup> <sup>i</sup>* <sup>2</sup>*ku*<sup>2</sup> = *u*2, φ*<sup>i</sup>* <sup>2</sup>*k*, where <sup>φ</sup>*<sup>i</sup>* <sup>2</sup>*k*(*t*1, *<sup>t</sup>*2) <sup>=</sup> *<sup>T</sup> <sup>i</sup>* 2*kK*<sup>1</sup> 2|*t*1*t*<sup>2</sup> . *u*<sup>2</sup> can be represented as *u*<sup>2</sup> = *u*<sup>20</sup> + *u*<sup>⊥</sup> <sup>20</sup>, where *<sup>u</sup>*<sup>20</sup> <sup>∈</sup> *<sup>H</sup>*<sup>1</sup> <sup>20</sup> and *<sup>u</sup>*<sup>⊥</sup> <sup>20</sup> <sup>∈</sup> *<sup>H</sup>*1<sup>⊥</sup> <sup>20</sup> are orthogonal, with

$$\mathcal{H}\_{20}^1 = \text{span}\left(\{\phi\_{2k}^i\}, i = 1, 2, \ k = 1, \dots, n^i\right),$$

and

$$\mathcal{H}\_{20}^{1\perp} = \{ \mu\_2 \in \mathcal{H} | T\_{2k}^i \mu\_2 = 0, i = 1, 2, k = 1, \dots, n^i \}.$$

Finally,

$$\begin{split} &\sum\_{i=1}^{2}\sum\_{k=1}^{n^i} \left( T\_{1k}^i u\_{10} + T\_{2k}^i u\_{20} - q\_k^i \right)^2 + \lambda\_1 \|u\_{10}\|\_{\mathcal{H}\_1^1}^2 + \lambda\_2 \|u\_{20}\|\_{\mathcal{H}\_2^1}^2 \\ &= \sum\_{i=1}^{2}\sum\_{k=1}^{n^i} \left( T\_{1k}^i (u\_{10} + u\_{10}^\perp) + T\_{2k}^i (u\_{20} + u\_{20}^\perp) - q\_k^i \right)^2 \\ &+ \lambda\_1 \|u\_{10}\|\_{\mathcal{H}\_1^1}^2 + \lambda\_2 \|u\_{20}\|\_{\mathcal{H}\_2^1}^2 \\ &\leq \sum\_{i=1}^{2}\sum\_{k=1}^{n^i} \left( T\_{1k}^i (u\_{10} + u\_{10}^\perp) + T\_{2k}^i (u\_{20} + u\_{20}^\perp) - q\_k^i \right)^2 \\ &+ \lambda\_1 \|u\_{10} + u\_{10}^\perp\|\_{\mathcal{H}\_1^1}^2 + \lambda\_2 \|u\_{20} + u\_{20}^\perp\|\_{\mathcal{H}\_2^1}^2 \end{split}$$

Therefore, the minimizer to (28) must belong to the subspaces *H*1 <sup>10</sup> and *<sup>H</sup>*<sup>1</sup> 20.

By plugging (29) into (28) and setting the gradient with respect to **<sup>c</sup>** to 0, we see that **<sup>c</sup>** is the solution to (30). -

## Self-consistent determination of the spike-train power spectrum in a neural network with sparse connectivity

## *Benjamin Dummer 1,2, Stefan Wieland1,2 and Benjamin Lindner 1,2\**

*<sup>1</sup> Theory of Complex Systems and Neurophysics, Bernstein Center for Computational Neuroscience, Berlin, Germany*

*<sup>2</sup> Department of Physics, Humboldt Universität zu Berlin, Berlin, Germany*

#### *Edited by:*

*Joshua H. Goldwyn, New York University, USA*

#### *Reviewed by:*

*Moritz Helias, Research Center Jülich, Germany Yu Hu, University of Washington, USA*

#### *\*Correspondence:*

*Benjamin Lindner, Department of Physics, Humboldt Universität zu Berlin, Philippstr. 13, Haus 2, 10115 Berlin, Germany e-mail: benjamin.lindner@ physik.hu-berlin.de*

A major source of random variability in cortical networks is the quasi-random arrival of presynaptic action potentials from many other cells. In network studies as well as in the study of the response properties of single cells embedded in a network, synaptic background input is often approximated by Poissonian spike trains. However, the output statistics of the cells is in most cases far from being Poisson. This is inconsistent with the assumption of similar spike-train statistics for pre- and postsynaptic cells in a recurrent network. Here we tackle this problem for the popular class of integrate-and-fire neurons and study a self-consistent statistics of input and output spectra of neural spike trains. Instead of actually using a large network, we use an iterative scheme, in which we simulate a single neuron over several generations. In each of these generations, the neuron is stimulated with surrogate stochastic input that has a similar statistics as the output of the previous generation. For the surrogate input, we employ two distinct approximations: (i) a superposition of renewal spike trains with the same interspike interval density as observed in the previous generation and (ii) a Gaussian current with a power spectrum proportional to that observed in the previous generation. For input parameters that correspond to balanced input in the network, both the renewal and the Gaussian iteration procedure converge quickly and yield comparable results for the self-consistent spike-train power spectrum. We compare our results to large-scale simulations of a random sparsely connected network of leaky integrate-and-fire neurons (Brunel, 2000) and show that in the asynchronous regime close to a state of balanced synaptic input from the network, our iterative schemes provide an excellent approximations to the autocorrelation of spike trains in the recurrent network.

**Keywords: neural noise, recurrent neural networks, non-Poissonian spiking, spike-train statistics, spike-train power spectrum**

## **1. INTRODUCTION**

Neurons in different parts of the nervous system respond to repeated presentation of the same stimulus with considerable trial-to-trial variability (van Steveninck et al., 1997). There are several true noise sources contributing to this variability: fluctuations of stochastic ion channels (Schneidman et al., 1998; White et al., 2000), unreliability of synaptic connections such as transmission failure and spontaneous release (Branco and Staras, 2009), and Johnson noise (Manwani and Koch, 1999). These are true noise sources in the sense that they result from the finite number of stochastic elements in the system, be it ionic channels, transmitter molecules, or charge carriers. In cases where synaptic input is absent, e.g., in the neural periphery, the statistics of spontaneous spiking is mainly shaped by channel noise (see e.g., Fisch et al., 2012 for an example); Johnson noise seems to be negligible in many cases (Manwani and Koch, 1999).

Besides these true noise sources there is another source of variability that is most likely dominating for neurons embedded in a network: the quasi-random input from other cells (Destexhe et al., 2003). In contrast to the aforementioned true noise sources, it is not *per se* clear what the input from other cells constitutes: mainly irregular uncontrollable fluctuations (London et al., 2010) or signals, possibly in a highly processed way (Stein et al., 2005; Droste et al., 2013; Masquelier, 2013). No matter how these fluctuations are interpreted, however, it appears reasonable that one may describe them in a stochastic framework and that the statistics of this irregular input is relevant for information transmission and processing in neural networks.

On the theoretical side, unstructured networks with random connections have been studied for a long time (Abbott and van Vreeswijk, 1993; Gerstner, 1995; van Vreeswijk and Sompolinsky, 1996; Brunel and Hakim, 1999; Fusi and Mattia, 1999; Brunel, 2000; Latham et al., 2000; Hansel and Mato, 2003; Leibold, 2004; Burkitt, 2006; Câteau and Reyes, 2006; Brunel and Hakim, 2008; Hennequin et al., 2012; Grytskyy et al., 2013; Ostojic, 2014). Besides various types of oscillatory and/or synchronous behavior, these networks typically also show asynchronous irregular firing if both excitatory and inhibitory connections are included, and excitation and inhibition in the network balance each other (van Vreeswijk and Sompolinsky, 1996). The irregular firing patterns observed in this asynchronous state resemble those of some cortical neurons seen in experiments (Bair et al., 1994; Compte et al., 2003).

An advanced mathematical treatment of stochastic activity in unstructured networks is based on the Fokker-Planck equation. The main assumption for this approach is that the input to the single cell can be described by white Gaussian noise, the mean and noise intensity of which is self-consistently determined by the firing rates of the neurons in the network. Put differently, the mean value and fluctuation intensity of the input spike trains reflect the statistics of the output and the connectivity of the network, where the latter is determined by the number and nature of the connections as well as the synaptic strengths. The Fokker-Planck approach has allowed for many insights into the transition between various states according to the emergence of oscillations and the degree of synchrony (Brunel and Hakim, 1999; Brunel, 2000; Brunel and Hansel, 2006). It has been recently extended to the study of strongly heterogeneous network states (Ostojic, 2014).

As mentioned above, a necessary approximation when using the Fokker-Planck approach in its simplest version is the assumption that the stimulus seen by a single neuron in the network is white Gaussian noise. This is usually justified by the diffusion approximation for a superposition of weakly correlated Poissonian spike trains. However, the spike trains generated by single neurons in the recurrent network are rarely Poissonian, i.e., they display a temporal correlation similar to the experimentally observed ones (Bair et al., 1994) or, equivalently, a non-flat spiketrain power spectrum. It is simple to show that the superposition of independent non-Poissonian spike trains inherits the correlations seen in the single spike train (Lindner, 2006). Furthermore, the non-Poissonian nature of spike trains can have severe consequences, e.g., for the output spike-train statistics (Ly and Tranchina, 2009; Schwalger et al., submitted) or for the propagation of signals in feedforward networks (Câteau and Reyes, 2006).

One way to deal with temporal correlations in the input is to extend the phase space of the Fokker-Planck equation by additional variables that can account for colored noise in the input. This has been done by Câteau and Reyes (2006) for the case of green noise (high-pass filtered noise) that arises by a presynaptic refractory period and it can be generalized and utilized to relate output spike-train statistics to temporal input statistics for a simple perfect integrate-and-fire neuron model (Schwalger et al., submitted). Another approach assumes a high degree of intrinsic or external uncorrelated noise that allows for a continuous rate-equation-like description of the activity in the neural network (see e.g., studies by Doiron et al., 2004; Lindner et al., 2005b; Pernice et al., 2011; Trousdale et al., 2012 for networks of integrate-and-fire neurons and the recent review by Grytskyy et al., 2013 for other network types). In this essentially linear description, a connection between input and output correlation matrix is easily derived but the main assumption of the approach, the linearization ansatz, is difficult to justify in general.

If the stochasticity of neural firing arises mainly from the network input, the following self-consistency problem emerges (cf. **Figure 1**). For any neuron randomly picked from a homogeneous recurrent network, the second-order statistics of the input spike

**FIGURE 1 | Basic problem addressed in this paper.** Excitatory (red) and inhibitory (blue) neurons interacting in a recurrent network (top) fire spike trains with a temporal correlation that can be characterized by the spike-train power spectrum. We focus on a homogeneous network, in the sense that excitatory and inhibitory neurons share the same firing rate and power spectrum. At the single-cell level (magnification at the bottom), a neuron is driven by a superposition of spike trains, the power spectra of which should be equal to the power spectrum of the neuron itself. This poses a self-consistency problem that we attempt to solve numerically in this paper in different approximations.

trains can be characterized by their input power spectra shown in the magnification **Figure 1** on the left. These spectra are in general not as flat as that of a (temporally uncorrelated) Poisson process. They should match the statistics of the neuron itself, in **Figure 1** represented again by the same non-flat power spectrum shown on the right. There are obvious generalizations possible if we think of different types of neural subpopulations of neurons sharing a common spike-train statistics (e.g., firing rates and power spectra), which all must be consistent with each other depending on the topology of the network. Even in the simple homogeneous version of the problem, the question has some interest on its own: What is the temporal correlation of a shot noise that would evoke a neural output with the same correlation statistics? Mathematically, it is not even clear whether such a solution exist and if so whether it is unique.

Lerchner et al. (2006) suggested a simple numerical procedure to determine a self-consistent autocorrelation statistics of spike trains in a sparse network using a Gaussian approximation. They focused on the case of network input that is balanced between excitation and inhibition and studied the dependence of the Fano factor on synaptic strength.

In this paper, we use similar self-consistent numerical procedures to determine the temporal correlations of single neuron activity in a sparse network of excitatory and inhibitory neurons in the asynchronous state. We employ exclusively the leaky integrate-and-fire (LIF) model (Lapicque, 1907; Gerstner and Kistler, 2002), that has been a standard choice in many studies of recurrent networks.

We use an iterative numerical scheme to determine the selfconsistent second-order statistics of spike trains in a recurrent neural network. In the *n*th step (henceforth referred to as the *n*th generation) of this procedure, we stimulate an LIF neuron in repeated trials with noisy input, the statistics of which is determined from the spike statistics of the previous generation. In order to generate the input, we employ two approximations. In one version, we use the ISI density of the LIF neuron to generate driving renewal spike trains for the next generation (renewal approximation). In an alternative version (equivalent to the original idea by Lerchner et al., 2006), we generate a Gaussian process that has the same power spectrum as the LIF spike train to generate the input for the next generation (Gaussian approximation).

For a parameter regime of balanced excitatory and inhibitory input from the previous generation, the spike-train power spectrum of the LIF neuron converges quickly over a small number of generations to a stationary spectrum. If the inhibitory component is too strong, however, our iterative scheme does not converge but displays strong oscillations in the firing rate as a function of the number of generations. As we will show, this instability can be understood already within the framework of the diffusion approximation.

We furthermore present results of extensive simulations for a sparse recurrent homogeneous network of excitatory and inhibitory LIF neurons, using parameters similar to the classical study by Brunel (2000). In the regime where our approximation scheme converges, we compare the power spectra to results from the renewal and Gaussian approximations. We find close agreement of power spectra for parameters of the Brunel setup for which the activity of neurons is asynchronous and the total input coming from the network is almost balanced. We conclude by discussing the implications of our results for a more faithful description of neural noise emerging in recurrent networks.

#### **2. MODELS AND METHODS**

#### **2.1. MODEL OF THE SINGLE NEURON AND SPIKE-TRAIN STATISTICS**

We consider a leaky integrate-and-fire model receiving an input current *I*(*t*) (here multiplied by the membrane resistance *R*) that obeys the following dynamics

$$
\tau \dot{\nu} = -\nu + RI(t). \tag{1}
$$

where the membrane time constant is chosen τ = 20 ms throughout this paper. Whenever the voltage reaches the threshold of *vT* = 20 mV, a spike time *ti* is registered, and after an absolute refractory period of τref = 2 ms the voltage is reset to a value *vR*, for which we use two different values (0 and 10 mV, see results). The current *I*(*t*) differs according to whether we consider a recurrent network or our self-consistent approximation schemes. In all cases considered, we numerically integrate Equation 1 with a simple Euler scheme using a time step of *t* = 0.1 ms. Please note that in all models studied in this paper, there is no Gaussian white noise, which would require a smaller time step.

The spike times defined by threshold crossings can be used to determine the statistics of the interspike interval (ISI) *Ii* = *ti* − *ti* <sup>−</sup> 1. The statistics inspected in this paper are (i) the mean interval -*Ii* (-· indicates an ensemble average ), which is related to the firing rate by ν = 1/-*Ii*; (ii) the coefficient of variation (CV)

$$C\_V = \frac{\sqrt{\langle (I\_i - \langle I\_i \rangle)^2 \rangle}}{\langle I\_i \rangle},\tag{2}$$

and (iii) the serial correlation coefficient among intervals that are lagged by an integer *k*:

$$\rho\_k = \frac{\langle (I\_i - \langle I\_i \rangle)(I\_{i+k} - \langle I\_{i+k} \rangle) \rangle}{\langle (I\_i - \langle I\_i \rangle)^2 \rangle}. \tag{3}$$

The neural spike train is represented by a sum of delta functions at the spike times

$$\alpha(t) = \sum\_{i} \delta(t - t\_i). \tag{4}$$

The spike-train power spectrum is computed from the Fourier transform of the spike train by

$$S(f) = \frac{\langle \tilde{\mathbf{x}} \tilde{\mathbf{x}}^\* \rangle}{T},\tag{5}$$

where the Fourier transform for the time window is defined by

$$
\tilde{\boldsymbol{x}}(f) = \int\_0^T dt e^{2\pi i \boldsymbol{f}t} \boldsymbol{x}(t). \tag{6}
$$

For the recurrent network, we assume that all neurons are statistically equivalent and that we can both average over realizations of initial conditions in the membrane voltage of the single cell *and* over different neurons when computing power spectra according to Equation 5 as well as all other spike-train measures employed in this work.

#### **2.2. RECURRENT-NETWORK MODEL**

We consider a connected random network of *N*<sup>E</sup> excitatory and *N*<sup>I</sup> = γ *N*<sup>E</sup> inhibitory LIF neurons as studied by Brunel (2000) in his model A. As the only topological constraint, excitatory and inhibitory neurons are assigned the same number *C*<sup>E</sup> and *C*<sup>I</sup> = γ*C*<sup>E</sup> of presynaptic excitatory and inhibitory neurons, respectively. Both neuron types follow the same single-cell dynamics; all parameter values of the LIF model are identical. This setup can still be regarded as homogeneous in the sense that in a large and sparse network power spectra of excitatory and inhibitory neurons should coincide, as verified numerically for all used parameter values. Spectra in recurrent networks presented here are averaged over 10<sup>3</sup> randomly picked neurons.

In the network simulations, the voltage variables *v* (*t*) with = 1, 2,..., *N*E(1 + γ ) all obey the same dynamics Equation 1 and fire-and-reset rule as explained above. The input current *I* (*t*) = *I* ,loc(*t*) + *I* ,ext(*t*) to the th neuron in Equation 1 consists of a local part

$$R I\_{\ell, \text{loc}}(t) = \tau \sum\_{j=1}^{C\_{\text{E}}} J \sum\_{i} \delta(t - t\_{\ell, i, j} - D)$$

$$-\tau \sum\_{j=1}^{C\_{\text{I}}} gJ \sum\_{i} \delta(t - t\_{\ell, i, j} - D), \tag{7}$$

comprising the input current from *C*<sup>E</sup> presynaptic excitatory and *C*<sup>I</sup> inhibitory neurons in the network. Here *g* is the relative strength of inhibitory amplitudes. The time instance *t* ,*i*,*<sup>j</sup>* denotes the *i*th spiking time of the *j*th presynaptic neuron of the th postsynaptic cell. The transmission delay is denoted by *D*. For the external input, we either consider a Poissonian background noise

$$RI\_{\ell, \text{ext}}(t) = \text{tr} \sum\_{j=1}^{C\_{\text{E}}} J \sum\_{i} \delta(t - t\_{\ell, i, j}) \,, \tag{8}$$

from an external population of excitatory neurons (to be consistent with Brunel, 2000) or a constant input current equal to the mean of the Poisson input:

$$R l\_{\text{ext}}(t) = C\_{\text{E}} l \nu\_{\text{ext}} \mathbf{r} \;. \tag{9}$$

We will use the constant external input current Equation 9 if not stated otherwise, in order to focus on noise (stochasticity) that is generated solely by the internal dynamics of the network itself.

For the standard network parameters, we follow (Brunel, 2000) by using γ = 0.25, *C*<sup>E</sup> = 103, *D* = 1.5 ms, *J* = 0.1 mV, but choose a larger network size of *N*<sup>E</sup> = 10<sup>5</sup> (Brunel, 2000 used *N*<sup>E</sup> = 104). Note that our value of γ implies that the input from the recurrent network is balanced if *g* = 4. Furthermore, we choose the constant external input such that *RI*ext(*t*) = 30 mV, which corresponds in Brunel (2000) to νext/νthr = 1.5, (νthr is the frequency of the external Poisson input needed to set the mean membrane potential to *vT* in the absence of local synaptic input). With this choice of parameters, the network is in the asynchronous firing regime for the range of values of *g* considered in our study (*g* ∈ [3.5, 5]).

#### **2.3. SELF-CONSISTENT DETERMINATION OF SPECTRAL STATISTICS**

The numerical procedure to determine the self-consistent spectral statistics uses essentially only a single model neuron in a number of repeated simulations. First, the neuron is stimulated by a combination of constant input and a Poisson process with given rate. A sufficient number of trials is carried out to reliably determine the output statistics of the neuron. This constitutes the output statistics of the *first generation* (the Poissonian drive can be regarded as the zeroth generation). In the next step we generate surrogate input to the neuron of the *second generation* according to one of the two approximations explained below. Again this is repeated for as many trials as required to obtain a reliable output statistics. The latter is used once more to generate surrogate data for the *third generation* and the whole procedure is repeated until the spike-train statistics converges, i.e., until the power spectrum of the *n*th generation does not differ significantly anymore from that of the (*n* − 1)th generation.

Our procedure is completely equivalent to simulating a feedforward network, in which layers correspond to the generations. There are two peculiarities compared to the usual setup of feedforward networks. First, in the way we approximate the input, all spatial correlations within a layer are neglected. Secondly, the number of layers is solely determined by the convergence of the spectra.

Because it is difficult to generate surrogate data with exactly the same statistics as the output of the previous generation, we employ two different approximations for the input, which are explained in the following subsections.

#### *2.3.1. Gaussian approximation for the input of the next generation*

As an extension of the diffusion approximation the local spiketrain input is approximated by a Gaussian noise η(*t*) ≈ *RI*loc(*t*), that is, however, not uncorrelated (white) as it would be in the diffusion approximation. The mean value is given by the constant current η = *C*E*J*(1 − *g*γ )ντ , which represents the average of the overall local input current. The power spectrum of the Gaussian noise equals the one of the summed spike trains of all presynaptic neurons. With the assumption of independent neurons this yields *S*η(*f*) = (*C*E*J*<sup>2</sup> + *C*I*g*2*J*2)τ <sup>2</sup>*Sx*(*f*), where *Sx*(*f*) is the spike-train power spectrum of the previous generation.

The approximation only requires to measure the power spectrum in each generation. Surrogate Gaussian input for the next generation that has this power spectrum can then be generated with standard algorithms (Billah and Shinozuka, 1990). Briefly, to generate a Gaussian time series η(*tj*) of *N* steps of size *t* with a prescribed power spectrum *S*η(*f*), we draw in Fourier space in each frequency bin two independent Gaussian random numbers η˜*r*(*fk*), η˜*i*(*fk*) with

$$\langle \tilde{\eta}\_m(f\_k) \rangle = 0, \ \langle \tilde{\eta}\_m(f\_k) \tilde{\eta}\_n(f\_\ell) \rangle = \frac{\delta\_{m,n} \delta\_{k,\ell}}{2\Delta f} \mathcal{S}\_\eta(f\_k), \ \ m, n \in \{r, i\} \tag{10}$$

where *fk* and *f* = (*Nt*)−<sup>1</sup> are center frequency and width of the *k*-th bin, respectively. By construction, the complex-valued sequence η˜(*fk*) = ˜η*r*(*fk*) + *i*η˜*i*(*fk*) is uncorrelated between frequency bins and has a variance proportional to the desired power spectrum. Transformation into the time domain (e.g., by fast Fourier transform) then yields the desired time series. Note that the Gaussian approximation assumes a high overall firing rate and a small synaptic efficacy (weight) and is expected to fail if one or only a few input spikes can already elicit an output spike.

The iterative procedure put forward by Lerchner et al. (2006) is similar in nature, drawing surrogate input statistics straight from the previous generation's autocorrelation function. As—unlike there—we consider uniform (i) firing thresholds and (ii) numbers of presynaptic excitary/inhibitory neurons, our LIF dynamics do not yield firing-rate heterogeneities in the network that would need to be accounted for. This in turn considerably simplifies the procedure and speeds up its numerical implementation.

#### *2.3.2. Renewal approximation for the input of the next generation*

For this approximation, we also measure the interspike interval histogram along with the spike-train power spectrum. This ISI histogram can be used to generate surrogate spike-train input for the next generation in form of renewal processes that have by construction the same interspike interval histogram. To this end, for each of the *C*<sup>E</sup> excitatory and *C*<sup>I</sup> inhibitory input spike trains, we assume an initial spike at *t* = −*T*<sup>0</sup> and draw in a large time window [−*T*0, *T*] a sufficient number of interspike intervals *Ii* such that *Ii* > *T* + *T*0. Partial sums of one, two, three etc. intervals then yield the first, second, etc spike time of the respective input renewal spike train. Although the intervals of different input spike trains are independent, all *C*<sup>E</sup> + *C*<sup>I</sup> renewal processes are initially synchronized by the common initial spike at *t* = −*T*0. To achieve a stationary asynchronous ensemble of renewal spike trains (Cox, 1962) , we use only the spike trains in the subinterval [0, *T*]. The necessary equilibration period *T*<sup>0</sup> can be estimated as *T*<sup>0</sup> ≈ (ν*C*<sup>2</sup> *<sup>V</sup>*)−<sup>1</sup> (for *CV* <sup>&</sup>lt; 1 as is the typical case in this paper).

As a smart alternative, we may start at *t* = 0 and use as the first spike time *t*<sup>1</sup> a sample of the so-called forward recurrence (FR) time, the probability density of which can be computed from the ISI density ρ(*t*) as follows (Cox, 1962)

$$
\rho\_{\rm FR}(t\_1) = \nu \int\_{t\_1}^{\infty} dt' \rho(t'). \tag{11}
$$

Thus, if we generate the first spike time *t*<sup>1</sup> from ρFR(*t*1) and all the following *ti* with *i* = 2, 3,... from drawing intervals according to ρ(*ti* − *ti*−1), we will also generate an equilibrium renewal spike train, avoiding the simulation period [−*T*0, 0] in the first method. We tested that both methods to generate an equilibrium renewal point process yield similar numerical results in our procedure.

Superposition of the *C*<sup>E</sup> excitatory renewal spike trains with amplitude *J* and *C*<sup>I</sup> spike trains with amplitude −*gJ* in the time window of [0, *T*] are used to stimulate the LIF model in the *n*th generation. Note that the superposition of the renewal spike trains is in general not a renewal process (Lindner, 2006) and thus there is no simple way to generate surrogate data for the superposition of the renewal spike trains directly instead of generating the single processes and summing them up. Although for special renewal processes (e.g., Gamma processes), efficient algorithms for the generation of such sums exists (Deger et al., 2012), our problem does not allow to specify the nature of the point process in advance. Hence, in particular for large *C*E, *C*I, the generation of renewal input becomes numerically inefficient.

We expect that the renewal approximation will work well if ISI correlations in the output spike train can be neglected. In contrast to the Gaussian approximation explained above there are no limitations regarding the spike amplitude and input rates. However, it is important to keep in mind that the renewal approximation cannot exactly yield what we are aiming at: a self-consistent second-order statistics because the generation of the surrogate data for the input is based on the ISI statistics and not on the second-order spike-train statistics. Only if also the output spike train is a renewal process, there is a unique relationship between power spectrum and ISI probability density (Stratonovich, 1967):

$$S(f) = \nu \frac{1 - |\tilde{\rho}(f)|^2}{|1 - \tilde{\rho}(f)|^2},\tag{12}$$

where ρ˜(*f*) is the Fourier transform of the ISI density ρ(*I*). By construction, Equation 12 yields the power spectrum for each of the surrogate input spike trains and is also proportional to the total sum of all independent input spike trains (Lindner, 2006). However, the power spectrum of the output spike train (sharing the same ISI density ρ(*I*)) does not obey Equation 12 unless all (linear but also non-linear) correlations among ISIs can be neglected. Briefly, output spectrum equals input spectrum only if the output spike train is also a renewal process. For finite ISI correlations, we can expect a discrepancy between the power spectrum of the surrogate input (superposition of renewal processes) and the power spectrum of the output spike train (which is in general non-renewal), even if our scheme has converged to a stationary output spike train. In contrast to the Gaussian approximation, it is more difficult to estimate when the renewal approximation will fail because it does depend on a property of the output (interval correlations) and not on the input (e.g., the size of amplitudes as for the Gaussian approximation).

#### *2.3.3. Convergence and uniqueness of the algorithms*

In general we consider Poisson spike trains as the input for the first generation. This input has only one parameter, the firing rate of the Poisson process. To see the difference between what we would obtain in the diffusion approximation, we use the firing rate determined in the network simulations below. The firing statistics of the first generation is then close to what we would expect to see in the diffusion approximation. Conveniently, differences between the converged power spectrum and the power spectrum of the first generation correspond to differences between the actual and the approximated output spectra in a theory based on the Poisson assumption.

We have tested in several cases that as long the procedure is stable (see below), the initial statistics does not matter and converged spectra are the same whether we start with asynchronous periodic input (*C*<sup>E</sup> + *C*<sup>I</sup> completely periodic spike trains with randomized initial spike times) or with Poisson input with a firing rate that differs significantly from the asymptotic value. So as far as numerical evidence in a limited parameter regime can tell, the procedure (if stable) converges to a unique spike-train power spectrum power spectrum.

One simple condition for the convergence of power spectra is that an even more essential statistics, the firing rate, converges. As our scheme can be regarded as a map, for which the firing statistics of the (*n* − 1)th generation determines that of the *n*th generation, we have to require that this map possesses a stable fixed point. Because the diffusion approximation captures this first-order statistics of the spike train fairly well (Brunel, 2000), we can employ the rate formula to estimate the map between the input rate (from the (*n* − 1)th generation) to the output rate (that of the *n*th generation by the well-known formula for the rate of a white-noise driven LIF neuron (Brunel, 2000)

$$\upsilon\_{\rm out} = \left(\tau\_{\rm ref} + \tau \sqrt{\pi} \int\_{\frac{\mu(\upsilon\_{\rm in}) - \upsilon\_{\rm R}}{\sqrt{2D(\upsilon\_{\rm in})}}}^{\frac{\mu(\upsilon\_{\rm in}) - \upsilon\_{\rm R}}{\sqrt{2D(\upsilon\_{\rm in})}}} d\varepsilon e^{z^2} \operatorname{erfc}(\mathbf{z})\right)^{-1} . \tag{13}$$

Here the constant input

$$
\mu(\nu\_{\rm in}) = \langle RI(t) \rangle = C\_{\rm E} l \tau \{ \nu\_{\rm ext} + (1 - \chi \, \mathbf{g}) \nu\_{\rm in} \} \tag{14}
$$

and the intensity of the white Gaussian noise

$$D(\nu\_{\rm in}) = \frac{C\_{\rm E}l^2 \pi (1 + \chi \,\text{g}^2) \nu\_{\rm in}}{2} \tag{15}$$

both depend on the input rate νin (note that we assumed just a constant external input, which does not contribute to the noise). A stable fixed point of the map should be characterized by the equality of input and output rates νout(νin) = νin, which become apparent as intersection points of the graph νout(νin) with the diagonal. Additionally, we have to require at this intersection point a slope |*d*νout/*d*νin| < 1 to ensure that small perturbations in the firing rate decay.

#### **3. RESULTS**

#### **3.1. SELF-CONSISTENT SPECTRUM USING TWO DIFFERENT ITERATIVE SCHEMES**

We begin with an example for which our procedure leads to a stable stationary output spike-train statistics (in terms of firing rate and spike-train power spectrum) and where both approximations yield very similar spectra. In **Figure 2** we show the power spectra of the selected generations using the renewal approximation in A, the Gaussian approximation in B, and compare the asymptotic spectra of both models in panel C. The number of presynaptic neurons corresponds here to the standard values of *C*<sup>E</sup> = 1000, *C*<sup>I</sup> = 250 used by Brunel (2000), while the strength of inhibition *g* = 4 is chosen such that the network input is balanced.

In the first generation the neuron is stimulated by a Poisson spike train with a self-consistent firing rate<sup>1</sup> according to the stable fixed point of Equation 13. The power spectrum of the first generation (solid lines in **Figure 2**) gives us what we would expect in the Poisson approximation of neural background activity: a spectrum with reduced power at low frequency, indicative of a stochastic process that one may refer to as a "green noise" (Guz and Sviridov, 1998). This spectrum agrees remarkably well with the analytical expression of the power spectrum of a white-noise driven LIF (Lindner et al., 2002) with the effective base current and noise intensity given by Equation (14) and (15), respectively (not shown).

On the contrary, the self-consistent power spectrum of the 15th generation is a narrow-band noise with strong peaks around frequencies equal to the firing rate or multiples of it. In the self-consistent picture, the neuron of the 15th generation is not

presynaptic environment (*C*<sup>E</sup> = 103, *C*<sup>I</sup> = 250) both the renewal approximation **(A)** and the Gaussian approximation **(B)** have converged to unique stationary spectra, which are compared in **(C)**. In the first generation, the neuron is stimulated by a constant input -*RI*(*t*) = 30 mV and *C*<sup>E</sup> + *C*<sup>I</sup> Poissonian spike trains of rate νin = 71 Hz [solution for the self-consistent firing rate Equation (13)] with amplitude *J* = 0.1 mV (excitatory synapses) and −*gJ* = −0.4 mV (inhibitory synapses). Note the rapid convergence of spectra for both approximations: the spectrum of the fifth generation differs only slightly from the result for the 15th generation.

driven by a spectrally flat noise but by a narrow-band noise with power around its firing rate that apparently leads to a much more regular spike train than an uncorrelated noise (the Poisson spike train) does.

In **Figures 3A,B** we show the rate and the CV as functions of the generation, respectively. We use Poisson processes to generate the input to the first generation, once with a firing rate close to the asymptotic one (νin = 71 Hz), once with a substantially lower rate (νin = 15 Hz). Apparently, the converged statistics after 15 generations do not depend on the initial value of the rate. While the firing rate does not change much over the generations, the CV drops from a value of 0.5–0.2. Hence the diffusion approximation (equivalent to the statistics of the first generation) leads to a reliable estimate of the self-consistent value of the firing rate but not of the CV. This discrepancy was already evident by looking at the power spectra at low frequency, which is largely determined by

<sup>1</sup>The resulting power spectrum of this first generation can be considered as the asymptotic spectrum resulting from an iterative scheme, in which we approximate the input as Poissonian spike trains that are solely determined by the output rate of the previous generation.

the CV according to *S*(0) = ν*C*<sup>2</sup> *<sup>V</sup>* (true only for a renewal process). **Figure 3F** illustrates the reason for the rapid convergence of our procedure over the generations in terms of the map for the firing rate Equation 13. The shallow dependence of the firing rate curve νout(νin) around the intersection point with the diagonal shown in **Figure 3F** implies that any initial perturbation from the fixed point (indicated by the magenta spot) approaches the fixed point monotonically over only a few generations (blue arrows).

Interestingly, although renewal and Gaussian approximations yield similar results for rate, CV, and power spectra, they differ in the stationary value of the first serial correlation coefficient ρ1, displayed in **Figure 3C**. This value is positive for the considered parameter set but in the renewal scheme we obtain only half the value of the correlation coefficient, which is observed when we use the Gaussian approximation and which is also close to the value observed in network simulations.

The map for the firing rate can also be used to understand why our procedure does not work for very strong inhibition. This case is illustrated in **Figures 3D,E** for *g* = 5, for which we observe oscillations in rate and CV that grow in amplitude over the generations (no instabilities are observed in the recurrent network for these parameters). Here the map for the firing rate still has

a fixed point but it is an unstable one, i.e., |*d*νout(νin)/*d*νin| > 1 at the fixed point and small perturbations from the fixed point grow in amplitude (cf. **Figure 3G**). Interestingly, the stability also depends on the size of the presynaptic environment, even if we fix the mean input from the previous generation because the number of synapses also determine the effective noise level Equation 15. For instance, a smaller presynaptic environment with *g* = 5, *C*<sup>I</sup> = 100, *C*<sup>I</sup> = 25 and *J* = 1 mV (leading to the same mean input as our standard parameter), the slope of the firing rate curve is still negative but its absolute value is smaller than one. Hence, here our procedure still yields a self-consistent spike-train power spectrum in this case (not shown).

For the parameter set in **Figure 2** both approximations yielded the same power spectrum because their respective assumptions, i.e., small amplitudes *J* for the Gaussian approximation and independence of ISIs for the renewal approximation, were sufficiently closely matched. Below in Section 3.3 we will show two examples, for which the two approximations result in visibly distinct power spectra because one of their respective assumptions is not obeyed.

## **3.2. SPECTRA IN RECURRENT NETWORKS**

We would like to compare our results for self-consistent spectra to those measured in a recurrent neural network. Here we use the model by Brunel (2000) (more specifically, Brunel's model A) within the parameter regime of asynchronous activity. Because we want to focus on the sparse limit of the model, in which input correlations can be neglected, we choose as a standard network size *N*E(1 + γ ) = 1.25 · 10<sup>5</sup> (exceeding the one used by Brunel, 2000 by a factor of 10). We clarify to what extend spike-train power spectra in the recurrent network depend on the transmission delay, the network size, and whether they are robust with respect to external noise.

In the approximation schemes introduced above, there is no synaptic delay *D*: the statistics of the (*n* − 1)th generation are measured during an ideally very large time interval, and stationary stochastic input with similar statistics is used to stimulate the neuron model in the *n*th generation; introducing here a delay would have no consequences for the spike statistics of the *n*th generation at all. In contrast, in the recurrent network, degree and character of synchrony in firing patterns depend strongly on *D* (Brunel, 2000).

It turns out, however, that for a range of synaptic delays in the asynchronous firing regime, single-neuron statistics are

**FIGURE 4 | Delay dependence of power spectra in recurrent networks.** Asynchronous regime. Parameters: *g* = 4.5, and *v*<sup>R</sup> = 10 mV.

independent of the particular choice of *D*. As shown in **Figure 4**, power spectra for delays that differ by a factor of 6 are very close to each other. Thus, in the iterative procedure, a self-consistent determination of the power spectrum in the asynchronous regime is possible without incorporating a delay *D*.

Apart from the synaptic delay, also the network size does not explicitly appear as a model parameter in our iterative procedures. However, implicitly we have assumed in both approximations that cross-correlations can be neglected, which in the recurrent network can be achieved (if at all) by a large network size. Hence, size is a concern and we have to check, how spike-train power spectra change as we change the system size at fixed number of input neurons. This is illustrated in **Figure 5**, where the smallest system size *N*<sup>E</sup> = 2000 implies with *C*<sup>E</sup> + *C*<sup>I</sup> = 1250 a non-negligible overlap of input neurons for any two neurons and thus significant cross-correlations among the neurons. However, for network sizes *N*<sup>E</sup> = 10<sup>4</sup> (as used by Brunel, 2000) and *N*<sup>E</sup> = 105, the spike train power spectra look very similar, justifying the choice of *N*E,<sup>I</sup> used in the following.

Two more features of the system are inspected in **Figure 6**: the robustness to external input and the dependence of spectra on the spike amplitude *J*. With respect to the latter, we use, besides our standard choice *J* = 0.1 mV with *C*<sup>E</sup> = 1000, *C*<sup>I</sup> = 250, also a larger amplitude of *J* = 1 mV with a reduced number of presynaptic neurons (*C*<sup>E</sup> = 100, *C*<sup>I</sup> = 25) such that the mean input from the network remains the same. Note that our change of parameters is different to that considered by Ostojic (2014), because we reduce the number of synapses when increasing the amplitude, avoiding in this way strong fluctuations in the population rate as seen by Ostojic (2014). Increasing the amplitude in our setting has the main effect of increasing the noisy input for the single neuron, which leads in our setup to a bursting behavior that becomes apparent by increased power at low frequency. Replacing the constant input current μ with an external Poissonian stimulus of the same mean generally does not alter the firing regime (Brunel, 2000) because this noise is only small compared to that coming from the recurrent network. In fact, for *J* = 1 mV the spectra with external Poisson spikes (dashed magenta line in **Figure 6**) and with a constant input of the same mean (blue line) do not differ at all. The effect of such an external noise is more visible for our standard choice: peaks in the power spectrum (dashed orange line) become wider and

the power at low frequency is increased compared to the spectrum with a constant external input current (red line). These are expected effects of external white noise on the power spectrum of a spike generator in the mean-driven regime (see e.g., Lindner et al., 2002).

After we have seen that power spectra in the recurrent network neither depend on the specific value of delays (as long as the existence of the asynchronous regime is ensured) nor on network size (as long as it is large enough), and that they do not change drastically with additional external noise, we turn now to the comparison of network spectra with the spectra from the self-consistent procedure.

#### **3.3. COMPARISON OF SPECTRA IN RECURRENT NETWORKS AND THE SELF-CONSISTENT SOLUTION**

Besides the comparison to the results of our iterative scheme, we use this section also to additionally inspect the variation of another parameter, the reset potential *vR*. So far we have chosen *VR* = 10 mV in accordance with (Brunel, 2000), corresponding to a voltage that is reset between the resting potential and the threshold. This is a reasonable choice for some cortical cells (Koch, 1999), but a reset closer to the equilibrium potential may be also appropriate for others. Hence, it is of interest how power spectra and also how our approximation schemes for them may depend on the choice of *vR*. We will thus use in all plots of this subsection *vR* = 0 mV as an alternative setup. Based on previous work (Vilela and Lindner, 2009) we can expect that with this value of the reset, we will observe a lower firing rate and also a lower CV than for *vR* = 10 mV.

Our main parameter to vary in the following is the relative strength of inhibition *g*. We start with a value of *g* = 3.5 (see **Figure 7**), which is close to the border of synchronization (Brunel, 2000). For *g* = 3.5, the spectra reveal strong peaks, i.e., although neurons still fire asynchronously, their spike trains are rather regular. The Gaussian approximation leads in our self-consistent procedure to a spectrum that captures the spike-train power spectrum of a neuron in the recurrent network very well. This holds true for both a reset value of *vR* = 0 mV (panel A) and for *vR* = 10 mV (panel B). As anticipated above the firing rate is higher for

*vR* = 10 mV and, consequently, also spectral peaks are located at higher frequencies.

In contrast to the Gaussian approximation, for *g* = 3.5 the renewal approximation used in our iterative scheme does not yield a power spectrum that closely matches the spectrum in the recurrent network. Peaks appear here at a somewhat higher frequency, and the neuron also fires at a somewhat higher rate. This discrepancy can be traced back to a non-renewal behavior indicated by the positive ISI correlations at lag one and two (cf. inset in **Figure 7B** for *vR* = 10 mV).

As we increase the relative strength of inhibition to *g* = 4, both approximations agree well with the spectrum measured in the recurrent network if we use the reset voltage of *vR* = 10 mV (cf. **Figure 8B**). This is not totally unexpected because for these parameters we found already an agreement of both approximations in **Figure 2C**. Because of the complementary assumptions made in the two approximations, an agreement of their self-consistent spectra is a strong hint that they both should work—**Figure 2C** can be taken as a confirmation of this.

Interestingly, if we choose the reset value at *vR* = 0 mV (cf. **Figure 8A**) and thus make the spike trains more regular, the renewal approximation shows again some disagreement with the power spectrum of recurrent network neurons. The Gaussian approximation, on the contrary, yields once more the correct spectrum.

For *g* = 4.5 (**Figure 9**), both renewal and Gaussian approximations agree with each other and with the network spectra, regardless of the value of the reset voltage. One might be tempted to think that this agreement is achieved because the spike-train statistics are closer to a renewal process. However, for this case we observe in both approximations as well as in the network simulations ISI correlations of the same order of magnitude as in **Figure 7**—only that correlations are negative in **Figure 9**, whereas they were positive in **Figure 7**.

What causes the failure of the renewal approximation in some of the cases considered above? Generally, it has become clear

**FIGURE 8 | Power spectra for balanced network input.** Results of recurrent network simulations and the two approximations from our iterative schemes for *g* = 4 (balanced local synaptic input). **(A)** *v*<sup>R</sup> = 0 mV [firing rate ν = 44.5 Hz, *CV* = 0.05] and **(B)** *v*<sup>R</sup> = 10 mV (firing rate ν = 70.4 Hz, *CV* = 0.19).

recurrent network simulations and the two approximations from our iterative schemes for *g* = 4.5 (inhibitory local synaptic input). **(A)** *v*<sup>R</sup> = 0 mV (firing rate ν = 28.1 Hz, *CV* = 0.088); **(B)** *v*<sup>R</sup> = 10 mV (firing rate ν = 34.8 Hz, *CV* = 0.27). Inset shows serial correlation coefficients for **(B)**.

that the self-consistent noisy current stimulus is temporally structured. It is a colored noise, that in general leads to a non-renewal spike train of the driven neuron model (Middleton et al., 2003; Lindner, 2004). In particular, a narrow-band noise as we observed in **Figures 7**, **8A** can lead to pronounced interval correlations (Bahar et al., 2001; Neiman and Russell, 2005; Bauermeister et al., 2013). Differences between the spectra in the recurrent network and that of our self-consistent renewal scheme are therefore not completely unexpected. Surprising is that the renewal scheme still works in some cases in which we saw pronounced ISI correlations.

So far, the Gaussian approximation worked well for all chosen parameters inspected. The reason for this is the small amplitude of postsynaptic potentials (*J* = 0.1 mV) we have used in all simulations. We finally show and discuss a case with a larger amplitude

(*J* = 1 mV in **Figure 10**). Because we do not want to change the mean input to the cell, we also reduce the number of synapses by a factor of 10. With a smaller number of synapses and a larger synaptic amplitude, we increase the noise in the system, which changes the shape of the power spectrum drastically, in particular, for the reset value of *vR* = 10 mV (**Figure 10**). Our main point with **Figure 10**, however, is that the renewal scheme in this case leads to a spectrum that is somewhat closer to the spectrum in the network than the Gaussian approximation for both values of the reset voltage. In this case, the assumption of the Gaussian approximation seems to be more severely violated than the assumptions of the renewal approximation are.

*v*<sup>R</sup> = 10 mV (firing rate ν = 65.7 Hz, *CV* = 1.98).

## **4. DISCUSSION**

The efforts in this paper were directed toward a better understanding of temporal correlations in recurrent neural networks. Here we focused on the simple case of a sparse homogeneous network, in which the autocorrelation of a single spike train is the only relevant temporal correlation of interest, i.e., crosscorrelations between neurons can be neglected. To this end we introduced and compared two iterative simulation schemes, one of which is a simplified and numerically more efficient version of the framework put forward by Lerchner et al. (2006). Both simulation schemes correspond to an infinitely sparse network, because all input spike trains are completely independent of each other and only share the same statistics. In this way we escaped from the trap of complete synchronization, seen previously in finite layer-feedforward networks (Wang et al., 2006), which does not adequately describe the asynchronous state in a recurrent network.

Our results demonstrate in line with previous results by Lerchner et al. (2006) that the power spectrum of a single neuron in an unstructured network in the asynchronous state may be determined in some cases in a self-consistent approximation using iterative simulations of essentially *only one neuron*. We offer, to the best of our knowledge, the first comparison of such selfconsistent spectral statistics with the respective statistics of the stationary state of the approximated LIF network. Moreover, we obtain strong numerical evidence from network simulations that these statistics do not vary with (a change of an uniform) synaptic delay as long as the latter yields an asynchronous state.

We showed that the iterative schemes (be it renewal or Gaussian one) do not work for too strong inhibition, because here instead of approaching a stationary spectrum, already the firing rate becomes unstable (an instability that is not present in the recurrent network), preventing a self-consistent determination of the spectrum. On the other hand, both schemes fail in any case for a non-sparse configuration (i.e., if *C*E/*N*<sup>E</sup> 1 is not obeyed), because cross-correlations between input neurons cannot be neglected anymore and, hence, approaches solely based on single-neuron statistics cannot reproduce the correct power spectrum as measured in the recurrent network. Finally, our approach requires that even in the sparse network no synchronization emerges. This implies e.g., that we cannot reproduce the spike-train power spectrum for *g* < 3 (dominance of excitatory input coming from the network), for which strong synchronization is observed (Brunel, 2000). Preliminary simulation results for the recurrent network indicate that upon the transition to the synchronous regime, a peak at the population frequency arises in otherwise unchanged spectra.

In the important case of asynchronous activity close to the balanced state, the two methods to generate surrogate input work best in distinct limits. The Gaussian approximation can accurately predict the spike-train statistics of a neuron in the recurrent network as long as the synaptic amplitudes are sufficiently small. The renewal approximation in turn shows systematic deviations because of ISI correlations that are typical if a neuron is driven by a correlated noise. It may work better than the Gaussian approximation and the similar framework of Lerchner et al. (2006), however, if the amplitude is larger, e.g., for a value of *J* = 1 mV that is still within the physiological range (Koch, 1999). Here the renewal approximation performs better because it maintains the pulsatile nature of spike-train input (shot noise). Because it is known that the shot-noise character of synaptic input may affect firing rate and response properties substantially (Richardson and Swarbrick, 2010), this limit of larger amplitudes is worthwhile additional exploration. In particular, more elaborate generators for surrogate spike trains with prescribed second-order statistics (Brette, 2009) should be employed in this case.

From a more abstract point of view, our self-consistent scheme boils down to the question of finding an input stimulus, temporally correlated in such a way that it evokes in a non-linear neuron model a spike train with the very same temporal correlation. It is clear that without further constraints, this problem could have several solutions. Here we showed that under the special constraint of a Gaussian input statistics, the iterative scheme converges in a parameter regime close to the so-called balanced state to a unique second-order statistics. At the moment it is not clear how one could prove the existence and uniqueness of the solution mathematically.

The iterative scheme for the determination of self-consistent spectra can be extended in several directions. For the sake of comparison to the classical study by Brunel (2000), we used in this paper current-based synapses with an instantaneous spike input. Preliminary results show that the self-consistent determination can be applied to neurons with conductance-based instead of current-based input, including a first-order kinetics (low-pass filter) for the synapse. Another manageable extension would be to consider two populations of excitatory and inhibitory neurons with different synaptic and neural parameters, which in our framework would imply to simulate a single excitatory and a single inhibitory neuron receiving different inputs. Last but not least, it appears conceivable to determine the self-consistent cross-correlations between two neurons in iterative scheme(s) that employ simulations of *two uncoupled neurons* that receive correlated input characterized by the power and cross-spectra of the previous generation. Whether such a scheme can successfully reproduce the spike statistics may also depend on the specific connectivity and, in particular, on the amplitude of synaptic spikes, as it has been recently shown that in networks in which the number of presynaptic neurons scales with the network size, so-called spike echos additionally shape neural cross-correlations (Helias et al., 2013).

Our results can be regarded as a further step toward a more general theory of biological neural networks that takes the temporal structure of neural activity in the network more faithfully into account. Although in many instances, the Poissonian approximation may give a lot of insights and even a network of Poisson neurons may turn out to approximate the recurrent network reasonably well (Ostojic, 2014), there are also examples where exactly the temporal structure of spike trains matters (Câteau and Reyes, 2006). Helpful for analytical approaches would be formulas for the spectral spike-train statistics of simple IF models, which are driven by an Gaussian noise with an arbitrary (in particular, nonflat) power spectrum. Once a formula is known that provides the output spike-train power spectrum as a functional of the input power spectrum of the stimulating Gaussian noise, this functional could be regarded as a map, the fixed point(s) of which would yield the stationary solution(s) of our numerical procedure. So far, however, the problem of the spike statistics of a general IF model driven by arbitrary colored noise is an open issue in computational neuroscience. Most efforts have been focussed on the special case of low-pass filtered input noise (see e.g., Brunel and Sergi, 1998; Brunel et al., 2001; Brenner et al., 2002; Moreno-Bote and Parga, 2006; Alijani and Richardson, 2011) and only rarely more general forms of input correlations have been addressed analytically (Câteau and Reyes, 2006; Bauermeister et al., 2013).

Progress along these lines may nevertheless be possible, as a recent study by Schwalger et al. (submitted) illustrates: for the special case of a perfect IF model driven by a weak Gaussian noise with arbitrary input correlations, there exists a simple relation between the input correlation function (i.e., the Fourier transform of the input power spectrum) and the output spiketrain statistics such as the ISI probability density and the interval correlations. Other approaches for IF neurons with threshold noise (Lindner et al., 2005a) or for threshold-crossing devices (Tchumatchenko et al., 2010) also permit approximations for the relation between spike-train power spectrum and colored Gaussian input noise. Results like these may be useful to calculate the self-consistent power spectrum in a recurrent network at least semi-analytically by finding the fixed point of the non-linear relation between input and output correlation functions.

## **ACKNOWLEDGMENT**

We acknowledge help by and discussions with Tilo Schwalger at an early stage of this study. This work was supported by Bundesministerium für Bildung und Forschung grant 01GQ1001A.

## **REFERENCES**


Destexhe, A., Rudolph, M., and Paré, D. (2003). The high-conductance state of neocortical neurons *in vivo*. *Nat. Rev. Neurosci.* 4, 739–751. doi: 10.1038/nrn1198


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 25 May 2014; accepted: 13 August 2014; published online: 18 September 2014.*

*Citation: Dummer B, Wieland S and Lindner B (2014) Self-consistent determination of the spike-train power spectrum in a neural network with sparse connectivity. Front. Comput. Neurosci. 8:104. doi: 10.3389/fncom.2014.00104*

*This article was submitted to the journal Frontiers in Computational Neuroscience.*

*Copyright © 2014 Dummer, Wieland and Lindner. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Structured chaos shapes spike-response noise entropy in balanced neural networks

## *Guillaume Lajoie1,2,3\*, Jean-Philippe Thivierge4 and Eric Shea-Brown3,5*

*<sup>1</sup> Nonlinear Dynamics Department, Max Planck Institute for Dynamics and Self-Organization, Goettingen, Germany*

*<sup>2</sup> Bernstein Center for Computational Neuroscience, Max Planck Institute for Dynamics and Self-Organization, Goettingen, Germany*

*<sup>4</sup> School of Psychology and Center for Neural Dynamics, University of Ottawa, Ottawa, ON, Canada*

*<sup>5</sup> Physiology and Biophysics Department, University of Washington, Seattle, WA, USA*

#### *Edited by:*

*Benjamin Lindner, Bernstein Center for Computational Neuroscience, Germany*

#### *Reviewed by:*

*Gianluigi Mongillo, Paris Descartes University, France Peter J. Thomas, Case Western Reserve University, USA*

#### *\*Correspondence:*

*Guillaume Lajoie, Nonlinear Dynamics Department, Max Planck Institute for Dynamics and Self-Organization, Am Faßberg 17, Goettingen D-37077, Germany e-mail: glajoie@nld.ds.mpg.de*

Large networks of sparsely coupled, excitatory and inhibitory cells occur throughout the brain. For many models of these networks, a striking feature is that their dynamics are chaotic and thus, are sensitive to small perturbations. How does this chaos manifest in the neural code? Specifically, how variable are the spike patterns that such a network produces in response to an input signal? To answer this, we derive a bound for a general measure of variability—spike-train entropy. This leads to important insights on the variability of multi-cell spike pattern distributions in large recurrent networks of spiking neurons responding to fluctuating inputs. The analysis is based on results from random dynamical systems theory and is complemented by detailed numerical simulations. We find that the spike pattern entropy is an order of magnitude lower than what would be extrapolated from single cells. This holds despite the fact that network coupling becomes vanishingly sparse as network size grows—a phenomenon that depends on "extensive chaos," as previously discovered for balanced networks without stimulus drive. Moreover, we show how spike pattern entropy is controlled by temporal features of the inputs. Our findings provide insight into how neural networks may encode stimuli in the presence of inherently chaotic dynamics.

**Keywords: neural variability, chaotic networks, neural excitability, network dynamics, spiking stimulus responses**

## **1. INTRODUCTION**

If a time-dependent signal is presented to a network whose dynamics are chaotic and whose initial conditions cannot be perfectly controlled, how much variability can one expect in its responses? Such a scenario is central to questions of stimulus encoding in the brain.

In this article, we study population level spiking responses in a neural network model with sparse, random connectivity and *balanced* excitation and inhibition. Such models are ubiquitous in neuroscience, and reproduce the irregular firing that typifies cortical activity. Moreover their autonomous activity is known to be chaotic, with extremely strong sensitivity of spike outputs to tiny changes in a network's initial conditions (van Vreeswijk and Sompolinsky, 1998; London et al., 2010; Sun et al., 2010). Remarkably, in these autonomous systems, the chaos is invariant to the network scale (i.e., it is *extensive*): the same spectrum of Lyapunov exponents recurs regardless of network size, even when coupling remains localized (Monteforte and Wolf, 2010; Luccioli et al., 2012). Our goal is to add a stimulus drive, and understand the implications for the network spike patterns that result—a task made challenging by the fact that spikes are related to phase space dynamics in a highly non-linear way.

Intriguingly, when such chaotic networks respond to timedependent signals, they produce spiking that is less variable than one might expect (c.f. Molgedey et al., 1992; Marre et al., 2009; Rajan et al., 2010). In recent theoretical work, this has been attributed to low-dimensional chaotic attractors that "project" only intermittently to produce variable spiking in any given single cell (Lajoie et al., 2013). It is unclear how such chaos-induced "noise" affects neural activity in the brain. However, chaotic dynamics appears to be a general attribute of many large models of recurrent networks, a phenomenon that likely constrains biological network dynamics. Furthermore, stimulus-evoked spike data similar to that of chaotic models has been experimentally observed *in vivo*, where fluctuating sensory stimuli are repeatedly presented to an animal. Here, cortical neurons produce spikes with a wide range of variability, with some spikes repeatedly evoked with millisecond precision (Reinagel and Reid, 2000; Yang et al., 2008). Information theoretic methods suggest that this type of "intermittent noise" may permit information to be encoded in the spike patterns that single neurons produce over time (Reinagel and Reid, 2000; Tiesinga et al., 2008).

However, the impact of variability on network coding cannot be understood by extrapolating from single cells alone (Zohary et al., 1994; Abbott and Dayan, 1999; Averbeck et al., 2006; Schneidman et al., 2006; Ecker et al., 2011; Hu et al., 2014). Thus, to eventually understand how network chaos impacts coding, we need to capture the *multicell* spike train variability in chaotic networks—and relate this to well-quantified measurements at the level of single cells. Direct, sampling-based approaches to this

*<sup>3</sup> Applied Mathematics Department, University of Washington, Seattle, WA, USA*

problem will fail, due to the combinatorial explosion of spike patterns that can occur in high-dimensional networks. Another method is needed.

Studies of variability in recurrent networks typically address two distinct properties. On one hand, there is the question of spike-timing variability, often measured by binarized spike pattern entropy and usually studied for single cells or small cell groups (Strong et al., 1998; Reinagel and Reid, 2000; Schneidman et al., 2006). On the other hand, recent theoretical work investigates the dynamical entropy production of entire networks, quantifying the state space expansion globally (Monteforte and Wolf, 2010; Luccioli et al., 2012). It is not clear how these two quantities are related. Here, we extend the work of Lajoie et al. (2013) to bridge this gap, leveraging random dynamical systems theory to develop a direct symbolic mapping between phase-space dynamics and binary spike pattern statistics.

The result is a new bound for the variability of joint spike pattern distributions in large spiking networks that receive fluctuating input signals. This bound is in terms of spike-response noise entropy, an information-theoretic quantity that is directly related to dynamical entropy production. By verifying that the previous extensivity results of Monteforte and Wolf (2010) and Luccioli et al. (2012) continue to hold in the presence of stimulus drive, we show how the bound applies to networks of all sizes, and only depends on input statistics and single-cell parameters.

We then apply this bound to make two observations about the spike-pattern variability in chaotic networks. The first is that the joint variability of spike responses across large networks is at least an order of magnitude lower than what would be extrapolated from measurements of spike-response entropy in single cells, despite noise correlations that are very low on average. Second, we show that the spike-response entropy of the network as a whole is strongly controlled by the tradeoff between the mean (i.e., DC) and higher-frequency components of the input signals. Entropy increases monotonically with the mean input strength by almost an order of magnitude, even as network firing rates remain constant.

#### **2. MATERIALS AND METHODS**

#### **2.1. MODEL**

To develop these results, we use large random networks of *N* Quadratic Integrate-and-Fire (QIF) model neurons, as in Monteforte and Wolf (2010) and Lajoie et al. (2013). This single neuron model captures the normal form dynamics of Type I neurons, as found in cortex (Ermentrout, 1996). Moreover, we make use of a smooth change of coordinates that maps QIF hybrid dynamics to a phase variable on the unit circle (see Ermentrout, 1996 and appendix of Lajoie et al., 2013). This cell model is known as the "θ-neuron" and eliminates the need for artificial reset after a spike. This results in smooth dynamics with dimensionless units, a feature which will prove crucial for analysis (see **Figure 1A**). For reference, in a QIF model neuron with a time constant τ = 10 ms, one *time-unit* in the θ-coordinates corresponds to about 125 ms.

The state of each cell in the network is represented by a phase variable θ*i*(*t*) ∈ [0, 1] (*i* = 1,..., *N*) where 0 and 1 are identified (i.e., *S*1) and a spike is said to occur when θ*<sup>i</sup>* = 1 ∼ 0. In addition to internal dynamics which depend on coupling between neurons, the network receives a temporally structured input signal *I*(*t*), as described below.

The dynamics of the *i*th cell in the network are given by the equation

$$d\theta\_i = [F(\theta\_i) + Z(\theta\_i) \sum\_{j=1}^{N} a\_{ij} \mathbf{g}(\theta\_j) + \frac{\varepsilon^2}{2} Z(\theta\_i) Z'(\theta\_i)] dt \dots$$

$$+ Z(\theta\_i) \underbrace{[\eta dt + \varepsilon dW\_{i,t}]}\_{I\_i(\mathbf{t})dt} \tag{1}$$

where *F*(θ*i*) = 1 + cos (2πθ*i*), *Z*(θ*i*) = 1 − cos (2πθ*i*) and

representation of a network driven by a quenched collection of

mark spike times.

$$\mathcal{g}(\theta\_{\hat{l}}) = \begin{cases} d\left(b^2 - \left[\left(\theta\_{\hat{l}} + \frac{1}{2}\right) \text{mod } 1 - \frac{1}{2}\right]^2\right)^3; \theta\_{\hat{l}} \in \left[-b, b\right],\\ 0 & \text{;else} \end{cases}$$

is a smooth coupling function with small support around θ*<sup>j</sup>* = 1 ∼ 0, mimicking the rapid rise and fall of a synaptic current (*b* = 1/20, *d* = 35/32). The ε<sup>2</sup> term comes from an Ito correction (Lindner et al., 2003).

The network's input *I* = {*Ii*} *N <sup>i</sup>* <sup>=</sup> <sup>1</sup>, represented by the last term in (1), models a temporal stimulus. It is a collection of *N* independent signals *Ii*(*t*) = η + ε*dWi*,*t*/*dt* driving each neuron respectively, where the *dWi*,*t*/*dt* are quenched realizations of white noise—that is, scaled increments of the independent Wiener processes *Wi*,*<sup>t</sup>* (see **Figure 1B**). Note that the input's mean η controls the network's "excitability" and can take negative values (Ermentrout, 1996) while ε ≥ 0 controls the amplitude of input fluctuations. Both parameters are constant across all cells. We begin by investigating network (1) in the excitable regime with parameters η = −0.5 and ε = 0.5. **Figure 1C** shows an example trajectory of an isolated neuron θ*<sup>i</sup>* in this regime, driven only by its input *Ii*(*t*). Model (1) has been analyzed previously for uncoupled neurons (Ritt, 2003; Lin et al., 2009a), and for a series of gradually more complex networks in Lin et al. (2009a,b); Lajoie et al. (2013) (cf. Monteforte and Wolf, 2010).

We assign 20% of the *N* neurons to be inhibitory and 80% to be excitatory, meaning that outgoing weights of neuron *j* are either *aij* ≤ 0 or *aij* ≥ 0 respectively. The coupling matrix *A* = {*aij*}*i*,*<sup>j</sup>* <sup>=</sup> <sup>1</sup>,...,*<sup>N</sup>* is chosen randomly with mean *in-degree* κ such that each neuron receives on average κ incoming connections from independently chosen neurons, from each excitatory/inhibitory population. Here, |*aij*| ∼ *O*(1/ <sup>√</sup>κ) when non-zero, in accordance with classical *balanced state* coupling (van Vreeswijk and Sompolinsky, 1998). Throughout, we set κ = 20 (|*aij*| 0.2) but find that as long as κ *N*, our findings are qualitatively robust to the choice of κ. Two consequences of this connectivity will be important below. First, as the mean in-degree κ is the same for all neurons, the spiking statistics of single cells are fairly stereotypical on average across the network. This is evident in the spike rasters of **Figure 2A**. Second, the magnitude of inputs to single cells remains similar as network size *N* grows, because κ is fixed.

We emphasize that the collection *I* is a multi-dimensional signal and not stochastic noise. We study the solutions of (1) arising from distinct initial conditions (IC) but receiving the same input *I*. In contrast to a standard stochastic differential equation, this interpretation of system (1) is defined as a *random dynamical system* (RDS) (Kunita, 1990). As we will see below, RDS theory addresses questions of ensemble dynamics when a quenched "realization" of a stochastic process drives an underlying dynamical system. This framework enables us to ask questions about stimulus-response variability of a chaotic network due to any perturbation. For example, one might ask: What is the impact of deleting or adding a spike from some neuron(s) on the future spiking output of the network (c.f. London et al., 2010)? This scenario is equivalent to comparing the response from the network initialized at a given state to its response resulting from a second perturbed state, where the coordinates of some neurons are set to be at or away from their spiking phase. Our approach is a

(round markers); see text. From top to bottom: homogeneous poisson (blue), inhomogeneous poisson (red), network interactions (black). The bottom curve is a computation of <sup>1</sup> <sup>2</sup> *<sup>H</sup>*2*<sup>L</sup> noise* from a cell pair (diamond markers). Abscissa scale is 1/*L* to better visualize extrapolation of extensive regime to *L* → ∞ (left square marker). For all panels: η = −0.5, ε = 0.5, *N* = 500.

generalization of this formulation as we consider large ensembles of initial states, and study the differences between resulting trajectories in response to a fixed input. This will enable us to quantify the statistics of variability in network responses due to chaos.

## **2.2. SPIKE-RESPONSE NOISE ENTROPY AND DIRECT ESTIMATES**

To quantify spike pattern variability, we treat spike trains as binary time series. We discretize time in bins of width *t* small enough so that for a given cell, each bin contains at most a single spike. Throughout, we use time bins of width *t* = 0.05 time-units; we found that moderately different resolutions did not significantly affect our results. Let us define finite binary words for *K* neurons over *L* time bins starting at time *tl* = *lt* for some integer *l*: *SKL*(*tl*) = {*S<sup>k</sup> <sup>l</sup>* ,..., *<sup>S</sup><sup>k</sup> <sup>l</sup>* <sup>+</sup> *<sup>L</sup>* <sup>−</sup> <sup>1</sup>}*<sup>k</sup>* <sup>=</sup> *<sup>k</sup>*1,...,*kK* with *<sup>S</sup><sup>k</sup> <sup>j</sup>* ∈ {0, 1} (see **Figure 2A**).

The variability of the evoked spike response *SKL*(*tl*) is captured by the *spike-response noise entropy*

$$H\_{noise}^{KL}(I, t\_l) = \frac{-1}{L\Delta t} \sum\_{\text{SKL}} P(\text{SKL}(t\_l)|I) \log\_2 P(\text{SKL}(t\_l)|I) \tag{2}$$

where *P*(*SKL*(*tl*)|*I*) denotes probability of observing word *SKL*(*tl*) conditioned on input *I*, given a random initial state of the network. This quantity may also be referred to as *conditional response entropy*. It is normalized to have units of bits per time-unit (*bits*/*tu*), as opposed to bits per time-bin, and thus represents an *entropy rate* in continuous time. Since the inputs *I* and network dynamics are statistically stationary processes (Lajoie et al., 2013), it follows that the expected noise entropy rate of *KL* words conditioned on any *I* from the same input distribution—controlled by the parameters η and ε—can be obtained from a long time average on any single *I*<sup>∗</sup> (see e.g., Rieke et al., 1996; Strong et al., 1998):

$$H\_{noise}^{KL} = \int\_{I} P(I) H\_{noise}^{KL}(I, t\_l) = \lim\_{T \to \infty} \frac{1}{T} \sum\_{l=0}^{T-1} H\_{noise}^{KL}(I^\*, t\_l). \tag{3}$$

As demonstrated in Strong et al. (1998) and reviewed below, (3) can be used to estimate the true entropy rate of *K*-neuron groups considered when *L* → ∞. As we will see, this is only practical for small *K*—we will need other tools to understand this quantity for entire networks (*K* = *N*). Nevertheless, we begin by applying a direct sampling approach.

To estimate the probability terms in (2), we simulate network (1) in response to a randomly chosen, quenched *I*(*t*) for 10, 000 time units and 2000 "trials," distinguished by different ICs. Here, we wish to choose ICs from a distribution that best describes random network states, while being agnostic about its past. As discussed in Lajoie et al. (2013), we assume that system (1) possesses an ergodic stationary probability measure μ(θ), which is the steady state solution of the Fokker-Planck equation associated with (1). Thus, μ is the probability measure describing how likely we are to find the network in a particular state at any moment in time, given the history of any input *I* with identical statistics. We emphasize that μ serves only as an initial distribution, and that ensembles of "trial" trajectories as described above will have a very different distribution, as they are conditioned on a *fixed* input *I*. (See Lin et al., 2009a,b; Lajoie et al., 2013 for more details about this distinction).

To sample from μ, we first select seed ICs uniformly over the state space, and evolve each of these for a "burn" period of 50 time units, for which different inputs are presented. The resulting endpoints of these trajectories represent a new IC ensemble that approximates μ. From then on, all ICs are integrated using the same input *I* and we use this solution ensemble to study variability of spike-responses.

From these simulated network trajectories, we first discard the first 100 time-units to eliminate transient effects. We then extract the binary spike output of neurons across all trials (see **Figure 2B** for a single, network-embedded neuron example). Normalized cross-trial counts of *SKL* words in consecutive, non-overlapping *L*-windows serve as estimates of the probabilities *P*(*SKL*(*tl*)|*I*) in Equation (2).

## **3. RESULTS**

#### **3.1. SINGLE-CELL VARIABILITY**

We begin by computing noise entropy in the spike responses of single cells in the network. Using the estimation techniques described above, we compare the effect of chaos to that of commonly used independent noise models on noise entropy. This complements similar analysis in Lajoie et al. (2013), which used a different metric of spike reliability from trial to trial.

We start by randomly selecting a cell in our network and extract its binary spike output across many simulated trials (see **Figure 2B**). Using this data, we estimate *H*1*<sup>L</sup> noise* for word lengths up to *L* = 20 and plot the results in **Figure 2C** as a function of 1/*L*. A system with finite autocorrelation timescales is expected to produce entropy rates that behave extensively as *L* becomes sufficiently large. This is readily apparent in the linear decreasing trend in *H*1*<sup>L</sup> noise* as *L* grows, until a point where the estimate quickly drops due to insufficient sampling. Following Strong et al. (1998), we use the point of least fractional change in slope to extrapolate this extensive trend and obtain an estimate for lim*L*→∞ *H*1*<sup>L</sup> noise* (intersection with ordinates in **Figure 2C**). We verified that taking larger sample sizes—with *L* up to 30 and ensembles of up to 10, 000 trials– did not significantly affect our estimates.

Our estimate of lim*L*→∞ *H*1*<sup>L</sup> noise* is 1.12 *bits*/*tu*. We note that a "purely random," homogeneous poisson spike train with the same firing rate (0.8 *spikes*/*tu*) would have noise entropy *H*1*<sup>L</sup> noise* of 3.67 *bits*/*tu*. Thus, while chaotic dynamics produce variable spiking in single cells, the resulting noise entropy is much less than that of a totally random response, a fact also evident from the spike rasters in **Figure 2B**.

Part of the reason for this difference is simply the presence of the stimulus; inputs from other cells in the chaotic networks also play a role. To isolate the network effect, we repeat the sampling process above by simulating our chosen cell in isolation, keeping the input *Ii* intact but replacing the incoming spike trains it receives from upstream cells by two surrogate ensembles meant to isolate distinct statistical features of network activity. (i) *Homogeneous poisson* surrogates: independent, poisson distributed spike trains with rate matching the mean firing rate of corresponding upstream cells. (ii) *Inhomogeneous poisson* surrogates: produced by independently drawing a binary random variable in each *t*-bin, according to the time-dependent probability given by the normalized spike count of the corresponding network train across all original trials. For each new simulated trial, we draw independent surrogates. **Figure 2C** shows a 66% increase in noise entropy rate for the homogeneous surrogates, and about 30% for the inhomogeneous case.

Overall, we have shown that single, stimulus-driven cells in chaotic networks produce spike-response entropy significantly lower than that expected for single, stimulus-driven cells receiving poisson background inputs, as in many statistical models. We next seek to characterize spike entropy in the joint responses of multiple cells.

#### **3.2. MULTI-CELL VARIABILITY**

Our network is connected—albeit sparsely (κ *N*)—and it is not clear in advance how coupling interactions will impact the entropy rate of groups of cells. As a first step, we repeat the noise entropy estimate described above for a randomly selected pair of connected cells up to *L* = 10, and extrapolate lim*L*→∞ *H*2*<sup>L</sup> noise* from this data. The black lines in **Figure 2C** show *H*2*<sup>L</sup> noise*/2, normalized to units of bits per time-unit per neuron for comparison with *H*1*<sup>L</sup> noise*. Due to combinatorial explosion of possible spike patterns as more neurons are considered, we were unable to compute such estimates for *K* greater than 2. Nevertheless, it appears from the *K* = 2 case shown that interactions between neurons conspire to lower response noise entropy per neuron, if only by a small margin.

However, this margin could easily be missed. For a given neuron pair (*i*, *j*), consider the difference between the sum of independent cell entropy rates and their joint pair rate: δ*ij* = lim*L*→∞ *H*1*<sup>L</sup> noise*(*i*) <sup>+</sup> *<sup>H</sup>*1*<sup>L</sup> noise*(*j*) <sup>−</sup> *<sup>H</sup>*2*<sup>L</sup> noise*(*i*, *j*) . From 45 random pairs of neurons, we obtain the average δ*ij* = 0.012 bits/tu. This implies a relative difference of the order of *<sup>O</sup>*(10<sup>−</sup>2) when estimating the entropy rate of pairs of cells using their marginal, single-cell response distributions. We will see later these small differences compound significantly when considering the network as a whole (cf. Schneidman et al., 2006).

To quantify the extent of these interactions over space and time, we compute the Pearson correlation coefficient *cij*(*tl*) between the spiking probability of two cells *i* and *j* in time bin *tl*. That is, we measure the cells' instantaneous *noise correlation*. **Figure 3A** shows a typical histogram of *cij*(*tl*) across all neuron pairs of a network with *N* = 500 for a fixed *tl*, where pairs with zero spiking probability were discarded. We can see that at a fixed moment, correlations are weak and most cells are uncorrelated. Moreover, these correlations are not static: a high correlation between two cells in one time bin does not guarantee that they will be correlated in another. This is illustrated by **Figure 3B**, showing a histogram of *cij*(*tl*) across 10000 time-units between two randomly chosen connected cells.

We emphasize that this weak and highly dynamic correlation structure might easily be dismissed as negligible experimentally. If one would choose a single pair of cells and measure the temporal average of *cij*(*tl*) over 500 time units, one obtains an average of the order of 10−<sup>5</sup> (over 4950 cell pairs tested), and standard deviation of the order of 10−<sup>2</sup> (across the 4950 cell pairs.) In other words, each individual cell pair appears to be almost completely uncorrelated–at least on average. Below, we will show that the weak, transient dependencies that are in fact present among

*bits*/*tu* as a function of *N*. Slope *H*1 averaged over 20 random cells

panels: η = −0.5, ε = 0.5.

neurons nevertheless have a very strong impact on network-wide noise entropy.

To summarize, measures of entropy and correlations indicate that there are noticeable but weak dependencies in the spiking activity of connected pairs of cells. Scaling up from such dependencies to accurately describe the joint activity of an entire network is a notoriously difficult problem. We take an approach based on RDS in what follows. This approach will quantify the entropy *HNL noise* of the network as whole, as networks size *N* grows.

#### **3.3. A BENCHMARK FOR NETWORK ENTROPY**

To benchmark *HNL noise* for different network sizes, we first describe the joint network entropy that would be naively predicted by direct extrapolation from single cells. In other words, this is the estimate one would obtain by ignoring statistical interactions between neurons. As the entropy of a multivariate distribution is always smaller or equal to the sum of the marginal distributions' entropies, it follows that if *H*1 denotes the average of lim*L*→∞ *H*1*<sup>L</sup> noise* over all neurons, then *<sup>N</sup> <sup>H</sup>*1 ≥ lim*L*→∞ *<sup>H</sup>NL noise*. We estimate *H*1 by sampling lim*L*→∞ *H*1*<sup>L</sup> noise* from randomly chosen neurons in a network with *N* = 500 using the same technique as in **Figure 2C**. As the mean in-degree κ for incoming connections to each neuron is constant, we found that using an ensemble of 20 neurons randomly sampled from the full network, gave a good estimate for *H*1.

Unlike cell pairs, spiking statistics of single neurons are expected to be unchanged by network size *N* with fixed in-degree κ. We therefore use *H*1to extrapolate the extensive upper bound on network noise entropy *N H*1 as a function of network size *N*. **Figure 3C** shows this estimate, where the shaded area around the line denotes the extrapolation of two standard errors of the mean of *H*1 estimated in a network with *N* = 500. We verified by spot checks that single cell activity in networks of different sizes agree with this extrapolation (see markers in **Figure 3C**). Next, we leverage dynamical properties of our network to estimate how much reduction in entropy can be expected from the joint activity of entire networks in comparison to this naive extensive bound.

#### **3.4. DYNAMICAL ENTROPY PRODUCTION**

In what follows, we use symbolic dynamics to map between the phase space of our network and the set of binary spike trains. Consider trajectories θ(*t*) = (θ1(*t*),...,θ*N*(*t*)) of model (1), evolving on the *N*-dimensional torus T*N*. Recall that a spike from cell *i* occurs when θ*i*(*t*) = 1, and will lead to *S<sup>i</sup> <sup>l</sup>* = 1 in the corresponding time bin. Notice that the phase response curve *Z*(θ*i*) modulates the effect of any input on neuron *i*– whether that input comes from the signal *Ii*(*t*) or from network activity—and that it vanishes at θ*i*(*t*) = 1. This implies that a neuron becomes insensitive to any inputs when it is about to spike. Indeed, the Taylor expansion of neuron *i*'s dynamics about θ*<sup>i</sup>* = 1 is constant up to quadratic order: *d*θ*<sup>i</sup>* = [<sup>2</sup> <sup>+</sup> *<sup>O</sup>*((θ*<sup>i</sup>* <sup>−</sup> 1)2)]*dt* <sup>+</sup> *<sup>O</sup>*((θ*<sup>i</sup>* <sup>−</sup> 1)2)*dWi*,*t*. Based on this observation we make the approximation that for *t* small enough, neuron *i* spikes in the time bin [*t*, *t* + *t*] if and only if θ*i*(*t*) ∈ [1 − 2*t*, 1) (see next section for verification).

Thus, equipped, consider the following partition of the state space T*N*: <sup>∗</sup> = {γ0, γ1}*N*, built of Cartesian products of intervals γ<sup>0</sup> = [0, 1 − 2*t*) and γ<sup>1</sup> = [1 − 2*t*, 1) across all θ*i*'s. At any time *tl* = *lt*, the <sup>∗</sup>-address of θ(*tl*) determines the binarized spiking state of the network in time bin [*tl*, *tl* + *t*]: θ*i*(*tl*) ∈ γ<sup>0</sup> ⇒ *S<sup>i</sup> <sup>l</sup>* <sup>=</sup> 0 and <sup>θ</sup>*i*(*tl*) <sup>∈</sup> <sup>γ</sup><sup>1</sup> <sup>⇒</sup> *<sup>S</sup><sup>i</sup> <sup>l</sup>* = 1. In order to describe *L*long spike trains in terms of <sup>∗</sup>-addresses, we must understand how solutions θ(*t*) evolve with respect to <sup>∗</sup>. To this end, consider the discretized dynamics given by the transition maps *<sup>t</sup>*;*<sup>I</sup>* that send T*<sup>N</sup>* onto itself according to the flow of (1) from *t* to *t* + *t*. If θ(*t*) is a solution of (1), then *<sup>t</sup>*;*I*(θ(*t*)) = θ(*t* + *t*) where *t* refers to the resolution of our binary spike trains *SNL*. Note that the maps *<sup>t</sup>*;*<sup>I</sup>* depend on both *t* and *I*, are generally smooth with smooth inverses (diffeomorphisms) (Kunita, 1990), and together form a discrete RDS. For detailed geometric properties of the RDS defined by system (1), we refer the reader to Lajoie et al. (2013).

For what follows, it is convenient to reverse time and study spike trains and trajectories starting in the distant past leading up to *t* = 0. This representation is statistically equivalent to forward time since our network has stationary dynamics (Lajoie et al., 2013). Consider now the *l*-step inverse map: <sup>−</sup>*<sup>l</sup>* <sup>0</sup>;*I*. For any set *A* in the partition <sup>∗</sup>, its pre-image <sup>−</sup>*<sup>l</sup>* <sup>0</sup>;*I*(*A*) refers to all points in <sup>T</sup>*<sup>N</sup>* at time −*lt* that will be mapped to *A*, and consequently have the same spiking state at *t* = 0. Similarly, if both *A*<sup>0</sup> and *A*<sup>1</sup> are sets in <sup>∗</sup>, the intersection <sup>−</sup>*<sup>l</sup>* <sup>0</sup>;*I*(*A*0) <sup>−</sup>*l*+<sup>1</sup> <sup>0</sup>;*<sup>I</sup>* (*A*1) describes all points that will be mapped to *A*<sup>1</sup> at *t* = −*t* and *A*<sup>0</sup> at *t* = 0. It follows that any subset of the form *B* = *<sup>L</sup> <sup>l</sup>*=<sup>0</sup> <sup>−</sup>*<sup>l</sup>* <sup>0</sup>;*I*(*Al*) where *Al* ∈ ∗ captures all past network states at time *t* = ( − *L*)*t* leading to identical spiking sequences {*S<sup>i</sup>* <sup>−</sup>*L*,..., *<sup>S</sup><sup>i</sup>* <sup>−</sup>1, *<sup>S</sup><sup>i</sup>* <sup>0</sup>}*<sup>i</sup>* <sup>=</sup> <sup>1</sup>,...,*N*, when the same *I* is presented. Moreover, it is easy to show that the collections of all possible sets constructed as *B*, named the *join* of pre-images of <sup>∗</sup> and denoted ∨*<sup>L</sup> <sup>l</sup>* <sup>=</sup> <sup>0</sup> <sup>−</sup>*<sup>l</sup>* 0;*I* <sup>∗</sup>, is itself a partition of T*N*.

It follows that this new partition offers a one-to-one correspondence between its member sets and the space of all *SNL* spike trains. Note that many sets in this partition will be empty since not all spike sequences are accessible by the network. In fact, the number of non-empty sets remaining in ∨*<sup>L</sup>* <sup>−</sup> <sup>1</sup> *<sup>l</sup>* <sup>=</sup> <sup>0</sup> <sup>−</sup>*<sup>l</sup>* 0;*I* <sup>∗</sup> as *L* → ∞ represents the number of allowed infinite spike sequences. Furthermore, for a given *SNL* and its associated set *B*(*SNL*) ∈ ∨*<sup>L</sup>* <sup>−</sup> <sup>1</sup> *<sup>l</sup>* <sup>=</sup> <sup>0</sup> <sup>−</sup>*<sup>l</sup>* 0;*I* <sup>∗</sup>, the probability of observing spike pattern *SNL* can be stated as an initial state probability in the distant past: *P*(*SNL*|*I*) = *P*(θ( − *Lt*) ∈ *B*(*SNL*)).

As discussed above and in Lajoie et al. (2013), we assume that our RDS possesses an ergodic stationary probability measure μ. Recall that we assume random ICs forming our distinct trials are drawn from μ. It follows that lim*L*→∞ *P*(*SNL*|*I*) = μ(*B*(*SNL*)). Thus, if we let

$$h\_{\mu}(\Phi\_{t;I}, \Gamma^\*) = \lim\_{L \to \infty} -\frac{1}{L} \sum\_{\substack{B \in \vee^{L}\_{l=0} \Phi^{-l}\_{0;I} \Gamma^\*}} \mu(B) \ln \mu(B), \quad (4)$$

it follows that

$$\lim\_{L \to \infty} H\_{noise}^{NL} = \frac{\Delta t}{\ln 2} h\_{\mu}(\Phi\_{t;I}, \Gamma^\*). \tag{5}$$

For any dynamical system, the expression (4) measures the amount of uncertainty produced by chaotic dynamics if we can only observe the system with the precision given by the partition <sup>∗</sup>. This concept is generalized by the *Kolmogorov-Sinai entropy h*μ, also called *dynamical* or *metric* entropy (Ruelle, 1989; Greven et al., 2003), defined by

$$h\_{\mu} = \sup\_{\Gamma} h\_{\mu}(\Phi\_{t;I}, \Gamma) \tag{6}$$

where the supremum is taken over all finite partitions . This quantity is related to the Lyapunov spectrum λ<sup>1</sup> ≥ λ<sup>2</sup> ≥···≥ λ*<sup>N</sup>* of a dynamical system which measures rates of exponential divergence or convergence between trajectories. Lyapunov exponents λ*<sup>i</sup>* are expected to be well defined for our RDS in the sense that they rely on system parameters such as coupling strength and the mean and variance of inputs, but not on specific realizations of the inputs *I*(*t*) (Kifer, 1986). The authors of Ledrappier and Young (1988) showed that although the join of a partition depends on *I*, *h*μ does not and that under some ergodicity assumptions, the following entropy formula holds:

$$h\_{\mu} = \sum\_{\lambda\_i > 0} \lambda\_i. \tag{7}$$

If λ*<sup>i</sup>* are the Lyapunov exponents of the original system (1) computed over time-units instead of *t* time-steps, we get from (4), (5), (6), and (7) the following upper bound for noise entropy rate :

$$H\_{KS} \equiv \frac{1}{\ln 2} \sum\_{\lambda\_i > 0} \lambda\_i \ge \lim\_{L \to \infty} H\_{noise}^{NL} \tag{8}$$

which has units of bits per time-unit.

To evaluate this bound, we numerically compute the exponents λ*<sup>i</sup>* of system (1) and find that, as originally observed in Monteforte and Wolf (2010) and Luccioli et al. (2012) for autonomous networks, our driven system has a size invariant Lyapunov spectrum (see **Figure 3D**), which is insensitive to particular choices of random coupling matrix *A* (see Supplementary Material for details). This leads to a spatially extensive behavior of the bound *HKS*, as shown in **Figure 3C**.

Intriguingly, *HKS* is much smaller than estimates from *H*1. This reveals a central result for our driven chaotic networks: *joint spike patterns are (at least) an order of magnitude less variable than what would be predicted by observing the spike train statistics of single cells, despite averaged noise correlations across neurons that are very low*.

#### **3.5. RELATIONSHIP BETWEEN STATE SPACE PARTITIONING AND SPIKING PATTERNS**

The derivation of the *HKS* bound (8) relies on the simple assumption that neuron *i* will spike within *t* time-units if and only if θ*i*(*t*) ∈ γ<sup>1</sup> = [1 − 2*t*, 1]. As discussed above, this assumption holds in the limit of small *t*. We found that for simulated trajectories of 1000 time-units from network (1), only about 0.01% of all spikes violated the spiking assumption for *t* = 0.05. This number dropped to zero for *t* = 0.01. Such values are evidence that errors in relating spike train entropy estimates to entropy production in state space will be slight. In the present section, we verify this in detail.

To do so, we compare the spiking statistics and entropy estimates for the main model (1) with those for an analogous dynamical system, for which our partition-based spiking assumption holds exactly, by design. Consider the *piecewise* model analogous to system (1):

$$d\theta\_i = \left[\tilde{F}(\theta\_i) + \tilde{Z}(\theta\_i)\sum\_{j=1}^N a\_{i\bar{j}\bar{\mathcal{G}}}(\theta\_{\bar{j}}) + \frac{\varepsilon^2}{2}\tilde{Z}(\theta\_i)\tilde{Z}'(\theta\_i)\right]dt\dots$$

$$+ \tilde{Z}(\theta\_i)\underbrace{\left[\eta dt + \varepsilon dW\_{i,t}\right]}\_{I\_i(t)}\tag{9}$$

in which we replace the functions *F* and *Z* by the following piecewise-defined terms:

$$\tilde{F}(\theta\_i) = \begin{cases} 1 + \cos\left(2\pi\theta\_i\right) ; \theta\_i \in [0, 1 - 2\Delta t) \\ 2 \qquad ; \theta\_i \in [1 - 2\Delta t, 1) \end{cases}$$

$$\tilde{Z}(\theta\_i) = \begin{cases} 1 - \cos\left(2\pi\theta\_i\right) ; \theta\_i \in [0, 1 - 2\Delta t) \\ 0 \qquad ; \theta\_i \in [1 - 2\Delta t, 1) \end{cases}$$

It is easy to see that the partition-based spiking assumption holds exactly for the network defined by (9). However, notice that for *t* > 0, both *F*˜ and *Z*˜ are discontinuous functions of *S*<sup>1</sup> and that as a result, the Jacobian of (9) is ill-defined. Nevertheless, for practical purposes, we can simulate system (9) and approximate its Lyapunov spectrum, since there is only one discontinuity point per neuron and the probability of a finite-duration, discretized trajectory landing on such points is nil.

The purpose of model (9) is to assess the differences arising between the dynamics of our full ("normal") model, given by Equation (1), and the alternate ("piecewise") model above for which the spiking assumption is exact. We fix *t* = 0.05 as in the main text and begin by comparing single cell dynamics for the "normal" and "piecewise" models. **Figure 4** shows a simulated single cell trajectory from each model, with identical input *Ii* and identical incoming spike trains (extracted from a separate network simulation). This setup mimics the activity a single cell would receive when embedded in a network. Notice that apart from small discrepancies that sometimes arise between spike times, the two trajectories agree almost perfectly. When differences do arise, they are quite small. From a simulation yielding about 3000 spikes from both models, most corresponding spikes from the normal and piecewise models were indistinguishably close, down to the numerical solver's time-step. The maximal difference was about 0.02 time-units, smaller than a *t* time-bin.

**Figure 4B** shows the first 60 Lyapunov exponents of a network with size *N* = 500, simulated with both the normal (1) and piecewise (9) models. Since Lyapunov exponents depend on the Jacobian of a system, we expected the piecewise model to yield smaller exponents: its derivative is zero on the intervals [1 − 2*t*, 1). Nevertheless, this discrepancy is minimal and amounts to a difference of about 0.002 bits per neuron per time-unit in the

slope of the *HKS* estimates shown in **Figure 4C**. Finally, we empirically estimate the noise entropy bound *H*1, as described in the main text, for the piecewise model (9). Its value differed from the normal model estimate by about 0.01 bits per neuron per timeunit, well below the standard error of the mean of estimates from both models, as can be seen in **Figure 4C**.

In light of these tests, we are confident that the main result of the paper—a computable bound on spike-train noise entropy that is much lower than what would be extrapolated from single cells—is a robust phenomenon for networks of the type modeled by (1), rather than a consequence of a (seemingly tiny) approximation error.

## **3.6. NOISE ENTROPY PRODUCTION AS A FUNCTION OF INPUT STATISTICS**

Previous studies showed that the level of sensitivity emerging from chaotic network dynamics can be controlled by carefully chosen inputs (see Molgedey et al., 1992; Rajan et al., 2010 for different contexts). We verify if this is the case for our network. We first identify a range of input statistics—the mean η and fluctuation amplitude ε—that are comparable in that they all produce the same firing rate as for the "standard" parameter set used above (η = −0.5, ε = 0.5). These parameters lie along the level curve in **Figure 5A**. Note that the curve is parameterized so that η grows while ε decreases; thus, as we travel along it, we gradually shift the dynamics from the excitable, fluctuation-driven regime (η < 0) to an oscillatory, mean-driven one (η > 0). In particular, the last point evaluated corresponds to a purely autonomous regime (ε = 0) where the input *I* has no fluctuating component.

**Figure 5B** shows the first 200 Lyapunov exponents of a network with *N* = 500 along this level curve, and panel (c) gives the corresponding *HKS* values. A clear trend emerges: *HKS* increases monotonically as the system transitions from fluctuation- to mean-driven regimes, by almost an order of magnitude. Moreover, **Figure 5D** shows that, for the two extremes of the level curve, network noise entropy continues to be much smaller than that predicted from single cells, and that singlecell noise entropy appears to follow the same trends as *HKS*. We conclude that *spike pattern variability emerging from chaos is not a fixed property of a network, but can be strongly modulated by the mean and variance of network inputs.*

## **4. DISCUSSION**

Biological neural networks may operate in a chaotic regime, with irregular activity driven by a balance of fluctuating excitatory and inhibitory interactions. This network chaos is under vigorous study, fueled in part by possible roles for chaos in generating "target" spatiotemporal patterns (Sussillo and Abbott, 2009) and in enabling useful temporal processing of inputs (Buonomano and Maass, 2009; Laje and Buonomano, 2013). Here, we address a complementary question: How much variability (or "noise") will chaotic dynamics add to network responses?

We compute bounds on network spike-response entropy that give novel answers. In particular, we show that the noise entropy of multi-cell spike responses is at least an order of magnitude lower than would be naively extrapolated from from single-cell measurements, under the assumption that spike variability is independent from cell to cell. The direction of the comparison between noise entropy of single cell and multi-cell spike responses agrees with intuition provided by the shape of the Lyapunov spectrum, which indicates time-dependent chaotic attractors of lower dimension than phase space. Thus, the phase space dynamics of each neuron are not independent. What we quantify explicitly is the order-of-magnitude size of the effect, as it is manifested in the binary *spiking* outputs of the system—a fact which might seem especially striking given that pairs of spike trains appear to be very weakly correlated on average.

If one considers the level of noise entropy as an indicator of potential information contained in spike patterns, we show that balanced networks may be able to encode inputs stimuli using spike timing if these inputs contain strong enough temporal structure. This mechanism takes root in the complex noiseinteractions that chaos induces between neurons. The extensive nature of this phenomenon suggests that this mechanism is scalable with network size. Moreover, the strong dependence of entropy on the input signal's mean and variance indicate that

a network can operate in different "regimes" modulating the repeatability of spike patterns. This is in addition to known advantages of balanced networks, such as efficiently tracking changes in common, mean inputs with firing rates (van Vreeswijk and Sompolinsky, 1998)—which may encode coarser statistics about inputs at the population level.

parametrization. Markers: square: η = −1, ε = 0.69, star: η = −0.5, ε = 0.5, circle: η = 0.07, ε = 0. **(B)** Lyapunov spectra along contour

To formalize these notions, future work could seek to compute the mutual information between an input ensemble and a system's response. In order to estimate this quantity, one needs to compute the *total entropy* (Rieke et al., 1996) of spike patterns in addition to the noise entropy computed in this paper– which captures how many distinct spike outputs can be produced by the network, for any input *I*. This quantity can be thought of as noise entropy marginalized over the set of possible inputs. Estimating the total entropy in large networks is a difficult problem since it depends on the evolution of ensembles of trajectories driven by ensembles of inputs. In other words, one needs to capture the entropy of trajectories when system (1) is treated as a stochastic differential equation rather than a RDS, a distinction that introduces a variety of challenges.

Our results complement prior work on the behavior of sparse, balanced networks in the large *N* limit. Seminal results use meanfield approaches (e.g., van Vreeswijk and Sompolinsky, 1998), deriving successful estimates of population activity statistics such as the mean and the variance of firing rates. In this approach, self consistent equations are derived for representative single cells based on the assumption that, when *N* is sufficiently large and *k*/*N* is sufficiently small, the inputs to each neuron in the network can be approximated by independent gaussian noise. In contrast, we derive estimates for the impact of correlations among these individual cells. Interestingly, in both the classical and the present work, noise entropy scales extensively with *N*; here, the predicted rate of scaling would be lower, as even weak correlations between cells combine to create statistical dependencies—especially when network activity is conditioned on an input.

and *HKS* extrapolated from a network with *N* = 500, as are quantities

Finally, we expect that the *HKS* bound can be adapted to other neuron models, provided a state space partition linking dynamics to spike patterns can be derived. This could prove to be a powerful tool to investigate stimulus encoding as a function of many network attributes, such as spike-generating dynamics, connectivity, learning rules and input correlations.

## **FUNDING**

from all other panels.

This work was supported in part by an NIH Training Grant from University of Washington's Center for Computational Neuroscience (5R90DA03346103, 5T90DA03243602), a Bernstein Fellowship from the Bernstein Center for Computational Neuroscience, a postdoctoral scholarship from the *Fonds de Recherche du Québec*, the Burroughs Wellcome Fund for Scientific Interfaces, the NSF under grant DMS CAREER-1056125 and NSERC Discovery and CIHR operating grants. Numerical simulations were performed on NSF's XSEDE supercomputing platform.

## **ACKNOWLEDGMENTS**

The authors thank Fred Wolf, Yu Hu, and Kevin K. Lin for helpful insights. We also thank two anonymous reviewers for comments and suggestions that improved the manuscript.

## **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at: http://www.frontiersin.org/journal/10.3389/fncom. 2014.00123/abstract

## **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 16 May 2014; accepted: 11 September 2014; published online: 02 October 2014.*

*Citation: Lajoie G, Thivierge J-P and Shea-Brown E (2014) Structured chaos shapes spike-response noise entropy in balanced neural networks. Front. Comput. Neurosci. 8:123. doi: 10.3389/fncom.2014.00123*

*This article was submitted to the journal Frontiers in Computational Neuroscience.*

*Copyright © 2014 Lajoie, Thivierge and Shea-Brown. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Commentary on Structured chaos shapes spike-response noise entropy in balanced neural networks, by Lajoie, Thivierge, and Shea-Brown

## *Peter J. Thomas\**

*Department of Mathematics, Applied Mathematics and Statistics, Case Western Reserve University, Cleveland, OH, USA \*Correspondence: pjthomas@case.edu*

#### *Edited by:*

*David Hansel, University of Paris, France*

#### *Reviewed by:*

*Demian Battaglia, Max Planck Institute for Dynamics and Self-Organization, Germany Guillaume Lajoie, Max Planck Institute for Dynamics and Self-Organization, Germany*

**Keywords: spike-response noise entroy, balanced neural networks, information theory, chaos, mutual information**

#### **A commentary on**

**Structured chaos shapes spike-response noise entropy in balanced neural networks**

*by Lajoie, G., Thivierge, J.-P., and Shea-Brown, E. (2014). Front. Comput. Neurosci. 8:123. doi: 10.3389/fncom.2014.00123*

Animals with nervous systems generate complex adaptive behaviors in part through the computational capabilities arising from large networks of interconnected neurons in their brains (Churchland and Sejnowski, 1992). Although a full description of the nervous system would take into account the interactions of central circuits with sensory and motor systems (Chiel and Beer, 1997) it is more common to consider central circuitry in isolation. The individual nerve cells and synaptic junctions that comprise biological neural networks are spatially extended structures with fundamentally stochastic dynamics on a range of spatial and temporal scales (Andersen et al., 2006; Carnevale and Hines, 2006). Nevertheless, much progress has been made in understanding the repertoire of neural behavior through simplified deterministic one dimensional "phase" models such as the Ermentrout-Kopell canonical model (Ermentrout, 1996; Brown et al., 2004; Ermentrout, 2008)<sup>1</sup> .

Even if we restrict attention to isolated networks of deterministic, instantaneously coupled phase models, we confront significant challenges. The behavior of such networks can be chaotic, as evidenced by the divergence of nearby trajectories (positive Lyapunov exponents). If we consider such a "chaotic network" driven by a collection of input signals, it is natural to ask how the intrinsic variability related to the chaotic dynamics impacts the networks' computational capabilities. It is equally natural to view the system as a communications channel. With the input signals drawn from some specified ensemble, and the output taken as the spike trains of (some or all of) the neurons, the mutual information between the input and output ensembles would be of great interest. However, this quantity is difficult to obtain, either analytically or numerically.

In Lajoie et al. (2014), the authors further the analysis of information processing in chaotic deterministic networks by formulating a computationally tractable upper bound on the spike-train noise entropy, building on Monteforte and Wolf (2010) and Lajoie et al. (2013). They study a network of deterministic canonical Ermentrout-Kopell "theta" neurons (Ermentrout and Kopell, 1986) with an *ad-hoc* interaction function. The network connectivity is fixed, sparse and random. Each neuron is driven by a quenched white noise injected current input of the form *Ii*(*t*) = η + *dWi*,*t*/*dt*. As the authors (and others) have shown previously, the spontaneous activity (i.e., with = 0) in this class of networks exhibits chaotic behavior. It has been observed that applying an input to such networks (i.e., setting  > 0) can reduce the apparent irregularity of the spike train ensemble. The *spike train entropy* quantifies this reduction in variability; the authors obtain an upper bound on this quantity through a state space partitioning construction that takes advantage of the Kolmogorov-Sinai entropy, which is given in turn by the Lyapunov spectrum, which the authors estimate numerically. They show convincingly that the KS entropy of the spike trains is roughly an order of magnitude smaller than what one would expect from a naive estimate based on the single-cell noise entropy. Their results help make rigorous the observation that the application of a driving stimulus reduces the variability of the resulting spike trains, although the networks remain chaotic.

While this result is a substantive contribution, it is still some steps removed from telling us the mutual information *I*(*X* : *Y*) = *H*(*Y*) − *H*(*Y*|*X*) between an ensemble of inputs, *X*, and the corresponding ensemble of outputs, *Y*. The authors' result gives a bound on *H*(*Y*|*x*) for a specific realization of the frozen noise inputs *x* ∈ *X*. Because the system is ergodic, this estimate applies as well to the mean entropy *H*(*Y*|*X*) [as discussed in Lajoie et al. (2013)]. However, as the authors point out, one cannot replace the entropy *H*(*Y*) with *H*(*Y*|0), the entropy when the input fluctuations are switched off, since (as they convincingly demonstrate) turning on the input ( > 0) significantly changes the entropy. The entropy that would be needed for calculating the mutual information would be the spike train entropy

<sup>1</sup>As pointed out in Wolf et al. (2014), the choice of 1D model can strongly influence the entropy production properties of the resulting network.

for the ensemble unconditioned on a specific input—but with an ensemble of different white noises all with fixed > 0. It would be very interesting if one could investigate how *I*(*X* : *Y*) varied as a function of ; for instance, whether the mutual information changes smoothly or whether there is evidence for some kind of information processing phase transition. The authors' contribution provides a valuable step along the way to a deeper understanding of the impact of chaotic dynamics on computations in deterministic neural networks.

## **REFERENCES**


Wolf, F., Engelken, R., Puelma-Touzel, M., Weidinger, J. D. F., and Neef, A. (2014). Dynamical models of cortical circuits. *Curr. Opin. Neurobiol.* 25, 228–236. doi: 10.1016/j.conb.2014. 01.017

**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 02 October 2014; accepted: 08 February 2015; published online: 10 March 2015.*

*Citation: Thomas PJ (2015) Commentary on Structured chaos shapes spike-response noise entropy in balanced neural networks, by Lajoie, Thivierge, and Shea-Brown. Front. Comput. Neurosci. 9:23. doi: 10.3389/fncom. 2015.00023*

*This article was submitted to the journal Frontiers in Computational Neuroscience.*

*Copyright © 2015 Thomas. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Differential effects of excitatory and inhibitory heterogeneity on the gain and asynchronous state of sparse cortical networks

## *Jorge F. Mejias 1,2\* and André Longtin2,3*

*<sup>1</sup> Center for Neural Science, New York University, New York, NY, USA*

*<sup>2</sup> Department of Physics, University of Ottawa, Ottawa, ON, Canada*

*<sup>3</sup> Department of Cellular and Molecular Medicine, University of Ottawa, Ottawa, ON, Canada*

#### *Edited by:*

*Mark D. McDonnell, University of South Australia, Australia*

#### *Reviewed by:*

*Alberto Mazzoni, Scuola Superiore Sant'Anna, Italy Fabiano Baroni, Monash University, Australia*

#### *\*Correspondence:*

*Jorge F. Mejias, Center for Neural Science, New York University, 4 Washington Place, New York, NY 10003, USA e-mail: jorge.f.mejias@gmail.com*

Recent experimental and theoretical studies have highlighted the importance of cell-to-cell differences in the dynamics and functions of neural networks, such as in different types of neural coding or synchronization. It is still not known, however, how neural heterogeneity can affect cortical computations, or impact the dynamics of typical cortical circuits constituted of sparse excitatory and inhibitory networks. In this work, we analytically and numerically study the dynamics of a typical cortical circuit with a certain level of neural heterogeneity. Our circuit includes realistic features found in real cortical populations, such as network sparseness, excitatory, and inhibitory subpopulations of neurons, and different cell-to-cell heterogeneities for each type of population in the system. We find highly differentiated roles for heterogeneity, depending on the subpopulation in which it is found. In particular, while heterogeneity among excitatory neurons non-linearly increases the mean firing rate and linearizes the f-I curves, heterogeneity among inhibitory neurons may decrease the network activity level and induces divisive gain effects in the f-I curves of the excitatory cells, providing an effective gain control mechanism to influence information flow. In addition, we compute the conditions for stability of the network activity, finding that the synchronization onset is robust to inhibitory heterogeneity, but it shifts to lower input levels for higher excitatory heterogeneity. Finally, we provide an extension of recently reported heterogeneity-induced mechanisms for signal detection under rate coding, and we explore the validity of our findings when multiple sources of heterogeneity are present. These results allow for a detailed characterization of the role of neural heterogeneity in asynchronous cortical networks.

**Keywords: heterogeneity, asynchronous state, gain control, mean-field, cortical networks, signal detection**

## **1. INTRODUCTION**

Mathematical models of neurons and neural circuits have become, in the last couple of decades, a highly valuable tool to analyze and understand real neural systems, from single cell to behavior. Models are commonly used to test hypotheses or to support experimental observations, and their potential usefulness increases as their predictions are refined to account for the actual behavior of neurons (Gerstner and Naud, 2009). While it is not uncommon to see a high level of biophysical detail in singleneuron models, most of these details are usually neglected when modeling larger systems, such as neural circuits of thousands of neurons, for the sake of simplicity.

A particularly interesting case is the natural intrinsic variability found in the biophysical properties of neurons, which is averaged out in most theoretical and computational modeling studies. Real neural systems display a significant level of cell-tocell diversity at the neuron level, even among same-class neurons, as well as other differences at the subcellular or synaptic level (Bannister and Larkman, 1995a,b; Reyes et al., 1998; Hausser and Mel, 2003; Jinno et al., 2007). Experimental observations indicate that this type of structural heterogeneity has non-trivial effects on several neural information processing mechanisms. For instance, neural heterogeneity has been shown to have an impact in burst coding *in vivo* (Avila-Akerberg et al., 2010) and in envelope coding and non-linear responsiveness of the electroreceptors of weakly electric fish (Savard et al., 2011). The presence of a certain level of heterogeneity at the cell-to-cell level has also been recently reported to have a beneficial role for population coding (Marsat and Maler, 2010; Tripathy et al., 2013), and it can also induce the decorrelation of neuronal firing and the optimization of information content (Padmanabhan and Urban, 2010; Angelo et al., 2012; Urban and Tripathy, 2012). These experimental observations can not be explained by neural circuit models where, for instance, any given pyramidal neuron is perfectly identical to all the other pyramidal neurons in the system. Models which take into account the intrinsic heterogeneity of neural systems are, therefore, necessary to understand neural coding.

In response to this increasing body of evidence, a significant number of theoretical and computational studies, especially in the last years, have contributed to explaining the properties and dynamics of networks of heterogeneous neurons. In particular, the role of heterogeneity on synchronization has been extensively studied (Golomb and Rinzel, 1993; White et al., 1998; Neltner et al., 2000; Golomb et al., 2001; Denker et al., 2004; Talathi et al., 2008, 2009; Luccioli and Politi, 2010; Olmi et al., 2010; Brette, 2012; Mejias and Longtin, 2012). More recently, the effect of neural heterogeneities on neuronal correlations (Chelaru and Dragoi, 2008; Yim et al., 2013), detection of weak signals (Tessone et al., 2006; Perez et al., 2010) and different types of neural coding (Chelaru and Dragoi, 2008; Savard et al., 2011; Mejias and Longtin, 2012; Hunsberger et al., 2014) have drawn special attention as well. Novel approaches and mean-field approximations to tackle the problem of heterogeneity have also been recently proposed (Nicola and Campbell, 2013; Yim et al., 2013). These studies, however, typically focus on one type of cell (such as pyramidal neurons) and consider the presence of heterogeneity on this specific population. The possible—and potentially relevant interplay between populations of different cell types, each one of them presenting its own heterogeneity level, has remained a goal for future work and has not been fully addressed yet.

In this work, we analytically and numerically study the properties of a typical cortical circuit with cell-specific heterogeneity levels. Our basic circuit is constituted by a population of excitatory (i.e., pyramidal) neurons and a population of inhibitory neurons (i.e., interneurons), connected in a sparse manner. Both the excitatory and the inhibitory populations have their own independent level of intrinsic heterogeneity. This allows us to quantitatively study the effect of heterogeneity of a given population (excitatory, or inhibitory, or both) and to characterize its impact on the whole network dynamics. Our results indicate highly differentiated roles for heterogeneity, depending on the population in which it is introduced. In particular, heterogeneity among excitatory neurons (which we call here *excitatory heterogeneity*) non-linearly increases the mean firing rate of the whole network and linearizes the input/output f-I curves. On the other hand, heterogeneity among inhibitory neurons (*inhibitory heterogeneity*) may decrease the network activity level and induce divisive gain effects in the f-I curves of the excitatory population, providing an effective gain control mechanism to influence the flow of information across the network. We also compute the conditions for stability of the network activity and provide an extension of the recently described heterogeneityinduced mechanism which optimizes information transmission under rate coding (Mejias and Longtin, 2012). Our novel meanfield approach extends our previous theoretical results for fully connected excitatory neurons (Mejias and Longtin, 2012) to cortical-like sparsely connected networks of heterogeneous excitatory and inhibitory cells, providing a strong analytical tool to characterize the role of neural heterogeneity in cortical networks.

#### **2. MATERIALS AND METHODS**

We consider a sparsely connected network of *N* integrate-andfire neurons (see **Figure 1**), where any two given neurons are

unidirectionally connected with a probability - (the average number of synapses onto a given neuron is then *K* ≡ -*N*). A subset of this population is constituted by γ *N* excitatory neurons, while the remaining (1 − γ )*N* neurons in the network are inhibitory. A given neuron *i* is governed by the dynamics

$$
\tau\_m \frac{dV\_i(t)}{dt} = -V\_i(t) + RI\_i^{\text{ext}}(t) + RI\_i^{\text{net}}(t), \tag{1}
$$

where τ*<sup>m</sup>* is the neuron membrane time constant, *Vi* is the membrane potential of the *i* − *th* neuron in the network, *R* is the membrane resistance, and *Iext <sup>i</sup>* , *<sup>I</sup>net <sup>i</sup>* are the external and recurrent input to the *i* − *th* neuron, respectively. Each neuron *i* is assumed to fire an action potential (AP) every time *Vi* reaches a certain firing threshold, and after that the membrane potential is reset to *Vr* for a time period τ*ref* . The external and recurrent input to the *i* − *th* neuron are given, respectively, by

$$RI\_i^{\text{ext}}(t) = \mu\_i + \sigma \sqrt{\tau\_m} \xi\_i(t), \tag{2}$$

$$RI\_i^{net}(t) = \mathfrak{r}\_m \sum\_j \sum\_k f\_{ij} \,\delta(t - t\_j^k), \tag{3}$$

where μ*<sup>i</sup>* is a constant input, ξ*i*(*t*) is a Gaussian white noise of zero mean and unitary variance, σ is the noise intensity, *Jij* is the coupling strength of the synapse from neuron *j* to neuron *i* (considered zero if there is not such a synapse between both neurons), and the *k* − *th* spike from neuron *j* arrives at neuron *i* at *t k <sup>j</sup>* . The synaptic coupling strength between two neurons *i*, *j* takes the value *Jij* = *J*αβ, where α = {*E*, *I*} is a label denoting the population to which the postsynaptic neuron belongs, and β = {*E*, *I*} denotes the population to which the presynaptic neuron belongs. We define the external input to the network as μ*<sup>i</sup>* = μ (arriving at all excitatory neurons), and we also define a constant bias μ*<sup>i</sup>* = μ<sup>0</sup> for all inhibitory neurons.

In this framework, we assume that each neuron of the network is characterized by a different distance-to-threshold value, which may be related with several biophysical properties (such as the membrane resistance, the firing threshold, or extra non-linear considerations). We assume here that such heterogeneity in the distance-to-threshold corresponds to heterogeneity in firing threshold values, although heterogeneities in distance-tothreshold can be translated into heterogeneities in other kind of parameters in more sophisticated neuron models. In particular, each excitatory neuron has a firing threshold θ*E*,*<sup>i</sup>* which is randomly distributed following a Gaussian profile *PE*(θ*E*) with mean θ and standard deviation *wE*. Equivalently, each inhibitory neuron has a firing threshold θ*I*,*<sup>i</sup>* randomly chosen from a Gaussian distribution *PI*(θ*I*) with mean θ and standard deviation *wI* (see **Figure 1**). Such heterogeneity serves to reflect some of the variability in the individual excitability properties of neurons found in actual neural systems, while treating separately the heterogeneity of excitatory and inhibitory cells will allow us to discern the effects caused by each population. For convenience, we will define a *low-threshold neuron* as a neuron whose firing threshold value (or, more precisely, its distance between the threshold for spiking and its resting state in absence of input) is below the average for its population (i.e., excitatory or inhibitory). A *high-threshold neuron* will therefore have a firing threshold value which is higher than the average for its population.

In the following, and unless specified otherwise, we choose *K* = 200 connections (in simulations, - = 0.2 for *N* = 1000 neurons), γ = 0.8, τ*<sup>m</sup>* = 20 ms, *Vr* = 10 mV, σ = 3 mV, θ = 20 mV, τ*ref* = 5 ms, *JEE* = *JIE* = 0.05 mV, and *JEI* = *JII* = −0.08 mV. These parameter values are within the physiological range for cortical neurons, and similar values have been used in previous modeling studies (Brunel and Hakim, 1999; Brunel, 2000). When computing the response of the system (for instance, the network mean firing rate for a given heterogeneity value), we average over 10 trials (or simulation runs on a random realization of the connectivity matrix) of 10 s each. The results presented in this work hold for these other parameter choices as well.

Together with the numerical simulations of the neural network described above, we have obtained an analytical meanfield solution of the model, which is described in detail in the Supplementary Material (Section Mean-field approach). In short, we have employed the diffusion approximation in the input to a single IF neuron to compute its mean firing rate in steady state conditions (Tuckwell, 1989; Brunel, 2000). Since the input to any given neuron will depend on the activity of the whole network (due to the recurrent nature of the system), we can average over the heterogeneity and obtain a mean-field description of the excitatory and inhibitory network mean firing rate, which will be given, respectively, by ν*<sup>E</sup>* = *<sup>E</sup>*(ν*E*, ν*I*, *wE*, *wI*) and ν*<sup>I</sup>* = *<sup>I</sup>*(ν*E*, ν*I*, *wE*, *wI*) (see Section Mean-field approach in Supplementary Material for an explicit form of these functions). An analytical estimation of the stability of this solution has been obtained as well (see Supplementary Material). The heterogeneity parameters *wE* and *wI* have an important effect on the mean firing rates, and allow us to use this mean-field solution, together with numerical simulations of the network, to explore the properties of the system.

## **3. RESULTS**

## **3.1. EFFECT OF HETEROGENEITY ON MEAN FIRING RATE**

Our first step is to understand the effect of an increase of the level of cell-to-cell heterogeneity on the stationary firing rate of the neurons in the network. Due to input noise and the sparseness of the network (which leads to a different number of incoming connections for each neuron), the neurons in the network are not characterized by a common unique mean firing rate (even in the absence of threshold heterogeneity), but rather each neuron has an individual mean firing rate, distributed around an average value following a Gaussian-like profile. This can be seen in **Figure 2A** (in light red, for excitatory neurons) and **Figure 2B** (in light blue, for inhibitory neurons). When we consider some degree of heterogeneity in the neuron firing thresholds, this original distribution of firing rates becomes wider and spans over a large range of firing rate values. For instance, a heterogeneity level of only *wE* = 2 mV for excitatory neurons lead the excitatory firing rate distribution from the previous narrow, Gaussian-like profile to a broad, long-tailed distribution which contains firing rates from zero to even tens of Hertz (**Figure 2A**, dark red). The same effect is observed for the inhibitory population: an increase in *wI* from 0 to 2 mV leads from a narrow, peaked distribution (**Figure 2B**, light blue) to a long-tailed one (dark blue). Excitatory (inhibitory) heterogeneity has also an effect on the shape of the inhibitory (excitatory) firing rate distribution, although it is not as strong as the effect of heterogeneity in a given population on the firing rate distribution of that same population (not shown).

Heterogeneity has a significant effect not only on the shape of the distributions, but on the mean firing rate of the populations in the network as well. The case of a single excitatory population was already considered in a previous work (Mejias and Longtin, 2012), where it was shown that an increase of neural heterogeneity triggered the appearance of a group of low-threshold neurons with higher firing rates (similar to the long tail in the dark-colored distributions in **Figures 2A,B**), and this group produced an extra recurrent input on the high-threshold neurons forcing them to increase their firing rate. The effect had a strong collective component, since a simple firing rate increase in the low-threshold neurons would have been at least partially compensated by a decrease in the high-threshold neurons, were they not connected to each other. The overall recurrent activity generated by the low-threshold neurons contributed to avoid a sudden drop in the firing rate of high-threshold neurons, yielding an overall quadratic-like increment in the network mean firing rate as the heterogeneity level increased.

The situation is more complex in the present case, where we have two different and interconnected populations of neurons (the excitatory and the inhibitory population), and also one heterogeneity parameter for each population. The results from the mean-field approach as well as from the numerical simulations can be seen in **Figures 2C,D**. **Figure 2C** shows the effect of increasing the excitatory heterogeneity in the activity level of the system. Our mean-field prediction, which agrees very well with numerical simulations, shows that increasing the heterogeneity level of the excitatory population leads to a rise in both excitatory and inhibitory activity. This can be easily understood by considering that the increase of *wE* produces the effect in the excitatory

firing rate described above, and this in turn increases the input entering from the excitatory to the inhibitory neurons, rising the inhibitory firing rate as well. The increment in the inhibitory rate also modulates back the excitatory population, which implies that the effect of *wE* on the excitatory population is not as pronounced as for the case of an isolated excitatory population.

The effects of increasing the inhibitory heterogeneity are, however, qualitatively different from those produced by its excitatory counterpart. As **Figure 2D** shows, increasing the inhibitory heterogeneity produces a rise in the inhibitory activity but decreases the excitatory activity. The origin of this effect is that increasing *wI* leads to the appearance of low-threshold inhibitory neurons with high firing rates, which increases the firing rate of the inhibitory network. This in turn induces more inhibition in the excitatory population, which lowers its level of activity as a consequence. It is interesting to note that, due to the negative character of the feedback within the inhibitory population, the increment in the inhibitory firing rate with the inhibitory heterogeneity is only due to the appearance of low-threshold inhibitory neurons, which pull the average firing rate up. On the other hand, we have three different factors that pull this average down: (i) the appearance of high-threshold inhibitory neurons, (ii) the decay in the positive contribution of the excitatory firing rate, and (iii) the presence of negative feedback within the inhibitory population. Because of this, the increase in the inhibitory firing rate with *wI* is not as strong as the increase of the excitatory firing rate with *wE*, where the feedback is positive. In particular, the effect of *wE* on the excitatory firing rate is about twice as strong as that of *wI* in the inhibitory firing rate, as one can see from the differences in the scale of the vertical axis in **Figures 2C,D**.

## **3.2. HETEROGENEITY AS A GAIN CONTROL MECHANISM**

After observing the strong effect that heterogeneity has on the mean firing rate of a cortical network for a given external input, an immediate question follows: how does neural heterogeneity influence the general input–output properties of cortical circuits? A first approach to answering this question is to analyze the effect of heterogeneity on the input–output dependence, or f-I curve, of the neural network. The f-I curve of a given neural system gives the relationship between a slow (usually considered constant) input to the circuit and the *readout* or mean firing rate of that circuit. There are a number of biophysical mechanisms which are able to modify or control the shape of this curve (a strategy commonly referred to as gain control). One can typically distinguish between several types of gain control, the most common ones being subtractive, divisive, or non-monotonic gain control (see Mejias et al., 2014 for an example of a system able to display these three regimes).

In this section, we will use our mean-field approach, together with numerical simulations, to address the role of neural heterogeneity as a relevant factor for gain control. Our results are shown in **Figure 3**, where different f-I curves are plotted, for both the excitatory and the inhibitory populations and varying either *wE* or *wI*. The effect of the excitatory heterogeneity on the f-I curves of the system is shown in the top panels of the figure, where we can see that increasing *wE* linearizes the f-I curve for both the excitatory (**Figure 3A**) and the inhibitory (**Figure 3B**) populations. The effect of the excitatory heterogeneity is similar to the noiseinduced linearization effect in a recurrent spiking neural network (Sutherland et al., 2009). As in the case of temporal fluctuations in noisy input currents, the effect of cell-to-cell heterogeneity is particularly important around the onset of the f-I curve, when most of the neurons lie between the fluctuation-driven and the mean-driven regime.

The effect of increasing the heterogeneity level in the inhibitory population is, however, more complex. As we can see in **Figure 3C**, the increase of *wI* leads to a decrease in the excitatory firing rate for any given input value. Interestingly, the effect of *wI* on the excitatory f-I curve is of a divisive nature, meaning that the inhibitory heterogeneity can be used as a divisive gain control parameter to perform multiplicative and divisive operations in cortical computations. Such a divisive gain control effect holds

**the f-I curves of the system.** The input in the horizontal axis corresponds to the external excitatory input μ. The inhibitory bias remains at μ<sup>0</sup> = 17 mV at all times. Left panels show the effect on the excitatory population, while right panels do the same for the inhibitory population. In **(C)**, one can observe a clear divisive gain control of the excitatory f-I curve when *wI* varies. This divisive effect is not present in correspond to the *wI* = 0 case (red line) but rescaled by a constant factor to fit the other cases, indicating that the effect of *wI* can be described as divisive gain control. The constant factor ζ used to rescale the *wI* = 0 f-I curve was obtained for each case by minimization of the squared distance (see main text) and the resulting values are, for increasing *wI* , the following: 0.772, 0.531, 0.355, and 0.223.

for small and moderate values of the f-I curve, although as we move to strongly mean-driven conditions (i.e., μ ≥ 20 mV) the divisive control is lost (not shown). The main effect of increasing *wI* on the inhibitory population is an increase in the mean firing rate for a wide range of input values, as **Figure 3D** shows; no clear subtractive, divisive or linearization effect is apparent for the inhibitory cells as their heterogeneity is increased.

Due to the relative complexity of the model (and, in particular, its recurrent connectivity nature), it is not easy to obtain, even approximately, a theoretical proof of the divisive nature of the gain control effect of *wI*. Indirect measurements, however, can be used to test this hypothesis. For instance, in **Figure 3E** we have computed, using our mean-field approach, the f-I curve for the excitatory population and different values of *wI*. We have set μ<sup>0</sup> = 12 mV and *JEI* = −0.4 mV so that the divisive effect is stronger and more easily identifiable than in **Figure 3C** for the range of biases shown. By taking the f-I curve for *wI* = 0 (in red) and multiplying it by a given constant factor ζ , one can obtain an f-I curve for *wI* > 0. To do this systematically, we have defined the squared distance between the rescaled *wI* = 0 f-I curve [namely ζ *r*0(*I*)] and a given *wI* > 0 f-I curve [namely *rw*(*I*)] as

$$\Delta = \frac{1}{n} \sum\_{i=1}^{n} \left[ \zeta \left[ r\_0(i) - r\_{\text{w}}(i) \right] \right]^2,\tag{4}$$

where the subindex *i* runs over all the input values considered in the numerical evaluation of the curve, and *n* = 25 is the total number of these values. By systematically varying the rescaling constant factor ζ , we find the value of this factor that minimizes the squared distance between both curves. This fitting is possible for all values of *wI* considered, and the squared distance at the optimal rescaling factor is always small (<0.003). For instance, by multiplying the original (*wI* = 0) f-I curve by a factor of ζ = 0.772, we obtain an f-I curve that fits very well ( < 0.0022) the f-I curve for *wI* = 2 mV. The good overlap between the rescaled *wI* = 0 curve (circles in **Figure 3E**) using different multiplicative constants and the original mean-field solutions (in colored lines), as demonstrated by the small values of obtained and graphically displayed in **Figure 3E**, indicates that the observed effect is indeed divisive. Simulation results are not displayed for **Figure 3E** for an easier visual comparison with the rescaled curves, although simulations agree very well with the mean-field predictions as in the previous set of parameters (see **Figure 3C** for a reference). We have further assessed the goodness of fit by checking that the residuals for each fit are distributed around zero, with approximately two thirds of the data points falling within one standard deviation of the data distribution, as expected for zero-mean Gaussian statistics. Other quantities for measuring the goodness of fit, like a normalized version of the quantity used here (which prevents our fit to depend on the average firing rate), also give the same results.

#### **3.3. STABILITY AND PHASE DIAGRAM**

So far, we have described the behavior of our cortical network model by assuming fixed point conditions, which led us to asynchronous steady-state solutions of the dynamics. Spiking neural networks are known to display other non-linear dynamics for certain conditions, such as multistability (Compte et al., 2000; Wang, 2001), fast global oscillations (Brunel and Hakim, 1999; Brunel, 2000), or winner-take-all dynamics (Wang, 2002; Wong and Wang, 2006). Although our aim in this study is not to fully characterize this kind of behavior in heterogeneous cortical networks– which would require more advanced calculations– we can study the local stability of the dynamics of the network to map the regions in the parameter space where our conclusions hold. To accomplish this, we compute the Lyapunov exponents of our system (see Equation 15 in Supplementary Material) and estimate, for a given value of the heterogeneity parameters, the maximum external input μ for which the asynchronous steady-state is stable (i.e., all eigenvalues have a negative real part). This limit would give us a clear frontier between asynchronous (top panel in **Figure 4A**) and synchronous (bottom panel in **Figure 4A**) network mean firing rate.

**Figure 4B** shows this maximum external input as a function of the excitatory heterogeneity, for different values of inhibitory heterogeneity. We can observe that, for each *wI* value, the maximum external input decreases linearly with the level of excitatory heterogeneity, as in the previously studied case of a purely excitatory heterogeneous network (Mejias and Longtin, 2012). This indicates that networks with highly heterogeneous excitatory neurons are able to enter the synchronous regime with less external stimuli than for the homogeneous case. The observed early synchronization in heterogeneous networks arises due to the presence of low-threshold excitatory neurons. This subset of neurons has a higher firing rate, and therefore they generate a stronger recurrent input that makes them closer to the bifurcation point from asynchrony to synchrony. As a consequence, low-threshold neurons become synchronous with less external input and they in turn contribute to the early synchronization of the rest of the neurons in the network, as we observe in **Figure 4B**.

The effect of the inhibitory heterogeneity is much less significant (see **Figure 4C**), although one can distinguish a small increase in the maximum external input for large enough values of *wI*. This is to be expected, since a large *wI* value would increase the inhibitory firing rate, inducing a decrease in the excitatory firing rate that must be compensated with a higher external input. Therefore, the synchronization onset will be located at a higher external input value. For both panels (**Figures 4B,C**), numerical simulations (points) agree very well with our mean-field predictions (lines). For large heterogeneity values (*wE*, *wI* ≥ 2.5 mV), the quenched disorder together with the stochasticity of the system make it difficult to accurately detect the synchronization onset. To avoid this problem, we have restricted our analysis to situations in which both the excitatory and inhibitory heterogeneity levels were small (*wE*, *wI* < 2.5 mV).

We have also used our mean-field approximation to compute a *wE* − *wI* phase diagram of the behavior of the system, which is shown in **Figure 4D**. For both *wE* and *wI* small, the system is in the asynchronous regime. The asynchronous state continues being stable for increasing *wI*, since the subsequent increment in the inhibitory firing rate contributes to stabilize the network dynamics as explained above (see **Figure 4C**). Only when *wI* takes moderate values and *wE* is significantly increased, synchronous behavior appears in the network dynamics. As the inhibitory

**FIGURE 4 | Stability of networks of heterogeneous neurons. (A)** Examples of the excitatory network mean firing rate in the asynchronous (top) and synchronous (bottom) regimes. **(B)** Critical external input as a function of the excitatory heterogeneity *wE* , for μ<sup>0</sup> = 12 mV and different values of *wI* . An external input higher than the critical one will induce spontaneous synchronization in the network. Mean-field predictions (lines) agree with numerical simulations (symbols). **(C)** Critical external input as a function of *wI* , for μ<sup>0</sup> = 12 mV and different values of *wE* . **(D)** Phase diagram in the *wE* − *wI* space, obtained using the mean-field approach from **(B,C)**, for an external input of 17 mV and different levels of the inhibitory population bias μ0. For all panels, σ = 1 mV.

bias μ<sup>0</sup> takes larger values, the stabilizing effect of increasing *wI* reduces the area of the regime of synchronous dynamics. This is due to the fact that *wI* allows for a stronger modulation of the inhibitory firing rate when μ<sup>0</sup> is larger, as the activity of the inhibitory low-threshold neurons will be higher in this case.

It is interesting to highlight that the results presented here (together with other recent studies such as Mejias and Longtin, 2012) provide counter-intuitive situations where heterogeneity promotes synchronization rather than impede it (Borgers and Kopell, 2003; Denker et al., 2004). A comprehensive study of the contrast between our results and the dynamical mechanisms previously reported is, however, beyond the scope of this study and will be addressed in future work.

## **3.4. SIGNAL DETECTION**

Since both excitatory and inhibitory heterogeneity have a significant impact on the input–output characteristics for constant input, it is convenient to extend our analysis to consider the effect of heterogeneity in the transmission of more realistic, time-varying signals. In particular, previous work showed that the presence of a certain level of heterogeneity can optimize the transmission of slow signals under rate coding in excitatory populations (Mejias and Longtin, 2012). The phenomenon was also present in cortical-like networks with sparseness and inhibition, although in this more realistic case, no theoretical approximations were provided to support these claims.

The mean-field approximation presented in the Supplementary Material (see Section Mean-field approach) constitutes a useful tool to investigate these heterogeneityinduced resonances in cortical-like network models, and to evaluate the likelihood of this phenomenon to occur in real cortical circuits. We consider an external input constituted by

$$
\hat{\mu}(t) = \mu + \text{S}\_0 \sin \left(2\pi f\_\text{\textdd} \right),
\tag{5}
$$

with the first part of the r.h.s. being a constant input and the second part being a slow and weak modulation. Such a weak input modulation is able to drive the excitatory mean firing rate of the network under certain circumstances, a situation which is shown in the inset of **Figure 5B**. A convenient measurement to quantify this behavior is the zero-lag input–output covariance function, which is given by

$$C \equiv \langle \hat{\mu}(t) \nu\_E(t) \rangle - \langle \hat{\mu}(t) \rangle \langle \nu\_E(t) \rangle. \tag{6}$$

**Figure 5A** shows this input–output covariance as a function of the excitatory heterogeneity. For networks of homogeneous neurons, the modulation part of the signal is typically too weak to be noticed. For higher values of the excitatory heterogeneity, however, the sensitivity of the network to small inputs increases, due to the existence of a larger number of low-threshold excitatory neurons. As a consequence, the small input modulation is now able to strongly drive the output (i.e., the excitatory network mean firing rate). For even higher values of *wE* the overall activity of the network increases drastically and the variations due to input modulations become slightly weaker. This makes the signaldriven firing rate modulations too small compared to the baseline firing rate, and as a consequence the quality of signal transmission to a given linear readout system decreases. The overall effect is a bell-shape dependence of the input–output covariance with *wE* shown in **Figure 5A**, indicating that a certain non-zero level of

by *S*<sup>0</sup> = 0.5 mV and *fs* = 2 Hz. **(B)** Same as in **(A)**, but as a function of excitatory heterogeneity optimizes signal transmission, as in the

findings (symbols). Other parameters are *JEE* = *JIE* = 0.043 mV, *JEI* = *JII* = −0.06 mV, and μ = μ<sup>0</sup> = 15 mV, and the signal is characterized

simpler case of a purely excitatory population studied in Mejias and Longtin (2012). Furthermore, our novel mean-field approach for excitatory and inhibitory sparse populations closely follows the numerical results.

The effect of inhibitory heterogeneity on signal transmission is notably different from the situation explained above. As we can see from **Figure 5B**, the input–output covariance tends to be weaker in networks whose inhibitory neurons are highly heterogeneous. This is due to the fact that inhibitory heterogeneity causes an increase in the inhibitory firing rate, which reduces the sensitivity of the excitatory population to weak stimuli and therefore hinders its capacity for signal detection. This is consistent with recent experimental and theoretical findings which show that correlations between two neurons decrease as their firing rate decrease (de la Rocha et al., 2007), and it suggests that heterogeneity in inhibitory neurons may have an important role in decorrelation between input signals and neural activity.

We can also see that the detection of the signal is frequencydependent. In **Figure 5C** the input–output covariance as a function of *wE* is computed for different values of the signal frequency *fs*, with a small shift of the peak toward more heterogeneous networks as the input frequency is increased. This behavior was also observed for the case of one isolated excitatory population (Mejias and Longtin, 2012), and suggests that the ability of neural networks to efficiently detect and transmit signals of a given frequency range depends on the heterogeneity level of the network. The decrease of the signal detection as a function of the inhibitory heterogeneity also depends on the input frequency considered (**Figure 5D**), although for large enough *wI* the signal

since the mean-field predictions does not hold for high input frequencies. **(D)** Same as **(B)**, but for different input frequencies and only considering

the numerical results. **(C,D)** share the same color code.

## **3.5. MULTIPLE HETEROGENEITY SOURCES**

is not detected regardless of the frequency.

As a final remark, it should be noted that, in all of the simulations and analyses presented so far, either the excitatory or the inhibitory heterogeneity was varied, while the heterogeneity of the other population was kept fixed at a very low level (0.1 mV, which would correspond to a almost homogeneous population). In the situations in which the system under study behaves in a linear fashion and the effects caused by parameter variations are independent, this approach is convenient to systematically characterize the behavior of the system. The neural network under study, however, is known to display multiple kinds of non-linear behavior (such as, for instance, the non-linear dependence of the mean firing rate with the heterogeneity level, as shown in **Figures 2C,D**). It is, therefore, unclear whether one can infer the response of the system for arbitrary combinations of heterogeneity parameters from the curves and results presented in previous sections, in which mainly only one type of heterogeneity was analyzed at a time.

In order to test the validity of our results in more complex scenarios, we have jointly increased both heterogeneity levels (*wE* and *wI*) at the same time, and we numerically computed the excitatory mean firing rate of the network as a function of this *combined* heterogeneity level *W* (with *W* = *wE* = *wI*). To test the linearity of the system to the presence of multiple sources of heterogeneity, we also compute, using the mean-field solution, the changes in the excitatory firing rate due to only *wE* or only *wI* (while keeping the other heterogeneity level at zero value), and we add these two contributions together. The comparison is shown in **Figure 6**, where we can see that the simulation results closely follows the linear prediction up to values of the combined heterogeneity of ∼ 4 mV. This finding implies that the effects of multiple types of heterogeneity can add up linearly in some parameter regimes, such as the one in this preliminary investigation. Therefore, the results of the present work are also valid for more realistic situations in which different types of heterogeneity (i.e., excitatory and inhibitory) are simultaneously present in the system.

## **4. DISCUSSION**

The importance and roles of intrinsic neuronal heterogeneities on the dynamics of neural networks is starting to being uncovered in recent years (Marsat and Maler, 2010; Padmanabhan and Urban, 2010; Savard et al., 2011; Angelo et al., 2012; Tripathy et al., 2013). Although commonly disregarded in most modeling studies, an increasing level of attention has been drawn to the subject by theoretical and computational models as well (Golomb et al., 2001; Denker et al., 2004; Luccioli and Politi, 2010; Mejias and Longtin, 2012; Nicola and Campbell, 2013). In particular, novel theoretical frameworks for addressing the heterogeneity of neural systems

have been proposed recently. Yim and colleagues, for example, propose a theoretical approach especially useful for addressing the relationship between neural heterogeneity and neural correlations (Yim et al., 2013), and they sketch a possible explanation for recent evidence of a positive role of heterogeneity on population coding (Padmanabhan and Urban, 2010). In another recent work, Nicola and Campbell provide a set of mean-field approaches used to shed light onto a heterogeneity-induced change on the nature of the Hopf bifurcation responsible for burst generation (Nicola and Campbell, 2013).

The theoretical understanding of the effects of heterogeneity on neural systems is still a young problem, though, and only simple situations have been considered up to now. In this work, we have analytically and computationally studied the interplay between population-specific levels of cellular heterogeneity, an important problem that has not been properly addressed to date. Interestingly, the effects that excitatory heterogeneity produces on neural networks are quite different from the ones produced by inhibitory heterogeneity. Excitatory heterogeneity, as we have shown, non-linearly increases the network mean firing rate with respect to that of a homogeneous network, and the f-I curves of the system are linearized as well. In this sense, excitatory heterogeneity may be viewed as a classical *quenched disorder* in excitable systems, with similar effects on the f-I curve than that of pure noise (Doiron et al., 2001). On the other hand, the introduction of inhibitory heterogeneity induces an increase (with respect to homogeneous networks) in the inhibitory firing rate and a decrease in the excitatory firing rate, and a divisive modulation of the f-I curve as a result. Divisive gain control mechanisms is often assumed as a key operation for neural computations (Carandini and Heeger, 1994; Chance and Abbott, 2000), but biophysical mechanisms for such a modulation have been hard to identify, regardless of being network-based mechanisms (Holt and Koch, 1997; Doiron et al., 2001; Chance et al., 2002; Mejias et al., 2014) or cell-based mechanisms (Prescott and De Koninck, 2003; Mehaffey et al., 2005). The identification of neuronal heterogeneity in inhibitory populations as a biophysically realistic mechanism for multiplicative and divisive gain control constitutes one of the key achievements of the present study.

The analysis of the stability of the fixed point solutions of heterogeneous networks also provides useful information about the effects of heterogeneity on neural networks. Again, the effects of neural heterogeneity heavily depend on the population in which it is found. Excitatory heterogeneity leads to an easier spontaneous synchronization of the neural network, while inhibitory heterogeneity has a weaker effect and tends to slightly increase the robustness of the asynchronous state. This produces a rich repertoire of stability behaviors in neural networks, with the stability conditions of a given particular network depending on its balance between excitatory and inhibitory heterogeneity.

In recent works, an optimal information transmission has been shown to occur for heterogeneous populations of neurons (Marsat and Maler, 2010; Padmanabhan and Urban, 2010), and the presence of short-term synaptic plasticity has been suggested to increase the efficiency of coincidence detection in the presence of heterogeneity via the appearance of *optimal frequencies* (Mejias and Torres, 2008). These findings indicate that heterogeneity may have an important role in information transmission in neural systems. In the present study, we have demonstrated here that the optimization of signal detection by networks of heterogeneous neurons under rate coding, first described in Mejias and Longtin (2012), holds for the more realistic cortical-like network used here, by means of both numerical simulations and mean-field approaches. The improvement of signal detection in heterogeneous neural and excitable systems has been a recent focus of interest. For instance, Tessone et al. (2006) found that global synchronized events in response to weak, slowly modulated external signals can be optimized in heterogeneous networks, a result that has been also obtained in neural networks with electrical and chemical synapses (Perez et al., 2010). Global synchronized events in heterogeneous networks can also work at very short time scales, being triggered by fast input and allowing for an efficient temporal coding (Mejias and Longtin, 2012). Recent experimental work has also highlighted an optimization of population coding in networks of heterogeneous neurons (Marsat and Maler, 2010; Padmanabhan and Urban, 2010; Savard et al., 2011; Angelo et al., 2012), establishing a solid ground for neural heterogeneity as a key ingredient of neural coding.

Finally, it is worth noting that, although we have studied exclusively the case of heterogeneity in the distance-to-threshold of LIF neurons, our mean-field approach can be used to study heterogeneity in other parameters as well. Indeed, there are many potential biophysical sources of heterogeneity in neural systems, both at the network level (i.e., heterogeneity in the network connectivity, as in Olmi et al., 2010) and at the neuron level. In this second group, possible heterogeneity sources can be defined in terms of anatomical and morphological properties, or also at a functional level, including neuronal excitability (Tessone et al., 2006; Perez et al., 2010), different degrees of spike frequency adaptation (Hemond et al., 2008; Nicola and Campbell, 2013), or other biophysical properties (Padmanabhan and Urban, 2010; Tripathy et al., 2013), to name a few. Understanding their individual or joint role in neural dynamics will require future modeling work at different scales and levels of detail, for which mean-field approaches could be of great help.

#### **ACKNOWLEDGMENTS**

This work was funded by the Natural Sciences and Engineering Research Council of Canada (NSERC) and in particular its Discovery Accelerator program, and the Canadian Institutes for Health Research (CIHR). Jorge F. Mejias would like to thank Lidia Roman-Gonzalez for help with some figures.

#### **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at: http://www.frontiersin.org/journal/10.3389/fncom.2014. 00107/abstract

## **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 23 April 2014; accepted: 21 August 2014; published online: 12 September 2014.*

*Citation: Mejias JF and Longtin A (2014) Differential effects of excitatory and inhibitory heterogeneity on the gain and asynchronous state of sparse cortical networks. Front. Comput. Neurosci. 8:107. doi: 10.3389/fncom.2014.00107*

*This article was submitted to the journal Frontiers in Computational Neuroscience. Copyright © 2014 Mejias and Longtin. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Heterogeneity of heterogeneities in neuronal networks

## *Fabiano Baroni 1,2,3\* and Alberto Mazzoni <sup>4</sup>*

*<sup>1</sup> School of Psychological Sciences, Monash University, Clayton, VIC, Australia*

*<sup>2</sup> NeuroEngineering Laboratory, Department of Electrical and Electronic Engineering, University of Melbourne, Parkville, VIC, Australia*

*<sup>3</sup> Centre for Neural Engineering, University of Melbourne, Carlton, VIC, Australia*

*<sup>4</sup> The BioRobotics Institute, Scuola Superiore Sant'Anna, Pontedera, Italy*

*\*Correspondence: fabianobaroni@gmail.com*

#### *Edited by:*

*Mark D. McDonnell, University of South Australia, Australia*

#### *Reviewed by:*

*Roberto Fernández Galán, Case Western Reserve University, USA Benjamin Lindner, Bernstein Center for Computational Neuroscience, Germany*

**Keywords: integrate-and-fire neurons, excitation/inhibition balance, neuronal network dynamics, biophysical heterogeneity, dynamical heterogeneity, signal modulation, high-frequency oscillations**

**A commentary on**

**Differential effects of excitatory and inhibitory heterogeneity on the gain and asynchronous state of sparse cortical networks**

*by Jorge F. Mejias, André Longtin (2014). Front. Comput. Neurosci. 8:107. doi: 10.3389/fncom.2014.00107*

Neurons in the brain exhibit a broad spectrum of heterogeneities even within a given morphological or physiological class. In a recent modeling study, Mejias and Longtin investigated the effects of heterogeneity in the voltage threshold for spike generation on the dynamics of random networks of excitatory and inhibitory neurons (Mejias and Longtin, 2014), hence extending their previous results on purely excitatory networks (Mejias and Longtin, 2012).

The authors focused on the different effects of heterogeneity when incorporated in either the excitatory or the inhibitory population. A greater heterogeneity in the excitatory population increases the average firing rate of both neuron types, because the subset of most excitable excitatory neurons provides a positive feedback to the whole network. Conversely, when heterogeneity is included in the inhibitory population, only the average firing rate of inhibitory neurons increases, while the average firing rate of excitatory neurons decreases. This result can be explained by the presence of lowthreshold, highly excitable inhibitory neurons which tend to shift the average firing rate of the inhibitory population to higher levels, while increasing inhibitory currents in the network. While the silencing effect of an increase in inhibitory currents in the heterogeneous inhibitory population is overcome, on average, by the presence of highly excitable neurons, it dominates the dynamics of the homogeneous excitatory population, hence reducing the average firing rate of excitatory neurons.

While firing rates increase with heterogeneity of excitatory cells, potentially leading to run-away excitation in the absence of saturation or adaptation mechanisms, other features of network dynamics exhibit a non-monotonic dependence on heterogeneity. For example, the encoding of an oscillatory input signal is optimal when the network exhibits an intermediate level of heterogeneity, consistently with a recent experimental study (Tripathy et al., 2013). This behavior is reminiscent of stochastic resonance, a general phenomenon observed in excitable systems, whereby intermediate levels of noise enable optimal information encoding. In fact, heterogeneity can be considered as a form of spatial noise.

When discussing about heterogeneity in the nervous system, it is important to distinguish between biophysical heterogeneity, which relates to neuronal parameters (in simulations) or quantities that are static in the time scales of interest (in experiments), and dynamical heterogeneity, which refers to measures of ongoing neuronal activity such as firing rates and correlations. The relationships between the two can be usefully explored in both directions: while Mejias and Longtin explored the dynamical consequences of different levels of biophysical heterogeneity (bottom-up), others started from experimental observations of dynamical heterogeneity, and investigated neuronal models that are consistent with the observed dynamics (topdown, Koulakov et al., 2009; Roxin et al., 2011).

Crucially, different biophysical sources of heterogeneity can yield similar effects at the level of network dynamics. For example, the strongly skewed, lognormallike distribution of firing rates typically observed in large-scale neuronal recordings (recently reviewed in Buzsáki and Mizuseki, 2014) can be explained by models that include nonrandom connectivity among linear neurons (Koulakov et al., 2009), as well as by homogeneous networks with random connectivity among more realistic nonlinear neurons, due to the expansive nonlinearity of the *f-I* curve (that is, the superlinear increase in output firing rate *f* with increasing input current *I*) in the presence of noise (Roxin et al., 2011). Similarly, delay and synaptic time scale diversity yield equivalent effects (Biggio et al., 2013).

Further, neuronal heterogeneity can arise from different biophysical substrates, and how the effects of different sources of heterogeneity interact is unclear. In fact, heterogeneity has been observed in virtually every aspect of neuronal physiology where it has been investigated. These include intrinsic neuronal properties, which are mostly determined by ion channels' dynamics, such as neuronal excitability, propensity to bursting, amplitude and time course of spikefrequency adaptation, post-inhibitory rebound, and many more (e.g., Marder, 2011; Angelo et al., 2012); as well as properties related to the connectivity among neurons, such as dynamics and efficacy of synaptic transmission, density and size of dendritic spines, thickness and myelination of axons (e.g., Dobrunz and Stevens, 1997; Song et al., 2005; Wang et al., 2008).

Mejias and Longtin reported that, in their simulations, the effects of heterogeneity in the distance-to-threshold of excitatory and inhibitory neurons summed linearly when combined. However, we should expect that, in general, the effects of heterogeneity in different biophysical parameter might interact in a complex manner. For example, Roxin et al. reported that heterogeneity in synaptic weights can decrease the variability in firing rates caused by the expansive nonlinearity of the *f-I* curve, contrary to expectations of a linear interaction between these two sources of variability (Roxin et al., 2011). The effects of neuronal heterogeneity are expected to be more complex in more physiologically relevant settings, where different sources of heterogeneity coexist and are distributed in a highly non-random fashion (Yassin et al., 2010).

While biophysical heterogeneity is widespread at all levels of description, we believe that important insights into the relationships between biophysical and dynamical heterogeneities can be obtained using reductionist approaches, where the degree of biophysical diversity can be described by few parameters. For example, neuronal network studies that investigated the role of connectivity heterogeneity yielded important insights by focusing on random and scale-free connectivity (described by a single parameter), both of which display important dynamical differences with respect to homogeneous all-to-all connectivity. We propose that a similar approach could be fruitfully applied to other forms of biophysical heterogeneity, and ultimately result in useful taxonomies of the different sources of biophysical heterogeneity, describing the dynamical heterogeneities they result in and the interactions between their effects. This level of understanding would facilitate the conceptual integration of different results and eventually lead to basic functional principles of neuronal processing beyond area- or species- specific details.

## **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 16 October 2014; accepted: 21 November 2014; published online: 10 December 2014.*

*Citation: Baroni F and Mazzoni A (2014) Heterogeneity of heterogeneities in neuronal networks. Front. Comput. Neurosci. 8:161. doi: 10.3389/fncom.2014.00161*

*This article was submitted to the journal Frontiers in Computational Neuroscience.*

*Copyright © 2014 Baroni and Mazzoni. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## A Markov model for the temporal dynamics of balanced random networks of finite size

## *Fereshteh Lagzi\* and Stefan Rotter*

*Bernstein Center Freiburg and Faculty of Biology, University of Freiburg, Freiburg, Germany*

#### *Edited by:*

*Mark D. McDonnell, University of South Australia, Australia*

#### *Reviewed by:*

*Maurizio Mattia, Istituto Superiore di Sanità, Italy Jonathan D. Touboul, Collège de France, France Richard Hardstone, Vrije Universiteit Amsterdam, Netherlands*

#### *\*Correspondence:*

*Fereshteh Lagzi, Bernstein Center Freiburg and Faculty of Biology, University of Freiburg, Hansastraße 9a, 79104 Freiburg, Germany e-mail: fereshteh.lagzi@ bcf.uni-freiburg.de*

The balanced state of recurrent networks of excitatory and inhibitory spiking neurons is characterized by fluctuations of population activity about an attractive fixed point. Numerical simulations show that these dynamics are essentially nonlinear, and the intrinsic noise (self-generated fluctuations) in networks of finite size is state-dependent. Therefore, stochastic differential equations with additive noise of fixed amplitude cannot provide an adequate description of the stochastic dynamics. The noise model should, rather, result from a self-consistent description of the network dynamics. Here, we consider a two-state Markovian neuron model, where spikes correspond to transitions from the active state to the refractory state. Excitatory and inhibitory input to this neuron affects the transition rates between the two states. The corresponding nonlinear dependencies can be identified directly from numerical simulations of networks of leaky integrate-and-fire neurons, discretized at a time resolution in the sub-millisecond range. Deterministic mean-field equations, and a noise component that depends on the dynamic state of the network, are obtained from this model. The resulting stochastic model reflects the behavior observed in numerical simulations quite well, irrespective of the size of the network. In particular, a strong temporal correlation between the two populations, a hallmark of the balanced state in random recurrent networks, are well represented by our model. Numerical simulations of such networks show that a log-normal distribution of short-term spike counts is a property of balanced random networks with fixed in-degree that has not been considered before, and our model shares this statistical property. Furthermore, the reconstruction of the flow from simulated time series suggests that the mean-field dynamics of finite-size networks are essentially of Wilson-Cowan type. We expect that this novel nonlinear stochastic model of the interaction between neuronal populations also opens new doors to analyze the joint dynamics of multiple interacting networks.

**Keywords: Markov process, self-consistent noise model, balanced random network, nonlinear dynamics, networks of finite size, Wilson-Cowan type model**

## **1. INTRODUCTION**

Cortical neurons of behaving animals show highly irregular patterns of activity. One hypothesis for the source of such irregularity is the balance of excitation and inhibition in the steady state activity of the network (Softky and Koch, 1993; Bell et al., 1994; Shadlen and Newsome, 1994, 1998; Tsodyks and Sejnowski, 1995; van Vreeswijk and Sompolinsky, 1996). Experimental evidence in favor of this hypothesis suggest that excitation-inhibition balance is the principle of brain dynamics (Sanchez-Vives and McCormick, 2000; Shu et al., 2003; Haider et al., 2006; Okun and Lampl, 2008). The balanced state is an emergent self-consistent and stable solution of the temporal dynamics of the network (van Vreeswijk and Sompolinsky, 1996, 1998; Amit and Brunel, 1997b; Brunel, 2000). In other words, in a recurrent balanced network, both excitatory and inhibitory activity are shaped such that in cooperation with each other, they generate a stationary, selfconsistent input-output behavior on the level of the mean and the fluctuations. The collective activity of the involved neuronal populations include weakly correlated and irregular spike trains. Due to its stochastic appearance, this feature is referred to as "self-generated noise" or simply "noise" in this paper. In fact, these fluctuations are generated mainly by the complex recurrent interactions in the network, even in absence of any external source of noise (van Vreeswijk and Sompolinsky, 1996; Kriener et al., 2008). As in our model there is no external source of noise, the fluctuations are very likely due to deterministic chaos in a high-dimensional system (for details see Jahnke et al., 2009).

Temporal fluctuations of neuronal activity reflect brain processes. Transient activity of neuronal networks, for instance, correspond to different neural computations at different stages of a cognitive task (see for example Churchland et al., 2011) or the representation of information in the brain (Destexhe and Contreras, 2006). Fluctuations also influence sensory perception in the case of ambiguous input. This phenomenon has been modeled by a multi-stable noise-driven dynamical system in which activity fluctuations cause transitions between meta-stable fixed points (Moreno-Bote et al., 2007; Deco and Romo, 2008). It was shown that dynamic noise in a neuronal network also gives rise to different dynamical states of the network. Both theoretical and simulation studies have addressed the role of noise for dynamic stability, or for the emergence of oscillations in network dynamics (Brunel and Hansel, 2006; Ghosh et al., 2008; Touboul et al., 2011; Cai et al., 2012) which may have implications for brain functions. Therefore, to understand the functional properties of neuronal networks, it is essential to understand the dynamics of the time dependent variability in such systems. Theoretical studies of balanced random networks indicate that the fluctuations are essentially determined by two factors: neuronal correlations and the finite size of the network (Ginzburg and Sompolinsky, 1994; Brunel and Hakim, 1999; Brunel, 2000).

To address the fluctuations of the activity of a complex high-dimensional system such as a spiking neural network, a reduced low-dimensional description of network activity is needed. However, to compensate for the loss of degrees of freedom, an analytical treatment is needed such that the essential properties of the stochastic dynamics of the system fluctuations are preserved to an acceptable degree. Finding a suitable stochastic model to replace the spiking dynamics of a network is a challenge. One reason is that such networks are hybrid systems, as the membrane potential of each neuron is a continuous variable, and the spiking activities are discrete quantities. Secondly, an appropriate time scale has to be defined because the amplitude and the dynamics of the fluctuations depend on the temporal resolution. Thirdly, the stationary activity of the reduced model should maintain the statistical properties of the population activities, as their statistics are crucial for the switching dynamics of a network with two or more interacting populations (Bressloff and Newby, 2013). Finally, each neuron is a highly nonlinear element in a random network, due to its threshold and refractoriness. It has been hypothesized that in the thermodynamic limit, when the number of neurons becomes very large, the global dynamics is linearized due to the negative feedback from the inhibitory population (van Vreeswijk and Sompolinsky, 1996, 1998; Tetzlaff et al., 2012; Helias et al., 2013). However, in networks of finite size, as we show in this paper, the dynamics are not fully described by a linear framework.

An influential study on population dynamics of excitatory and inhibitory neurons has been performed more than 40 years ago (Wilson and Cowan, 1972). The authors considered an infinitely large number of neurons in each population such that a fraction of all neurons in the population are in the refractory state. They derived an *ad hoc* response function for the non-refractory neurons and using coarse-graining of activities in time, they derived a set of coupled ordinary differential equations. However, correlations and finite size fluctuations were not captured by this model. In essence, without fluctuations in the input, there are no output fluctuations in this model. In fact, most theoretical studies of population interactions using a mean-field approach (see e.g., Gerstner, 1995; Amit and Brunel, 1997a,b; Brunel, 2000; Aviel and Gerstner, 2006; Kriener et al., 2008; Toyoizumi et al., 2009; Cardanobile and Rotter, 2011; Ledoux and Brunel, 2011; Ostojic and Brunel, 2011) have not addressed the consequences of the finite size of the network and/or pairwise correlations for the temporal dynamics of the population activities in a self-consistent way.

There are a few explicit or implicit suggested approaches to study finite size population dynamics. The first type of studies are based on deterministic equations using mean-field approaches that are derived by an external source of noise (Kriener et al., 2008; Toyoizumi et al., 2009; Tetzlaff et al., 2012). Seminal studies on balanced random networks show that a stochastic input is not needed for network fluctuations and the noise in the system is self-generated as a result of recurrent activity and stochastic spiking of neurons (van Vreeswijk and Sompolinsky, 1996, 1998). Therefore, this approach cannot describe the temporal dynamics in a self-consistent way. Moreover, a deterministic set of equations with additive noise (Kriener et al., 2008) cannot reproduce the state dependent fluctuations of the network activity, as we will show in our study.

A second class of studies considered independent neurons with Poisson statistics (Brunel and Hakim, 1999; El Boustani and Destexhe, 2009). This approach could lead to unrealistic number of spike counts in a short time bin. Moreover, with the assumption of uncorrelated neurons, neural Poisson statistics result in network Poisson statistics. As we show in this article, the statistics of the population activities in balanced random networks are not Poissonian. Exploiting a general escape noise model in a point process framework, Spiridon and Gerstner (1999) derived an integral equation for the population activity of a fully coupled network. According to their approach, the finite size effect would show up as a multiplicative noise term in the original equation. As we show later, our analysis supports these results to some extent.

A third approach tries to describe either the dynamics of each neuron (Ohira and Cowan, 1993, 1995; Soula and Chow, 2007; El Boustani and Destexhe, 2009) or the network (Touboul and Ermentrout, 2011; Buice and Chow, 2013a) by a Markov process. The latter has the problem that possible jumps of the Markov process are limited to the immediate neighbors of each state; meaning that the number of active neurons in each population can either increase or decrease by one. The former approach seems to be able to better capture the dynamics and statistics of the network. Soula and Chow (2007) assumed a Markov model in discrete time with a time scale in the range of membrane time constant in the case of instantaneous synapses. The transition rate of a typical neuron in the network is calculated from the stationary firing rate and includes the net amount of excitation in the system. However, balanced random networks operate on a much faster time scale compared to that of a single neuron. Also, the interaction between excitation and inhibition, and the consequences of the negative feedback on temporal dynamics are not analyzed in this study. In another study, following a similar approach in continuous time, a master equation for the activity of a balanced network with current-based and conductance-based synapses was derived (El Boustani and Destexhe, 2009). This method needs the static transfer function of a single neuron that maps input rates to output rates.

Finally, population density approaches based on a conservation law imposed on the probability flux (the number of neurons is constant) (for details see Knight, 1972; Abbott and van Vreeswijk, 1993; Treves, 1993; Knight et al., 1996; Omurtag et al., 2000; Sirovich et al., 2000; Haskell et al., 2001; Nykamp and Tranchina, 2001; Mattia and Del Giudice, 2002) are yet another way of deriving stochastic dynamics from deterministic equations. In a study on the temporal dynamics of the interaction between excitation and inhibition in networks of finite size, an eigenfunction expansion of the essentially nonlinear Fokker-Planck equation resulted in a set of coupled ordinary differential equations (Mattia and Del Giudice, 2002, 2004). However, the noise term included in the model to account for the finite size of the network, was a white noise of fixed amplitude. Recently, Buice and Chow (2013a,b), using a population density method and moment hierarchies of the equation, derived path integrals of a moment generating functional. To get a time dependent correlation function of the system, by introducing a small perturbation, they applied a linear expansion of the equation of moments (Buice and Chow, 2013a,b).

In this paper, we aim at describing statistics and dynamics of finite-size fluctuations in a balanced random network of excitatory and inhibitory neurons self-consistently, such that the temporal dynamics of the network is driven by the interactions in the network. Moreover, a high correlation between the excitatory and inhibitory population, in their stochastic representation in the model, has to be preserved. To this end, a typical neuron in the network is modeled by a two-state Markov system that its transition probabilities needs to be derived dependent on the network activity states. Our model is based on the state space analysis of numerical simulations of interactions between the excitatory and the inhibitory population in a large balanced network, in the regime dominated by inhibition.

Systematic analysis of the two-dimensional population spike counts shows that no dynamic model with additive Gaussian noise can fully describe the temporal dynamics of the network activity. Specifically, the more excitation and the less inhibition is recruited in the network at any given point in time, the higher is the variance of the self-generated noise. It will be demonstrated in this paper that a stationary external input results in nonlinear interactions between excitation and inhibition. Moreover, we will show that the self-generated and state dependent noise emerges naturally as a result of the finite number of neurons in the network. Furthermore, the suggested two-state Markov model is capable of producing a heavy-tailed (positively skewed) distribution of excitatory and inhibitory spike counts, a property of balanced random networks that is considered here for the first time. We show that this heavy-tailed distribution can be well approximated by a log-normal distribution.

#### **2. MATERIALS AND METHODS**

In this section, we first describe our assumptions about the structure and the parameters of the network. Then, we show how to reconstruct the dynamic flow based on the results of spiking network simulations. In Section "Estimation of the dynamic flow underlying the mean-field dynamics" we first suggest a Markov model to represent the collective activity of neuronal populations, with transition probabilities inferred from numerically simulated networks of spiking neurons. The objective of this study is to identify a suitable bin size to represent the temporal dynamics of the network fluctuations. Then, in Section "Markov model for the mean-field and stochastic dynamics" we introduce a two-state Markov model for each neuron, termed the "Active-Refractory Markov" (ARM) model. A method to find the mean-field equations and a self-consistent noise model based on the Markovian single-neuron dynamics is introduced at the end of this section.

#### **2.1. NETWORK STRUCTURE AND PARAMETERS**

The network under study is composed of 10,000 excitatory and 2500 inhibitory neurons, similar to the network studied by Brunel (2000). Each neuron receives local inputs from randomly chosen fraction of the excitatory and the inhibitory population (10% each). An external Poisson process of rate 25 spikes/ms mimics the input from other brain areas. The neurons are modeled by a leaky integrate-and-fire (LIF) dynamics with pulse-like post-synaptic currents (PSC), and exponential post-synaptic potentials (PSP). Therefore, the dynamics of neuron *i* in the network, regardless of whether it is excitatory or inhibitory, satisfies

$$\tau \dot{\nu}\_i(t) = -\nu\_i(t) + \tau \sum\_{j=1}^{N} J\_{\vec{l}\vec{j}} \mathbf{S}\_{\vec{j}}(t - t\_d) \tag{1}$$

where *Sj*(*t*) = *<sup>k</sup>* δ(*t* − *t j <sup>k</sup>*) is the spike train of neuron *j*, which is seen by the postsynaptic neuron with a delay of *td*, and integrated by it with a membrane time constant τ = 20 ms. *Jij* is the amplitude of the post synaptic potential (PSP), *J* = 0.1 mV for the available excitatory synapses to each neuron and −*gJ* = −0.6 mV for inhibitory synapses. *N* is the total number of neurons in the network. The membrane potential of each neuron, once reached the threshold at θ = 20 mV, is reset to *v*reset = 10 mV and a spike will be generated. The membrane potential remains at *v*reset for a refractory period of 2 ms. These parameters are identical to those studied in Brunel (2000) and represent a simplified model for one cubic millimeter of neocortical tissue. The results of our study are valid for a wide range of biologically realistic parameters, as far as the network is in the inhibition dominated regime. Numerical simulations of the network were all conducted in NEST (Gewaltig and Diesmann, 2007) with a time resolution of 0.01 ms and a minimal synaptic delay *td* equal to the time resolution. The network simulation time was 100 s. As a result, a total number of 10<sup>7</sup> data points for each population in a histogram with a bin size of 0.01 ms was available for further data analysis.

## **2.2. ESTIMATION OF THE DYNAMIC FLOW UNDERLYING THE MEAN-FIELD DYNAMICS**

We reconstructed the flow corresponding to the mean-field dynamics from a simulated time series of the spiking network activities. To this end, for each possible combination (*i*, *j*) of excitatory spike count *i*, and inhibitory spike count *j*, the "state" of the system, we first collected the corresponding derivatives by computing the increments in successive time bins divided by the time bin, for both excitatory and the inhibitory spike counts. Taking the mean of the encountered derivatives for each state gave the average direction into which the system moved forward in time. This way, for each state visited in the simulation, a velocity vector is obtained, yielding a vector field that approximates the flow in a two-dimensional state space. Calculating the variance of the state dependent derivatives results in the state dependent variance of the self generated noise (fluctuations) in the system (see Results for more details).

## **2.3. MARKOV MODEL FOR THE MEAN FIELD AND STOCHASTIC DYNAMICS**

As mentioned in the beginning of this section, we considered two different Markov models for different purposes. First, for the selected time bin *dt*, chosen such that a Markov model is capable of reproducing the power spectrum of the network activity with the highest fidelity, a state transition matrix was inferred entirely based on the data obtained from a spiking network simulation. It is important to stress that the Markov model for the selected bin size *dt* provides a compressed description of the network dynamics, assuming that only the most recent bin, and no longer account of the history of the network activity, determines the dynamics of the network at any given point in time. The analysis described here explores the limits of such a description of the the large-scale dynamics, and in this paper we refer to it as "Markov chain analysis." However, deriving the transition matrix from a self-consistent analysis is difficult. Therefore, we looked for a more analytically tractable approach in a second step. This approach, which is the main focus of this study, is based on a two-state Markov description of individual neurons. As shown in this article, this model is able to describe both the dynamics and the statistics of the collective activity of the network in a self-consistent way. The transition rates of this model are also estimated form the spiking network simulations.

#### *2.3.1. Markov chain analysis*

In this section we address the question whether we can identify a time scale such that a Markov model can describe the time dependent dynamics of the network sufficiently well. To that end, based on the joint time series of the two population spike counts, a matrix of transition probabilities for any possible jump from state (*i*, *j*) at time *t* to state (*i* - , *j* - ) at time *t* + *dt* was inferred. Starting from an arbitrary initial condition, a stochastic signal was then generated which was able to match the dynamics and statistics of the two populations PSTH sufficiently well (**Figures 1B–D**). However, as the power spectra of the stochastic implementation of the Markov model were not exactly identical to those of the spiking network simulations, even for the optimal step size *dt* (see next subsection), we conclude that the Markov property is only approximately satisfied. This approximation to the complicated dynamics of the system provides a simple model for the stochastic behavior of the network. It is important to check which properties of the system can be explained by a Markov model of the type discussed here. Some limits of the model due to this assumption are described in the results and will be further analyzed in the Discussion Section.

#### *2.3.2. Selection of the bin size*

To extract the activity of the two neuronal populations, we consider a variant of the Peri-Stimulus Time Histogram (PSTH) of the spike trains of the excitatory and inhibitory neuronal populations. To find a suitable time resolution such that the dependency of the dynamics on the past history is essentially reduced to the most recent time bin, we looked at the autocorrelation functions of the PSTH of each population. As commonly done, we considered the first zero-crossing of the autocorrelation function as a first estimate for the time scale of the dynamics. As **Figure 1A** shows, however, the two populations have slightly deviating time scales. In the range of these two time scales, we explored a set of bin sizes that were integer multiples of the temporal resolution for the spiking network simulations (0.01 ms). For each value, we reconstructed the transition probabilities from the data, as explained in Section "Markov chain analysis." Using these transition probabilities, a stochastic signal for both populations was generated and the power spectra of the signals were compared with those of the spiking network simulations extracted at the same bin size. We found that a bin size of 0.1 ms provides power spectral densities best matching to those obtained from spiking network simulations for both populations (**Figure 1D**). This yields a reasonable time basis for the dynamics, based on numerical experiments, in line with the Markov assumption. Due to the discrete nature of spike counts in each bin, the dynamics in this paper is analyzed in discrete time, with a discrete noise model corresponding to the finite size of both neuronal populations.

## *2.3.3. Two state Active-Refractory Markov model*

The goal is to derive a Markov model the parameters of which can be interpreted in terms of neural dynamics, and that captures the finite size effects. The model should also capture the properties of the spiking network simulation on the level of mean-field as well as the transient fluctuations. We came up with a twostate Markov model, where each neuron is assumed to generate its spikes independently of the other neurons, given the input from the rest of the network. Moreover, it is assumed that each neuron's membrane potential falls into one of two classes: close to threshold (active state) or far away from threshold (refractory state). Hence, we call this process the "Active-Refractory Markov" (ARM) model. A schematic of the neuron model is depicted in **Figure 2**. Transitions from the active to the refractory state can be of two different types. Either the membrane potential of a neuron decays due to the membrane leak [Equation (1)], or the neuron receives inhibitory input spikes, or the neuron fires a spike itself and the membrane potential is reset. The former is described by the β branch; the latter is due to the γ branch in the model (**Figure 2**). More specifically:


In general, α, β, and γ are nonlinear functions of the state (spike counts). In our model, it turns out that α is an exponential function of a linear combination of spike counts; β and γ are constants estimated from the data. Note that the definition of a state in the ARM model deviates from the definition in the Markov chain model described in the previous section. In the latter, states are just spike counts, and transition probabilities are

curves) corresponding to a spiking network simulation with *v*reset = 0 mV. The low frequency dynamics are not very well captured by the Markov model; however, the overall shape of the spectrum, as well as the frequency beyond which the power drops, are well preserved. There are several peaks in the low frequency regime, corresponding to slow network oscillations. **(D)** Power spectral density for a simulation with *v*reset = 10 mV. In contrast to the case of stronger reset, **(C)**, there are fewer peaks in the low frequency range.

also defined in terms of spike counts. In the ARM model, however, the actual states are the occupation numbers of the active and refractory pools. These can neither be directly observed nor inferred from numerical simulations of the spiking neuronal network (SNN), but for our analysis we found a way work around this problem. All neurons in the network under study have the same in-degree and, consequently, share the same input statistics. Therefore, the transition rates are assumed to be identical for excitatory and inhibitory neurons. Neurons are assumed to make their state transitions, in particular spike firing, independently of each other, given their inputs.

The network is comprised of an excitatory and an inhibitory population. A population of size *N* will have *A* neurons in the active state and *N* − *A* neurons in the refractory state. Considering identical and independent transition rates for all neurons, our approach was to take neurons out of a state based on binomial distributions with the number of neurons in each state corresponding to the occupation numbers. The transition probability is given by multiplying the transition rate with *dt*, assuming that this is a small number. This results in the following stochastic description of the system

$$
\Delta A(t) = A(t + dt) - A(t) = \Delta A^+(t) - \Delta A^-(t) \tag{2}
$$

In equation (2), *A*+(*t*) and *A*−(*t*) are the increment and decrement from the active pool *A* that indicate the number of incoming neurons from the refractory pool, and the number of outgoing neurons to the refractory pool, respectively. *A*−(*t*) has a binomial distribution with parameters *A*(*t*) and (β + γ )*dt* as the number of available neurons and the probability of selection, respectively. Therefore,

$$\varrho\left(\Delta A^{-}(t) = \mathbf{x}\right) = \binom{A(t)}{\mathbf{x}} \left( (\boldsymbol{\beta} + \boldsymbol{\nu})dt \right)^{\mathbf{x}} \left( 1 - (\boldsymbol{\beta} + \boldsymbol{\nu})dt \right)^{A(t) - \mathbf{x}} \tag{3}$$

where *A*(*t*) *x* = *C<sup>x</sup> <sup>A</sup>*(*t*) is the binomial coefficient. In this model, the spike count *S* is determined by the number of neurons in the active state and the transition rate γ in two steps: First, the total number of outgoing neurons from the active pool is calculated from a binomial distribution with probability (β + γ )*dt*. This quantity is exactly equal to *A*−(*t*). Second, from *A*−(*t*), the neurons that are actually firing a spike are drawn with another binomial distribution with probability <sup>γ</sup> <sup>γ</sup> <sup>+</sup><sup>β</sup> . Therefore,

$$p\left(\mathbf{S}(t) = \mathbf{z}\right) = C\_{\Delta A^{-}(t)}^{z} \left(\frac{\mathcal{V}}{\mathcal{V} + \beta}\right)^{z} \left(\frac{\beta}{\mathcal{V} + \beta}\right)^{\Delta A^{-}(t) - z} \tag{4}$$

A similar expression as the one given in equation (3), a binomial distribution with rate α, describes the number of incoming neurons to the active pool at time *t*.

$$p\left(\Delta A^{+}(t) = \mathbf{y}\right) = C\_{N-A(t)}^{\mathcal{Y}} (\alpha dt)^{\mathcal{Y}} (1 - \alpha dt)^{N-A(t)-\mathcal{Y}} \tag{5}$$

With *A* neurons in the active state and *N* − *A* neurons in the refractory state at time *t*, the dynamic equation describing the expected value of the dynamics of *A* is

$$\frac{d}{dt}\mathbb{E}[A(t)] = \alpha(N - \mathbb{E}[A(t)]) - (\beta + \gamma)\mathbb{E}[A(t)]\tag{6}$$

It is important to point out that the number of neurons in the active state, *A*(*t*), is not observable, as we record only spikes. To describe the dynamics of the network it is, therefore, easier to describe the temporal dynamics in terms of observables, like the number of spiking neurons *S*(*t*). As mentioned before, *S* is the integer number of spikes generated in the γ branch, where E[*S*] = E[*A*]γ *dt*. It is straightforward to rewrite equation (6) as a function of *S* and get an equation describing the temporal dynamics of the spike counts

$$\frac{d}{dt}\mathbb{E}[\mathcal{S}(t)] = \alpha(\mathcal{N}\boldsymbol{\chi}\,dt - \mathbb{E}[\mathcal{S}(t)]) - (\boldsymbol{\beta} + \boldsymbol{\gamma})\mathbb{E}[\mathcal{S}(t)] \tag{7}$$

For better readability, we will drop the E[. ] operator that indicates the expected values of *A* and *S*. As all neurons are assumed to have the same membrane potential dynamics, determined by equation (1), and receive the same number of inputs regardless of their identity, the two populations have identical transition rates and therefore equation (7) holds for both populations. The only difference between the two populations, however, is the total number of neurons included in each of them. We denote the number of excitatory neurons by *Ne* and the number of inhibitory neurons by *Ni*. Excitatory and inhibitory spike counts generated at time *t* are given by *Se*(*t*) and *Si*(*t*), respectively. A nonlinear regression analysis of equation (7) applied to data from a spiking network simulation shows that α is an exponential function of recent spike counts, and that β and γ are well approximated by constant rates (see the Result Section for more details). Therefore, the two-dimensional mean-field has the following dynamics

$$\begin{cases} \dot{S}\_{\epsilon}(t) &= \exp\left(c\_{0} + c\_{1}S\_{\epsilon}(t) + c\_{2}S\_{i}(t)\right) \left(N\_{\epsilon}\gamma \, dt - S\_{\epsilon}(t)\right) \\ &- (\beta + \gamma \,)S\_{\epsilon}(t) \\ \dot{S}\_{i}(t) &= \exp\left(c\_{0} + c\_{1}S\_{\epsilon}(t) + c\_{2}S\_{i}(t)\right) \left(N\_{i}\gamma \, dt - S\_{i}(t)\right) \\ &- (\beta + \gamma \,)S\_{i}(t) \end{cases} \tag{8}$$

All the unknown parameters of the ODE system (8) can be estimated from the vector field extracted from simulated data. For this purpose, the regression analysis was performed with Python, using general purpose least-square optimization available in the SciPy library (Jones et al., 2001).

The exponential shape of α can be qualitatively justified. As pointed out in previous studies (Brunel and Hakim, 1999; Ricciardi et al., 1999; Brunel, 2000), the time dependent distribution of the membrane potential of a typical integrate and fire neuron, under the assumption of stochastic input and small PSPs, follows the Fokker-Planck equation with a drift and a diffusion term. In general, these two terms are functions of the recurrent activity of the network and therefore result in a nonlinear partial differential equation. However, under the assumption of Gaussian white noise input that is independent of the activity of the network, the steady state solution of equation (9), with appropriate boundary conditions, characterizes the stationary distribution of the membrane potential of a typical leaky integrate-and-fire (LIF) neuron. The dynamics of the distribution of the membrane potential is therefore

$$\tau \frac{\partial}{\partial t} p(\nu, t) = -\frac{\partial}{\partial \nu} \left[ \left( -\nu + \tau \sum\_{k} r\_{k}(t) f\_{k} \right) p(\nu, t) \right] \qquad (9)$$

$$+ \frac{1}{2} \left[ \tau \sum\_{k} r\_{k}(t) f\_{k}^{2} \right] \frac{\partial^{2}}{\partial \nu^{2}} p(\nu, t)$$

$$= -\frac{\partial}{\partial \nu} \left[ (-\nu + \mu) p(\nu, t) \right] + \frac{\sigma^{2}}{2} \frac{\partial^{2}}{\partial \nu^{2}} p(\nu, t)$$

where *rk*(*t*) is the firing rate of the pre-synaptic source *k* and *Jk* is the corresponding PSP to source *k*. μ and σ are the first and second moments, respectively, of the external input to the neuron. Equation (9) is based on the conservation of the probability flux of the membrane potential and could be rewritten in the following form:

$$
\pi \frac{\partial}{\partial t} \mathfrak{p}(\nu, t) = -\frac{\partial}{\partial \nu} \Phi(\nu, t), \tag{10}
$$

where (*v*,*t*) represents the probability flux and shows the probability mass crossing any arbitrary *v* per unit of time *t*. The stationary time independent solution of *p*(*v*) without considering refractoriness and with appropriate boundary conditions at threshold θ, *v*reset and *v* = −∞ is (Gerstner and Kistler, 2002)

$$p(\nu) = \begin{cases} \frac{c\_1}{\sigma} \exp\left(-\frac{(\nu-\mu)^2}{\sigma^2}\right) & \nu < \nu\_{\text{reset}}\\ \frac{c\_2}{\sigma} \exp\left(-\frac{(\nu-\mu)^2}{\sigma^2}\right) \int\_{\nu}^{\theta} \exp\frac{(\mathbf{x}-\mu)^2}{\sigma^2} \,d\mathbf{x} & \nu\_{\text{reset}} < \nu < \theta \end{cases} (11)$$

Intuitively, active and refractory states of a neuron are linked to high and low membrane potentials, respectively. The probability to encounter a potential exceeding a certain value, therefore, should be related to the rate α that describes a transition from refractory to active in the ARM model. The transition rate α, therefore, is proportional to the time dependent flux of the membrane potential crossing the border between the active and the refractory state. For simplicity we can approximate the time (state) dependent flux with the steady state flux plus some fluctuations. These fluctuations are determined by the spike counts of the activity: the larger the excitatory spike counts and the smaller the inhibitory spike counts are, the larger the probability flux crossing the border between the active and the refractory state is. As in equation (10), is related to the voltage integral of *p*(*v*, *t*), the time dependent flux will be proportional to the local behavior of the Cumulative Density Function (CDF) of the membrane potential distribution. In our Result section, it is argued that for a wide range of the membrane potential between *v*reset and θ, the stationary Cumulative Density Function (CDF) is locally well approximated by an exponential function, matching the functions we saw in numerical simulations of SNNs based on the LIF neuron model. A formal derivation of the link between the LIF and the ARM model, however, is mathematically involved and beyond the scope of the present paper.

#### **3. RESULTS**

In this section, numerical results from the simulation of large but finite balanced random networks are illustrated, which all imply either the nonlinear dynamics of interactions, or the non-Gaussian and state dependent nature of the self-generated noise. A comparison between the spiking network simulation results and simulation of the Active-Refractory Markov (ARM) model is made to show the limits and strengths of the model.

#### **3.1. MARKOV CHAIN INFERRED FROM TIME SERIES**

A numerical study of the network shows that with a time bin of 0.1 ms the most essential features of the system under study are recovered by a Markov chain model where the probabilities for state transitions are all extracted from the data. The vector field obtained from the spike counts (**Figure 1B**) and the power spectral density of both populations modeled with a Markov chain (**Figure 1D**) are similar to those obtained from the simulation of the spiking network.

The autocorrelation function (ACF) of the excitatory and inhibitory spike counts (**Figure 1A**) reflect the memory of the system. It has been reported previously (Tetzlaff et al., 2012), and is confirmed again in our study, that the time constant of the decay is smaller for the inhibitory population, although the input to excitatory and inhibitory neurons are statistically the same. This behavior was hypothesized to be related to the negative feedback contribution of the inhibitory population in the large-scale dynamics of the network (Tetzlaff et al., 2012). To examine the Markovian nature of the dynamics, a good choice for the time bin of the histograms is the point where the autocorrelation has its first zero-crossing. For a simulation with a temporal resolution of 0.01 ms, this point is roughly 0.07 ms for the inhibitory population and 0.20 ms for the excitatory population, according to **Figure 1A**. Since a unique time scale for the model and simulations is needed, we chose 0.1 ms to construct the PSTH of both populations.

The low frequency power spectra of the populations are not very well captured by the Markov model. This may be caused by the fact that there is a dependence on the past spiking activity of the two populations, which cannot be reflected by the Markov process employed here. We wanted to find out which features of the system are nevertheless recoverable by a Markov process. The low frequency behavior of the system depends on the distance between *v*reset and the spiking threshold θ (**Figures 1C,D**). The bigger this distance is, the more peaks in the low frequency part of the system appear. Refractoriness can also change the shape of the power spectrum in these frequency ranges (Franklin and Bair, 1995; Mar et al., 1999; Spiridon and Gerstner, 1999). It is obvious that a Markov process due to its lack of memory cannot capture this phenomenon.

## **3.2. TRANSITION PROBABILITIES OF THE ACTIVE-REFRACTORY MARKOV (ARM) MODEL**

All neurons in the network under study are statistically the same. Therefore, identical transition probabilities α, β, and γ were imposed for excitatory and inhibitory neurons. A nonlinear regression to estimate the parameters of equation (8) given the excitatory and inhibitory spike counts and their temporal derivatives from the time series results in an exponential link function for α. The function β exhibits a slight negative dependency on the excitatory spike counts (**Figure 3C**-dots). Ignoring this does not visibly affect the simulation results of the model (data not shown). Therefore, it was assumed to be a constant parameter. An exponential function for α and constant parameters for β (solid lines in **Figure 3**) and γ result in *c*<sup>0</sup> = −0.046,

*c*<sup>1</sup> = 0.032, *c*<sup>2</sup> = −0.152, β = 7.78 and γ = 0.325 (all quantities in spikes/ms) as the estimated parameters of the model in equation (8). Interestingly, *c*2, the coefficient of the inhibitory spike counts is roughly 5 times bigger in amplitude compared with *c*1. This was expected since IPSP = −*g* EPSP. In the stochastic implementation of the ARM model, the transition rate α is a function of random variables *Se* and *Si*. Therefore, the ARM model can be interpreted as a type of "doubly stochastic" point process.

As mentioned in the Methods Section, for a wide range of membrane potentials *v* the CDF of the membrane potential can be approximated by an exponential function (**Figure 4B**). For values of the membrane potential close to threshold, however, the CDF does not match the fitted exponential function very well. This sub-exponential behavior is also observed in the data (dots in **Figures 3A,B**). This similarity suggests that a better approximation for α might come from the integral of the analytical solution of equation (9), but a formal mathematical analysis of this idea is beyond the scope of this paper. Moreover, β can be assumed as the leak term in the dynamics of leaky integrate and fire model due to its role in taking the value of the membrane potential away from the membrane threshold when there is no other inputs to the neuron. Inhibitory inputs in general can influence this parameter as well, however, for the sake of simplicity, we assumed it to be a constant. γ is a constant rate determining the number of spiking neurons at any given time interval *dt*. This term is proportional to the outgoing probability flux from the threshold θ in equation (11).

In comparison to the well-known Wilson-Cowan model (Wilson and Cowan, 1972), the ARM model assumes an exponential, instead of a sigmoidal, function as a transfer function for low input level. The reason is that in the balanced network, the firing rates of the neurons are low and therefore the activity of the populations are far from saturation. The ARM model can be considered as a special case of Wilson-Cowan model that was suggested for the dynamics of fluctuations around the mean firing rates of the populations for time-dependent inputs. The advantage of our model is that it can generate the statistics of the noise from the mean-field dynamics of the system in a self-consistent way.

#### **3.3. STATE SPACE ANALYSIS OF SELF-GENERATED NOISE**

Each pair of excitatory and inhibitory spike counts that are observed in the same time bin define a state in the state space. Spiking network simulations show that the future evolution of states during a particular trajectory of the system is highly dependent on the current state of the network. Therefore, the increment or decrement of the spike counts, as well as the derivative estimated from this (which are basically the difference between the spike counts at successive points in time divided the time interval 0.1 ms) are state dependent (**Figures 5A,B**). For each state, the distribution is heavy-tailed (positively skewed) and has a higher variance in excitation (compare the color-bars of **Figures 6A,B**). For a particular value of inhibitory spike counts, the variance increases as excitatory spike counts increase. Also, for a particular value of excitatory spike counts, increasing inhibitory spike counts results in less variability of the derivative of the spike counts for both excitatory and inhibitory populations. In other words, the more net excitation is in the system, the higher is the variance of the derivatives of the population spike counts. This statistical property of the network dynamics indicates that the noise model cannot be that of additive Gaussian white noise, as otherwise the variability in derivatives would be identical for the entire state space. The reason is that for any stochastic system governed by an equation of type *x*˙ = *f*(*x*) + ξ , if the variance of *x*˙ is the same for the entire state space, ξ could be considered as an additive white noise where the variance of the noise ξ is not state dependent. Otherwise, the noise term has to be state dependent, or non-additive. Multiplicatively interacting point processes are an explicit model for the interaction between neurons (Cardanobile and Rotter, 2010), with non-additive and non-Gaussian noise.

**hint for state probability transition** *α***. (A)** Stationary PDF of the membrane potential of a leaky integrate and fire neuron. The value of the membrane potential that separates the active from the refractory state is somewhere

In the ARM model the transition probabilities are generally state dependent. Therefore, each state has a different distribution of the spike rate increments (derivatives). The model shows the same pattern of state dependent variance, however, the variance is not as high as that of spiking network simulations (**Figures 6C,D**). Particularly, for the excitatory spike counts the variance generated in the model is smaller by a factor of 10 (**Figures 6A,C**). However, this fact does not affect the distribution of the spike counts in the populations drastically (**Figure 9**) and the normalized correlation functions (**Figure 8**) are still recovered with a high accuracy.

**Figure 5** shows that for some states in the state space, the distribution of the derivatives of the spike counts is reproduced by the ARM model with very high fidelity. Specifically for excitation, however, the model does not seem to generate enough variability in the excitatory spike counts. In **Figures 5C,D** the overlap between the distributions of the derivatives generated from SNN and ARM in the entire state space are represented by a number between 0, indicating no overlap, and 1, corresponding to a complete overlap, respectively. To calculate the overlap, first the state dependent distribution of the derivatives both for the model and the simulation data were normalized. Then the two distributions were compared, and for each possible value of the derivative, the minimum between the two distributions was determined. The integral over the minima is a number between 0 and 1 corresponding to the overlap. It is clear that the performance of the model in terms of variability of the inhibitory activity is very good (**Figure 5D**), however, the model is not as good in generating large enough variability in the derivative of the excitatory population. This might be related to the larger number of excitatory

is observed in the case of inhibition. **(C)** Variance of spike count derivatives for the excitatory population in the stochastic implementation of the ARM model. The same pattern as **(A)** is observed but with a 10 times reduced variance. **(D)** Variance of spike count derivatives for the inhibitory population in the ARM model. The scale of the state dependent variance of noise is the same as in **(B)**.

neurons compared to inhibitory ones, and therefore the bigger influence of the pairwise correlations among excitatory neurons that is ignored in the ARM model. The ARM model is based on the assumption that neurons, given the network is in a certain state, perform spike transition independently. This leads us to use binomial distributions to describe the transitions between the two neuronal states. This results in a linear scaling of the variance with the population size. As shown in **Figures 6C,D** (note the different scales of the color bars), the variance of the derivatives in both populations differ by a factor of 4, which is exactly the ratio of the excitatory and inhibitory population sizes. However, in the spiking network simulation, the variance of the derivatives for the excitatory population is much bigger than the inhibitory population (**Figures 6A,B**). We conclude that due to the linear relationship between the variance and the population size, the ARM model systematically underestimates the variance of the activity for the larger subpopulation in the network. This is a drawback of the model and in the discussion section we will suggest ways to overcome this problem. In the next section it is shown that the portion of the state space visited in a stochastic simulation of the model is less spread, and we attribute this fact to the reduced variability in the increments of the excitatory spike counts in the model.

### **3.4. NONLINEAR ISOCLINES**

The state space of the system reconstructed from the simulated data (vector field shown in **Figure 7**) is a good representation of the mean-field dynamics that represents smooth transition of the average fluctuations toward the fixed point. Starting from an arbitrary initial condition in the state space, the mean-field dynamics leads the trajectory to a stable fixed point. However, due to recurrent activity and the finite size of the network, the trajectory driven by the mean-field is continuously perturbed and the result is a quasi-stochastic signal that only on average follows the

**FIGURE 7 | Nonlinear isoclines and dynamic flow (vector-field) in the state space of excitatory and inhibitory spike counts. (A)** Left: Mean of the excitatory spike count derivatives; three levels of contour lines, −200, 0, 200 are shown in black. The isoclines are clearly nonlinear. Right: Mean of the inhibitory spike count derivatives; the contour lines are shown for −50, 0, 50. For inhibition, the nonlinearity is not dominant. **(B)** Mean of the derivatives of the excitatory and inhibitory spike counts in the ARM model. The isoclines

are similar to those in a spiking network simulation. **(C)** Vector field extracted from a numerical simulation of spiking neuronal network (SNN; left) and simulation of the ARM Markov model (ARM); (middle: analytical model, right: stochastic simulation of the model). Parameters of the model were chosen such that the vector fields on the left and in the middle are identical. There is a good match between the vector fields and the nullclines in the simulation and in the model.

mean-field dynamics. In other words, the vector field describes how a trajectory evolves on average.

The vector field has one component for each population representing the mean increment (or derivative) of the spike counts, given the two-dimensional state of the joint population. The excitatory and inhibitory components of the average state dependent derivatives are shown in **Figure 7A**. Isoclines in the state space represent contour lines of a particular value of the derivative. In **Figure 7A**, the isoclines corresponding to values −200, 0, 200 of the excitatory spike count derivative, and the −50, 0, 50 isoclines for the inhibitory spike count derivative are depicted in black. The inhibitory isoclines seem to be a linear function of excitatory and inhibitory spike counts. The isoclines for the derivatives of the excitatory spike counts are non-linear with a negative dependency of inhibition on excitation when there are only few excitatory spikes and relatively more inhibitory spikes are available. The slope of the dependency becomes positive when the number of excitatory spikes increases. The nullclines (0 isocline) of the system are the solutions of the two dimensional mean-field equations. An immediate conclusion from the shape of the nullclines is that the dynamics of a finite size network of excitatory and inhibitory neurons is nonlinear. More theoretical evidence for the nonlinearity of the dynamics, based on the system characteristic equation analysis is provided in the Supplementary Material.

The ARM model with parameters estimated form the vector field of the spiking network simulation reproduces the flow with very high fidelity (**Figure 7C**, middle). The stochastic implementation of the ARM model generates a comparable vector field with a similar shape of the nullclines (**Figure 7C**, right). However, the spread of the excitatory spike counts is reduced and therefore, compared to the spiking network simulation results, the variance of the excitatory spike counts is smaller (**Figure 9B**). The low variance of the excitatory spike counts may be explained by the state dependent distribution of the derivatives of the excitatory spike counts which the model generates (**Figure 5A**, red curve compared to the blue). In comparison with the spiking network simulation, the variance of this distribution is reduced. We conjecture that if the excitatory spikes are not only generated from a binomial distribution, but are also correlated with the recently generated inhibitory spike counts, the variance of this distribution will increase. Another way of generating excitatory spike counts from the occupation number would be to consider the two dimensional log-normal distribution of the spike counts in the state space.

#### **3.5. CORRELATION FUNCTIONS OF THE TWO POPULATIONS**

The balanced random network with stationary input is a highly recurrent system. A high correlation between excitation and inhibition nevertheless results in a decorrelated input to single neurons, because the recurrent inhibitory input cancels the effect of recurrent excitation (Renart et al., 2010). In an inhibition-dominated network, this provides an input to each neuron such that the mean membrane potential is below threshold and only fluctuations of the input cause threshold crossing and therefore result in a very low rate and irregular spiking activity of a neuron in the network. The high correlation between excitation and inhibition manifests itself in the cross-correlation between the spiking activity of the excitatory and inhibitory population (**Figure 8C**). The ARM model is capable of producing a high correlation function between excitation and inhibition, and the time scale of the correlation function is the same as for the network simulations.

The autocorrelation function of the excitatory and inhibitory population activities in the simulation of the spiking neuronal network has a very sharp decay in the range of the time scale selected for the Markov model (0.1 ms). For the excitatory population, the autocorrelation is close to zero after 0.2 ms and the ARM model reproduces the same correlation function (**Figure 8A**). We conclude that the model is able to capture the temporal dynamics of the excitatory population on such a short time scale. The autocorrelation of the inhibitory spiking activity decays faster and exhibits a small undershoot after 0.1 ms. The ARM model shows the same time-scale of correlation decay, however, the undershoot is more pronounced (**Figure 8B**). The shorter time scale of the inhibitory auto-correlation function compared to the excitatory one is also represented in the power spectral density of the excitatory and inhibitory populations, illustrated in **Figures 1C,D**. The excitatory population concentrates most of its power in a relatively narrow low frequency band compared to the inhibitory population.

### **3.6. HEAVY TAILED DISTRIBUTION OF SPIKE COUNTS**

Some previous studies of the temporal dynamics of interacting populations assume a Gaussian distribution of the activity around the fixed point solution of a low-dimensional system (Kriener et al., 2008; Tetzlaff et al., 2012; Helias et al., 2013), however, our numerical study shows that a log-normal distribution provides a good fit to the spike counts observed in the the spiking neuronal network (SNN) simulation (**Figure 9A**). Note that the population spike counts are bounded from above (due to the finite size of the system), a property that is not reflected by the

log-normal distribution. Nevertheless, it was found to provide a good approximation for the range of data actually observed in our network simulations. Beyond the upper bound, the probability of spike counts is zero and the log-normal distribution is not valid. Therefore, strictly speaking, the distribution of such a bounded random variable is not heavy-tailed but we use this term for the positively skewed distribution of spike counts. The parameters μ and σ of the fitted log-normal distribution are 3.214, 0.664 and 1.938, 0.582 for the excitatory and inhibitory spike counts, respectively. The ARM model with the parameters extracted from the mean-field flow which was reconstructed from simulated data can capture this statistical property of the system (**Figure 9B**). The corresponding parameters of the ARM model are 3.405, 0.405 and 2.036, 0.539 for the excitatory and inhibitory spike counts, respectively. A log-normal distribution fits to the distribution of the spike counts generated in spiking network simulation and in the stochastic implementation of the ARM model very well, much better than possible alternative right-skewed distributions (gamma distribution, negative binomial distribution; data not shown). Insets of **Figure 9** show a fitted normal distribution to the envelope of the distribution of the logarithm of the spike counts. The fact that spike counts are integer numbers needs to be taken into account in the fitting process. We conclude from these results that the spike counts of the population activity embedded in a network do not follow Poisson statistics, as the mean and the variance of the spike counts are not identical.

A log-normal distribution of activities in a different context, when there are inhomogeneous degree distributions or other quenched noise in the system, have been reported in different studies (Roxin et al., 2011; Mizuseki and Buzsáki, 2013; Buzsáki and Mizuseki, 2014). Outside the neuroscience literature it has been claimed that log-normality could be an emergent feature of competition among subgroups of individuals (Halloy, 1998; Halloy and Whigham, 2005). For a balanced random network with the same in-degree for all neurons, however, to the best of our knowledge, it is the first time that this statistical behavior is reported. We believe that this emergent feature is tightly related to the large-scale dynamics of the interacting excitatory and inhibitory populations. The state dependence of the transition rates, and the doubly stochastic nature of the Markov system that follows from it, might provide a formal explanation for this result.

#### **3.7. UNIVERSALITY IN BALANCED NETWORKS OF DIFFERENT SIZE**

The network under study is a strongly connected network with identical in-degrees for all neurons. This implies that each neuron receives inputs from a fixed fraction of each population in the network. To keep the mean and the variance of the activity in the network limited, when the size *N* of the network tends to infinity, we scaled the synaptic weights by √ 1 *N* . We studied the network size effect on the variance of fluctuations and the nullclines of the reduced dimension system for 3 different networks of total size 7500, 12,500, and 20,000 neurons where in all these cases 20% of the neurons were inhibitory. The corresponding EPSP amplitude was 0.13, 0.1, and 0.08, respectively. To see whether there is any universal feature in the population dynamics of such networks, we normalized the spike counts of each population by the size of the population. This helps us to study dynamics of the fraction of the active neurons irrespective of the size of the network. To check the size invariance property of the ARM model, we rescaled equation (8) by the corresponding population size for the excitatory and inhibitory population. Introducing new variables *Xe* = *Se Ne* and *Xi* <sup>=</sup> *Si Ni* , equation (8) could be rewritten in the following form

$$\begin{cases} \dot{X}\_{\varepsilon}(t) &= \exp\left(c\_{0} + \iota\_{1}^{\prime}X\_{\varepsilon}(t) + \iota\_{2}^{\prime}X\_{i}(t)\right)\left(\boldsymbol{\upchi}dt - X\_{\varepsilon}(t)\right) \\ &- (\boldsymbol{\upbeta} + \boldsymbol{\upchi})X\_{\varepsilon}(t) \\ \dot{X}\_{i}(t) &= \exp\left(c\_{0} + \iota\_{1}^{\prime}X\_{\varepsilon}(t) + \iota\_{2}^{\prime}X\_{i}(t)\right)\left(\boldsymbol{\upchi}dt - X\_{i}(t)\right) \\ &- (\boldsymbol{\upbeta} + \boldsymbol{\upchi})X\_{i}(t) \end{cases} (12)$$

In equation (12), the coefficients *c*- <sup>1</sup> and *<sup>c</sup>*- <sup>2</sup> depend on the population sizes. The variables *X* are confined between 0 and 1 and comparing the dynamics of the networks with different sizes is reduced to a comparison between transition rates α, β, and γ . In our study, we were not able to identify an exact relationship between these parameters and the single neuron and connectivity parameters. However, **Figure 10** shows that these parameters change such that the equation is invariant to network size.

As illustrated in **Figure 10**, the nullclines have the same shape irrespective of the size of the network, but the variability that is reflected in the total visited area of the state space decreases as the network size increases. The suggested active refractory model can reproduce the vector field of the spiking network simulation for different network sizes, provided that the parameters of the model are correctly estimated. Therefore, our conclusion is that the nonlinearity in the interaction between populations for stationary input and the heavy-tail (positively skewed) distribution of the activity are both universal properties of strongly connected networks of finite size.

#### **3.8. IS THE STATIC NONLINEAR TRANSFER FUNCTION RESPONSIBLE FOR THE NONLINEAR DYNAMICS?**

Some studies of network dynamics assume that the stationary input-output transfer function of a neuron could provide a good description even for the time dependent activity of the network (Ledoux and Brunel, 2011; Pernice et al., 2012; Tetzlaff et al., 2012; Ostojic, 2014). The firing rate of a neuron in these models is typically approximated by the differential equation

$$\text{tr}\dot{r}\_i(t) = -r\_i(t) + \mathbf{F}(\mathbf{r}(t))\tag{13}$$

where **F** is the static nonliner function and **r** is the vector of the firing rate of other neurons in the network. This model, however, can be a good approximation for the temporal dynamics of the network only at large time scales, when the network activity is filtered over time and fluctuations have relatively low amplitude. Under these conditions, a linear approximation of the dynamics does provide a good match with the data. In our study, however, the time scale was chosen based on the time scale of the autocorrelation function of the network activity such that a wide range of frequencies contributes to the fluctuations. For *dt* = 0.1 ms, we demonstrate that the neural static nonlinearity cannot follow the delicate nonlinearity of the isoclines (**Figure 11A**) and the log-normality of the spike counts (**Figures 11B,C**). The nullclines of this model, after transforming the rates to spike counts, are almost linear functions of the respective spike counts (**Figure 11A**). Comparing the cumulative distributions of the spike counts generated from SNN, ARM and the input-output nonlinearity in equation (13) illustrates the discrepancy between the latter and the former. In fact, assuming that the instantaneous rates are equal to the stationary rates results in an overdispersed spike count distribution that does not match with that of SNN. Therefore, this assumption is obviously not correct for small time bins. These results show that system (13) is not valid on small time scales. On the other hand, as shown in the previous sections, the ARM model reproduces statistics and dynamics similar to the SNN data.

### **4. DISCUSSION**

In this study, we highlighted some properties of the large scale dynamics of finite-size balanced random networks in the inhibition dominated regime. It was shown that a linear system with additive Gaussian white noise cannot capture the statistics and dynamics of the two neuronal populations, and a more sophisticated model that represents the nonlinearity of the interactions and the statistics of the activity in a self-consistent way was suggested. We showed that a two state Markov process that the states of which reflect the coarse-grained membrane potential of a neuron, along with appropriate state-dependent transition probabilities, can reproduce the dynamics and statistics of a finite size network in the stationary balanced state in a satisfactory way. The state-dependent transition probabilities were all inferred from the

numerical simulation of a spiking network of leaky integrate-andfire neurons with stationary input. The approach was based on a reconstruction of the mean-field dynamics flow. The deterministic dynamics was complemented by a state-dependent stochastic component, assuming independent spiking in all neurons. This essentially leads to a binomial noise model, approximating Poisson statistics in very small time bins. On the level of the meanfield, the Active-Refractory Markov (ARM) model resembled the vector field extracted from the data with high fidelity. The population correlation functions of the spiking network simulation and the stochastic implementation of the model shared the same dynamical behavior.

We do not claim that the temporal dynamics of the finite size system is Markovian but a Markov model provides a good approximation. In general, the transition probabilities of the model depend on an unbounded history of the network activity (Cessac, 2011). However, we showed that a Markov model with a carefully chosen time scale is able to reflect some major statistical and dynamical properties of a balanced random network with fixed in-degrees. In the first part of the paper we explored the Markov properties of the network dynamics, and the size of time bins was chosen based on the similarity between the power spectra of a Markov model and the spiking network simulation. Then, we specifically considered a two-state Markov model for each neuron and estimated the state dependent transition probabilities from the simulated data. A self-consistent description that also accounts for the fluctuations of the finite-size system was obtained.

As outlined in detail in the Results section, there is a distinct similarity between the mean-field equations derived from the ARM model and the well-known Wilson-Cowan model (Wilson and Cowan, 1972) for the joint dynamics of excitatory and inhibitory neuronal populations. In the Wilson-Cowan model, the population response function, which gives the expected proportion of active neurons in a population as a function of the overall excitation in the system, derives from a cumulative unimodal density function and therefore typically has a sigmoidal shape. Using the Fokker-Planck equation to obtain the stationary distribution of the membrane potential and neglecting the absolute refractory time, a unimodal distribution will emerge. As discussed in this paper, the integral of the density function gives the transition probability from the refractory to the active state, the rate α in our model, as it reflects the local probability flux between the refractory and the active state. The local behavior of this function, for a wide range of the membrane potential between the threshold and reset, is well approximated by an exponential function and this approximation is quite good in comparison with spiking network simulations. A more precise result might come out if the CDF of the solution of equation (9) with free parameters is used for α. As shown in **Figure 4**, the CDF is a sigmoid function due to its unimodality. It is important to mention that the parameters of the model do not directly come from the CDF, but they should be estimated from the simulated data. An exponential function for the monotonically increasing CDF provides a good approximation for a wide range of network parameters and external input levels, as long as the network activity is asynchronous and irregular. Another comparison between the two models could be done on the kernel that is applied on the total excitation level in the network. In Wilson and Cowan (1972) an exponential kernel was used, which is justified by the impulse response of a leaky integrate and fire neuron.

The self-consistent noise model is an important feature of the model suggested in this article. The noise distribution is specific for each state, due to a deterministic dependency of the transition rates on the most recently generated spikes. Surprisingly, the state-dependent noise is such that it creates a strongly skewed distribution of the spike counts, similar to what is observed in the numerical simulations of spiking networks. The variance of the fluctuations generated by the model is, however, slightly smaller than the variance of the spike counts in the spiking network simulations. This is probably caused by the symmetric and relatively narrow state-dependent distribution of the noise in the excitatory population (**Figure 5A**). As mentioned in the result section, this problem is due to the assumption of independent spiking of individual neurons in each pool, implying the emergence of binomial distributions to describe the size of the neuronal populations that undergo a transition. As the variance of the outcome will be proportional to the total number of available neurons in each state, the variance of the increments will be directly proportional to the population size. One way of coping with this problem would be to create a correlation between the noise distributions in the two populations. Another way would be to consider the spike count distributions of the excitatory population given the spike counts of the inhibitory one. This way the heavy tail (positively skewed) distribution will come out automatically.

The nonlinear isoclines of the vector field provide strong evidence for a nonlinear interaction between the two neuronal populations. In particular the nullcline of the excitatory population exhibits very nonlinear behavior in the regime of small spike counts. Close to the fixed point of the flow, the excitatory nullcline changes direction. This property of the network can only be explained by a nonlinear model. It was already shown by Wilson and Cowan (1972) that different shapes of the nullclines emerge due to the asymmetry between excitation and inhibition, and because of their different signs in the argument of the population response function. In our model, however, the parameters of the two differential equations are the same, but different signs of the coefficients of the excitatory and inhibitory spike counts are enough to yield different shapes of the nullclines. A similar shape of the nullclines was in fact obtained in a model of the thalamo-cortical response transformations in the Barrel cortex of rodents (Pinto et al., 2003). We also checked whether the shape of the nullclines is invariant with regard to the size of the network. To compare the nullclines of networks of different sizes, however, it was necessary to normalize the activities of both populations. After normalization, the nullclines of networks with different sizes were almost identical, and it seems justified to claim that the nonlinear interaction is a universal property of strongly connected balanced networks. Since our method is based on the reconstruction of the dynamic flow, we conclude that our model can successfully capture the dynamics due to its size-invariance property.

It has been shown in van Vreeswijk and Sompolinsky (1996, 1998) that balanced random networks are extremely fast in following the temporal dynamics of an external input. This means among other things that the power spectral density of "spontaneous" network activity has a very broad frequency range and, as a result, the autocorrelation functions of the populations decay sharply. Our suggested model was successful in reproducing this fast temporal dynamics. Experimental evidence and theoretical studies suggest that dynamic responses of neocortical neurons are much faster when multiplicative input noise is imposed, compared to the case of additive noise (Lindner and Schimansky-Geier, 2001; Silberberg et al., 2004; Boucsein et al., 2009). We hypothesize that the multiplicative nature of the self-generated noise in balanced random network could also contribute to fast neuronal responses and sharp autocorrelation functions. Furthermore, an emergent property of balanced networks in the asynchronous irregular state is the high correlation between excitatory and inhibitory population activity, which was also successfully retrieved by our model. This is due to the fact that the same mean-field provides input to both populations. In other words, log (α) is a linear function of excitatory and inhibitory activity at any given point in time, with a positive weight for excitation and a negative weight for inhibition. This type of dependency makes a major contribution to correlating excitatory and inhibitory spike counts.

Recently, some studies have featured heavy-tail distributions of various phenomena in the brains of different species, supposedly a robust and important aspect of cortical computation (for a review see Buzsáki and Mizuseki, 2014), often approximated by log-normal distributions. Neurons in the auditory cortex (Deweese and Zador, 2008) and in the hippocampus and enthorinal cortex (Mizuseki and Buzsáki, 2013) of rats exhibit a log-normal distribution of firing rates. In Mizuseki and Buzsáki (2013) it is also shown that during the bursting activity state of the network, the fraction of all recorded neurons that fire a spike, for either stimulus-evoked or spontaneous activity, display a log-normal distribution. This, in turn, might be attributed to the log-normal distribution of the synaptic weights. Similarly, high density micro-electrode recordings of the human brain during sleep, in combination with a separation of excitatory and inhibitory cellular activities based on the spike wave-forms, has shown a log-normal distribution of firing rates (Peyrache et al., 2012). A theoretical study (Roxin et al., 2011) has shown that for a randomly connected network of excitatory and inhibitory LIF neurons with a random number of external inputs for each neuron and random synaptic efficacies, using an exponential *f*-*I* curve to describe the single neuron dynamics and a Gaussian distribution of inputs to each cell, a log-normal distribution of neural firing rates arises. In our study, however, all neurons had the same statistical inputs without any quenched variability, resulting in identical firing rates for all neurons in the network. However, the distribution of the spike counts for a large simulation time with small time bins was shown to be well fitted by a log-normal distribution. The suggested model was also able to represent this emergent statistical property of the network. This might emerge due to the exponential dependency of the transition probability α with suitably chosen parameters. In the theory of complex systems, the log- normal distribution is considered to be a universal statistical property of many natural systems (Halloy, 1998; Halloy and Whigham, 2005; Kobayashi et al., 2011). In particular, Kobayashi et al. (2011) showed that if the history of each component of the system defines its present state, log-normality becomes emergent. This could be tested on a network of multiplicatively interacting point processes that mimic the behavior of LIF neurons (Cardanobile and Rotter, 2010, 2011). However, a rigorous analysis shows that the Gibbs distribution is a unique invariant probability measure of the system under stationary input (Cessac, 2011). It might be interesting to investigate the underlying reason for the emergence of the log-normal distribution analytically.

We would like to stress once more the importance of choosing the right time scale for the ARM model. Due to the Markov assumption, the time bin must be chosen such that the influence of the past activity of the network on the transition rates (its memory) is minimal. This time scale is typically small, and we observe that a wide range of the power spectrum of the spiking activity is preserved. We conclude that the time dependent dynamics of the network is quite accurately captured. It was also shown in our paper that the typical input-output static nonlinearity of a neuron is neither suited to reproduce the nonlinear activity of the network, nor the log-normal distribution of the spike counts.

Markov models have been suggested before as models of the temporal dynamics of finite size networks (Soula and Chow, 2007; El Boustani and Destexhe, 2009; Cessac, 2011; Buice and Chow, 2013a). In Soula and Chow (2007), an approach similar to ours was proposed assuming a statistically homogeneous network of excitatory neurons. The difference to our work is that an active neuron is by definition a neuron that emits a spike, therefore only one transition probability from the silent to the active state is needed. This probability is proportional to the steady-state firing rate of the neuron. Also, a full-blown theoretical framework was introduced to calculate the first and second moment of fluctuations in the network. Our model, however, is different in the sense that the interaction between the excitatory and inhibitory population is taken into account, and that a high correlation between the activities of the two populations is preserved. Moreover, a heavytail distribution of the activity emerges as a result of the dynamical interaction between excitation and inhibition. However in Soula and Chow (2007) the distribution of the single population activity is symmetric due to the lack of inhibitory population. Our model on the other hand considers a neuron in the active state if its membrane potential is above some unspecified value of the membrane potential between threshold and rest. It takes the effect of leak into account by assuming a potential transition from the active to the refractory state without emitting a spike. El Boustani and Destexhe (2009) followed the same approach in continuous time for a sparse random network of excitatory and inhibitory neurons. Assuming quasi-stationarity, they derived the first two moment equations of the activity using the static input-output transfer function of a typical neuron in the network. However, Poisson statistics for each neuron was assumed and the transition function was calculated based on the mean activity of the network; therefore, the resultant distribution of the activity was Gaussian.

The main underlying assumption of the ARM model is the two-state Markovian single-neuron dynamics. For an appropriate choice of the time step, it was shown in the present study that the statistics and dynamics of recurrent networks of leaky integrateand-fire neurons can be captured by the model. However, we neglected the role of absolute refractoriness in the dynamics of the membrane potential. Refractoriness shapes the low frequency range of the population dynamics (Mar et al., 1999; Spiridon and Gerstner, 1999). It could be modeled by introducing a chain of refractory states and thereby, increasing the dimensionality of the model (Toyoizumi et al., 2009). In the ARM model, absolute and relative refractoriness together, effectively, will have a wide distribution (for a relevant study, see Deger et al., 2010). Another aspect of our study is that synaptic transmission delays were neglected in the model as well as in our network simulations. In general, the transition probabilities depend on an unbounded past (Cessac, 2011) and delayed feedback makes the system non-Markovian (Vidybida and Kravchuk, 2013). It might be interesting to investigate whether a delayed α rate operating on the current pool of refractory neurons can represent the dynamics of the spiking network simulations. Furthermore, the important assumption of the ARM model is that all neurons statistically behave the same, because they all have the same in-degree. This allows us to reduce the dimensionality of the large scale dynamics and come up with a simple two state stochastic model of the system. If there is any inhomogeneity in the system, this model will not be a good candidate. Maybe for a network with homogeneous subpopulations, each component could be modeled by the ARM presented in this paper, with suitable parameters.

Population density methods are promising approaches for dimensionality reduction in dynamic networks. Including finitesize effects and correlations in the model, however, is a challenge. Deterministic density equations describing the temporal dynamics of finite-size networks were derived by using an eigenfunction expansion of the Fokker-Planck equation (Mattia and Del Giudice, 2002, 2004). Using a stochastic and deterministic approach, Buice and Chow recently suggested a mean-field equation and moment hierarchies of a density equation to obtain corrections arising from the finite size of the system and from correlations which are basically due to heterogeneities in the system (Buice and Chow, 2013a). For a homogeneous network, using the effective action approach of field theory (Buice and Chow, 2013c) and system size expansion around the mean-field density function (Buice and Chow, 2013b), they derived moment equations leading to the dynamics of mean and covariance in the network. Our suggestion for future exploration of the field is that a nonlinear set of Fokker-Planck equations for each population might be recast in the form of ARM model suggested in this paper. The more general model will have nonlinear and state dependent transition parameters that could be analytically derived and the stochastic behavior will emerge as a result of the finite size of the system.

Finally, to extend the model to cover the more general case of non-stationary and time dependent input, it is necessary to investigate the precise role of the external input in the ARM model. There are at least two possibilities: it could either be reflected in γ , or it could be included in the transition rate from the refractory to the active state, α. We suggest that the same data analysis method that we applied in this study might also help in determining the role of external time-dependent input in the model. Furthermore, it is possible to test whether this model can capture the dynamics of more than two interacting populations. However, this will be more challenging, as the dynamics of these type of networks are not necessarily stationary in time. Particularly, under certain conditions, switching dynamics between populations might arise (Litwin-Kumar and Doiron, 2012).

## **AUTHOR CONTRIBUTIONS**

Mathematical analysis and numerical experiments of this study were conceived and designed by Fereshteh Lagzi and Stefan Rotter. Numerical simulations and data analysis were performed by Fereshteh Lagzi, and supervised by Stefan Rotter. The paper was written by Fereshteh Lagzi and Stefan Rotter.

#### **ACKNOWLEDGMENTS**

We thank Fatihcan Atay and Björn Schelter for helpful discussions. We thank the developers of the simulation software NEST (see http://www.nest-initiative.org) and the maintainers of the BCF computing facilities for their assistance throughout this study. We would like to acknowledge the use of the computing resources provided by the Black Forest Grid Initiative. This work was supported by the German Ministry of Education and Research (BFNT Freiburg∗Tübingen, grant 01GQ0830) and the German Research Foundation (DFG, grant EXC 1086). The article processing charge was covered by the open access publication fund of the University of Freiburg.

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 25 May 2014; accepted: 20 October 2014; published online: 03 December 2014.*

*Citation: Lagzi F and Rotter S (2014) A Markov model for the temporal dynamics of balanced random networks of finite size. Front. Comput. Neurosci. 8:142. doi: 10.3389/ fncom.2014.00142*

*This article was submitted to the journal Frontiers in Computational Neuroscience. Copyright © 2014 Lagzi and Rotter. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## **APPENDIX**

#### **A.1. PROPERTIES OF A LINEAR SYSTEM WITH GAUSSIAN NOISE**

In this section we highlight the main properties of a linear system and we will show that the temporal dynamics of finite size networks does not suggest a linear model. To demonstrate this, we consider a general two-dimensional linear system and will use two sources of Gaussian white noise to drive the excitatory and the inhibitory population. Then, we will discuss the features of this model due to the linear nature of the system and those which emerge due to the external drive. We assume in the following that *ye*(*t*) and *yi*(*t*) represent the instantaneous firing rates of the excitatory and inhibitory population. We consider an arbitrary communication delay in population interactions, which need not to be identical. Without loss of generality, we assume zero initial conditions for both populations. A coupled twodimensional linear system in its general form is represented in the following way

$$
\dot{\mathbf{y}}\_{\varepsilon}(t) = a\_{\varepsilon\varepsilon}\mathbf{y}\_{\varepsilon}(t-d) + a\_{\varepsilon i}\mathbf{y}\_{i}(t-d) + a\_{\varepsilon}I\_{\varepsilon}(t)
$$

$$
\dot{\mathbf{y}}\_{i}(t) = a\_{i\varepsilon}\mathbf{y}\_{\varepsilon}(t-d) + a\_{i\overline{i}}\mathbf{y}\_{i}(t-d) + a\_{i}I\_{i}(t) \tag{A1}
$$

Taking the Laplace transform of the above equation results in

$$sY\_{\varepsilon}(s) = a\_{\varepsilon\varepsilon}e^{-s d}Y\_{\varepsilon}(s) + a\_{\varepsilon i}e^{-s d}Y\_{i}(s) + a\_{\varepsilon}I\_{\varepsilon}(s)$$

$$sY\_{i}(s) \ = a\_{i\varepsilon}e^{-s d}Y\_{\varepsilon}(s) + a\_{i\varepsilon}e^{-s d}Y\_{i}(s) + a\_{i}I\_{i}(s) \tag{A2}$$

which in matrix form is

$$\begin{aligned} \begin{bmatrix} Y\_{\epsilon}(s) \\ Y\_{i}(s) \end{bmatrix} &= \begin{bmatrix} a\_{\epsilon\epsilon}e^{-sd} - s & a\_{\epsilon i}e^{-sd} \\ a\_{i\epsilon}e^{-sd} & a\_{i\bar{\imath}}e^{-sd} - s \end{bmatrix}^{-1} \begin{bmatrix} -a\_{\epsilon}I\_{\epsilon}(s) \\ -a\_{\bar{\imath}}I\_{i}(s) \end{bmatrix} \\ &= \frac{1}{G(s)} \begin{bmatrix} a\_{\bar{\imath}\epsilon}e^{-sd} - s & -a\_{\epsilon i}e^{-sd} \\ -a\_{i\epsilon}e^{-sd} & a\_{\epsilon e}e^{-sd} - s \end{bmatrix} \begin{bmatrix} -a\_{\epsilon}I\_{\epsilon}(s) \\ -a\_{\bar{\imath}}I\_{i}(s) \end{bmatrix}, \end{aligned} \tag{A3}$$

where

$$G(s) = \left(a\_{il}e^{-sd} - s\right)\left(a\_{ee}e^{-sd} - s\right) + a\_{ic}a\_{ci}e^{-2sd}$$

is called the characteristic equation of the linear system and appears in the denominator of both *Ye*(*s*) and *Yi*(*s*). The peaks in the power/amplitude spectra show the location of the zeros of the characteristic equation of the system. Therefore, the poles of the two components are identical and result in the same location of peaks in the power spectrum of *ye*(*t*) and *yi*(*t*). This statement is true in the more general case, when the inputs are different and non-Gaussian, since the two inputs play a role in shaping the power spectrum of both signals. The conclusion that we draw from this analysis is that if the power or amplitude spectra of two mutually interacting signals do not have the same peaks, then the interaction between them cannot be linear.

Equation (A3) shows that a coupled system as such behaves like a low-pass filter and for white noise input, the output would just be a filtered version of the noise. Since the distribution of the input is symmetric about its mean, the distribution of the output is also symmetric. Therefore, a non-symmetric distribution of the output must be interpreted as either a sign of nonlinearity of the system or the non-Gaussian nature of the input.

#### **A.2. NON-IDENTICAL POLES IN THE AMPLITUDE SPECTRUM OF EXCITATORY AND INHIBITORY SPIKE COUNTS**

A balanced network of excitatory and inhibitory population, with the same characteristics of the network introduced in the Methods section, but with a synaptic delay of 1.5 ms and the total simulation time of 100 s, was simulated. The amplitude spectra of the excitatory and the inhibitory population are illustrated in **Figure A1**. In the low frequency regime of the dynamics, the peaks of the excitatory and inhibitory amplitude spectra are identical. However, for the high frequency part of the spectra the locations of the poles of the two populations are slightly shifted with respect to each other. This fact shows that the dynamics of the system that describe the temporal activity of the population interactions cannot be linear.

**FIGURE A1 | Amplitude spectral density of spike counts with a bin size of 0***.***1 ms and** *d* **= 1***.***5 ms.** The entire spectrum of excitatory spike counts in bins of 0.1 ms is plotted in blue. The power spectrum of inhibitory spike counts under the same condition is plotted in red. Due to less number of neurons in the inhibitory population, its amplitude is lower than that of excitation. Dashed lines are plotted at frequencies corresponding to the peakes in the power spectra. For low frequencies, the peaks of the two spectra are the same. In the high frequency range the peaks in the amplitude spectrum of the inhibitory spike counts are slightly shifted with respect to those of excitatory ones.