Edited by: Nicolas Brunel, Centre National de la Recherche Scientifique, France
Reviewed by: Daoyun Ji, Baylor College of Medicine, USA; Jaime De La Rocha, Institut D'Investigacions Biomèdiques August Pi i Sunyer, Spain; Alex Roxin, Centre de Recerca Matemàtica, Spain
*Correspondence: Birgit Kriener, Center for Learning and Memory, University of Texas at Austin, 1 University Station C7000, Austin, TX 78712-0805, USA e-mail:
This article was submitted to the journal Frontiers in Computational Neuroscience.
†These authors have contributed equally to this work.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Random networks of integrate-and-fire neurons with strong current-based synapses can, unlike previously believed, assume stable states of sustained asynchronous and irregular firing, even without external random background or pacemaker neurons. We analyze the mechanisms underlying the emergence, lifetime and irregularity of such self-sustained activity states. We first demonstrate how the competition between the mean and the variance of the synaptic input leads to a non-monotonic firing-rate transfer in the network. Thus, by increasing the synaptic coupling strength, the system can become bistable: In addition to the quiescent state, a second stable fixed-point at moderate firing rates can emerge by a saddle-node bifurcation. Inherently generated fluctuations of the population firing rate around this non-trivial fixed-point can trigger transitions into the quiescent state. Hence, the trade-off between the magnitude of the population-rate fluctuations and the size of the basin of attraction of the non-trivial rate fixed-point determines the onset and the lifetime of self-sustained activity states. During self-sustained activity, individual neuronal activity is moreover highly irregular, switching between long periods of low firing rate to short burst-like states. We show that this is an effect of the strong synaptic weights and the finite time constant of synaptic and neuronal integration, and can actually serve to stabilize the self-sustained state.
The sustained activity of populations of spiking neurons, even in the absence of external input, is observed in many circumstances, amongst them spontaneously active neurons in cell cultures (see e.g., Marom and Shahaf,
Several possible explanations of how neuronal networks can generate and sustain activation of subpopulations of neurons have been put forward in the past, amongst them persistent activation by thalamo-cortical and cortico-cortical loops, intrinsic cellular bistability, or attractor states of local recurrent networks (Wang,
Of particular interest is the question of what constitutes the
Griffith (
In most of these models, attractor states are characterized by rather constant individual firing rates and homogeneous population activity. In experimental investigations of sustained states in prefrontal cortex during working memory (Marder et al.,
Here, we demonstrate that LIF neurons with current-based synapses can sustain highly irregular activity at moderate rates provided the coupling between them is sufficiently strong (see also the preprint by Gewaltig,
We demonstrate by simple arguments how the competition between the mean and variance of the neuronal input as a function of synaptic strength leads to a non-monotonic firing-rate transfer in the network. Thus, by increasing the synaptic coupling strength the system can become bistable, and in addition to the quiescent state a second stable fixed point at moderate firing rates, the SSAI state, can emerge by a saddle-node bifurcation. The population activity in this SSAI state is characterized by inherent population fluctuations and highly irregular spiking of individual neurons.
We show that the high irregularity in the activity of individual cells is induced by the large fluctuations of the neuronal input currents which keep the membrane potential far away from threshold for long times and induce firing at close to maximal rate when there is a large occasional suprathreshold transient. Hence, the firing-rate activity of individual neurons is basically binary. In particular, it demonstrates that highly irregular individual neuron firing and stable sustained activity states are indeed compatible and do not necessitate extra sources of variability, such as additional external noise or cellular bistability.
The substantial population fluctuations on the other hand lead to a constant perturbation of the network activity from the SSAI-attractor. We show how taking this into account in a simple escape rate model can explain the observed lifetimes of the persistent activation as a function of the network coupling parameters
The paper is organized as follows: In Section 2 we will shortly outline the neuron and network model as well as the data analysis techniques used in this paper. In Section 3.1 we present the essential features of the SSAI-state in strongly coupled networks, and then explain the mechanism underlying its emergence and irregularity in Section 3.2. Section 3.3 discusses the effect of synaptic weight distributions on the emergence of SSAI. In Section 3.4 we show how a stochastic rate model can capture the distribution of lifetimes observed in simulations, and in Section 4 we finally summarize and discuss our results.
We study balanced random networks (van Vreeswijk and Sompolinsky,
Populations | Three: excitatory (E), inhibitory (I), external input ( |
|
Connectivity | Random convergent connectivity with probability ϵ | |
Neuron model | Leaky integrate-and-fire (LIF), fixed voltage threshold, exact integration scheme (Rotter and Diesmann, |
|
Synapse model | α-shaped post-synaptic current (PSC) | |
Input | Independent Poisson spike trains | |
E,I | LIF neuron | |
Eext | Poisson generator | |
{E,I} | E ∪ I | Random convergent |
Eext | E ∪ I | Non-overlapping 1→ 1 |
Name | Leaky integrate-and-fire neuron with α-shaped PSCs | |
Subthreshold dynamics | ||
Spiking | If |
|
1. Set spike time |
||
2. Emit spike with time-stamp |
||
Postsynaptic currents | ||
Poisson generators | Spike times |
{100000, 5000} | Number of excitatory neurons ( |
|
γ |
Number of inhibitory neurons | |
ϵ | {0.01, 0.1} | Connection density ( |
τm | 20 ms | Membrane time constant |
20 GΩ | Membrane resistance | |
20 mV | Firing threshold | |
0 mV | Reset potential | |
τref | 2 ms | Refractory time |
∈ [0.1, 4.5] mV | Peak-amplitude of excitatory PSP( |
|
∈ [0.12, 5.44] pA | Amplitude of excitatory PSC( |
|
∈ [4., 8.] | Relative inhibitory coupling strength | |
1.5 ms | Synaptic delay | |
τsyn | 0.5 ms | synaptic time constant |
νext | 1000( |
Rate of external Poisson stimulus |
1000 ms | Stimulus duration |
Though all results we present below hold for a very broad class of balanced random networks, all neurons in the simulations presented here received the same number of excitatory and inhibitory synapses, i.e.,
Finally, we assume that the coupling strength is parametrized by the peak-amplitude
We emphasize that the main results do not crucially depend on the network density or the fine details of the weight and degree distribution.
The dynamics of the subthreshold membrane potential
with membrane time constant τ
resulting in a post-synaptic potential
Here, τsyn is the synaptic time constant, whereas
To initially activate the network, in Figures
and a filter kernel as defined by Equation 3. Note that in these cases the external input is only delivered during the period
In
Whenever
For each parameter pair (
The population rate is estimated by the temporal average of the population spike count per time bin Δ
where
where
To estimate the coefficient of variation (CV) of inter-spike intervals (ISI), we compute the ISI of
are computed and averaged over all neurons
To estimate the pairwise correlations between neurons, we removed the stimulus period
The correlation coefficients c
The spiking activity of the network is inherently fluctuating and chaotic. To estimate the response function of the network we thus assume that the instantaneous population rate ν(
where the noise ξ(
where the average is taken over all
In many simplified integrate-and-fire neuron models that receive temporally fluctuating input current from a pool of presynaptic neurons, the probability to emit a spike is determined by two key properties of this integrated input: its mean and variance with respect to the firing threshold. In essence, the output rate of such a neuron will depend on the probability that the free membrane potential is suprathreshold.
This is the essence of models as proposed in Abeles (
where μ = μ[
where τ denotes a characteristic memory time constant, e.g., the membrane time constant.
We investigate the transition in the dynamic behavior that random networks of inhibitory and excitatory LIF neurons undergo when the synaptic coupling strength
To characterize the dynamical features of the SSAI state, we first analyze the lifetime, firing rate, irregularity and correlations in dependence of coupling strength
The lifetime of the SSAI increases rapidly from zero to more than 1000 s (i.e., networks stay active for the whole duration of the simulation) within a narrow band in the parameter space spanned by
Figure
In summary, for wide regions of the
Several earlier studies suggested that the self-sustained asynchronous-irregular activation we observe here is impossible in balanced random networks with current-based synapses (Kumar et al.,
Inspection of the membrane potential traces of neurons in SSAI states reveals that they fluctuate strongly (on the order of volts, rather than millivolts, depending on the amplitude of input current variance), only limited by the threshold for positive values and the maximally possible inhibitory input for negative values, which depends on the dynamical state of the system.
If we consider the free membrane potential
Moreover, due to these extreme fluctuations the neuron reset amplitude becomes almost negligible due to the occasional massive net-excitatory input transients, and as long as the free membrane potential
The full self-consistent dynamics of self-sustained activity states is hard to assess because of the non-linear input-output relation of LIF neurons and the non-Poissonian nature of the compound input spike trains that characterizes the SSAI-state. To address the spiking irregularity in the case of strongly weighted input spikes, we thus now consider a simplified scenario where we assume that the incoming spike trains are independent stationary Poisson processes, implying a CV of unity for the input spike trains.
Already in this case,
The spiking activity, coefficient of variation CV, population spike count, free membrane statistics, and pairwise spike train correlation coefficient c
Figure
In Figure
This demonstrates how the full recurrent network amplifies weak pairwise correlations and irregularity of spiking, yielding much larger population fluctuations, wider free membrane potential distribution, and higher CV of ISIs compared to what is expected from the Poisson-input assumption. Moreover, as the variability increases, also firing rates increase. For the Poisson-driven ensemble the average rate is 36 s−1 (Figure
From the observations of the last two sections, we will now derive a simple dynamical model to analyze the basic mechanism underlying the saddle-node bifurcation that leads to the emergence of a second stable fixed-point at finite rate, i.e., the self-sustained state. As discussed in Section 3.2.2, if
To derive the time that
with
The PSP(
The probability
All neurons in expectation spike at the same rate, such that Equation (18) can in analogy to the Abeles model Equation (15) be used to estimate an upper bound 〈ν(
Because μ and σ are functions of ν(
where μ
Moreover, we can assess the critical parameters for which (i) there exists a 〈ν(
where the final condition is necessary for stability.
Figure
For increasing
In the Abeles model, a smaller fixed-point rate corresponds to a smaller area of the free membrane potential above threshold, i.e., smaller
For example, evaluation of Equations (17) shows that μ[
From the full spiking network, however, we saw that there is a wide range of
To test how well the reduced two-state approach performs compared to actual spiking neurons, we simulated a population of LIF neurons with balanced Poisson inputs to mimic a network of size
In order to mimic the self-consistent state, these Poisson inputs had a rate ν0 = νsim0 that was numerically tuned such that the
Within the simplified two-state Abeles model approach followed here, we cannot only derive the self-consistent firing rate, but also the approximate distribution of the population spiking activity. The probability for any neuron to be in the active state and fire with rate
The expected number and variance of counts in a time bin Δ
We indeed find very good agreement for Poisson-driven LIF neurons with ν0 = νsim0 and
The two-state firing rate approximation for Poisson-driven LIF neurons is thus a valuable tool to gain qualitative insight into the basic mechanisms that underlie SSAI in random networks of excitatory and inhibitory spiking neurons.
So far we considered networks where all excitatory synapses are weighted by the same weight
If we assume that all synaptic weights are distributed according to some excitatory and inhibitory weight distribution
with expectation value across network realizations E
Many experimental studies report lognormally distributed synaptic weights
For this type of weight distribution we obtain
How this increased input variance in terms of the parameters
If we fix the average values of the excitatory and inhibitory coupling strengths E
The key effect of increasing the variance of the free membrane potential in this way, while keeping the mean fixed, is a decrease in the critical average coupling strength for the saddle-node bifurcation to occur. This is exemplified in Figure
The gray lines show the same setup for E
Similar effects are expected from every manipulation that increases the variance of the free membrane potential, while keeping the mean approximately fixed, as well as manipulations increasing the mean for fixed or increasing variance, e.g., by varying the number of synaptic inputs
So far we analyzed the occurrence, variability and irregularity in terms of a reduced two-state Abeles-type model. But can we understand the transition from finite to virtually infinite lifetimes in the fully recurrent networks when the synaptic coupling strength increases?
As shown in the previous sections large population-rate variability is an inherent feature of self-sustained activity states. So the system perpetually perturbs itself and can substantially deviate from the high rate fixed-point ν0. If the basin of attraction is smaller than the characteristic fluctuation size, the system can escape the attractor and run into the trivial attractor at zero rate. Inspection of
cf. Equation (19), as a function of the input rate ν (Figure
The upper panel in Figure
To relate these findings from the two-state Abeles-type model with Poisson input to the full recurrent SSAI, we perform the analogous analysis with some examples of the data we obtained from the systematic large-scale simulations discussed in Figures
Such estimated response functions Δν are shown in Figure
In the cases where the synapses are sufficiently strong to sustain persistent activity, we see that the distribution may be well approximated by a Gaussian centered at the upper fixed-point of the response function. This observation thus motivates the following simple stochastic model for the rate: We assume that the rate at any time is distributed normally with a mean given by the fixed-point of the response function. Both the response function and the width of the distribution are functions of the network and neuron parameters.
The probability to observe a given rate ν is thus,
where ν0 = ν0(
From the observations of network response functions we can also see that there is indeed typically another (unstable) fixed-point λ close to the trivial fixed-point at zero. For the purpose of the stochastic rate model, we assume that if the rate fluctuates to a value less than λ, the network activity will move toward the trivial fixed-point at zero rate and cease.
From the probability distribution above, we can calculate the probability for the rate to be below λ, i.e.,
We conclude that the lifetime for the self-sustained network activity will be inversely proportional to the probability for the network activity to cease,
where τ0 is a constant (see also El Boustani and Destexhe,
We validate the stochastic model approach Equations (29), (30) by estimating the values for ν0, λ and σ, as well as the lifetimes
The values for the parameters ν0 and λ as a function of
We note that a saddle-node bifurcation as predicted from the Abeles-type two-state model Equation (19) is also predicted from the diffusion-approximation (Brunel,
Local cortical circuits can sustain elevated levels of activity after removal of the original stimulus or in total absence of external drive. Moreover, this ongoing activity is often characterized by highly fluctuating individual firing rates. In contrast to previous beliefs (see e.g., Kumar et al.,
We analyzed and identified simple mechanistic explanations for these activity features. The emergence of a stable attractor at non-zero rates is due to a saddle-node bifurcation: At sufficiently large synaptic efficacy, two fixed-points with finite rate exist in addition to the quiescent mode. These modes exist even when there is no external input to the network. The intermediate low-rate fixed-point is always unstable, while the fixed-point at higher rate can be long-lived with a lifetime rapidly increasing with synaptic efficacy.
Using a simple stochastic rate model, we have shown that the lifetime is determined by a trade-off between the size of the basin of attraction of the high-rate fixed-point and the intrinsic variance of the network activity in this state. The stochastic model explains the lifetime over a wide range of network parameters.
The saddle-node bifurcation appears also in the simplified analytical models introduced by Siegert (
We note that quantitatively the two-state model yields good agreement with the observed SSAI-states, if the amplitude of the free membrane potential fluctuations is large, and their mean and variance are known. The latter can be measured in simulations, but in practical terms they are hard to assess. For other cases, such as the Poisson-driven LIF-ensemble shown in Figures
Still, our model nicely shows that high variability of the spiking activity of individual neurons, pronounced population fluctuations, and stable persistent activity can go together well (see also Druckmann and Chklovskii,
Broad membrane potential distributions as observed here are not very physiological and not possible for neurons with conductance-based synapses (Kuhn et al.,
Yet, also in networks of leaky integrate-and-fire neurons with conductance-based synapses self-sustained activity states occur for broad parameter ranges of excitatory and inhibitory conductances (Vogels and Abbott,
In the Supplementary Material Section
We moreover note that clamping the membrane potential of LIF neuron with current-based synapses at a minimal value to avoid unbiological hyperpolarization leads to a shift of the saddle-node bifurcation line to smaller
Asynchronous, highly irregular self-sustained activity, even in comparably small, yet strongly coupled networks, does thus not crucially depend on the synapse model, nor on extremely large subthreshold membrane potential fluctuations, but it is mainly a consequence of the large input fluctuations generated by the highly variable neuronal activities and the strong synaptic weights.
We emphasize that a comparably small fraction of strong weights suffices to permit self-sustained activity (see Teramae et al.,
The reduced Abeles-type model already shows that the critical average coupling strength for the saddle-node bifurcation decreases, if the variance of the weight-distribution increases. For the extreme case of mostly very weak synapses and few very strong synapses, the reduced model predicts SSAI to occur for small average coupling strength on the order of
Song et al. (
The emergence of a self-sustained activity state is not the only intriguing dynamical effect caused by the presence of strong synapses. As pointed out in many studies, strong coupling in complex networks can lead to a breakdown of linearity and give rise to new collective phenomena, such as pattern formation, oscillations or traveling waves (see e.g., Amari,
The presence of strong synapses was shown to lead to spike-based aperiodic stochastic resonance, and thus reliable transmission of spike patterns, in an optimal self-sustained background regime in networks of conductance-based LIF neurons (Teramae et al.,
Analogous to our observations, Ostojic (
Ostojic, as well, explains the effects in random networks by the breakdown of the linear response approximation and the non-linear network amplification of heterogeneous perturbations (see detailed discussion in the Supplementary Material Section
The amplification by the recurrent network is also the reason that underlies the strengthening of irregularity and population fluctuations that we observe, e.g., in Figures
The existence of strong synapses in recurrent neuronal networks as observed in experiments thus leads to a plethora of interesting dynamical properties that just start to be explored, and analysis of how circuits can make use of their presence computationally is an important topic of future research.
Birgit Kriener and Håkon Enger conceived and performed simulations, mathematical analysis, and data analysis. Birgit Kriener, Håkon Enger, Tom Tetzlaff, Hans E. Plesser, Marc-Oliver Gewaltig, and Gaute T. Einevoll wrote the manuscript.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
We gratefully acknowledge funding by the eScience program of the Research Council of Norway under grant 178892/V30 (eNeuro), the Helmholtz Alliance on Systems Biology, the Helmholtz Association in the Portfolio theme “Supercomputing and Modeling for the Human Brain,” the Jülich-Aachen Research Alliance (JARA), and EU Grant 269921 (BrainScaleS). All network simulations were carried out with NEST (
The Supplementary Material for this article can be found online at:
1If we assume absolute refractoriness, i.e., the neuron loses the input during that period, the dynamics becomes biased toward higher rate because the neuron stays at