**SPIKING NEURAL NETWORK CONNECTIVITY AND ITS POTENTIAL FOR TEMPORAL SENSORY PROCESSING AND VARIABLE BINDING**

**Topic Editors Julie Wall and Cornelius Glackin** 

COMPUTATIONAL NEUROSCIENCE

#### *FRONTIERS COPYRIGHT STATEMENT*

© Copyright 2007-2014 Frontiers Media SA. All rights reserved.

All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.

The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.

Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.

Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.

As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.

All copyright, and all rights therein, are protected by national and international copyright laws.

The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.

Cover image provided by Ibbl sarl, Lausanne CH

**ISSN** 1664-8714 **ISBN** 978-2-88919-239-7 **DOI** 10.3389/978-2-88919-239-7

## *ABOUT FRONTIERS*

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

## *FRONTIERS JOURNAL SERIES*

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing.

All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

## *DEDICATION TO QUALITY*

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view.

By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

## *WHAT ARE FRONTIERS RESEARCH TOPICS?*

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area!

Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

## **SPIKING NEURAL NETWORK CONNECTIVITY AND ITS POTENTIAL FOR TEMPORAL SENSORY PROCESSING AND VARIABLE BINDING**

Topic Editors: **Julie Wall,** Queen Mary, University of London, UK **Cornelius Glackin,** University of Hertfordshire, UK

The most biologically-inspired artificial neurons are those of the third generation, and are termed spiking neurons, as individual pulses or spikes are the means by which stimuli are communicated. In essence, a spike is a short-term change in electrical potential and is the basis of communication between biological neurons. Unlike previous generations of artificial neurons, spiking neurons operate in the temporal domain, and exploit time as a resource in their computation. In 1952, Alan Lloyd Hodgkin and Andrew Huxley produced the first model of a spiking neuron; their model describes the complex electro-chemical process that enables spikes to propagate through, and hence be communicated by, spiking neurons.

Since this time, improvements in experimental procedures in neurobiology, particularly with in vivo experiments, have provided an increasingly more complex understanding of biological neurons. For example, it is now well understood that the propagation of spikes between neurons requires neurotransmitter, which is typically of limited supply. When the supply is exhausted neurons become unresponsive. The morphology of neurons, number of receptor sites, amongst many other factors, means that neurons consume the supply of neurotransmitter at different rates. This in turn produces variations over time in the responsiveness of neurons, yielding various computational capabilities. Such improvements in the understanding of the biological neuron have culminated in a wide range of different neuron models, ranging from the computationally efficient to the biologically realistic. These models enable the modelling of neural circuits found in the brain.

In recent years, much of the focus in neuron modelling has moved to the study of the connectivity of spiking neural networks. Spiking neural networks provide a vehicle to understand from a computational perspective, aspects of the brain's neural circuitry. This understanding can then be used to tackle some of the historically intractable issues with artificial neurons, such as scalability and lack of variable binding. Current knowledge of feed-forward, lateral, and recurrent connectivity of spiking neurons, and the interplay between excitatory and inhibitory neurons is beginning to shed light on these issues, by improved understanding of the temporal processing capabilities and synchronous behaviour of biological neurons. This research topic aims to amalgamate current research aimed at tackling these phenomena.

# Table of Contents

*05 Spiking Neural Network Connectivity and Its Potential for Temporal Sensory Processing and Variable Binding* Julie Wall and Cornelius Glackin *07 Metabolic Efficiency With Fast Spiking in the Squid Axon* Abdelmalik Moujahid and Alicia d'Anjou *15 Learning and Prospective Recall of Noisy Spike Pattern Episodes* Karl Dockendorf and Narayan Srinivasa *26 Anisotropic Connectivity Implements Motion-Based Prediction in a Spiking Neural Network* Bernhard A. Kaplan, Anders Lansner, Guillaume S. Masson and Laurent U. Perrinet *40 Spiking Timing Regulation on the Millisecond Scale by Distributed Synaptic Plasticity at the Cerebellum Input Stage: A Simulation Study* Jesús A. Garrido, Eduardo Ros and Egidio D'Angelo *58 Toggling Between Gamma-Frequency Activity and Suppression of Cell Assemblies* Christoph Börgers and Bryan Walker *69 Rapid, Parallel Path Planning by Propagating Wavefronts of Spiking Neural Activity* Filip Jan Ponulak and John J. Hopfield *83 Perspectives for Computational Modelling of Cell Replacement for Neurological Disorders* James B. Aimone and Jason P. Weick *90 Generalized Activity Equations for Spiking Neural Network Dynamics* Michael A. Buice and Carson C. Chow *100 Stable Learning of Functional Maps in Self-Organizing Spiking Neural Networks With Continuous Synaptic Plasticity* Narayan Srinivasa and Qin Jiang

## Spiking neural network connectivity and its potential for temporal sensory processing and variable binding

#### *Julie Wall <sup>1</sup> \* and Cornelius Glackin2*

*<sup>1</sup> Multimedia and Vision Research Group, School of Electronic Engineering and Computer Science, Queen Mary, University of London, London, UK <sup>2</sup> Adaptive Systems Research Group, Department of Computer Science, University of Hertfordshire, Hatfield, Hertfordshire, UK*

*\*Correspondence: julie.wall@qmul.ac.uk*

#### *Edited by:*

*Misha Tsodyks, Weizmann Institute of Science, Israel*

**Keywords: cell assembly, spiking neural network, spike timing, biological neurons, learning, connectivity, sensory processing**

The most biologically-inspired artificial neurons are those of the third generation, and are termed spiking neurons, as individual pulses or spikes are the means by which stimuli are communicated. In essence, a spike is a short-term change in electrical potential and is the basis of communication between biological neurons. Unlike previous generations of artificial neurons, spiking neurons operate in the temporal domain, and exploit time as a resource in their computation. In 1952, Alan Lloyd Hodgkin and Andrew Huxley produced the first model of a spiking neuron; their model describes the complex electro-chemical process that enables spikes to propagate through, and hence be communicated by, spiking neurons. Since this time, improvements in experimental procedures in neurobiology, particularly with *in vivo* experiments, have provided an increasingly more complex understanding of biological neurons. For example, it is now well-understood that the propagation of spikes between neurons requires neurotransmitter, which is typically of limited supply. When the supply is exhausted neurons become unresponsive. The morphology of neurons, number of receptor sites, amongst many other factors, means that neurons consume the supply of neurotransmitter at different rates. This in turn produces variations over time in the responsiveness of neurons, yielding various computational capabilities. Such improvements in the understanding of the biological neuron have culminated in a wide range of different neuron models, ranging from the computationally efficient to the biologically realistic. These models enable the modeling of neural circuits found in the brain.

In recent years, much of the focus in neuron modeling has moved to the study of the connectivity of spiking neural networks. Spiking neural networks provide a vehicle to understand from a computational perspective, aspects of the brain's neural circuitry. This understanding can then be used to tackle some of the historically intractable issues with artificial neurons, such as scalability and lack of variable binding. Current knowledge of feed-forward, lateral, and recurrent connectivity of spiking neurons, and the interplay between excitatory and inhibitory neurons is beginning to shed light on these issues, by improved understanding of the temporal processing capabilities and synchronous behavior of biological neurons. This research topic spans current research on neuron models to spiking neural networks and their application to interesting and current computational problems. The research papers submitted to this topic can be categorized into the following major areas of more efficient neuron modeling; lateral and recurrent spiking neural network connectivity; exploitation of biological neural circuitry by means of spiking neural networks; optimization of spiking neural networks; and spiking neural networks for sensory processing.

Moujahid and d'Anjou (2012) stimulate the giant squid axon with simulated spikes to develop some new insights into the development of more relevant models of biological neurons. They observed that temperature mediates the efficiency of action potentials by reducing the overlap between sodium and potassium currents in the ion exchange and subsequent energy consumption. The original research article by Dockendorf and Srinivasa (2013) falls into the area of lateral and recurrent spiking neural network connectivity. It presents a recurrent spiking model capable of learning episodes featuring missing and noisy data. The presented topology provides a means of recalling previously encoded patterns where inhibition is of the high frequency variety aiming to promote stability of the network. Kaplan et al. (2013) also investigated the use of recurrent spiking connectivity in their work on motion-based prediction and the issue of missing data. Here they address how anisotropic connectivity patterns that consider the tuning properties of neurons efficiently predict the trajectory of a disappearing moving stimulus. They demonstrate and test this by simulating the network response in a moving-dot blanking experiment.

Garrido et al. (2013) investigate how systematic modifications of synaptic weights can exert close control over the timing of spike transmissions. They demonstrate this using a network of leaky integrate-and-fire spiking neurons to simulate cells of the cerebellar granular layer. Börgers and Walker (2013) investigate simulations of excitatory pyramidal cells and inhibitory interneurons which interact and exhibit gamma rhythms in the hippocampus and neocortex. They focus on how inhibitory interneurons maintain synchrony using gap junctions. Similarly, Ponulak and Hopfield (2013) also take inspiration from the neural structure of the hippocampus to hypothesize about the problem of spatial navigation. Their topology encodes the spatial environment through an exploratory phase which utilizes "place" cells to reflect all possible trajectory boundaries and environmental constraints. Subsequently, a wave propagation process maps the trajectory between the target or multiple targets and the current location by altering the synaptic connectivity of the aforementioned "place" cells in a single pass. A novel viewpoint of the state-of-the-art for the exploitation of biological neural circuitry by means of spiking neural networks is provided by Aimone and Weick (2013). In their paper, a thorough and comprehensive review of modeling cortical damage due to stroke is provided. They argue that a theoretical understanding of the damaged cortical area post-disease is vital while taking into account current thinking of models for adult neurogenesis.

One of the issues with modeling large-scale spiking neural networks is the lack of tools to analyse such a large parameter space, as Buice and Chow (2013) discuss in their hypothesis and theory article. They propose a possible approach which combines mean field theory with information about spiking correlations; thus reducing the complexity to that of a more comprehensible ratelike description. Demonstrations of spiking neural networks for sensory processing include the work of Srinivasa and Jiang (2013). Their research consists of the development of spiking neuron models, initially assembled into an unstructured map topology. The authors show how the combination of self-organized and STDP-based continuous learning can provide the initial formation and on-going maintenance of orientation and ocular dominance maps of the kind commonly found in the visual cortex.

It is clear that research on spiking neural networks has expanded beyond computational models of individual neurons and now encompasses large-scale networks which aim to model the behavior of whole neural regions. This has resulted in a diverse and exciting field of research with many perspectives and a multitude of potential applications.

## **REFERENCES**

Aimone, J. B., and Weick, J. P. (2013). Perspectives for computational modeling of cell replacement for neurological disorders. *Front. Comput. Neurosci*. 7:150. doi: 10.3389/fncom.2013.00150


*Received: 14 November 2013; accepted: 02 December 2013; published online: 19 December 2013.*

*Citation: Wall J and Glackin C (2013) Spiking neural network connectivity and its potential for temporal sensory processing and variable binding. Front. Comput. Neurosci. 7:182. doi: 10.3389/fncom.2013.00182*

*This article was submitted to the journal Frontiers in Computational Neuroscience. Copyright © 2013 Wall and Glackin. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Metabolic efficiency with fast spiking in the squid axon

## *Abdelmalik Moujahid\* and Alicia d'Anjou*

*Computational Intelligence Group, Department of Computer Science, University of the Basque Country UPV/EHU, San Sebastián, Spain*

#### *Edited by:*

*Julie Wall, Queen Mary, University of London, UK*

#### *Reviewed by:*

*Julie Wall, Queen Mary, University of London, UK Christoph Borgers, Tufts University, USA*

#### *\*Correspondence:*

*Abdelmalik Moujahid, Computational Intelligence Group, Department of Computer Science, University of the Basque Country UPV/EHU, 20018 San Sebastián, Spain.*

*e-mail: jibmomoa@gmail.com*

Fundamentally, action potentials in the squid axon are consequence of the entrance of sodium ions during the depolarization of the rising phase of the spike mediated by the outflow of potassium ions during the hyperpolarization of the falling phase. Perfect metabolic efficiency with a minimum charge needed for the change in voltage during the action potential would confine sodium entry to the rising phase and potassium efflux to the falling phase. However, because sodium channels remain open to a significant extent during the falling phase, a certain overlap of inward and outward currents is observed. In this work we investigate the impact of ion overlap on the number of the adenosine triphosphate (ATP) molecules and energy cost required per action potential as a function of the temperature in a Hodgkin–Huxley model. Based on a recent approach to computing the energy cost of neuronal action potential generation not based on ion counting, we show that increased firing frequencies induced by higher temperatures imply more efficient use of sodium entry, and then a decrease in the metabolic energy cost required to restore the concentration gradients after an action potential. Also, we determine values of sodium conductance at which the hydrolysis efficiency presents a clear minimum.

**Keywords: Hodgkin–Huxley model, action potential, neuron metabolic energy, sodium entry, overlap load, regular-spiking cells**

## **1. INTRODUCTION**

The generation of action potentials in mammalian neurons involves the flux of different ions such as sodium, potassium, and calcium across the cell membrane. In this process, the electrochemical gradients are partially altered and must be restored by ion pumps which move ions from one side of the membrane to the other at the expense of energy. Re-establishing the concentration gradients after electrical discharges demands most of the energy used for neuronal metabolism (Laughlin et al., 1998; Laughlin, 2001; Shulman et al., 2004; Siekevitz, 2004). This requirement for metabolic energy has important implications for the brain's evolution and function (Attwell and Laughlin, 2001), and the availability of energy may impose a limit on neural activity taking into account that the brain has very small energy reserves (Ames III, 2000). It is, however, generally accepted that energy metabolism is highly organized within cells resulting in energetically efficient mechanisms that transfer energy from the site of generation to the processes that require it (Ames III, 2000; Belanger et al., 2011).

On other hand, all energy used for neural metabolism is finally transformed into heat (Ames III, 2000), and the metabolic brain activation appears to be the primary cause of heat production. Because neural properties are temperature-dependent, potential imbalance between heat production and dissipation could lead to overheating and aberrant functioning (Falk, 1990; Koch, 1998; Kiyatkin, 2007). Studying the relationship between temperature, firing frequency, sodium entry, and the energy cost required to generate an action potential using neuron models like the Hodgkin–Huxley model (Hodgkin, 1975) will provides a useful framework for addressing these issues.

The Hodgkin–Huxley model representing the dynamics of the squid giant axon continues to be the most frequently used model to study the dynamics and other properties of actual neurons. Based on biophysical considerations about the nature of the Hodgkin–Huxley model, we have recently found an analytical expression of the electrochemical energy involved in the dynamics of the model, which provides a new approach for estimating the energy consumption during the resting and active states of neurons (Moujahid et al., 2011). This energy function was used as a measure to evaluate the metabolic energy consumption of a neuron to maintain its signaling activity and to estimate the metabolic cost of transmitting information between neurons.

This approach, contrary to other methods (Attwell and Laughlin, 2001; Lennie, 2003), does not require ion counting for estimating the metabolic energy consumption of the generation of action potentials, and gives us the opportunity to check in the Hodgkin–Huxley model which ion counting gives the correct metabolic energy consumption. In this work we investigate the impact of ion currents overlapping on the number of adenosine triphosphate (ATP) molecules required to restore the concentration gradients after an action potential in the Hodgkin–Huxley model. Because the observed overlap is temperature-dependent, we have computed the number of ATP molecules per action potential and its corresponding energy cost at different values of temperature. Both the classic study by Hodgkin and Huxley of the squid axon (Hodgkin, 1975), and other recent works (Attwell and Laughlin, 2001; Lennie, 2003) assume that the action potential requires four times Na<sup>+</sup> charge compared to the charge needed for the change in voltage. This waste of Na<sup>+</sup> charge, and accordingly metabolic energy, is the result of extensive overlap between inward Na<sup>+</sup> and outward K<sup>+</sup> during the generation of action potentials.

However, it has been demonstrated that mammalian central neurons, characterized by action potentials similar to those of the squid giant axon, are significantly more efficient in generating action potentials (Carter and Bean, 2009).

We show in this work that increased firing frequencies induced by higher temperatures in the Hodgkin and Huxley model imply more efficient use of sodium entry and metabolic energy. The paper is organized as follows. In section 2, the dynamics and electrochemical energy of the Hodgkin and Huxley model are introduced. In section 3 we discuss the overlap of ion currents and energy efficiency as a function of temperature in the squid axon. Finally, conclusions are drawn in section 4.

## **2. MATERIALS AND METHODS**

## **2.1. THE HODGKIN–HUXLEY NEURON ENERGY**

In the original Hodgkin–Huxley model (Hodgkin and Huxley, 1952), the dynamics governing the membrane potential is given by:

$$\begin{aligned} C\,\dot{V} &= -\text{g}\_{Na}m^3h(V - E\_{Na})\\ &- \text{g}\_K n^4(V - E\_K) - \text{g}\_l(V - E\_l) + I,\end{aligned} \tag{1}$$

where *V* is the membrane potential in mV, *C* the membrane capacitance density in μF/cm2, *I* is the total membrane current density in μA/cm2. *gNa* , *gK*, and *gl* are the maximal conductances per unit area for ion and leakage channels, and *ENa*, *EK*, and *El* are the corresponding reversal potentials.

The gating variables *m*, *h*, and *n*, representing, respectively, sodium channels activation and deactivation variables, and potassium channels activation variable, obey the standard kinetic equation *x*˙ = α*x(*1 − *x)* − β*xx*, (*x* = *m, h, n*), where α*<sup>x</sup>* and β*<sup>x</sup>* are voltage-dependent variables. For sodium channels, the activation and deactivation rates are given by,

$$\begin{aligned} \alpha\_m(V) &= (2.5 - 0.1V) / (\exp(2.5 - 0.1V) - 1), \\ \beta\_m(V) &= 4 \exp(-V/18), \\ \alpha\_h(V) &= 0.07 \exp(-V/20), \\ \beta\_h(V) &= 1 / (\exp(3 - 0.1V) + 1). \end{aligned}$$

and for potassium channels,

$$\alpha\_{\boldsymbol{n}}(V) = (0.1 - 0.01V) / (\exp(1 - 0.1V) - 1),$$

$$\beta\_{\boldsymbol{n}}(V) = 0.125 \exp(-V/80).$$

In this work we have used for these parameters the standard constant values given in **Table 1** (Gerstner and Kistler, 2002).

The ion currents of sodium, potassium, and leakage (mainly chloride) correspond, respectively, to the three first terms in the right hand of the Equation (1), and are generated in response to a change in the respective ion conductances.

**Figure 1** shows in part **(A)** the shape of the sodium and potassium currents corresponding to a particular action potential. The sodium current is negative but has been depicted with a positive sign for a better appreciation of the great extent of its overlapping



*The membrane capacitance density is C* = 1*µF/cm2. The voltage scale is shifted so that the resting potential vanishes.*

with the potassium current. Note that as sodium and potassium currents are both of positive charges but moving in opposite directions of the cell's membrane they neutralize each other to the extent of their mutual overlap. The sodium charge that is not counterbalanced by simultaneously flowing potassium charge is much smaller for a greater overlap.

The unbalanced current load, represented in **Figure 1B**, consists of two components which occur, respectively, during the depolarizing and hyperpolarizing phases of the membrane potential action. The integral of the first component of this unbalanced load gives the net Na<sup>+</sup> ion charge that is not counterbalanced by simultaneously flowing K<sup>+</sup> crossing into the membrane during the rising of the action potential. The integral of the total unbalanced ionic current is directly proportional to the number of ATP molecules required to restore the resting potential.

For the action potential represented in **Figure 1B** generated for an external stimulus *I* = 13μA/cm2, the total sodium charge transfer computed as the integral of Na<sup>+</sup> current was 1168 nC/cm2 which agree with the estimate of 1098 nC/cm2 reported recently in Sengupta et al. (2010). Neutralized currents that account for the overlapping of Sodium and potassium fluxes give rise to an excessive overlap charge of about 1092 nC/cm2. This overlap has been calculated as the difference between the total Na<sup>+</sup> current and the depolarizing unbalanced component of Na<sup>+</sup> current. As stated in the work of Hodgkin, the squid action potential is very inefficient in the sense that it requires a fourfold Na<sup>+</sup> charge compared to the minimum charge necessary to depolarize a pure capacitor (Hodgkin, 1975). The efficiency of sodium entry during the generation of action potential in the squid axon at different temperatures is discussed in section 3. The values of sodium and overlap load reported above correspond to a temperature of 6.3◦C.

To estimate the energy consumption necessary to restore the resting potential in the Hodgkin–Huxley model, we have used a new approach not based on the ion counting method. Following previous works of finding energy functions of neuron models of chaotic dynamics (Sarasola et al., 2004, 2005; Torrealdea et al., 2009), we have deduced for the model given by Equation (1) an energy function representing the analytical expression of the electrochemical energy involved in its dynamics. The procedure followed to find this energy has been reported in detail in Moujahid et al. (2011), and is summarized below.

It is well known that the Hodgkin–Huxley equation given by Equation (1) expresses an electrical circuit consisting of capacitor *C* and three Na, K, and L ionic channels, where *gNa* , *gK*, and *gl* are the maximal conductances, and batteries stand for the Nernst potentials of their corresponding ions. If *V* is the membrane potential, the total electrical energy accumulated in the circuit at a given moment in time is,

$$H(t) = \frac{1}{2}CV^2 + H\_{\text{N}\_d} + H\_K + H\_l,\tag{2}$$

where the first term in the summation gives the electrical energy accumulated in the capacitor and represents the energy needed to create the membrane potential *V* of the neuron. The other three terms are the respective energies in the batteries needed to create the concentration jumps in sodium, potassium, and chloride. The electrochemical energy accumulated in the batteries is unknown. Nevertheless, the rate of electrical energy provided to the circuit by a battery is known to be the electrical current through the battery times its electromotive force. Thus, the total derivative with respect to time of the above energy will be,

$$
\dot{H}(t) = CV\dot{V} + i\_{N\_d}E\_{N\_d} + i\_KE\_K + i\_lE\_l. \tag{3}
$$

where *ENa*, *EK*, and *El* are the Nernst potentials of the sodium, potassium and leakage ions in the resting state of the neuron. And *iNa* , *iK*, and *il* are the ion currents of sodium, potassium and leakage, given by,

$$i\_{N\_a} = \lg\_{Na} m^3 h (V - E\_{Na}),$$

$$i\_K = \lg\_K n^4 (V - E\_K),$$

$$i\_l = \lg (V - E\_l),\tag{4}$$

If we substitute Equations (1) and (4) in Equation (3), we have for the energy rate in the circuit,

$$\dot{H} = V I - \mathbb{g}\_{N\_d} m^3 h (V - E\_{N\_d})^2 - \mathbb{g}\_K n^4 (V - E\_K)^2 - \text{g} (V - E\_l)^2,\tag{5}$$

which provides the total derivative of the electrochemical energy in the neuron as a function of its state variables. The first term in the right hand summation represents the electrical power given to the neuron via the different junctions reaching the neuron and the other three terms of the summation represent the energy per second consumed by the ion channels. This equation permits evaluation of the total energy consumed by the neuron and also gives information about the consumption associated to each of the sodium, potassium, and leaking channels.

**Figure 2A** reports the time course of the electrochemical energy consumption in nJ/s corresponding to sodium ion channel for the particular action potential (dashed line) generated at an external constant stimulus *I* = 13μA/cm2. This energy consumption is given by the term *gNam*<sup>3</sup>*h(V* − *ENa )*<sup>2</sup> [see Equation (5)]. The total metabolic consumption given by the sum of the last three terms in Equation (5) is reported in **Figure 2B**. The last three terms of the energy derivative are negative definite, corresponding to an actual energy dissipation of energy, but has been represented as a positive consumption.

The total consumption of energy per second at the ion channels required to generate one particular action potential must be replenished by the ion pumps and metabolically supplied by hydrolysis of ATP molecules in order to maintain the neuron's activity. The higher demand of metabolic energy associated to the generation of spikes is clearly visible, for instance at *I* = 13μA/cm2 which is the value used to generate the particular action potential analyzed previously, the average of the total metabolic consumption depicted in **Figure 2B** is about 11.4 μJ/s per membrane unit area. This consumption must be replenished by metabolic ATP supply. The number of ATP molecules per membrane unit area hydrolyzed by the Na+/K<sup>+</sup> ATPase pump to extrude the Na<sup>+</sup> load can be deduced from the amount of Na<sup>+</sup> ions crossing the membrane during an action potential, operating

**FIGURE 2 | (A)** Electrochemical energy consumption corresponding to sodium ions at an action potential (dashed line) generated for an external stimulus *I* = 13μA/cm2. Values of membrane potential are scaled with a factor of 200. **(B)** Total metabolic consumption corresponding to three ion channels at the same action potential. Magnitudes refer to nJ/s. Vertical dashed line in panels **(A)** and **(B)** separates between the rising and falling phase regions.

with a ratio of 3Na<sup>+</sup> per ATP (Attwell and Laughlin, 2001; Crotty et al., 2006).

The ratio of the total metabolic consumption (in J/s) to one third of the number of Na<sup>+</sup> load through the membrane expressed in electronvolts per ATP represents the efficiency of the ATP hydrolysis measured as the free energy provide by the hydrolysis of one molecule of ATP. We will show that our calculation of the actual energy consumption and the number of ATP molecules involved in the generation of an action potential are consistent with relevant data in the literature and that the Hodgkin–Huxley model produces accurate estimates of energy consumption.

## **3. RESULTS**

## **3.1. ION CURRENTS OVERLAPPING**

The overlap of ion currents in the Hodgkin–Huxley model decreases as the temperature increases and the impact of overlap on the number of ATP molecules required per action potential can be analyzed rescaling the model equations to include the temperature-dependence. In this work, we have adopted the original assumption of Hodgkin and Huxley multiplying the rates of change of the activation *m*, *n*, and inactivation *h* gating variables by a factor *k* = 3*(T*−6*.*3*)/*10, *T* [◦C] (Chandler and Meves, 1970).

To illustrate the overlap decrease with increasing temperature, **Figure 3** shows instantaneous Na<sup>+</sup> and K<sup>+</sup> currents elicited by a single spike at, respectively, 6.3◦C and 18.5◦C. To perform the simulation we have rescaled the current equations to different temperatures between 6.3◦C and 18.5◦C which is the range of temperatures in the original study of the squid giant axon by Hodgkin and Huxley. It should be noticed that for higher temperature the firing regime in the Hodgkin–Huxley model can only be maintained at large values of the injected current.

To quantify the current overlap we have adopted two different measures. Following (Attwell and Laughlin, 2001), we calculated the dimensionless charge separation as the Na<sup>+</sup> charge that is not counterbalanced by simultaneously flowing K<sup>+</sup> charge (depolarizing component of the unbalanced load depicted in **Figure 1B** divided by total Na<sup>+</sup> charge per action potential). The relationship between charge separation and temperature is illustrated in the inset of **Figure 4A**.

As it can be appreciated, charge separation shows a 2.98-fold increase with increasing temperature and varies from 0.0652 at 6.3◦C to 0.1942 at 18.5◦C. At this temperature, Alle et al. (2009) studying mossy fibers of the rat hippocampus report an average charge separation of 0.769. The respective consumptions per action potential reflect this different overlap. At 18.5◦C with an injection current *I* = 13 μA/cm2, the Hodgkin–Huxley model of the squid giant axon demands 0*.*68 × 1012ATP/cm2 to produce one action potential (see **Table 2**), while according to Alle et al. (2009), mossy fibers of the rat hippocampus demand only 0*.*32 × 1012ATP/cm2 per action potential.

The collected results for the number of ATP molecules per unit membrane area to produce one action potential related to charge separation are depicted in **Figure 4A**. As it can be seen the increase in separation implies a 3.54-fold decrease in the number of ATP molecule/cm2. At 6.3◦C the Na<sup>+</sup> charge transfer per unit membrane area of an action potential in the squid axon is about 1168 nC/cm2, consuming 2.43 1012 ATP molecules/cm2. While at 18◦C, the Na<sup>+</sup> load is 346 nC/cm2 consuming 0.72 1012 ATP molecules/cm2. So, high frequency firing induced by high temperature appears to be more efficient in the use of Na<sup>+</sup> entry. **Table 2** reports details of values achieved for these measures at different values of temperature.

The other measure used in this work to quantify the current overlap has dimension of charge and is computed following (Crotty et al., 2006) as the difference between the total Na<sup>+</sup> load and the depolarizing component of the Na<sup>+</sup> load. **Figure 4B** shows the relationship between the total Na<sup>+</sup> load and the overlap load measured in nC/cm2. As it can be seen the total Na<sup>+</sup> charge increases linearly with overlap load with a slope close to unity. That is, the overlap load is positively correlated with the total Na<sup>+</sup> charge that crosses the membrane resulting in a decisive factor when analyzing the efficiency. The same behavior is observed when considering the relationship between the total unbalanced load computed as the sum of Na<sup>+</sup> and K<sup>+</sup> currents, and the overlap load.

**FIGURE 4 | (A)** The number of ATP molecules required per action potential versus charge separation. In the inset charge separation measured as Na<sup>+</sup> charge that is not counterbalanced by simultaneously flowing K<sup>+</sup> charge divided by total charge per action potential. **(B)** The relation between the total Na<sup>+</sup> load and the overlap charge, calculated as the difference between the total Na<sup>+</sup> current and its depolarizing component.


**Table 2 | Simulation values at different temperatures between 6.3◦C and 18.5◦C of the overlap between N+ and K+ ion currents in the squid giant axon.**

*The degree of overlap is measured as the difference between the total Na*<sup>+</sup> *load and the depolarizing component of the Na*<sup>+</sup> *load per action potential. Their corresponding values of firing frequency, consumption rate, ATP molecules, and Na*<sup>+</sup> *entry per spike are also given. All extensive magnitudes refer to cm*<sup>2</sup> *of membrane.*

**Figure 5** reports, in the right axis, the collected values of the total unbalanced load related to the overlap load. And, in the left axis, the unbalanced component of Na<sup>+</sup> load crossing the membrane during the rising phase as a function of overlap. At 6.3◦C the total unbalanced load is 175 nC/cm2 which is 2.3 times the depolarizing unbalanced Na<sup>+</sup> load, while at 18◦C both measures provide close values around 65 nC/cm2. We observe that for a decrease of overlap load between its maximum value achieved at 6.3◦C and values around 560 nC/cm2 corresponding to a temperature of about 12◦C, the depolarized unbalanced load undergoes only a slight decrease. However, further increase of temperature causes the overlap to decrease by 2.12-fold resulting in a 1.13-fold decrease of the depolarizing unbalanced Na<sup>+</sup> load.

## **3.2. ENERGY EFFICIENCY**

The calculated values of the electrochemical energy involved in the dynamics of the Hodkin–Huxley model according to our method are reported in **Figure 6**. Both the total metabolic consumption and the metabolic consumption in the ionic channels show a decreasing pattern as the temperature increases. At 6.3◦C, the total action potential energy cost is around 152 nJ/cm2, 45%

of which is consumed in the sodium channel, and the rest is mainly consumed in the potassium channel. While at 18◦C the energy consumed in both channels represents 49% of the total metabolic consumption. At this temperature the total metabolic consumption experiences a 3.35-fold significant decrease.

The total cost of one pump's cycle, that pumps three Na<sup>+</sup> ions out of the cell and two K<sup>+</sup> ion in, is computed as the ratio of the total metabolic consumption [given by the last three terms of Equation (5)] to one third of the total number of Na<sup>+</sup> load. According to our calculations, the liberated free energy by hydrolyzing one ATP molecule, defined as hydrolysis efficiency, seem to be independent of temperature and shows values around 0.39 eV which is close to other estimates in the literature (Nelson, 2004; Sinkala, 2006).

However, if we consider only the depolarizing metabolic consumption in the Na<sup>+</sup> channel associated to the depolarizing unbalanced Na<sup>+</sup> load, the hydrolysis efficiency seem to be affected by temperature showing a parabolic shape with a minimum around 0.38 eV (see **Figure 7A**). This minimum is observed for a temperature around 13◦C corresponding to a firing frequency of about 138 Hz.

The same behavior (see **Figure 7B**) was observed when studying the depolarizing efficiency as a function of the sodium channel conductance. The range of variation of the ion channel conductances was obtained multiplying the maximum conductances *gNa*, *gK*, and *gl* in the Hodgkin–Huxley model by the expression *k* = 1*.*5*(T*−6*.*3*)/*<sup>10</sup> for temperatures ranging between 6.3◦C and 18.5◦C according to Chandler and Meves (1970). The maximum depolarizing efficiency occurs for a sodium channel conductance value around 160 mS/cm<sup>2</sup> which is close to the biological density conductance. Crotty et al. (2006), studying the energetic cost that arise from an action potential using computational models of the squid axon as a function of sodium channel densities, showed that the energy cost associated with the action potential produce a convex

**FIGURE 7 | (A)** The ratio of the depolarizing Na<sup>+</sup> energy consumption to one third of the depolarizing unbalanced Na<sup>+</sup> load as a function of temperature. **(B)** The ratio of the depolarizing Na<sup>+</sup> energy consumption to one third of the depolarizing unbalanced Na<sup>+</sup> load as a function of sodium channel conductance. The range of variation of the Na<sup>+</sup> channel conductance was obtained multiplying the maximum conductances in the Hodgkin–Huxley model by the factor *k* = 1*.*5*(T*−6*.*3*)/*<sup>10</sup> for temperatures ranging between 6.3◦C and 18.5◦C.

curve with a minimum around the same value reported above.

We have seen that the fast-spiking regime in the Hodgkin– Huxley model induced by higher temperatures implies more efficient use of sodium entry, and gives rise to energy efficient action potentials characterized by less overlap between sodium and potassium currents. Since this firing regime could be also induced by raising the external stimulus, it would be interesting to investigate whether the same fast-spiking regime achieved, for one hand, by raising the temperature, and for the other hand, by increasing the external current, contributes in the same way in reducing both the energy consumption and the overlap load. To do so, we have computed the energy consumption required to generate an action potential and the corresponding firing frequency for different values of the external stimulus I. We have considered values of I ranging between 13μA/cm2 and 40 μA/cm2.

As it can be appreciated in **Figure 8** (left panel), the influence of higher stimulus on the firing frequency in the squid giant axon is more significant only for higher temperatures. In fact, at a given temperature, raising the external current causes the firing frequency to increase by 1.46-fold, while at a given external stimulus, the firing frequency experiences a 2.96-fold increase with increasing temperature. On the other hand, according to our results, the fast-spiking regime induced by higher stimuli is less efficient in generating action potentials expending more energy compared with the same fast-spiking regime induced by higher temperatures. For example, at *T* = 8◦C and a relatively high external current *I* = 39μA/cm2 (see circle markers in **Figure 8**), the firing frequency of the squid axon is of about 127 Hz, and are necessary 106.75 nJ/cm2 to generate one action potential. While at a higher temperature *T* = 12◦C and a low current *I* = 13μA/cm2 (see square markers in **Figure 8**) corresponding to the same firing regime (i.e., *F* = 127 Hz), the energy consumption is only

83.24 nJ/cm2. This 0.78-fold decrease in energy consumption is accompanied by a 0.76-fold decrease in overlap load between the spike-generating Na<sup>+</sup> current and delayed rectifier K<sup>+</sup> current. The corresponding values of overlap load are 740.83 nC/cm2 and 563.92 nC/cm2, respectively. This confirms again that the overlap load of voltage-gated currents of Na<sup>+</sup> and K<sup>+</sup> dominates energy efficiency, and efficient action potentials have little overlap.

## **4. DISCUSSIONS**

Our results with simulated spikes of the squid axon show that increased firing frequencies induced by higher temperatures imply more efficient use of sodium entry due mainly to the reduced overlap load between inward Na<sup>+</sup> current and outward K<sup>+</sup> current. This, corroborates what has been reported recently in Carter and Bean (2009); Sengupta et al. (2010), and Alle et al. (2009), i.e., the most energy efficient action potentials are those generated by Na<sup>+</sup> and K<sup>+</sup> currents that have substantially reduced overlap.

The values of sodium entry are close to the original values calculated by Hodgkin and Huxley (1952), and are in nice agreement with values reported recently by Sengupta et al. (2010). At 6.3◦C corresponding to 75 Hz we obtain a sodium influx of 12.12 pmole/cm2 per spike (1168 nC/cm2) while at 18◦C and 206 Hz the sodium influx is 3.58 pmole/cm2 per spike (346 nC/cm2) which means a 3.38-fold decrease in sodium entry corresponding to a 2.75-fold increase in firing frequency.

Regarding the energy consumption associated to the generation of action potentials in the squid axon, we have found that the hydrolysis of one ATP molecule liberates a free energy with optimum values that range from 0.37 eV achieved for a sodium conductance around 160 mS/cm2 and when only the depolarizing components of both energy consumption and Na<sup>+</sup> load are considered. To a value of about 0.39 eV produced when considering the total metabolic consumption associated to the total Na<sup>+</sup> load. As stated before, these values of hydrolysis efficiency are in nice agreement with other estimates, which confirms that our method of calculation of the actual energy consumption by the pump and the number of ATP molecules involved are consistent with other data in the literature.

Also, we have found that the fast-spiking regime in the Hodgkin–Huxley model induced by a higher stimulus appears to be less efficient in generating action potentials expending more energy compared with the same fast-spiking regime induced by a higher temperature. Accordingly, the reduction in the overlap of the Na<sup>+</sup> and K<sup>+</sup> currents is less when the firing frequency is raised by rising the external stimulus. Carter and Bean (2009), have suggested that the primary determinant of differences in Na<sup>+</sup> entry efficiency among neurons is their different action potential shapes. Indeed, the shape of the action potential in the squid axon at a given frequency is different depending on whether it has been generated by raising the temperature or the external stimulus.

The findings of this work were validated using others Hodgkin–Huxley-like model neurons, in particular, we have considered the simplest model of regular spiking cells in neocortex which consists of sodium and potassium currents responsible for generating spikes, and an additional slow voltage-dependent potassium current responsible for spike-frequency adaptation. This model generates action potentials which capture the typical firing characteristics of regular spiking neurons in ferret visual cortex *in vivo* (Pospischil et al., 2008). To carry out the comparison, we performed the same experiment as for the squid giant axon, i.e., rescaling the model equations given in Pospischil et al. (2008) to include the temperature-dependence of the membrane ionic conductances, using values that range between 20◦C and 40◦C which corresponds to the normal range of temperature in these cells. The external current was varied between 1.5 and 5μA/cm2. The results (see **Figure 9**) show a behavior qualitatively

and external stimulus.

similar to that we observed in the squid axon. i.e., less energy is spent per action potential at higher temperatures than at lower ones due mainly to the reduced overlap between sodium and potassium currents. And, the efficiency of action potentials is more dependent on temperature than on the external stimulus. At *T* = 20◦C and an external high stimulus of about 4.75μA/cm2, the regular spiking cell fires at a frequency of about 46.5 Hz, and requires 90.5 nJ/cm2 to generate one action potential. While, for a higher temperature *T* = 36*.*5◦C and relatively a small current *I* = 2*.*75μA/cm2, making the cell to fire at the same firing regime (i.e., frequency of about 46.5 Hz), the energy consumption is 29 nJ/cm2 representing only one third of the energy expended when considering higher stimulus. This difference in energy

## **REFERENCES**


required to generate action potentials in the squid giant axon as well as in the regular-spiking model of cells in neocortex is lower

at higher temperatures. Also, we found that for these cells, the fast-spiking regimes induced by higher temperatures are more energy efficient than those induced by higher stimuli. Finally, we think that the approach considered in this work could bring a new framework to analyze the relationship between energy consumption, temperature, and firing frequency in neuronal tissues since it could be potentially used for more relevant models of mammalian brains.

consumption is due to a significant reduction of the overlap load

Our principal findings were that the energy consumption

Hodgkin-Huxley neurons. *Phys. Rev. E* 83, 031912.

that decreases from 506 to 88 nC/cm2.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 03 September 2012; accepted: 29 October 2012; published online: 15 November 2012.*

*Citation: Moujahid A and d'Anjou A (2012) Metabolic efficiency with fast spiking in the squid axon. Front. Comput. Neurosci. 6:95. doi: 10.3389/ fncom.2012.00095*

*Copyright © 2012 Moujahid and d'Anjou. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

## Learning and prospective recall of noisy spike pattern episodes

## *Karl Dockendorf and Narayan Srinivasa\**

*Information and System Sciences Lab, Center for Neural and Emergent Systems, HRL Laboratories LLC, Malibu, CA, USA*

#### *Edited by:*

*Julie Wall, University of London, UK*

*Reviewed by:*

*James B. Aimone, Sandia National Laboratories, USA Juan G. Avina-Cervantes, University of Guanajuato, Mexico*

#### *\*Correspondence:*

*Narayan Srinivasa, Information and System Sciences Lab, Center for Neural and Emergent Systems, HRL Laboratories LLC, 3011, Malibu Canyon Road, Malibu, CA 91388, USA e-mail: nsrinivasa@hrl.com*

Spike patterns *in vivo* are often incomplete or corrupted with noise that makes inputs to neuronal networks appear to vary although they may, in fact, be samples of a single underlying pattern or repeated presentation. Here we present a recurrent spiking neural network (SNN) model that learns noisy pattern sequences through the use of homeostasis and spike-timing dependent plasticity (STDP). We find that the changes in the synaptic weight vector during learning of patterns of random ensembles are approximately orthogonal in a reduced dimension space when the patterns are constructed to minimize overlap in representations. Using this model, representations of sparse patterns maybe associated through co-activated firing and integrated into ensemble representations. While the model is tolerant to noise, prospective activity, and pattern completion differ in their ability to adapt in the presence of noise. One version of the model is able to demonstrate the recently discovered phenomena of preplay and replay reminiscent of hippocampal-like behaviors.

**Keywords: spiking, STDP, learning, sequences, prospection, preplay, replay, memory**

## **INTRODUCTION**

The CA3 region of the mammalian hippocampus is a typical example of a recurrent neural network *in vivo* (Hasselmo et al., 1995; Rolls, 2000; Kobayashi and Poo, 2004; Kesner, 2007; Li et al., 2010). It is known these recurrent networks are well-suited for learning pattern sequences and pattern completion (Treves and Rolls, 1992; Hopfield, 1995; Káli and Dayan, 2000; Gold and Kesner, 2005; Leutgeb et al., 2007; Yassa and Stark, 2011). While in general neural networks (Graves et al., 2009) have shown their ability to perform tasks on highly corrupted data, spiking neural network (SNN) models are often sensitive to input scale and must be carefully tuned to generate the desired output (Buonomano, 2005). These issues are amplified in recurrent SNNs where instabilities can result in cascades of activity even with slight input perturbations (Gerstner and Kistler, 2002). Furthermore, biological networks are exposed to a great degree of input variability that can cause many simulated SNNs to fail.

In order to address input variability, recent model implementations showed that it is possible to globally scale the synaptic weight update on afferent synapses so as to constrain the cumulative weight to an artificial limit. This helps to also maintain stability in the model (Song et al., 2000; Van Rossum et al., 2000). However, presynaptic spike timing is primarily maintained locally at each synapse without evidence *in vivo* for an instantaneous global rescaling of all afferent synaptic weights. One biologically plausible solution is to incorporate homeostatic regulation. Several forms of homeostasis exist in biological networks that occurs at many timescales and is critical for the stability of these networks (Malinow and Malenka, 2002; Renart et al., 2003; Turrigiano and Nelson, 2004; Deeg, 2009; Turrigiano, 2011). Moreover, modeling homeostasis allows for a self-adjustment and overall scaling of input synaptic weights to neurons in a network and can help compensate for this input variability.

Learning of pattern sequences through the use of plasticity has been studied before (Berthouze, 2000; Arthur, 2006). These methods have relied on learning associations with sequences of on-going reliable and repetitive background activity. However, repeatable sequences have only been detected during task-related activity (Itskov et al., 2011). Moreover, the sequencing of activity (Diba and Buzsaki, 2007) is likely to be based on self-sustaining recurrent pathways of activity in the network due to previous association with sensory and spatial activity.

In this paper, we present three variations of a recurrent SNN for learning spatiotemporal patterns of activity using spiketiming dependent plasticity (STDP) with homeostatic regulation of activity. This model can learn to integrate patterns into pattern sequences from multiple noisy presentations and complete previously learned patterns from partially available data. Furthermore, stable pattern learning can be achieved with synaptic weight changes despite relative differences in intensity of the input patterns. The network demonstrates both *prospective* firing of activity and pattern completion while maintaining stability without the need to balance the input weights directly. The lessons learned from the development of these models are discussed and potential opportunities for handling their limitations are noted.

## **MATERIALS AND METHODS SIMPLE NETWORK MODEL**

The simple network model is comprised of a recurrent population of Izhikevich neurons (Izhikevich, 2007a) with the default

parameters from **Table 1**. Each neuron connects to a fraction (*pe* and *pi*) of all other excitatory and inhibitory neurons (see **Figure 1**) with both excitatory and inhibitory connections having a random delay (τ*e*delay and τ*i* delay). The network is constructed such that for each excitatory connection, an inhibitory connection is also made between those neurons. For simplicity, one neuron is used for both connections, however, in an alternate larger model with different input scheme these dual paths are implemented with separate inhibitory and excitatory neurons. The recurrent network structure itself is random but balanced such that each neuron has an equal number of efferent and afferent connections and all neurons have the same number of connections.

Neuron dynamics (Izhikevich, 2007a) are described by three variables, membrane voltage (*v*), recovery (*u*), and excitatory



scaling of afferent synaptic conductances (*s*, long-term homeostatic parameter):

$$\nu' = \frac{1}{C}(k(\nu - \nu\_r)(\nu - \nu\_t) - \mu + I\_{\text{syn}} + I\_{\text{noise}}) \tag{1}$$

$$u' = a(b(\nu - \nu\_r) - u) \tag{2}$$

$$s' = \frac{\alpha\_D}{\mathfrak{r}\_{\alpha}}\tag{3}$$

where *I*noise is zero mean gaussian noise with σ = 80 pA, ω*<sup>D</sup>* is the target firing rate, and τ*<sup>w</sup>* is the time constant. *I*syn is composed of inputs from *N* excitatory and *M* inhibitory synapses:

$$I\_{\rm syn} = \sum\_{j=1}^{N} s \cdot \lg\_{\hat{\jmath}}(E\_t - \nu) + \sum\_{k=1}^{M} \lg\_k(E\_i - \nu) \tag{4}$$

where *gj* and *gk* are the excitatory and inhibitory synaptic conductances respectively. Upon action potentials (*v* ≥ *vp*),

$$\nu = c, \ u = u + d, \text{ and } s = s - \frac{1}{\mathfrak{r}\_{\alpha}} \tag{5}$$

Integration was performed using Euler's method for all variables [except *v* where a hybrid method (Izhikevich, 2010) was used] with a simulation time step of 0.5 ms.

The network model is designed to have inhibitory and excitatory connectivity between neurons so that the effective pairing between any group of co-active neurons (an ensemble) and another neuron scales from net inhibitory to net excitatory for a given ensemble. Therefore, a direct connection from a neuron to another neuron maybe strongly excitatory, while the indirect connectivity between the presynaptic and postsynaptic neuron maybe inhibitory depending on the currently active ensemble. A network with the aforementioned properties can then associate positively or, by default, negatively each neuron with each ensemble. As long as the coding of the inputs and relative connectivity of the network spread the representation of ensembles such that these net effects from ensembles to any neuron can be modulated, new patterns can be learned while minimally effecting the coding of other ensembles (see Results). Thus, these networks exhibit the ability to learn or recall spike patterns whether they code for sequential spatial, sensory, or other data.

Two different forms of synaptic plasticity are used for the model, one for excitatory and one for inhibitory synapses. The inhibitory synapse rule is symmetrical, and functionally implements a rule where co-active neurons reduce their inhibitory coupling, but neurons that fire independently have strong inhibitory connectivity. Inhibitory plasticity uses an inverted top-hat shaped symmetric STDP curve, which is similar to a Mexican-hat plasticity curve (Caporale and Dan, 2008; Srinivasa and Jiang, 2013). Upon presynaptic or postsynaptic action potential:

$$
\Delta w\_i = \begin{cases}
8\_i^{\text{LTP}} & \text{if } \left| t\_{\text{post}} - t\_{\text{pre}} \right| < \mathbf{r}\_i^{\text{LTP}} \\
8\_i^{\text{LTP}} & \text{else if } \left| t\_{\text{post}} - t\_{\text{pre}} \right| \le \mathbf{r}\_i^{\text{LTP}} \\
0 & \text{otherwise}
\end{cases} \tag{6}
$$

where *w* is constrained to 0 ≤ *wi* ≤ *w*max *<sup>i</sup>* , <sup>τ</sup>LTD *<sup>i</sup>* is the time window of long-term depression (LTD) for inhibitory STDP. The parameters δLTD *<sup>i</sup>* and <sup>δ</sup>LTP *<sup>i</sup>* correspond to the change in synaptic weights for inhibition during LTD and long-term potentiation (LTP), respectively. The terms *t*pre and *t*post correspond to the time at which presynaptic and postsynaptic spike events occur.

Complementary to the inhibitory weight changes, excitatory changes are asymmetric and strengthen synapses that contribute to the causal activation of the postsynaptic neuron and weaken those that are activated in reverse order unless the postsynaptic neuron activates again. Excitatory plasticity follows a tripletbased STDP rule (Pfister and Gerstner, 2006) and is described by the dynamics when presynaptic or postsynaptic action potentials occur:

$$
\Delta\omega\_{\varepsilon} = \begin{cases}
(\ $\_{\varepsilon}^{\text{LTP}} + \triangle\omega\_{\varepsilon}^{\text{LTP}})e^{\frac{l\_{\text{pre}} - l\_{\text{post}}}{t\_{\text{eff}}^{\text{LP}}}} & \text{if } t\_{\text{post}} - t\_{\text{pre}} \ge 0 \\
\$ \_{\varepsilon}^{\text{LTP}}e^{\frac{l\_{\text{pre}} - l\_{\text{post}}}{t\_{\text{eff}}^{\text{LP}}}} & \text{if } t\_{\text{post}} - t\_{\text{pre}} < 0
\end{cases} (7)$$

where *w* is constrained to 0 = *wi* = *w*max *<sup>i</sup>* and *w*LTD *<sup>e</sup>* is the change due to the last depression event. The parameters δLTD *<sup>e</sup>* and δLTP *<sup>e</sup>* correspond to the change in synaptic weights for excitation during LTD and LTP, respectively and τLTP *<sup>e</sup>* is the time window of LTP for excitatory STDP. Excitatory synaptic weights are initialized with values from a uniform distribution on the interval [0, 0.1 *w*max *<sup>e</sup>* ).

#### **ALTERNATE NETWORK MODEL**

An alternate model of 2000 excitatory and 500 inhibitory neurons was used for some simulations with several other modifications detailed below. Excitatory neurons were recurrently connected with other excitatory neurons with probability *pee* and with the inhibitory neurons with probability *pei*. Inhibitory neurons were recurrently connected with other inhibitory neurons with probability *pii* and with the excitatory neurons with probability *pie*, see **Table 2**.

The alternate model used Izhikevich's fast spiking (FS) model for inhibitory neurons and increased the membrane capacitance, *C*, of the neuron model to 500 pF for the excitatory neurons. Furthermore, the excitatory reversal potential, *Ee* was increased to 40 mV to prevent excitation lock that could occur with the increased membrane capacitance. Excitatory-excitatory and excitatory-inhibitory connections used the aforementioned triplet STDP rule, while inhibitory-excitatory connections used **Table 2 | Alternate model parameter modifications.**


**Table 3 | Preplay model alterations.**


the inverted top-hat STDP rule and inhibitory-inhibitory connections used a non-inverted top-hat STDP rule. These inhibitory connections were also changed to an order of magnitude faster conductance decay time constant. A single inhibitory feedback neuron with long decay time constant was added with connections to and from all excitatory neurons; these connections were non-plastic.

## **PREPLAY MODEL**

Patterns of place cell activity are preplayed in forward order or replayed in reverse order at the beginning and end, respectively, of a linear track (De Almeida et al., 2007). Similarly, one version of our model demonstrates preplay and replay of activity. Although less biologically plausible, this model used the same input and recurrent connection scheme and was the same size as the simple model with a few exceptions. Firstly, each neuron was the regularly spiking simple four parameter model (Izhikevich, 2003) but with membrane capacitance 1000× greater, the *a* parameter 1000× smaller, and the *d* parameter 10× smaller. Secondly, weight changes were larger with an inverted window for inhibitory STDP (this was allowable since the network was more stable due to higher capacitances and slower postspike recovery). The parameter settings for this model are summarized in **Table 3**.

## **NETWORK INPUT**

Two network input structures were considered: a 1-to-1 input neuron to recurrent neuron model and a many-to-many input model where input neurons spread output onto three recurrent neurons and each recurrent neuron receive 12 connections from input neurons. 1-to-1 network inputs are used with the simple model and preplay model; the many-to-many network input structure is used with the alternate model. In the simulations with 1-to-1 inputs, each neuron receives simulated spiking input through a single large, non-plastic, excitatory synapse from an input neuron so that a spatiotemporal episode can be strongly forced onto the downstream recurrent network model. This strong influence results in the recurrent network having a high likelihood to spike in a similar manner as the upstream inputs. This input drive is modeled as noisy frequency modulation of input spikes as shown in **Figure 2**.

Twelve episodes were constructed each consisting of 25% of the network and segmented into 30 temporal steps. These episodes were presented in blocks of four episodes each and consisting of 100% of the network for the 1-to-1 network input structure. In each block, the episodes were presented at random (from a uniform distribution) and with overlap of four temporal steps. The current step is presented with the last few patterns in decreasing intensity each low frequency period. Neurons comprising each temporal step are activated simultaneously (with one high frequency period of jitter) for each active spatiotemporal step. As each episode is presented, one neuron out of each temporal step is selected at random to be removed from that presentation of the episode.

In the alternate model, the episodes are encoded in the same manner. However, the input neuron allotment to each episode was performed without regard for overlap and redundant usage during each group of episodes (see **Figure 1**). Additionally, a larger network was used and as a result each temporal step consisted of 14 neurons as opposed to 4.

In the preplay model, during each episode presentation each temporal step was repeated 7 times before proceeding to the next temporal step, whereas in the simple and alternate models each step occurred once per episode.

For all models, noise was injected into the network by adding spontaneous action potentials from the input neurons at a rate of ν = 0*.*05 Hz per neuron in addition to the somatic gaussian current input. A single, fast inhibitory, input neuron projects to the entire recurrent network with a fixed, but varied time delay (1–4 ms). The inhibitory input neuron supplies the network with high frequency modulated action potentials at a rate of 250 Hz.

## **SIMULATION AND ANALYSIS**

Simulations were performed using a custom C/C++ MPIbased simulator and run 2–9× real time on a 2 GHz quad-core i7 MacBook Pro with 8 GB of RAM. Spike data and weights were analyzed using MATLAB. Local linear embedding (LLE) of weights was performed using the MATLAB Toolbox for Dimensionality Reduction freely available at http://homepage.tudelft.nl/19j49/Matlab\_Toolbox\_for\_ Dimensionality\_Reduction.html

## **RESULTS**

The results are summarized for the models as follows: the simple model with the complete usage and non-overlapping allotment of input neurons, the alternate model with the random allotment of

episode is split into 30 temporal steps, each associated with a few input neurons. Groups of four temporal steps are presented sequentially in time (in successive high frequency, 1/49 s, blocks) with the current step being

spike). In each successive low frequency block, the current temporal step is advanced by one, similar to activity reported in Maurer and McNaughton (2007).

inputs, and the preplay model with varied input types. In most cases, inputs are partially presented, noisy, and probabilistic (in their occurrence) resulting in a disproportionate use of input neurons (**Figures 1**, **2**). It was necessary to use the alternate input mapping when using random allotment to prevent the growth of homeostatic scaling and bursting (and recruitment of the entire network) in the recurrently connected neurons not assigned to an ensemble (data not shown). This connection strategy ensures that most recurrent network neurons will receive some input, since the fraction of neurons without active inputs is small. Additionally, a many-to-many input architecture amplifies the amount of noise by increasing the number of input noise sources to each neuron in the recurrent network, which prevent the uncontrolled growth of the homeostatic parameter through direct activation of the neuron and indirectly by promoting random changes in its weights that may result in the "unused" neuron becoming active in a random ensemble without direct stimulation.

## **LEARNING EPISODES**

The use of low and high frequency modulated inputs force short, high frequency bursts of spikes that replicate a sequential long time scale procession in temporal steps. After learning, the sequential inputs activate subsequent nodes prospectively (see **Figure 3**). The net inhibitory effect of on-going firing limits prospective firing to a handful of future temporal steps. The network demonstrates a cascade of activity during each low frequency cycle. This cascade is preempted by the intermittent high frequency inhibition and terminated by an increase in low frequency inhibition and due to refractoriness of the neurons. The beginning of the next low frequency cycle is marked by the input stimulus that excites another cascade of activity.

Activity in the network is initially the direct result of activated input neurons whether from noise or as part of the episode. As the network adapts to the input episodes, the background noise is suppressed and missing components of the repeated episode are

**FIGURE 3 | Prospective spiking activity in the simple model.** Example raster plot of activity in the recurrent network after training of an episode in the 1-to-1 network. Neuron have been resorted by the mean time of activity in the interval. Spikes near and to the left of the blue line are, in general, prospective activity.

activated along with the neurons that are soon to be activated. Background noise is suppressed due to recurrent inhibition and the down regulation of input scaling so as to counter the up regulation of recurrent connections between members of ensembles. The co-activation of neurons over several presentations allow for the learning of an episode on the basis of partial activity of member neurons in the ensemble when the partial pattern is reactivated as shown in **Figure 4**.

With 1-to-1 input structure, the simple network neurons are excitable relative to their history of activity as expected. With the many-to-many input structure, the alternate network neurons that are active in fewer ensembles are excited stronger than other neurons in an ensemble. Neurons become active prospectively firing prior to the presentation of input that occurs in subsequent temporal steps. Neurons, also, complete missing portions of the patterns presented. The amplitude of the pattern completion and prospective activity is less than that of neurons directly activated by episodic input (see **Figure 5**). The prospective and pattern completion firing rates formed a smooth curve while the directly stimulated activity was far greater. This leads to the notion that the prospective firing and pattern completion are related by a common mechanism in this model, and that the prospective activity is pattern completion of the representation forward in time. However, see Noise Effects for differences in the learning of these phenomena.

## **SYNAPTIC WEIGHT CHANGES**

To examine learning at the synapses, LLEs of the excitatory and inhibitory weight spaces were performed over the course of learning. Briefly, LLEs map high dimensional spaces into neighborhood preserving low dimensional spaces (Roweis and Saul, 2000). The globally mapped nature of the low dimensional embedding makes them appropriate for visualizing data and relative trajectories in high dimensional spaces. Excitatory and inhibitory dimensionality reductions were performed separately due to the different time constants and learning rules involved. L2 distance in the weight space served as a poor indicator of learning (data not shown) and asymptotically demonstrated strong relation to the simulation time difference between weight vectors during learning.

Instead, the distance in a few dimensions of LLE space appeared to be a better indicator of changes in the network. For example, in a representative example, comparing three sequential sets of learning in the LLE space results in the expected non-linear adaptation and convergence of weights of recurrent network weights in the simple model. In this 3D LLE, these adaptations move along three nearly linear trajectories, each of which are nearly orthogonal (see **Figure 6**). This shows that given a global embedding into a low dimensional space, the weight vector trajectory moves from neighborhood to neighborhood in such a way that global changes are not discordant. During adaptation, the targeted correlated activity reduces inhibition, which in

**FIGURE 4 | Activity in alternative network model.** Instantaneous firing rate of the input neurons and recurrent network during an episode with many-to-many connections. Smoothed firing rate (binned in 1/ 7 s and smoothed with a causal exponential kernel, τ = 1*/* 7 s, over 3 bins) demonstrates an example of recurrent network activity (right) leading the input neuron activity (left) throughout the

progression of an episode near the **(A)** beginning, **(B)** middle, and **(C)** end. Recurrent neurons are matched to each input neuron and are redundantly represented for each input neuron they receive synapses from leading to the appearance of more background noise. Color scale ranges from dark blue to dark red representing a 0–20 Hz firing rate, respectively.

**FIGURE 5 | Pattern prospection and completion in the simple model.** Instantaneous spike firing rates (in 1/ 7 s window) relative to the current step in the episode for the **(A)** simple model, **(B)** preplay model, and **(C)** preplay model with slower progressing stimulation with increased overlap. Negative lags are yet to be stimulated. Green triangles, black squares, and blue circles represent the average firing rate of recurrent neurons that are soon to be, currently being, and are not but would normally be stimulated via input neurons. Thus, they highlight prospective, active, and pattern completing neural activity, respectively. Bars indicate the 5–95% percentiles of firing rates for each marker.

and **(B)** inhibitory recurrent synapses] to 3 dimensional space using local linear embedding. Black points are the initial state of the weights. Red points indicate the learning of the first group of 4 patterns. While green and blue represent the learning of the second and third patterns,

finding process that blends between neighboring points (and thus episodes) at the tail of each segment of the episodic learning except the last one. **(C,D)** Excitatory and inhibitory, respectively, weight space reduction for the preplay model.

turn enables faster spiking and thus faster weight changes (see **Figure 7**). Weight adaptation rate peaks and slows as the pattern is learned and weights reach their maximum or minimum values. This coincides with a peak in the combined degree of prospectivity (i.e., look ahead time window) and pattern completion ability of the network (see **Figure 5**). Increasing the number of episodes (redundant allocation) that were being learned resulted in the trajectory of the learning in the first three LLE dimensions to appear less orthogonal (data not shown). This implies that learning of the groups of patterns does not interfere with each other when the subsequent uses of neurons are in completely new ensembles; however, pairwise reuse of neurons within the STDP windows violates this condition.

#### **NOISE EFFECTS**

To define a metric to quantify network prospectivity and pattern completion, the firing rate of neurons for up to 2 temporal steps into the future of the episode or the neurons from the current step that were removed from the input are averaged and compared with the activity of the directly stimulated neurons.

$$R = \frac{\mu\_{\text{(proportion)} \text{ completion}} - \mu\_{\text{background}}}{\mu\_{\text{active}} - \mu\_{\text{background}}} \tag{8}$$

where μ is the mean firing rate. Given that the inputs are not active for prospection or completion, this relative activity measure is expected to be less than 1 as homeostatic input scaling regulates neuron activity based on inputs as well as recurrent connections. Remarkably, the network is tolerant to large amounts of background noise for pattern completion but less so for prospective firing (see **Figure 8**). In contrast, the noise has little effect on the initial learning rate, but affects the final quality of learning (see **Figure 8**).

#### **PREPLAY AND REVERSE REPLAY**

The preplay network began to demonstrate prospection sooner than other models (on the second presentation of an episode however each step is presented 7× longer than the other models

subsequent groups. Each learning curve represents the mean performance across four different simulation runs with four noise levels (*n* = 4) within a group. The error bars represent the standard deviation in *R*prospection for that group at various episode durations.

presented), and, in general, produced better and more stable pattern completion and prospective activity than the other models presented in this paper. Although more divergent from biological underpinnings, this model was able to demonstrate preplay and replay of activity in the form of self-sustaining sequences in forward or reverse, respectively. This is accomplished by reactivating the ensemble at the beginning (preplay) or end (replay) of the spatiotemporal sequence after learning has occurred. The episode is reactivated by short gamma frequency bursts of 1–4 spikes from each neuron in the initial or end segment of the episode (two temporal steps were used). This could be considered reminiscent of the upstream spiking activity due to the sensory input at the beginning or end of a linear track. The preplay of activity (see **Figure 9**) can be viewed as a full recall of a previous memory. The recall of previous memories based on partial patterns means that the results of various behaviors can be predicted given a similar sensory match to other experiences. The model also exhibits various successful durations of the preplay and replay of the entire episode that is qualitatively similar to the results to other recent research (Foster and Wilson, 2006; Diba and Buzsaki, 2007).

However, there are several limitations of this version of the model. This model aggressively suffered interference and learned more slowly the second and third sets of patterns. The sequence preplay speed did not occur on biologic timescales. The activity propagated at a much slower rate due to the high membrane capacitance resulting in long time constants and the slow inhibitory currents. The preplay propagation speed increased when using the fast stimulation protocol used for the other models due to depression of inhibitory weights. The capacitance and the localized slow inhibitory currents proved to be the reason for stability in this model; reducing the capacitance, as in the other

**FIGURE 8 | Effect of noise on the learning.** Performance of the simple model through learning with incomplete patterns is shown here. Solid lines are the metric, *R*, for prospection. Dashed lines are the metric, *R*, for pattern completion. Note, the similar slope of the curves initially as compared to divergence in the final value relative to noise, ν. ν is expressed in terms of per neuron noise spikes in Hz. A significant amount of noise tolerance exists considering the input signals are an average of 6 spikes per presentation of each episode (which equates to a brief instantaneous firing rate of 10 Hz but averages out to 0.3 Hz over the course of the learning trial). The error bars represent the standard deviation in either *R*propsection or *R*completion for various episode durations. The results also show that the network is more robust to noise for pattern completion (solid lines) compared to prospective recall (dashed lines).

models, results in less stability, yet, faster response times in the cascade or sequencing of activity. When forming symmetric excitatory connections between neurons (in the triplet STDP case, and needed for reverse replay), faster response times gives the network the ability to recruit a major portion of the network and this in turn leads to bursting activity in the network. A better method of achieving a balance between the stability-sustainability trade-off in the model could come from compartmental models and more robust and targeted inhibition.

The preplay model was only able to demonstrate replay when the triplet STDP rule was used in conjunction with a temporal blurring of the input patterns as opposed to the sharp probability peaks seen in **Figure 2**. A preplay model with traditional couplet STDP could learn, form prospective activity, and demonstrate preplay, however, reverse replay could never be achieved even when using shifted or noisy windows (Babadi and Abbott, 2010). Another interesting aspect of this model was the fleeting fragments of episodes were reignited (primarily forward and on the same time scale as preplay) when not driven by an episode pattern but still receiving random noise inputs (supplementary video available online).

#### **DISCUSSION**

We demonstrated recurrent SNNs capable of learning episodes operating with missing, noisy, and unbalanced data was demonstrated. This learning is demonstrated by the prospective firing of neurons attributed to subsequent stimuli and the completion of missing portions of the patterns. This prospective firing is excited by recurrent connections from neurons in the currently active ensemble. The active ensemble contains neurons that are components of temporally adjacent (both previous and subsequent) ensembles; however, the previous neurons were just active and are generally in a refractory state and moderately inhibited.

These recurrent spiking models provide a means to store patterns and recall or even predict them given a previously encoded pattern (Lisman and Redish, 2009). In the many-to-many input coding of the alternate model, the input to recurrent network mapping results in a randomized and more distributed encoding. This spreads activation through the recurrent neurons to preferentially recruit neurons that are not utilized or rarely utilized.

Our model maintains stability through strong high frequency inhibition to limit the network activity to be cyclic and by introducing the input pattern such that its driving force terminates at the peak of the low frequency cycle enabling recurrent activity to trail-off in the second half of the cycle. Therefore, the network is stable as long as network activity is always decaying which can be ensured by limiting the upside of excitatory weights, the downside of inhibitory weights, and using a time constant for inhibition that is slower than excitation. This balance enables the scaling of weights to modulate, on a neuron-to-neuron basis, the duration of time for which the net recurrent activity is excitatory before turning inhibitory. Setting the weight ranges so that the maximum duration is half of the low frequency cycle designates the spiking activity due to inputs to become inhibitory to the network and thus a stabilizing force (see **Figure 10**).

Other models exist which relate learning and sequences of spiking activity (O'Keefe and Recce, 1993; Rao and Sejnowski, 2001; Buonomano, 2005; Lisman et al., 2005). Another method proposed by Buonomano (2005), uses the scaling of the presynaptic component of the weight to learn a time delay and sequence a spiking pattern. However, those authors admit the difficulty with learning multiple different patterns. This work does not use the pattern correcting and forward lookup circuitry proposed

in Lisman et al. (2005). However, this simple recurrent network is able to do both. As a result though, there is a tendency in this model to bring temporal associations forward and backward in the sequence learning—a separate pattern correction and completion network may solve this issue. Regardless, a spiking network that can perform free recall of multiple episodes has yet to be demonstrated in a simulated network that first learned the multiple episodes concurrently.

red and black lines show the time point at which inhibition begins to exceed

## **LINK TO BIOLOGY**

excitation.

Recurrent neural networks have been found throughout the brain (Rao and Sejnowski, 2001; Kobayashi and Poo, 2004; Buzsaki, 2006). Specifically, our network employs neural architecture design that can produce hippocampal-like behaviors including reactivation, preplay and replay (Pavlides and Winson, 1989; Wilson and McNaughton, 1994; Louie and Wilson, 2001; Andersen et al., 2006; Rasch and Born, 2007; O'Neill et al., 2008; Dragoi and Tonegawa, 2010; Gupta et al., 2010; Buhry et al., 2011). This enables the network to automatically load balance densely coded downstream networks through homeostatic regulation and sparse upstream coding. Furthermore, the low and high frequencies used here are similar to the theta and gamma rhythms found in the hippocampus (Buzsaki et al., 1992; O'Keefe and Recce, 1993; Skaggs et al., 1996; Penttonen et al., 1998; Bragin et al., 1999; De Almeida et al., 2007; Lenck-Santini and Holmes, 2008; Pastalkova et al., 2008; Zhang et al., 2011; Penley et al., 2012).

However, this network is appreciably simpler than the one found in the hippocampus, including the neural dynamics, learning rules, and homeostasis. The 80:20 ratio for excitatory to inhibitory connections is lower than that found in the CA3 region of the hippocampus but similar to cortical areas (Buzsaki, 2006). Moreover, we believe that the memory capacity of this network to be markedly smaller than of a more complex network where feedback inhibition is present in the form of multiple independent neurons and target specific regions of the principal cell's dendrites, soma, and axon. Also, the mammalian hippocampus is thought to encode through the sparse representation of the dendate gyrus (DG) and recall using a direct entorhinal cortex to CA3 pathway that contain several smaller connections as opposed to the DG-CA3 pathway (Treves and Rolls, 1992; Nolan et al., 2011) which was adopted here.

## **FUTURE WORK**

Stability in this model is fragile with large weight changes, meaning that increased excitation or increased recruitment can easily lead to cascades of activity that result in bursts like many recurrent networks (Wagenaar et al., 2006). Once the network bursts, the network continues to burst due to both the positive feedback effect of triplet STDP and the negative feedback effect of inhibitory STDP during bursting. Separating late and early phase LTP (Adams and Dudek, 2005) may prevent infrequent occurrence of bursts from having these catastrophic ramifications. However, this may not be the case since an analogous development occurs *in vivo*; for example, tetanic hippocampal stimuli evolve into epilepsy (Sanchez et al., 2006). Alternatively, the use of reward-based learning methods may increase the speed and the

## **REFERENCES**


together. *Neural Plast.* 2011:203462. doi:10.1155/2011/203462


specificity with which learning occurs by identifying new patterns and promoting learning via neuromodulation (Izhikevich, 2007b; O'Brien and Srinivasa, 2013).

In the preplay iteration of this model, there is evidence that this rudimentary kind of network supports replay and preplay at compressed timescales (Diba and Buzsaki, 2007). Due to the use of the triplet based rule, symmetric excitatory connections are formed between neurons allowing for both the forward or reverse propagation of activity. However, the triplet rule does not promote the robust symmetric connections found in the hippocampus with short bursts of post-then-pre-synaptic spikes. We believe that the use of a more phenomenological rule (Markram et al., 2012) would promote further symmetry in learning and robust replay on par with preplay without requiring as much blurring and overlap of spike trains. Further investigation of preplay is necessary to determine if the compression factor seen *in vivo* (Euston et al., 2007) occurs as a natural result of propagating neural activity without theta frequency resets.

## **ACKNOWLEDGMENTS**

Work performed under DARPA contract (HRL0011-09-C-001) with HRL under the SyNAPSE program. The views expressed are those of the authors and do not reflect the official policy or position of the Department of Defense or the U.S. Government.

ripples. *Nat. Neurosci.* 10, 1241. doi: 10.1038/nn1961


*Mach. Intell.* 31, 855–868. doi: 10.1109/TPAMI.2008.137


of STDP and dopamine signaling. *Cereb. Cortex* 17, 2443–2452. doi: 10.1093/cercor/bhl152


comprehensive overview. *Front. Synaptic Neurosci.* 4:2. doi: 10.3389/ fnsyn.2012.00002


mechanisms in cortical circuit refinement. *Annu. Rev. Neurosci.* 34, 89–103. doi: 10.1146/annurevneuro-060909-153238


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 08 March 2013; accepted: 03 June 2013; published online: 21 June 2013.*

*Citation: Dockendorf K and Srinivasa N (2013) Learning and prospective recall of noisy spike pattern episodes. Front. Comput. Neurosci. 7:80. doi: 10.3389/ fncom.2013.00080*

*Copyright © 2013 Dockendorf and Srinivasa. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

## Anisotropic connectivity implements motion-based prediction in a spiking neural network

#### *Bernhard A. Kaplan1,2, Anders Lansner 1,2,3, Guillaume S. Masson4 and Laurent U. Perrinet <sup>4</sup> \**

*<sup>1</sup> Department of Computational Biology, Royal Institute of Technology, Stockholm, Sweden*

*<sup>2</sup> Stockholm Brain Institute, Karolinska Institute, Stockholm, Sweden*

*<sup>3</sup> Department of Numerical Analysis and Computer Science, Stockholm University, Stockholm, Sweden*

*<sup>4</sup> Institut de Neurosciences de la Timone, UMR7289, Centre National de la Recherche Scientifique & Aix-Marseille Université, Marseille, France*

#### *Edited by:*

*Julie Wall, Queen Mary University of London, UK*

#### *Reviewed by:*

*Jean-Baptiste Masson, Institut Pasteur, France Elad Ganmor, Weizmann Institute of Science, Israel*

#### *\*Correspondence:*

*Laurent U. Perrinet, Institut de Neurosciences de la Timone, Centre National de la Recherche Scientifique & Aix-Marseille Université, 13385 Marseille Cedex 5, France e-mail: laurent.perrinet@univ-amu.fr*

Predictive coding hypothesizes that the brain explicitly infers upcoming sensory input to establish a coherent representation of the world. Although it is becoming generally accepted, it is not clear on which level spiking neural networks may implement predictive coding and what function their connectivity may have. We present a network model of conductance-based integrate-and-fire neurons inspired by the architecture of retinotopic cortical areas that assumes predictive coding is implemented through network connectivity, namely in the connection delays and in selectiveness for the tuning properties of source and target cells. We show that the applied connection pattern leads to motion-based prediction in an experiment tracking a moving dot. In contrast to our proposed model, a network with random or isotropic connectivity fails to predict the path when the moving dot disappears. Furthermore, we show that a simple linear decoding approach is sufficient to transform neuronal spiking activity into a probabilistic estimate for reading out the target trajectory.

**Keywords: motion detection, motion extrapolation, probabilistic representation, predictive coding, network of spiking neurons, large-scale neuromorphic systems**

## **1. INTRODUCTION**

#### **1.1. PROBLEM STATEMENT**

In a dynamical world, prediction is a highly relevant evolutionary advantage. This is crucial in sensory systems, as the raw data that is processed is most often noisy, and possibly ambiguous or distorted. Take for example the task performed by the primate visual system of tracking the trajectory of a moving object and accurately moving the eyes in order to stabilize the image on the retina. The image of the object may be blurred, or the measure of its velocity may depend on its geometry instead of its trajectory. Another problem occurs when the object is occluded, or simply when the observer blinks. It is an advantage to be able to predict the position and speed of the object at the end of this blanking period. This problem is classically referred to as *motion extrapolation* (see **Figure 1**). While predictive coding mechanisms may have different aspects and occur at different levels ranging from the retina to higher level areas (Gollisch and Meister, 2010), we will focus on this particular phenomenon as prototypical example.

Particularly in primates, object motion information is extracted along a cascade of feed-forward cortical areas, where primary visual area (V1) extracts local motion information that is integrated in extra-striate middle temporal (MT) and medial superior temporal (MST) areas (Newsome et al., 1988). MT and MST process visual motion and oculomotor signals driving pursuit (see Ilg, 1997 for a review) and are therefore key elements in motion extrapolation. Specifically, we will focus on the dynamics of neural activity during the period without informative sensory input (to which we will refer as the blank) and just after its reappearance. Indeed, the capacity of the dynamics to transform such fragmented input into a correct, continuous representation is a major pressure on the evolution of the visual system (Gollisch and Meister, 2010). It was shown in the monkey visual system that neural activity was mostly absent during the blank in lower areas of the visual hierarchy while it was maintained in some higher level areas (Assad and Maunsell, 1995). More precisely, neural activity in MT is driven by the motion of the dot and quickly devolves to spontaneous activity during a blank, while activity in its efferent area MST is maintained to the level of neural activity expected if the dot was not blanked when there is no retinal image motion. This can happen during a transient image occlusion (Newsome et al., 1988) or while tracking an imaginary target covering the visual field outside of the receptive field currently being recorded (Ilg and Thier, 2003). Similar sustained activity during target occlusion has been found in primate posterior parietal cortex, and it is linked to image motion prior to target disappearance (Assad and Maunsell, 1995), that is, by a predictive signal.

Motion extrapolation is also seen in lower level neuronal structures, such as the retina (Berry et al., 1999), and calls for a more generic computational framework. However, direct evidence for such neural mechanisms is still lacking. Before proposing a solution using a connectivity pattern based on motion-based prediction, we will first review some existing experimental and theoretical evidence. Along this study, our aim is to provide a basis for future applications of neuromorphic hardware (Schemmel et al., 2010; Brüderle et al., 2011).

**FIGURE 1 | The motion extrapolation problem.** Sensory input, such as the smooth motion of a dot in visual space, may be perturbed by disruption of sensory drive, like when the eye blinks during a visual stimulation. It is essential that some mechanisms may fill this blank: this defines the motion extrapolation problem. We first define the problem by parameterizing a generic input and its perturbation. **Left:** The input is a Gaussian hill of activity in a topographically organized space, moving on a straight trajectory. We show here a snapshot in time of the input (blue) and the resulting input activity to the network (gray) for a period of 400 ms. This corresponds for instance to the activation of a low-level visual area to a single dot represented by a bell-shaped hill of activity (blue blurred circle). In addition, this input carries information about the motion of the object (blue arrow) and drives neurons which have a close

## **1.2. NEURO-PHYSIOLOGICAL CORRELATES OF PREDICTION FOR MOTION EXTRAPOLATION**

At the neural level, it seems that the topography of neural representation is an essential constraint to prediction. Indeed, it is more efficient that populations of neurons that represent similar parameters should be adjacent. This is due to the cost of wiring neurons (length and volume of axon and dendrites) Chklovskii et al. (2002) but also due to the limited speed of information propagation in neural wires. Such aspect is particularly acute on the surface of the cerebral cortex and this hypothesis has been an efficient construct to understand the organization of visual areas (Miikkulainen et al., 2005). This is also implemented in other cerebral structures and species such as the convergence of inputs from place cells in the hippocampus of rats that code for path integration of body position in an environment (McNaughton et al., 2006). Physiological evidence shows that similar mechanisms are present in the deep superior colliculus of primates allowing for the integration of the belief on the position of a visual target in visual space for the guidance of saccadic or smooth eye movements (Krauzlis, 2004). Here, we will focus on low-level visual areas based on the neurophysiology of the macaque brain (V1, MT and MST), but we will keep a rather generic formulation to explore the functional role of some key parameters.

Neurons in such areas receive connections from neighboring neurons in the same cortical area (local connectivity) but also respectively by feed-forward or feed-back connections from lower or higher areas. Focusing on area MT, early physiological studies in macaque monkey identified this area as a specialized module for visual motion processing (Dubner and Zeki, 1971; Allman et al., 1973). This involves extracting speed and direction of the moving object. MT neurons respond selectively to visual motion selectivity in position and velocity (gray arrows). **Right:** We show the time course of the input for one representative neuron (denoted by the yellow star in the left panel). **Top:** The blue trace shows the envelope of the inhomogeneous Poisson process that creates the input spike train. For 0 ms *< t* ≤ 200 ms and 600 ms *< t* ≤ 800 ms the stimulus is blanked, that is, that all neurons in the sensory layer receive input from a Poisson process with the same rate. We permuted the input vector fed into the network among all the cells in the network for each time step during the blank. Black vertical lines indicate input spikes. **Bottom:** Histogram of the input spike train with a bin size of 50 ms. This shows clearly the missing information during the blank. We define the goal of solving the motion extrapolation problem as representing the prediction of information on motion (speed and position) during the blank.

and are tuned for local speed and direction of luminance features moving in their receptive fields (Maunsell and Van Essen, 1983). Concerning motion integration, Pack and Born (2001) have shown that the temporal dynamics of behavior can correspond with the firing rates of MT neurons. They showed that neuronal responses quickly progress from local to global motion direction in about 100 ms, suggesting that such integrative mechanisms are dynamical and progressive. These results pinpoint the key role of MT neurons in local motion analysis and global motion integration. Area MT and MST receive feed-back connections that may modulate the activity of their neurons (Salin and Bullier, 1995). However, these connections (mostly myelinated) introduce constant delays and are mostly related to higher level contextual modulations. Provided that motion extrapolation is implemented in one single cortical area, a finely structured set of diffusive mechanisms would be required. A potential candidate is naturally the dense network of lateral interactions found in subcortical and cortical structures involved in sensory processing and sensorimotor control. Of particular relevance is the role of the connectivity pattern in the emergence of a solution to this problem. In this paper we will focus on a smaller spatio-temporal scale and study the role of lateral, intra areal (mostly unmyelinated) connections.

A possible correlate of prediction may lay in the traveling waves of neural activity that may be observed on the cortical surface. Bringuier et al. (1999) was the first to show a precisely tuned synaptic integration field (Bringuier et al., 1999) [see (Sato et al., 2012) for a review]. Theoretical studies suggest that for such waves to exist, there should exist some specific anisotropy connectivity pattern (Bressloff and Coombes, 1998). It is established that the speed of propagation of activity along these mostly unmyelinated connections is of the order 0.1–0.4 m/s but there is an ongoing debate on their selectivity. In the primary visual cortex, a set of patchy connections in the long-range horizontal connections found in superficial layers of cortex (Bosking et al., 1997) that preferentially connect columns with similar orientation preference has been observed in ferrets. This is consistent with the fact that columnar interactions determine horizontal propagation of recurrent network activity in neocortex (Wester and Contreras, 2012). It has also been observed that activity in cat V1 spreads anisotropically for all orientation columns (Chavane et al., 2011). Anisotropies in the connectivity pattern necessarily lead to a wide range of traveling wave parameters (speed, direction) and introducing inhomogeneities can in addition lead to more complex wave profiles and possibly even wave propagation failure (Bressloff, 2001). However, the function of these traveling waves, and therefore the underlying structure of the intracortical connectivity, is mostly unknown.

## **1.3. EXISTING NEUROMORPHIC MODELS FOR PREDICTION**

There have been numerous attempts at modeling generic predictive neural mechanisms. Here, we review some prototypical examples at different modeling levels, from a more abstract level to a neuromorphic implementation.

Following the idea of the Kalman filter as an adaptive predictive filter and extending the work of Montagnini et al. (2007), Bogadhi et al. (2011) proposed a hierarchical recurrent Bayesian framework to understand the behavioral response to motion extrapolation as observed in smooth pursuit eye movements. Indeed, probabilistic inference has been successful in explaining motion perception to a variety of stimuli (Weiss et al., 2002) under the hypothesis that sensory areas use predictive coding as a generic neural computation (Rao and Ballard, 1999). They are somewhat similar to engineering models proposed earlier (Nowlan and Sejnowski, 1995) but allow for a more explicit formulation of the underlying hypothesis. Such a framework accommodates uncertainty in the motion information of the measurement likelihoods (Weiss et al., 2002; Stocker and Simoncelli, 2006; Hedges et al., 2011). Representing uncertainty in the measurements and prior expectation gives a simple, yet powerful framework to investigate the predictive behavior of the system, and offers the possibility to optimally adapt to changes in the measurements. The approach from Bogadhi et al. (2011) allows for a mix of prediction and measurement based on their reliability measured from their respective variances. The combined estimate is used to drive the pursuit response. The hierarchical framework allows investigation of the adaptive behavior of pursuit as well as the role of prediction on motion integration as observed in pursuit responses. Such Bayesian models give a generic account of the motion extrapolation mechanism but do not provide a neural implementation.

A direct translation could in theory be performed by a probabilistic population code approach (Beck et al., 2008). This requires that neural responses represent probability distributions and assume "Poisson-like" spike response variability. Under that hypothesis, one could derive from a Bayesian model the architecture of a network of spiking neurons. Another approach is to use a global and generic functional cost for the problem (such as the free-energy of a system designed to track a dot) and derive the optimal system. Such endeavors allow one to propose a hierarchical neural architecture (Friston, 2009), which predicts behavioral results under visual occlusion for control and schizophrenic patients (Adams et al., 2012). Such models are in essence similar to other modeling approaches where neural activity is represented by average firing rate on a cortical map (forming a so-called neural field). Such models were successful in accounting for a large range of classical and non-classical receptive field properties of V1 including orientation tuning, spatial and temporal frequency tuning, cross-orientation suppression, surround suppression, and facilitation and inhibition by flankers and textured surrounds (Spratling, 2010). Similar models were applied to problems specific to motion detection and a link can be drawn between such solutions and classical solutions drawn in computer vision (Tlapale et al., 2010). However, these models do not take advantage of the specificity of computing with spiking neurons, that is the dual property of being able to integrate information and detect synchrony in the input.

Some models propose solutions related to motion extrapolation using neuronal networks (spiking and non-spiking). A recent model of spiking units (Lim and Choe, 2008) explains the phenomenon of the flash-lag effect (Nijhawan, 2008) by a motionextrapolation mechanism provided by facilitating synapses, but acts on the single cell level only. Baldo and Caticha (2005) present a feed-forward network of leaky integrate-and-fire (LIF) neurons performing prediction, but that does not account for the role of recurrent connectivity abundant in cortical networks. In this regard, Liu and Wang (2008) proposed a more realistic recurrent network but focused on a binary decision task, whereas we aim at a more generic solution for the problem by studying prediction performance for a spectrum of possible directions. Recently, Jancke and Erlhagen (2010) used a recurrent neural field model to explain visual illusions like the Fröhlich, the flash-lag, and the representational momentum effects. Our approach is similar to theirs in the sense that the mechanism for motion-extrapolation can be seen in spreading activation to surrounding neuronal populations, but differs fundamentally in the way that connections are set up, as connection selectivity for directional tuning is not considered in their model.

As an intermediate observation, we see that though there exist a wide spectrum of models, a common feature is that these models use diffusive mechanisms implemented by the connectivity to propagate predictive information (probabilities, population activity, spikes) from a local to a global scale. The richness of behaviors is then mostly obtained by using different types of neurons (for instance by varying their polarity excitatory or inhibitory, or the time constant of the synapses), which implements complex non-linear mechanisms such as gain control. This may be sufficient to account for motion extrapolation. However, it should also be highlighted that all these models assume a prediction in all directions and therefore that the connectivity is *a priori* isotropic. We challenge this assumption by introducing anisotropy in the connectivity as a key mechanism transporting motion information in a coherent manner.

#### **1.4. OUR APPROACH: AN ANISOTROPIC CONNECTIVITY PATTERN IMPLEMENTING MOTION-BASED PREDICTION**

At the behavioral level, Yuille and Grzywacz (1989) have shown that motion integration in humans is highly dependent on the smoothness of the trajectory of the stimulus. Humans can detect a target dot moving in a smooth trajectory embedded in randomly moving dots, while the target dot is not distinguishable from noise in each frame separately. Introducing a preference for smooth trajectories, the activity from local motion detectors are made more coherent in space and time and this globally lowers the threshold for detecting stimuli moving in smooth versus segmented trajectories. In particular, during a transient blanking, it is most likely that such processes (along with the knowledge that the sensory input was indeed blanked and not definitively removed) underlie motion extrapolation. For instance, when a moving target disappears, smooth pursuit eye movements continue at the same velocity during the initial period of occlusion (Bennett and Barnes, 2003). Therefore, it seems that neural computations take advantage of the information about motion, but it is yet not clear how this can be done efficiently in a network of spiking neurons.

At an abstract level, a preference for temporal coherency of motion can be defined in a probabilistic framework. This was formulated theoretically by Burgi et al. (2000), who proposed a neural field implementation including local to short-range connectivity. However, it lacked the precision needed to efficiently represent realistic input sequences. In our earlier work (Perrinet and Masson, 2012), we implemented an efficient prior for smooth trajectories to investigate different aspects of spatio-temporal motion integration. Particularly, this model focused on the aperture problem and proposed that local, diffusive predictive coding is sufficient to infer global motion from local, ambiguous signals. The aperture problem is a challenging problem to study integration of local motion information (Pack and Born, 2001). The model proposed that instead of specific mechanisms such as line-ending detectors, the gradual spatio-temporal integration of motion relies on prediction based on the current knowledge of motion in terms of its velocity and position (motion-based prediction). Compared to previous models the main difference of this implementation is that, it is possible to predict that information about motion velocity at a known position will be transported in the direction given by the velocity.

Indeed in motion-based prediction, the retinotopic position of the velocity of motion is an essential piece of information that allows routing information and allow implementation of predictive coding on smooth trajectories. By including explicitly the dependence of local motion signals between neighboring times and positions knowing the current speed along a smooth trajectory, incoherent features should get canceled out, while coherent information should get progressively enhanced. As such, this context-dependent, anisotropic diffusion in the probabilistic representation of motion also results in the formation of a tracking behavior favoring temporally coherent features. Such a model was recently extended to account for motion extrapolation (Khoei et al., 2013) and has been able to replicate some behavioral results from Bogadhi et al. (2011). Our goal here is to show that the idea of motion-based prediction [as described in Perrinet and Masson (2012)] can be implemented in a generic network of spiking neurons through anisotropic connectivity and that this is sufficient to solve a motion extrapolation task. The novelty compared to previous studies is the transition from an abstract, probabilistic framework to a spiking neural network and the link between anisotropic connectivity to motion-extrapolation, a task of functional relevance. Of course, we will not exclude that other complementary solutions may exist, but we will argue that it constitutes one of the simplest solutions for a network of spiking neurons. For that purpose, we will use a classical implementation of recurrent networks using conductance-based integrate-and-fire neurons with three prototypical connectivities: random, isotropic or anisotropic. While the consequence of nonhomogeneous connectivities has been somewhat explored (Voges and Perrinet, 2012), it is—up to our knowledge—the first study of the functional consequence of anisotropic connections in a large-scale neural network.

## **1.5. OBJECTIVES AND OUTLINE**

This paper is organized in the following order: First, we develop a network of spiking neurons with the connectivity directly drawn from the probabilistic modeling framework proposed for the solution to the aperture problem (Perrinet and Masson, 2012), and that was extended to the motion extrapolation problem (Khoei et al., 2013). We will include in Section 2.1 details on structure and implementation of the model but also details from the experimental and numerical aspects.

Then, we report results in Section 3 from simulations where we studied the network response to a disappearing moving dot under three different connectivities: random, isotropic or anisotropic.

Finally in the discussion (Section 4), we interpret these results in the light of current knowledge on probabilistic inference and dynamical systems, and we will discuss the limitations of the current study along with suggestions for future work.

## **2. METHODS**

#### **2.1. NEURON PARAMETERS**

Simulations were performed with PyNN (Davison et al., 2008) as interface to the NEST simulator (Gewaltig and Diesmann, 2007) on a Cray XE6 system using 96 cores. For analysis we used python modules numpy (Oliphant, 2007), scipy (Oliphant, 2007) and visualization was performed using matplotlib (Hunter, 2007). Neurons were simulated as LIF neurons with conductance based synapses. The membrane potential *V* of a neuron with index *k* obeys the following equation:

$$C\_m \frac{dV\_k}{dt} = \mathcal{g}\_l \left( E\_l - V\_k(t) \right) + \sum\_j \left[ \mathcal{g}\_{j,k,E}(t) \left( E\_E - V\_k(t) \right) \right]$$

$$+ \mathcal{g}\_{j,k,I}(t) \left( E\_I - V\_k(t) \right) \Big|,\tag{1}$$

where *<sup>j</sup>* is the index of the sources, *gj, <sup>k</sup>(t)* <sup>=</sup> *wj, <sup>k</sup>* · exp −*<sup>t</sup>* <sup>−</sup> *tspike* τ*p* is the synaptic conductance time course with *p* ∈ {*E, I*}: τ*<sup>p</sup>* are the synaptic time constants, and *Ep* is the reversal potential for excitatory (*p* = *E*) and inhibitory (*p* = *I*) synapses respectively, *gl* is the constant leakage conductance, and *El* the leakage or resting potential. When the membrane potential *V* is above the threshold


voltage *Vthresh* = −50 mV a spike is emitted and *V* is set to *Vreset* = −70 mV for a refractory period of τ*refrac* = 1 ms. **Table 1** lists the parameters used for both excitatory and inhibitory neurons. The cell and synapse parameters have been chosen to be in a similar range as seen in experimental studies [see Table 3 in the study by Rauch et al. (2003)] to allow for comparison between future modeling and experimental studies. The model is in principle not dependent on the cell parameters and different parameters would not change the fundamental outcome, but returning of other parameters like connection strengths would be necessary. The initial values of the membrane potentials are drawn from a normal distribution around *Vinit* = −65 mV with a standard deviation of 10 mV.

#### **2.2. TUNING PROPERTIES**

The model is inspired by retinotopic cortical areas like areas V1 or V5/MT in primates. In our model, each neuron has four tuning properties: *(xi, yi, ui, vi)* parameterizing the center of the spatial receptive field of neuron *i* at position *x<sup>i</sup>* = *(xi, yi)* and its preferred direction *vi* = *(ui, vi)*. The width of this receptive field defines the tuning selectivity of neurons and is parameterized by β*<sup>X</sup>* and β*V*, respectively for space (*x* and *y*) and velocity (*u* and *v*). The spatial receptive fields are arranged in a hexagonal grid to optimally cover the input space which is set to span a 1 × 1 area in arbitrary units. As we will implement a network size of approximately 10<sup>4</sup> neurons, this will in practice correspond to a spatial scale of the order of millimeters.Velocities should therefore be in the range of m s<sup>−</sup>1.

In order to have receptive fields for all possible directions (up to a certain maximum velocity of approximately |*vmax*| = 4*.*0 ms<sup>−</sup>1) at all positions, the midpoint of each of the 100 hexagonal grid cells contains neurons with ten different preferred velocities for ten different angles, hence 100 different preferred directions per hexagonal grid cell. The lengths of preferred directions are distributed according to a distribution favoring low velocities (Weiss et al., 2002) with a logarithmic scale for the speed according to Weber's law (Stocker and Simoncelli, 2006). In order to avoid boundary effects, both spatial dimensions are closed and continuous. This leads to a horn torus as input space, i.e., if a stimulus leaves the 1 × 1 space it reappears on the opposite side (so-called "pac-man topology"). This topology holds also for the network connectivity, e.g., connections reaching beyond the virtual border at *xtarget* = 1 will be wrapped around. After all tuning properties are set, they get dispersed to account for natural variability (Paik and Ringach, 2011).

#### **2.3. INPUT STIMULUS**

A classical way of studying motion extrapolation is by presenting a moving target that travels behind an occluder for a short period of time. A seminal study used timing estimation by asking participants to make a button press response at the time they judge the occluded target to have reached a particular point (Rosenbaum, 1975). Motion extrapolation can be carried out for lateral motion with the target moving across the fronto-parallel plane, or for approaching motion, when the object moves toward the observer (DeLucia, 2004). Herein, we investigate visual, lateral motion extrapolation only. For simplicity, we study the network's response to a moving dot stimulus and the network's ability to predict the trajectory of the dot when it disappears behind an obstacle producing a blank gap in the input signal.

From the definition of the tuning properties of a neuron *i*, we may model the response to a moving dot as an inhomogeneous Poisson process with a parametrically defined envelope. Indeed, we will use the following input stimulus *Li(t)* as the envelope for a Poisson process with a maximum of 5 kHz (when *Li(t)* reaches 1) and a time step of 0.1 ms:

$$L\_i(t) = \exp\left(-\frac{\|\vec{\mathbf{x}}\_{\text{stim}}(t) - \vec{x}\_i\|^2}{2\mathfrak{\beta}\_X^2} - \frac{\|\vec{\nu}\_{\text{stim}} - \vec{\nu}\_i\|^2}{2\mathfrak{\beta}\_Y^2}\right) \tag{2}$$

where *x<sup>i</sup>* is the neuron's receptive field central position, *vi* the neuron's preferred direction, *xstim(t)*, (*vstim*) is the position (direction) of the moving dot (see **Figure 1**). As the trajectory of the dot is rectilinear and constant, we have

$$
\vec{\chi}\_{\rm stim}(t) = \vec{\chi}\_{\rm stim}(0) + \vec{\nu}\_{\rm stim} \cdot t \tag{3}
$$

The resulting inhomogeneous Poisson spike train is connected to the respective neurons via one excitatory synapse of strength *winput* = 5 nS. This formalization allows to study for the different roles of theses parameters. In particular, the tuning width may play an important role as it is known that in low level visual areas (such as the retina), receptive fields are small (position is accurate, motion is imprecise) while in higher level areas motion is more finely represented, while position is less precise (as the receptive fields' size increase). In the rest, all neurons have the same tuning width defined by β*<sup>X</sup>* = 0*.*15 and β*<sup>V</sup>* = 0*.*15*s* <sup>−</sup>1. The β*X, <sup>V</sup>* values have been set so that a reasonable part of the network receives sufficient input from the moving dot stimulus. Increasing β*X, <sup>V</sup>* would make the dot appear broader, whereas smaller β*X, <sup>V</sup>* would make a smaller fraction of the network respond to the stimulus. Changes in the β*X, <sup>V</sup>* parameters would not change the working concept of the model, but would require a retuning of connectivity parameters like number of connections and connection strengths.

For simplicity, we studied only networks in which excitatory neurons receive input because inhibitory neurons primarily provide a normalization mechanism in our model, even though this might not reflect real cortical circuits (Frégnac et al., 2003). All neurons receive additional noise in form of Poisson spike trains with a rate of *fnoise* = 2 kHz injected via excitatory and inhibitory synapses with a weight of *wnoise* = 4 nS to simulate the input from external networks. For all simulations of this paper, the network was stimulated with a dot moving at a speed of *vstim* = 0*.*5 *s* −1 from left to right (and an initial position defined by *xstim*(t=0) = *(*0*.*1*,* 0*.*5*)*). Crucially, during the blank phase, the stimulus vector in the network was permuted randomly at each time step among the whole excitatory population, so that the selectivity of the input was completely lost while keeping a similar input average frequency as compared to phases when stimulus is active.As the input vector is shuffled the network does not receive a coherent continuous input signal. During the blank phase, cells that are well-tuned to the stimulus receive input spike trains with a larger inter-spike-interval leading to a decrease in the effective input. This is due to the integration of post-synaptic potentials on the membrane in the context of LIF neurons. Compared to an empty input vector, this input vector shuffling during the blank phase elevates the mean membrane potential of the population slightly which can help the network to fill the blank phases with meaningful input.

#### **2.4. NETWORK CONNECTIVITY**

The network study consists of an excitatory population with *Nexc* = 13000 neurons and an inhibitory population with *Ninh* = 2520 neurons (that is, with a ratio of 16*.*2% inhibitory cells over the whole population). Both populations are mutually and recurrently connected in one of the following ways which will be explained in the next sections: randomly, isotropically or anisotropically.

#### *2.4.1. Random and isotropic connectivities*

Connections within and between populations can be set up in an isotropic manner that does not depend on the source or target neuron's tuning properties. When neurons are connected in this way, connection probabilities are computed according to:

$$p\_{\vec{\eta}} = p\_{\text{max}} \cdot \exp\left(-\frac{d\_{\vec{\eta}}^2}{2 \cdot \sigma\_X^2}\right) \tag{4}$$

where *pmax* is a normalizing factor and *dij* = *<sup>x</sup><sup>i</sup>* <sup>−</sup> *<sup>x</sup><sup>j</sup>* represents the distance (in visual space) between both neurons. *pmax* is set so that the total number of connections between two populations drawn isotropically is equal to an overall probability of *pk, <sup>l</sup>*, *(k, l)* ∈ {*E, I*}. The connection probabilities utilized are: *pEE* = 0*.*5%, *pEI* = 2%, *pIE* = 2%, *pII* = 1%.

Weights are drawn from a normal distribution with mean μ*iso w* and standard deviation σ*iso <sup>w</sup>* = 0*.*2 · μ*iso <sup>w</sup>* . The value of μ*iso <sup>w</sup>* is set so that the expected sum of incoming weights equals a certain target value *wkl* specific to the type of the source and target population (*k, l* ∈ {*E, I*}): *wEE* = 0*.*3μS, *wEI* = 1*.*8μS, *wIE* = 0*.*8μS, *wII* = 0*.*15μS (if not stated differently).

Delays are drawn from a normal distribution with a mean value μ*iso* <sup>δ</sup> <sup>=</sup> 3 ms and a standard deviation of <sup>σ</sup>*iso* <sup>δ</sup> = 1 ms. Selfconnections have been discarded. A completely random connectivity may be then achieved by setting σ*<sup>X</sup>* to a sufficiently big value (relative to the scale of the spatial period). This results in a flat, uniform probability of connection over the whole population.

#### *2.4.2. Anisotropic, motion-based prediction connectivity*

Inspired by motion-based prediction (Perrinet and Masson, 2012), we may define a connectivity by wiring neurons that are linked by a smooth trajectory with a higher probability. Connectivity will then be specifically anisotropic as it provides a mechanism for motion-based prediction by diffusing motion information across the network in a forward, asymmetric manner. Specifically, we will take advantage of the latency that exists between neurons in the same cortical area and use that parameter to connect cells matching a smooth trajectory. The motivation underlying this formula is based on the idea that smooth trajectories are more likely seen in natural scenes and are promoted by the network connectivity. If the target position is situated at the position where the source neuron predicts the stimulus to be in a certain time τ*ij* and if the target neuron predicts the stimulus to move in a similar direction *vj* as the preferred direction of the source neuron *vi*, the source neuron connects with a high probability to the target neuron.

As a consequence, the connection probability is computed from the tuning properties of the source neuron *i* and target neuron *j* according to the sampling of the prior defined in Perrinet and Masson (2012):

$$p\_{i,j} = p\_{\text{max}} \cdot \exp\left(-\frac{\left\|\vec{x}\_{i,j}^\* - \vec{x}\_j\right\|^2}{2 \cdot \sigma\_X^2}\right) \cdot \exp\left(-\frac{\left\|\vec{\nu}\_i - \vec{\nu}\_j\right\|^2}{2 \cdot \sigma\_V^2}\right) \quad (5)$$

$$
\vec{\mathbf{x}}\_{i,j}^{\*} = \vec{\mathbf{x}}\_{i} + \vec{\nu}\_{i} \cdot \mathbf{r}\_{i,j} \tag{6}
$$

$$\mathbf{r}\_{ij} = \frac{\|\vec{x}\_i - \vec{x}\_j\|}{\|\vec{v}\_i\|} \tag{7}$$

In this formulation, *x*<sup>∗</sup> *<sup>i</sup>, <sup>j</sup>* is the position predicted for a motion that would leave the source neuron's receptive field (therefore from position *x<sup>i</sup>* and with velocity *vi*) after a latency τ*ij*. Then, parameter τ*ij* corresponds to the expected latency knowing the respective position and velocity of source and target neurons. In Equation (5), the parameters σ*<sup>X</sup>* and σ*<sup>V</sup>* determine the strength of the tuning properties of motion-based prediction. Unless stated otherwise, we will use σ*<sup>X</sup>* = 0*.*1 and σ*<sup>V</sup>* = 0*.*1−<sup>1</sup> (see **Figure 2**). Note that the precision of prediction in the velocity domain (that is σ*V*) determines a scaling factor for the degree of anisotropy: the lower σ*<sup>V</sup>* is, the more the outgoing connections of a neuron are aligned with the preferred direction of the source neuron. Note also that only σ*<sup>V</sup>* includes the predictive prior on velocity and that we may retrieve an isotropic connectivity by setting σ*<sup>V</sup>* to a sufficiently high value.

The probabilities are then sorted and each target neuron receives input from 0*.*5% of the source neurons that have the highest connection probability. Those 0*.*5% highest probabilities are converted to connection weights so that the sum of incoming weights per neuron equals a certain target value *wkl* specific to the type of the source and target population (*k, l* ∈ {*E, I*}): *wEE* = 0*.*20μS (for motion-based connectivity and *wEE* = 0*.*25μS for direction-based connectivity), *wEI* = 1*.*8μS, *wIE* = 0*.*8μS, *wII* = 0*.*15μS (these values are only for the example networks and might differ depending on the exact implementation and require an adjustment for different network sizes).This means the connectivity becomes deterministic (based on the tuning properties of the source and target cell) and the term probability refers only to the overall selection of source cells in the network.

**FIGURE 2 | From aniso- to iso-tropic connectivities.** We propose that the local connectivity pattern of lateral connections play a crucial role in solving the motion-extrapolation problem. In order to show that, we compared different connectivity patterns in response to the blanked input (see **Figure 1**). In the three panels, we show the incoming and outgoing connections for the same single neuron (as marked by the yellow diamond) for different connection rules. The preferred direction of that neuron is shown by the yellow arrow. Cells targeted by the yellow cell are marked with black circles and cells projecting to the yellow cell are marked with red triangles. The relative weights of incoming (outgoing) connections are indicated by the size of the source (target) neuron, respectively. The preferred direction of source and target neurons is shown by solid arrows. Connection delays are color coded. **Left:** Motion-based prediction anisotropic connectivity. Inspired by previous work on motion-based prediction (Perrinet and Masson, 2012), we propose a first pattern of connectivity based on connecting a source neuron to a target neuron if and only if the position and velocity of the target is compatible with a smooth trajectory that would originate from the source neuron. The strength of this

## *2.4.3. Anisotropic, direction-based prediction connectivity*

However, if we use the previous equation to connect cells [Equation (5)], and scale our network realistically, it appears that latencies depend on the velocity coded by the cells, and in turn, this leads to unrealistically high delay values with the range of velocities we used. As a consequence, we defined another way of setting up the connectivity which only take into account the angle between source and target cells and the angle between the directions coded for by the source and target cells. It is therefore independent of the preferred speed (modulus of velocity) of the neurons and on the latency used to connect the cells.

We use von Mises probability distribution functions to define the tuning in the range of all directions:

$$p\_{i,j} = p\_{\text{max}} \cdot \exp\left(\frac{\cos\left(\vec{\mathbf{x}}\_{\hat{i}} - \vec{\mathbf{x}}\_{i}, \vec{\nu}\_{\hat{i}}\right)}{\sigma\_X^2}\right) \cdot \exp\left(\frac{\cos\left(\vec{\nu}\_{\hat{i}}, \vec{\nu}\_{\hat{j}}\right)}{\sigma\_V^2}\right) \tag{8}$$

The first term guarantees that information spreads in the direction that is preferred by the source cell (and where σ*<sup>X</sup>* gives approximately the width of tuning in radians). The second term ensures that information is passed only to cells that code for motion moving in a similar direction as preferred by the source cell (and where similarly σ*<sup>V</sup>* gives approximately the tuning width in radians). Note that in position-velocity space, the probability of connection is maximal in the direction given by the preferred velocity of the source cell and centered on the position prediction is parameterized by the width of the lateral tuning selectivity and here we show the prototypical pattern for σ*<sup>X</sup>* = 0*.*3 and σ*<sup>V</sup>* = 0*.*3 ms−<sup>1</sup> (as used for the simulations). **Middle:** Direction-dependent connectivity. To create a more realistic connectivity pattern, we used the same rule but independently of speed (i.e., the modulus of velocity), but only as a function of the direction of motion. The target neuron is connected if and only if its direction is close to the source's direction and if its position is predicted to be in the direction given by the source neuron. Additionally, to account for physiological constraints on lateral interaction, only connections within a radius of *rConn* = 0*.*10 or latencies shorter than 100 ms are allowed. This leads to a more local connectivity and smaller connection delays compared to the previous connectivity. We show here the resulting connectivity pattern for σ*<sup>X</sup>* = 0*.*3 and σ*<sup>V</sup>* = 0*.*3 ms−<sup>1</sup> with the motion-dependent connectivity and σ*<sup>X</sup>* = 1*.*0 and σ*<sup>V</sup>* = 1*.*0 ms−<sup>1</sup> with the direction-dependent connectivity. **Right:** An isotropic connectivity pattern was chosen as a control. There is no prediction in velocity space, but we still predict that activity should diffuse locally, as the connection probability drops with the distance between cells.

of that cell's receptive field. The density therefore defines a cone around this half-line, defined by widths σ*<sup>X</sup>* and σ*<sup>V</sup>* [see middle panel in **Figure 2**]. Note that this formulation may be derived from the formulation of motion-based prediction by lowering the strength of prediction on the radial component of velocity. As such, this connection probability gives a similar mechanism for promoting smooth trajectory, and provides the diffusion of motion information in the direction detected by the network. A comparison of these two network connectivities is visualized in **Figure 2**.

Whereas encoding and decoding of direction information is now largely understood in various neuronal systems, how the human brain accurately represents speed information remains largely unknown. Speed tuned neurons have been identified in several early cortical visual areas in monkeys. However, how such speed tuning emerges is not yet understood. A working hypothesis is that speed tuned neurons non-linearly combine motion information extracted at different spatial and temporal scales, taking advantage of the statistical spatiotemporal properties of natural scenes. Furthermore, the population code underlying perceived speed is not yet elucidated and therefore we are still far from understanding how speed information is decoded to drive and control motor responses or perceptual judgments. As a consequence, such a connectivity profile will serve as a further control to test if restraining the prediction to direction is sufficient to solve the motion extrapolation problem.

## **2.5. CHOICE OF PARAMETERS**

The number of receptive fields has been set so that the fourdimensional space of tuning properties is covered with a reasonable density of cells. Decreasing the number of receptive fields would decrease the number of cells in the network and would impede the diffusion of information between cells. This is because the weight of connections is sensitive to the distribution of cells in the tuning property space, and a over-sparsely populated tuning property space can lead to unwanted effects for activity spread in the network. The parameters describing receptive field sizes, β*<sup>X</sup>* and β*V*, determine the distribution of the input signal in the network. They have been chosen so that a small part of the network receives a sufficient amount of excitation that brings this small part above the spike threshold and initiates the spread of activity, and by that, the diffusion of motion-information in the network. Increasing β*X, <sup>V</sup>* would make the stimulus appear fuzzier and, consequently the extrapolation task more difficult. A decrease of β*X, <sup>V</sup>* would make the stimulus appear sharper. But it would not necessarily make the task easier since the source of activity would be smaller and the seed for the diffusion of information could possibly be too small to propagate through the network, depending on the network connectivity parameters. The parameters determining the network connectivity *pk, <sup>l</sup>* and *wk, <sup>l</sup>*, *(k, l)* ∈ {*E, I*} were chosen to be in a range comparable to physiological values for large networks. Especially the connection weights needed to be fine tuned to solve the motion-extrapolation task. Redistributing the tuning properties could easily lead to instabilities, i.e., that the trajectory could not be extrapolated, and too high weights could lead to an explosion of activity in the network.

## **2.6. PREDICTION READOUT**

A crucial issue when trying to map a Bayesian inference algorithm to a network of spiking neurons is to understand how probability can be expressed in terms of neural activity. Herein, we applied a simple vector averaging method to infer the prediction about stimulus position and direction from the activity of the excitatory population. Indeed such decoding scheme may be justified as a simple implementation of probabilistic codes as done by Beck et al. (2008). Their approach requires several assumptions which are not guaranteed in our model: First of all, neurons are assumed to have Poisson-like spiking statistics, which is obviously not true in our model since activity is strongly driven by the stimulus and hence neuronal activity is not Poisson-like (see 3). Secondly, they assume that network activity is uncorrelated on timescales of 50 ms, which is likewise not realistic for our model. Furthermore, their approach works on probability distributions gained over several trials, which could principally be done with our model, but it is computationally more expensive than the single-trial vector-averaging method described above. However, this provided a decoding approach which seemed to robustly represent the activity in the network.

In particular, we used a similar formulation as the decoding framework proposed for neurons in area MT (Jazayeri and Movshon, 2006). Indeed, the definition of our model fits well to their implementation. In both models, the activity of sensory neurons is pooled in a simple additive feed-forward architecture, In contrast to their model, we extend the application beyond the angle of motion and apply the readout framework to position and direction. More precisely, the tuning properties are in the exponential family and tile uniformly the position-velocity space. Thanks to the definition of the tuning selectivity of the neurons in the network, the position and velocity corresponding to the Maximum Likelihood estimation corresponds to the average over all neurons of each central tuning parameter weighted by the activity of the neuron (independently of β*<sup>X</sup>* and β*<sup>V</sup>* as they are uniform for all neurons). To define a continuous activity at each time bin a weight *pi(t)* is defined for each excitatory neuron *i* based on the number of output spikes fired during a time bin *t*:

$$p\_i = n\_i(t) \Big/ \sum\_{i}^{N\_{\text{exc}}} n\_i(t) \tag{9}$$

where *ni(t)* is the number of spikes fired by neuron *i*. The time bin size was set to 50 ms, but it could be chosen differently without qualitative changes.

Such decoding schemes are classically implemented on unbounded variables. However, we defined space on a torus in order to avoid edge effects. Hence, the network average must be computed for circular quantities (Mardia and Jupp, 2009): The idea behind Equation 10 is that in order to compute the mean of a circular quantity, the position and direction first need to be transformed into an angle, which is then projected to the 2D unit circle where the arithmetic mean is computed. After that, the angle that the mean position forms is transformed back from an angle to space. For positions, this takes the form:

$$\chi\_{\rm pred}^{\rm net}(t) = 1 + \frac{1}{2\pi} \arctan 2 \left( \sum\_{i}^{N\_{\rm ec}} p\_i(t) \sin \left( 2\pi \mathbf{x}\_i - \pi \right),$$

$$\sum\_{i}^{N\_{\rm ec}} p\_i(t) \cos \left( 2\pi \mathbf{x}\_i - \pi \right) \right) \tag{10}$$

where *xi* is the center of the spatial receptive field of neuron *i* (the same formula is applied to compute *ypred(t)*). The subtraction of *pi* in the sin and cos functions is necessary to map the interval of position which is between 0 and 1 to the interval of *(*−π*,*π*)* required for the projection of position on the unit circle.

Similarly, for reading our the direction of the stimulus predicted by the network:

$$\nu\_{\rm pred}^{\rm net}(t) = \frac{1}{\pi} \arctan2 \left( \sum\_{i}^{N\_{\rm euc}} p\_i(t) \sin \left( \pi \nu\_i \right), \sum\_{i}^{N\_{\rm euc}} p\_i(t) \cos \left( \pi \nu\_i \right) \right) \tag{11}$$

The difference between Equations 10 and 11 lies in the fact that positions are bound to be between 0 and 1, whereas directions can be negative and larger than 1 or −1, which changes the transformation to and from angles for position and direction. Taken together, this gives an easy decoding scheme from the neural activity to a probabilistic read-out.

#### **3. RESULTS**

## **3.1. ANISOTROPIC DIFFUSION TRANSPORTS INFORMATION DURING THE BLANK**

Recurrent excitatory connectivity is a candidate for providing mechanisms for motion-based prediction. In order to prove the functionality of our approach, we show results for single example networks of three different connectivity types applied to the recurrent excitatory population: isotropic, anisotropic motionbased (speed tuning dependent) and anisotropic direction-based (speed tuning independent). For simplicity, the other connectivities are set up according to the isotropic scheme. Networks using the anisotropic scheme for one or more of the other pathways (*E* − *I*, *I* − *E*, *I* − *I*) could be tuned to perform similarly (not shown).

The connectivity in our model is mainly controlled by two parameters: the sum of incoming weights and the number of connections received by a cell. The sum of incoming weights for excitatory-excitatory connections have been tuned so that the activity initiated by the stimulus is strong enough to propagate through the network when the stimulus is turned off after 400 ms of input driven activation. Weights from the excitatory to the inhibitory population have been chosen so that inhibitory neurons exhibit a reasonable level of average activity of approximately 5 Hz when the stimulus is driving the excitatory population. The role of inhibitory to excitatory connections is to balance the network activity when the stimulus is active after the blank. In contrast to balanced random networks, the inhibitory to excitatory weights could not be set to high values that compensate for the higher number of excitatory neurons as in previous models [e.g., (Brunel, 2000; Morrison et al., 2007)] because strong isotropic inhibitory feedback would silence excitatory neurons in the vicinity and impede the propagation of motion information during the blanking period. The interplay between the excitatory and inhibitory populations is crucial for balancing network activity, but more importantly, for suppressing activity that creates false predictions about the target trajectory. All connectivity parameters were tuned so that the spread of activity within the excitatory population is strong enough to fill a realistically long blanking period, where the average duration of a single blink is between 100 and 400 ms (Schiffman, 2001).

Before stimulus onset, the network input consists of background noise that persistently drives the network at low firing rates. The stimulus spike trains are dispersed over the whole excitatory population (see **Figure 3**). As this input is not coherent, the type of connectivity has no effect on the activity before onset. The stimulus activates the network between 200 and 600 ms of the simulation before another blank phase of 200 ms in which the input is equal to the phase before stimulus onset. Neurons that are well tuned to the stimulus fire at very high rates (temporarily up to 250 Hz) when the stimulus is present. This is due to the strong input stimulus and the amplification by the recurrent excitation. The average firing rate of neurons being active at least once during the simulation grows to up to 20 Hz when the stimulus is persistently present.

**FIGURE 3 | Rasterplot of input and output spikes.** The raster plot from excitatory neurons is ordered according to their position. Each input spike is a blue dot and each output spike is a black dot. While input is scattered during blanking periods (**Figure 1**), the network output shows shows some tuned activity during the blank (compare with the activity before visual stimulation). To decode such patterns of activity we used a maximum-likelihood estimation technique based on the tuning curve of the neurons.

During the blank phase the global network activity drops rapidly to a low average rate of approximately 2 Hz and those neurons that convey the remaining motion information fire approximately 5–15 spikes during the blanking period, as individual output rates remain elevated to levels of 25–75 Hz. Due to the anisotropic connectivity the activity triggered by the stimulus propagates through the network in the direction that was initiated by the target (see **Figures 3**, **4**).

We observed that there needs to be a balance between stimulus induced excitation and the recurrent excitation: When recurrent excitation is too strong, the internal neural dynamics dominate over the activity triggered by the stimulus and it is likely that false tracking behavior occurs, i.e., network activity spreads too fast in the direction of the stimulus and the predicted trajectory gets ahead of the target. When recurrent excitation is not strong enough, the network activity fails to fill the blank by its own dynamics like in the network with isotropic connectivity.

The connectivity parameters σ*<sup>X</sup>* and σ*<sup>V</sup>* need to be chosen differently for the two anisotropic networks, because their role in determining the connection probability between cells is slightly different according to Equations 5 and 8. For motion-based (*MB*) connectivity we used σ*MB <sup>X</sup>, <sup>V</sup>* = 1, and for direction-based (*DB*) σ*DB <sup>X</sup>, <sup>V</sup>* = 0*.*5. When σ*<sup>X</sup> <* σ*<sup>V</sup>* two main effects could be observed. As the network connectivity is more specific in the spatial domain, the prediction performance in the target direction tends to be lower and in the target position gets more precise. But the prediction can get ahead of the stimulus because excitation spreads to fast along the predicted trajectory. In the opposite case, when σ*<sup>V</sup> <* σ*X*, the prediction of target direction gets more precise. Since the connectivity is spatially more distributed, the network is less likely to fill the blank because excitation is diluted across space.

## **3.2. READING-OUT THE POPULATION RESPONSE**

Jazayeri and Movshon (2006) presented a framework for a generic representation of likelihoods of sensory stimuli by neural activity. Here, we used a similar approach (see 2.6) which allows us to transform the binary spiking activity into a continuous valued representation of probability about the target motion. By these means, the activity of individual neurons can be interpreted as time-continuous confidence measure.

The phase before stimulus onset is like a prior probability. It is dominated by noisy activity that seems uniform and does not converge into a coherent probability distribution (see **Figure 4**). At stimulus onset, the network activity increases instantaneously and the probability distribution changes into a meaningful representation of motion information. During the blank period, the network activity drops rapidly (sometimes more gradually) and the probability distribution becomes more noisy, but changes less dramatically. Hence, despite the overall decrease in activity, information is not lost when the stimulus disappears. Instead, activity continues to propagate through the network, driven by the anisotropic connectivity. When the stimulus reappears the network activity grows again and continues to grow up to an average rate of 20 Hz until it is counterbalanced by the inhibitory feedback.

#### **3.3. MOTION-BASED PREDICTIVE ANISOTROPIC DIFFUSION SOLVES MOTION EXTRAPOLATION**

In order to get a global estimation of the motion information we combine the probability estimates of individual neurons as described earlier (see 2.6) by a linear weighting of their timevarying activity. This provides a single valued, time-continuous prediction, i.e., readout signal of target position and direction. We will now compare the maximum confidence response for the three different connectivities to the exact same input in order to investigate the effect of the network connectivity on the readout signal.

Before stimulus onset the readout signal of all three networks follows the same noisy time course (see **Figure 5**). After stimulus onset and after the blank, all three estimations coincide with the actual target position and are very close to the target direction. This shows that our simple linear decoding approach is sufficient to translate the network activity into a meaningful readout signal.

The difference between the three networks can be seen during the blank phase. During this period, the readout signal of

**FIGURE 5 | Comparison of prediction performance for the different connectivities.** The performance of direction (**top**) and position (**bottom**) prediction as decoded from the network activity is shown (see Equations 10, 11). First and second columns show the horizontal and vertical components, respectively, while the last column shows the mean squared error of the predicted position with respect to the known position of the target. The color of the lines correspond to

the different connectivities presented in **Figure 2**: motion-based prediction (solid blue), direction-dependent prediction (dashed green), isotropic (dash-dotted red). While an isotropic connectivity clearly fails to predict the fate of the input during the blank, we show here that the anisotropic connectivities may efficiently solve the motion extrapolation problem, even with an approximate solution such as the direction-based prediction.

the network with the isotropic connectivity returns to the noisy time course, just like before the stimulus onset. In contrast, readout from the networks with anisotropic connectivity continues to give a precise estimation of the target position and direction of motion, as can be seen from the low prediction error (see the right most columns of **Figure 5**). The readout from the direction-based connectivity is less accurate than the motion-based connectivity, but it still shows that the direction-based diffusion mechanism allows for inference of the target position and direction during the blank phase. Thus, it can solve the motion extrapolation problem.

In some experiments, we observed that direction prediction appears to be biased toward higher velocities - especially during the blank. Improvements to the connectivity rule might be necessary to gain a "perfect" prediction performance (zero root-mean-square error). The reason for the drift toward higher velocities can be seen as an unbalanced distribution of incoming weights. Neurons with higher velocities are more seldom and hence have less cells with similar tuning properties in their vicinity. Due to the fact that all cells receive the same sum of incoming weights, the comparatively few cells that project to cells with high preferred velocities do this via few, strong connections, possibly leading to the observed drift and instabilities in the network dynamics. This could possibly be solved by improving how probabilities are mapped to connection weights, e.g., by introducing a non-linearity that prohibits weights above a certain value. Nevertheless, it was not our objective to present an optimal ad-hoc connection algorithm that gives perfect prediction performance, but to prove the fact that anisotropic, tuning property-based connectivity could be an important mechanism achieving motion-based prediction.

## **3.4. CONCLUSION**

The comparison of the prediction performance of the three different networks shows two main points. Anisotropic connectivity provides a mechanism for the diffusion of motion information, which is relevant to predict future trajectories in noisy environments where the flow of information is interrupted frequently. Also, our simple approach to read out network activity linearly is sufficient to solve the given task, and does not require knowledge about probability distributions gained over many trials.

## **4. DISCUSSION**

## **4.1. SUMMARY AND COMMENTS**

Following our previous study (Perrinet and Masson, 2012), we have confirmed that anisotropic diffusion of information is a sufficient mechanism to realize motion-based prediction as tested by the moving-dot blanking experiment. We have studied the role of different anisotropic and isotropic connectivity patterns and have shown that network connectivities that take into account the tuning properties of neurons and prefer smooth trajectories are more efficient in predicting the trajectory of a disappearing moving stimulus than isotropic networks. The main contribution of this study is to show that anisotropic diffusion of motion information can be implemented in networks of spiking neurons and thus could be a generic mechanism for motion-based prediction. Furthermore we have presented, to the best of our knowledge, the first model for motion prediction using spiking neurons and selective anisotropic connectivity that is inspired by a probabilistic framework.

The presented model is certainly limited and unrealistic in many ways. We have intentionally chosen a simple model (in terms of neuronal and synaptic features) to focus on the effects of connectivity patterns and to explore possible future applications for neuromorphic hardware systems (Schemmel et al., 2010; Brüderle et al., 2011). One of the main limitations consists of using long interconnection delays that are necessary to achieve its main function in the current state of the model. According to the spatial scale and the conduction delays of cortical networks, the connection delays resulting from our model are not on the same order of magnitude. Assuming that the concept of anisotropic diffusion of information operates in motion processing networks, neuronal mechanisms other than such long delays would be required to achieve the desired predictive function. One reason for long delays is likely the size of our network and that we are sub-sampling neurons in comparison to realistic cortical network sizes. In larger networks it would be sufficient to have recurrent excitation working on a more local scale and long-range connectivity would implement the transport of an expectation signal, possibly in the subthreshold domain. We have successfully explored one possibility to relieve the need for long delays by constraining the connectivity to more local scales, which reduced the required delays by more than one order of magnitude. Instead of the axonal delays as employed by our model, dendritic delays, long synaptic time constants (provided e.g., NMDA or GABA-B currents), or a combination of those three mechanisms could be used to implement the same principle. In summary, larger network sizes and longer synaptic time constants would likely help to realize our approach with shorter, more realistic connection delays. The fact that both models with longer and shorter delays show very similar performance could be a hint for the generality of the idea of motion information being transported by anisotropic connectivity.

## **4.2. CONTEXT OF EXISTING MODELS**

Based on early ideas by Hubel and Wiesel (1962), models employing anisotropic connectivity have been used to describe orientation selectivity (Finette et al., 1978) and its general use for visual information processing of static images with non-spiking units (Rybak et al., 1991). Models that are more similar to ours in motion coherence and the representation of motion trajectories [e.g., (Burgi et al., 2000; Jancke and Erlhagen, 2010)] do not use anisotropy in setting up the network connectivity. Other continuous recurrent network models have been used for various tasks like spatial working memory (Compte et al., 2000) and categorical discrimination with veridical judgment of motion (Liu and Wang, 2008). Our model works on a different level, but combines the functional features of previous models in the sense that motion trajectories are represented in a spiking and probabilistic way. Prediction signals are transported through recurrent connectivity, but none of the earlier models has shown that anisotropy could be a key element for this.

The dynamics of our network show some similarity to synfire chains (Prut et al., 1998) and we believe a model with similar dynamics and functionality could be implemented by attractor networks making use of a columnar organization (Lundqvist et al., 2006) that is prominent in motion processing areas like are V5/MT (Albright et al., 1984). Work by Bressloff (2001) has shown that weak heterogeneities in excitable neural media can lead to wave propagation failure. We have shown that in principle, it is favorable to have heterogeneity (i.e., the anisotropy in connectivity) to promote the spread of activity in a meaningful way. Still, there is much experimental evidence showing that cortical networks can spread activity in the form of traveling waves (Sato et al., 2012), and it is believed that long-range horizontal connections might be one of the underlying mechanism. It is arguable how well the conclusions of the Wilson-Cowan formalism used by Rybak et al. (1991); Bressloff (2001); Jancke and Erlhagen (2010) can be translated into the context of spiking networks, especially if the neuronal and synaptic machinery get more complex. We leave it for future analysis to determine if our network model could show behaviors similar to ones observed in experiments [for a review on traveling waves see Sato et al. (2012)].

We applied a simple vector averaging strategy to decode the position and motion direction from the network's response (Georgopoulos et al., 1986), but the optimal way of decoding is up to debate. Several studies (Priebe and Lisberger, 2004; Pack et al., 2005) suggest a vector averaging approach with a bias term to estimate speed, but more recent experiments suggest that "perceived speed is not based on a labeled-line interpretation of MT cells" (Krekelberg et al., 2006). Alternative decoding approaches involve a winner-take-all mechanism (Liu and Wang, 2008) or probabilistic codes (Beck et al., 2008). We may use an existing method to decode the optimal estimate from the population of neurons as is done in real neural data (Jazayeri and Movshon, 2006), though our goal here was to show in a simple way that anisotropic and selective connectivity could be of great importance for motion prediction.

Based on the results of our model, we predict that the connectivity in higher cortical motion-processing areas like MT or MST is not isotropic, but that effective connectivity between cells depends on their tuning properties. A sign of this anisotropic, tuning-property based connectivity could possibly be seen in future experiments similar to those in Guo et al. (2007) in the form of an anticipatory signal in cells that "expect" to receive stimulus input via the recurrent network connections.

## **4.3. OUTLOOK AND FUTURE WORK**

After having shown a proof-of-concept for the idea that motionbased prediction can be achieved through anisotropic connectivity, many problems could be explored by the presented framework. One of the most urgent challenges in our view is the question how the recurrent connectivity can develop in a selforganized and robust manner. In order to integrate our model into the visual hierarchy, we need to understand how the tuning properties introduced in our model could be constructed through either feed-forward connections from lower cortical areas, through recurrent mechanisms that shape the desired properties, or both. Similarly important is the question of how a selective connectivity involving the inhibitory population influences the effective receptive field sizes along with the performance and stability of the presented framework. Another future challenge future is to use our probabilistic framework of spiking neurons for more realistic input and toward real-world applications.

## **ACKNOWLEDGMENTS**

This work is supported by projects FACETS-ITN (EU funding, grant number 237955) and "BrainScaleS" (EU funding, grant number FP7-269921). The simulations were performed on resources provided by the Swedish National Infrastructure for Computing (SNIC) at PDC Centre for High Performance Computing (PDC-HPC) at KTH, Stockholm. Code to reproduce figures and supplementary material are available on github and on the corresponding author's website at http://invibe.net/LaurentPerrinet/Publications/Kaplan13. The authors thank the reviewers for the feedback and Phil Tully for proofreading.

## **REFERENCES**


*Physica D* 155, 83–100. doi: 10.1016/S0167-2789(01)00266-4


the cat's visual cortex. *J. Physiol.* 160:106.


of primates. *J. Neurosci.* 15, 1195–1214.


cortical networks based on spatially realistic connectivities. *Front. Comput. Neurosci.* 6:41. doi: 10.3389/fncom.2012.00041


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 21 April 2013; accepted: 25 July 2013; published online: September 2013. 17*

*Citation: Kaplan BA, Lansner A, Masson GS and Perrinet LU (2013) Anisotropic connectivity implements motion-based prediction in a spiking neural network. Front. Comput. Neurosci. 7:112. doi: 10.3389/fncom.2013.00112*

*This article was submitted to the journal Frontiers in Computational Neuroscience.*

*Copyright © 2013 Kaplan, Lansner, Masson and Perrinet. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Spike timing regulation on the millisecond scale by distributed synaptic plasticity at the cerebellum input stage: a simulation study

## **Jesús A. Garrido1,2\*, Eduardo Ros <sup>3</sup> and Egidio D'Angelo1,4\***

<sup>1</sup> Neurophysiology Unit, Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy

<sup>2</sup> Consorzio Interuniversitario per le Scienze Fisiche della Materia, Pavia, Italy

<sup>3</sup> Department of Computer Architecture and Technology, University of Granada, Granada, Spain

<sup>4</sup> Brain Connectivity Center, IRCCS Istituto Neurologico Nazionale C. Mondino, Pavia, Italy

#### **Edited by:**

Julie Wall, University of London, UK

#### **Reviewed by:**

Carson C. Chow, National Institutes of Health, USA Simon Davidson, University of Manchester, UK

#### **\*Correspondence:**

Jesús A. Garrido and Egidio D'Angelo, Brain Connectivity Center, IRCCS Istituto Neurologico Nazionale C. Mondino, Via Mondino 2, Pavia 27100, Italy. e-mail: jesus.garrido@unipv.it; dangelo@unipv.it

The way long-term synaptic plasticity regulates neuronal spike patterns is not completely understood.This issue is especially relevant for the cerebellum, which is endowed with several forms of long-term synaptic plasticity and has been predicted to operate as a timing and a learning machine. Here we have used a computational model to simulate the impact of multiple distributed synaptic weights in the cerebellar granular-layer network. In response to mossy fiber (MF) bursts, synaptic weights at multiple connections played a crucial role to regulate spike number and positioning in granule cells. The weight at MF to granule cell synapses regulated the delay of the first spike and the weight at MF and parallel fiber to Golgi cell synapses regulated the duration of the time-window during which the firstspike could be emitted. Moreover, the weights of synapses controlling Golgi cell activation regulated the intensity of granule cell inhibition and therefore the number of spikes that could be emitted. First-spike timing was regulated with millisecond precision and the number of spikes ranged from zero to three. Interestingly, different combinations of synaptic weights optimized either first-spike timing precision or spike number, efficiently controlling transmission and filtering properties. These results predict that distributed synaptic plasticity regulates the emission of quasi-digital spike patterns on the millisecond time-scale and allows the cerebellar granular layer to flexibly control burst transmission along the MF pathway.

**Keywords: spiking network, spike timing, cerebellum, granular layer, LTP, LTD**

## **INTRODUCTION**

By operating in a continuously changing environment, neuronal networks have evolved precise processes regulating the number and positioning of spikes (Rieke et al., 1999). Spike timing has been revealed in afferent sensory pathways and in cortical networks (Mackevicius et al., 2012), in which millisecond-scale correlations among neurons are thought to improve information storage capacity and computational capabilities (Petersen et al., 2009; Eldawlatly and Oweiss, 2011; Kimura et al., 2011). Spike timing can be controlled by long-term synaptic plasticity (Nieus et al., 2006), which regulates the strength and dynamic properties of synaptic connections. Nevertheless, it is not clear how differential distribution of synaptic weights could fine-tune spike timing in central networks expressing multiple distributed forms of synaptic plasticity.

The cerebellum has long been proposed to operate as a "timing machine" (Eccles, 1967) and a "learning machine" (Ito, 2006), but the intrinsic nature of these operations has not been resolved yet. Interestingly, the cerebellum controls motor behavior with millisecond precision (Timmann et al., 1999; Osborne et al., 2007), so it is expected that its computations are performed on a comparable or even faster time-scale. There are indications, mostly

derived from cellular investigations in rat cerebellar slices, that the granular layer (see **Figure 1**) is capable of exerting a close control on spike timing (D'Angelo and De Zeeuw, 2009). The granule cells (GrCs) generate brief spike bursts in the axon initial segment, which are almost instantaneously (<0.3 ms) transmitted to the dendrites and to synapses on the axonal ascending branch (Diwakar et al., 2009, 2011). Moreover, the mossy fiber (MF) – GrC EPSCs have extremely fast kinetics [rise time <1 ms (Silver et al., 1992)] and can therefore excite the GrCs with high temporal precision (Cathala et al., 2005). Finally, GrCs are endowed with specific ionic mechanisms capable of controlling the delay and persistence of spike emission (D'Angelo et al., 2001). A theoretical analysis has revealed that half of the information carried by MF spike trains is retransmitted by GrCs as first-spike delay with millisecond precision and half as spike frequency (Arleo et al., 2010). Interestingly, long-term potentiation (LTP) and long-term depression (LTD) have been shown to regulate both first-spike delay and spike frequency through different mechanisms (Nieus et al., 2006). The outstanding timing capabilities of this system have been summarized into the "time-window" hypothesis, which considers how these mechanisms compete with feed-forward synaptic inhibition mediated by Golgi cells (GoCs) in order to control spike emission

The basic one (black lines) includes the excitatory pathway (MF-GrC), feed-forward inhibitory loop (MF-GoC-GrC) and the feed-back inhibitory loop (GrC-GoC-GrC). The extended versions add the GrC-SC-GoC-GrC loop (green line) and the GoC-GoC inhibitory connection (blue lines) (Barmack and Yakhnitsa, 2008; Hull and Regehr, 2012).

from the GrCs during a period of a few milliseconds after MF burst discharge (D'Angelo and De Zeeuw, 2009; D'Angelo et al., 2013; D'Angelo et al., submitted).

A precise understanding of spike timing in the granular layer needs to consider long-term synaptic plasticity not just at the MF – GrCs synapse but also at other synapses (Gao et al., 2012). Recent results indicate that LTD may indeed occur at the parallel fiber (PF) – GoC synapse (Robberechts et al., 2010), while evidence for long-term synaptic plasticity in the feed-forward MF – GoC – GrCs inhibitory loop is still lacking. However, forms of adaptation may occur at the GoC – GrC synapse (Rossi et al., 2006; Brandalise et al., 2012). The other important element to consider is how network inhibition is organized. Different reports have indicated the presence of stellate cell (SC) – GoC inhibitory synapses (Casado et al., 2000), of GoC – GoC inhibitory synapses (Hull and Regehr, 2012), and of GoC–GoC gap junctions (Vervaeke et al., 2010). This complex connectivity could have an impact on spike timing.

Given the large number of variables, it is desirable that a theoretical and computational analysis is anticipated to predict how a distributed regulation of synaptic weights could control spike timing. To this aim, we have constructed a granular-layer spiking network model accounting for different hypotheses on connectivity in the inhibitory loops and for different assets of weights in circuit synapses. In response to burst stimulation of the MFs, GrCs in the model showed a permissive time-window regulated by inhibition. Variations in the weights at multiple synapses effectively tuned the windows boundaries and the efficiency of inhibition, regulating the delay to first-spike, and the overall number of spikes

emitted. Therefore, distributed plasticity can implement an extensive and flexible control over spike timing and bursting on the millisecond time-scale, which could have important implications for cerebellar mechanisms of function.

## **MATERIALS AND METHODS**

## **OVERVIEW OF THE COMPUTATIONAL MODEL**

A granular-layer model was built using the EDLUT simulator (http://edlut.googlecode.com) (Ros et al., 2006). This model accounts for the topology of connections and neuronal elements included into the realistic granular-layer model developed by Solinas et al. (2010). However, since the aim of this work was to investigate the influence of synaptic weights at multiple connections, neurons were simplified using leaky integrate-and-fire (LIF) models, and synapses were simplified using exponential models (Gerstner and Kistler, 2002). The source code of the model is available at ModelDB (https://senselab.med.yale.edu/modeldb/ ShowModel.asp?model=149913).

The differential equations were solved using the fourth order Runge–Kutta method with a fixed time-step of 100µs. Simulations of 4-s activity trials were carried out on a 152-node cluster located at the *Centro de Investigación en Tecnologías de la Información y las Comunicaciones* (*CITIC*) in the University of Granada.

## **THE NETWORK STRUCTURE**

**Figure 1** shows the model of the granular layer including 5177 single-compartment cells. Network topology was developed in two different steps. Initially, the number of constitutive elements was calculated following anatomical studies of cell densities in the granular layer (Korbo et al., 1993). Then, we have connected those elements respecting convergence-divergence ratios and connectivity rules based on biology (Eccles, 1967; Harvey and Napper, 1988, 1991).

The granular-layer model which has been simulated in this work is composed of the following elements:


Since this work is focused on temporal properties, while neglecting specific topologies like the center-surround organization imposed by lateral inhibition [see (Solinas et al., 2010)], all connections have been generated randomly (with the only restriction of avoiding duplication of source-target cell pairs).

## **THE NEURON AND SYNAPSE MODELS**

Neurons were modeled using modified versions of the LIF model (Gerstner and Kistler, 2002; Gerritz et al., 2011). In the LIF model, membrane potential (*V* <sup>m</sup>) is computed through the differential equation (Eq. 1A), which accounts for the effect of chemical synapses [including AMPA, NMDA, and gamma-aminobutyric acid (GABA) receptors] and resting conductance (*G*rest),

$$\begin{aligned} \text{C\_m} \frac{dV\_{\text{m}}}{dt} &= \text{g}\_{\text{AMPa}}\left(t\right) \left(E\_{\text{AMPa}} - V\_{\text{m}}\right) \\ &+ \text{g}\_{\text{NMDA}}\left(t\right) \text{g}\_{\infty, \text{NMDA}}\left(V\_{\text{m}}\right) \left(E\_{\text{NMDA}} - V\_{\text{m}}\right) \\ &+ \text{g}\_{\text{GABA}}\left(t\right) \left(E\_{\text{GBA}} - V\_{\text{m}}\right) + G\_{\text{rest}}\left(E\_{\text{rest}} - V\_{\text{m}}\right) \end{aligned} \tag{1a}$$

$$\text{g}\_{\infty, \text{NMDA}}(V\_{\text{m}}) = \frac{1}{1 + e^{-\alpha V\_{\text{m}}} [\text{Mg}^{2+}] / \beta} \tag{1b}$$

Where *C*<sup>m</sup> denotes the membrane capacitance, *E*AMPA, *E*NMDA, and *E*GABA are the reversal potentials of each synaptic conductance and *E*rest is the resting potential. The conductances *g*AMPA, *g* NMDA, and *g* GABA integrate all the contributions received through individual synapses for each of the receptor types (AMPA, NMDA, and GABA, see below). Finally, Eq. 1B determines *g*∞,NMDA(*V* <sup>m</sup>), the gating function of the NMDA channels accounting for voltagedependent magnesium block (Jahr and Stevens, 1990), with α = 62V−<sup>1</sup> , [Mg2+] = 1.2 mM and β = 3.57 mM (adapted from Gabbiani et al., 1994).

The synaptic conductances have been modeled following Eqs 2A–C:

$$\mathbf{g\_{AMPA}}(t) = \begin{cases} 0 & , \quad t < t\_0 \\ \mathbf{g\_{AMPA}}(t\_0) \ e^{-(t-t\_0)/\tau\_{\text{AMPA}}} & , \quad t \ge t\_0 \end{cases} \tag{2a}$$

$$\mathfrak{g}\_{\text{NMDA}}(t) = \begin{cases} 0 & , \quad t < t\_0 \\ \mathfrak{g}\_{\text{NMDA}}(t\_0) \, e^{-(t-t\_0)/\tau\_{\text{NMDA}}} & , \quad t \ge t\_0 \end{cases} \tag{2b}$$

$$\text{g\_{GABA}}\left(t\right) = \begin{cases} 0 & , \quad t < t\_0\\ \text{g\_{GBA}}\left(t\_0\right)e^{-(t-t\_0)/\tau\_{\text{GABA}}} & , \quad t \ge t\_0 \end{cases} \tag{2c}$$

Where *t* denotes the simulation time and *t* <sup>0</sup> denotes the time at which an input spike is received. *g*AMPA and *g* NMDA represent AMPA (α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid) and NMDA (*N*-methyl d-aspartate) receptor-mediated conductance respectively, which provide excitation, and *g* GABA represents the GABA receptor-mediated conductance, which provides inhibition. τAMPA, τNMDA, and τGABA are the decaying time constants of each receptor type. The synaptic conductances were modeled as decaying exponential functions, which provide a reasonable accuracy as well as computational efficiency (Ros et al., 2006).

The parameters of each cell type (Eqs 1A–C) and synaptic receptor (Eqs 2A–C) have been chosen to model granule cell, stellate cell, and Golgi-cell dynamics (see **Table 1**) (Silver et al., 1996; Tia et al., 1996; Nusser et al., 1997; Rossi and Hamann, 1998).

**Table 2** shows the synaptic connections which have been included in the developed model, the convergence and divergence ratios, and the different kinds of receptors under consideration. While both AMPA and NMDA receptors have been included in

#### **Table 1 | Parameters of the model cells.**


These parameters have been obtained from the following references: GrC (D'Angelo et al., 1993, 1995, 1998, 2001; Gabbiani et al., 1994; Nieus et al., 2006), SC (Häusser and Clark, 1997; Carter and Regehr, 2002; Kreitzer et al., 2002; Chavas and Marty, 2003; Suter and Jaeger, 2004), and GoC (Forti et al., 2006; Solinas et al., 2007a,b). Synaptic connectivity is defined in **Table 2** and synaptic conductances are reported in**Table 3**.

#### **Table 2 | Synaptic connections in the network model.**


This table reports the convergence and divergence ratios and the receptors implemented in the model.The GoC inhibitory connections (SC-GoC and GoC-GoC) are included only when explicitly mentioned.The convergence and divergence ratios have been taken from the literature (Nieus et al., 2006; Solinas et al., 2007a,b, 2010; Mapelli et al., 2009; Hull and Regehr, 2012).

MF-GrC synapses, only AMPA transmitters have been implemented at the remaining excitatory synapses (MF-GoC, GrC-GoC, and GrC-SC). This decision emerges from available evidence suggesting, through *in situ* hybridization studies, that adult GoCs do not express NR2B (Ottersen and Storm-Mathisen, 2000). Moreover, patch-clamp recording experiments suggest that NR2Dcontaining receptors are expressed only extra-synaptically on these cells (Cull-Candy et al., 2001; Brickley et al., 2003).

#### **THE SYNAPTIC WEIGHTS**

Synaptic weights have been derived from previous network models (Maex and Schutter, 1998; Solinas et al., 2010). As a strategy, we first established the weight of MF-GrC connections and then adjusted the others to calibrate network responses. Reflecting physiological determinations, the simulated network generated singlets in control and increased its output to doublets and triplets when inhibition was reduced or the MF-GrC synapse was potentiated (Mapelli and D'Angelo, 2007; Roggeri et al., 2008; Andreescu

et al., 2011; Diwakar et al., 2011). We have defined three different plasticity states at the MF-GrC connection: LTD, control, and LTP (D'Errico et al., 2009). The weights in control derived from global peak conductance values (i.e., the maximum conductance resulting from the simultaneous activation of all the synapses impinging on a single GrC) measured in neurophysiological recordings. In the control state, the peak conductance of AMPA-receptor channels at each MF-GrC synapse was set at 3.49/4 = 0.87 nS (Gabbiani et al., 1994). With an AMPA/NMDA ratio of 9.94, the peak conductance of NMDA-receptor channels at each MF-GrC synapse was set at 0.87/9.94 = 0.087 nS. In the LTP (LTD) state, peak conductance at the MF-GrC connection was allowed to increase (decrease) by 30%, i.e., to 1.131 nS (0.609 nS) (Maffei et al., 2002; D'Angelo et al., 2005). The inhibitory synaptic weights at the GoC-GrC synapse were also set similar to those reported experimentally (Mapelli et al., 2009), therefore implementing an appropriate excitatory/inhibitory balance in GrCs. Once the GrC synaptic weights had been established, the model allowed investigating the full parameter space including all the synaptic weights in the inhibitory loops (that is, MF-GoC, GrC-GoC, and GoC-GrC), which ranged between the boundaries reported in **Table 3**. Throughout the paper, the AMPA/NMDA ratio was maintained constant and only the AMPA synaptic weight has been indicated in the figures.

It should be noted that in this paper we did not implement intrinsic learning mechanisms but rather we investigated how multiple changes at different synapses could modify the network functional state. Therefore, synaptic weights have been systematically changed either independently or in various combinations. The implementation of network learning rules for distributed synaptic plasticity goes beyond the present aims and represents a further step that will be required in order to investigate network self-organization.

#### **STIMULATION PROTOCOL**

In order to analyze the response of the network to realistic patterns, our model was stimulated with 100 Hz MF spike bursts (Mapelli and D'Angelo, 2007; D'Angelo and De Zeeuw, 2009). The individual spike times were extracted from a Gaussian distribution. In order to avoid GrC saturation, some preliminary simulations were run to study the influence of two main parameters: the firing probability of each single MF and the standard deviation of firing times. By lowering firing probability, the response of the GrCs was reduced. Similarly, by increasing sparseness of MF input activity, the amount of response decreased due to the lack of coincidence between incoming spikes. A 5-Hz basal random MF activity was included based on previous investigations (Solinas et al., 2010). In aggregate, the stimulation protocol was composed of MF bursts with a maximum of three spikes burst and average inter-spike frequency of 100 Hz. Spikes were generated with a probability of 0.7 followed a Gaussian time distribution with SD 1 ms.

## **ANALYSIS OF SIMULATED RESULTS: EXCITATORY/INHIBITORY BALANCE**

Since GrCs encode information both in the delay of the first spike and in the spike count, we have evaluated the peri-stimulus histogram (PSTH) in response to the first spike of the MF burst and the number of spikes (usually singlets, doublets, or triplets) elicited


The table reports the synaptic weights in control, LTD, and LTP states for each connection. The configurations are based on previous works (Maex and Schutter, 1998) and on preliminary simulations. The synaptic connections traditionally described in the literature (MF-GrC, MF-GoC, GoC-GrC, and GrC-GoC) have been studied conjointly (i.e., simulating all the possible four-dimensional weight combinations), while the GoC inhibitory connections (SC-GoC and GoC-GoC) have been added to simulations separately.

in response to the whole burst. In order to determine the activity state of neurons under combined excitatory and inhibitory synaptic drive, we have defined the GrC excitatory/inhibitory conductance balance using the following equation.

$$E/\text{Ig balance}\left(t\right) = \frac{1}{\#P} \sum\_{i \in P} \left( \sum\_{j \in E\_i} \left( \text{gAMA}\_{i,i,j}\left(t\right) + \text{gNMAD}\_{i,i,j}\left(t\right) \right) \right)$$

$$+ \sum\_{j \in I\_i} \text{gGARA}, i, j \text{ ( $t$ )} \tag{3}$$

where *E/I g* balance (*t*) represents the average excitatory/inhibitory balance at time *t*, *P* represents the set of cells under study. *E<sup>i</sup>* and *I<sup>i</sup>* represent all the sources of excitation and inhibition. *g*AMPA*,i,j*(*t*) and *g* NMDA*,i,j*(*t*) represent the AMPA-receptor and NMDA-receptor-mediated conductances at the *j-*th excitatory synapse reaching the cell *i*, as previously defined in Eq. 3. Similarly, *g* GABA,*i,j*(*t*) represents the GABA-receptor conductance at the *j-*th inhibitory synapses reaching the cell *i*. In Eq. 4, the excitatory conductances (AMPA and NMDA) have been considered with negative sign, and the inhibitory conductance (GABA) has been considered positive. Indeed, a negative value at the conductance balance indicates the predominance of excitation over inhibition in the cell.

#### **RESULTS**

In order to investigate the consequences of distributed synaptic plasticity on spike timing, we have analyzed the impact of weights at the different synapses of the granular layer in a spiking network model (**Figure 1**). Background activity was generated by random low-frequency activity in MFs (average frequency = 5 Hz) and three-spike 100-Hz bursts were then used to elicit network responses to impulsive stimulation.

The response of single neuronal elements is represented in **Figure 2A**. With control weight settings [see **Table 3** (Solinas et al., 2010)], GrC responses manifested a marked dependence on activity in the inhibitory loops. When the inhibitory loops

**FIGURE 2 | Neuronal responses and excitatory/inhibitory conductance balance in the granular-layer model.** Response of the model to a stimulation burst composed of three spikes at 100 Hz over a 5 Hz background activity generated by random low-frequency firing in the MFs. **(A)** Electrical response of single GrCs and GoCs in control (inhibition ON) and in the absence of inhibition (inhibition OFF). In each panel, four traces are shown superimposed and arrows indicate the mossy fiber stimulation time. Note that the same seeds are used for initializing the

were active, the GrCs usually generated a single spike followed by sub-threshold membrane potential changes, while in the absence of inhibition the GrCs generated multiple spikes in response to burst stimulation (D'Angelo et al., 1995, 2001). This is an implementation of the *time-window* effect, which predicts that the feed-forward inhibitory loop passing through the MF-GoC-GrC connections can curtail the GrC response (D'Angelo and De Zeeuw, 2009). The GoCs efficiently followed the input bursts, as expected from their high reactivity to synaptic inputs through the MFs (Kanichay and Silver, 2008). GoC activity increased when GoC-GrC transmission was blocked, since GrC disinhibition enhanced activity at PF synapses. Thus, the major functional properties of the circuit revealed in electrophysiological experiments were captured by the model. Nevertheless, it must be noted that

theta-frequency auto-rhythmicity of GoCs was not implemented (Forti et al., 2006).

The regulation of GrC responsiveness by inhibition was reflected by the average GrC synaptic conductance, which provided an effective measure of the Excitatory/Inhibitory balance (*E/I g balance*). Following a MF burst, the *E*/*I g* balance showed an initial negativity (net excitatory conductance) followed by a large positivity (net inhibitory conductance), which was eliminated by switching-off GoC-GrC transmission (**Figure 2B**).

The behavior of the entire network is represented in **Figure 3A**. Random low-frequency activity in MFs generated sparse GrC spikes with an average frequency of about 0.1 Hz (Solinas et al., 2010) without remarkably engaging GoCs. MF bursts raised the probability of spike emission from GrCs up to around 25%, which

**FIGURE 3 | Response patterns in the granular-layer model.** Response of the model to a stimulation burst composed of three spikes at 100 Hz over 5 Hz background activity generated by random low-frequency firing in the MFs. **(A)** Response of network elements in control conditions. (Top) Raster plots of activity recorded in the MF (left), GrC (center), and GoC (right) populations, respectively. Note the three-spike bursts in MFs with 100-Hz average frequency and firing probability 0.7. GrC activity rises sharply in response to the first spike in the burst generating a single output spike. Then GrC activity is inhibited by the GoCs, which keep on firing in response to all the three MF burst spikes. (Bottom)

Peristimulus-time histograms (PSTH) in MFs (left), GrCs (center), and GoCs (right) respectively. Note the single PSTH peak in GrCs in response to the first MF burst (bin size 1 ms). **(B)** PSTHs taken from GrCs and GoCs around the first MF spike are shown over-imposed on expanded scale. The Input/Output offset (I/O offset) represents the time with respect to the MF stimulation. Given the probabilistic distribution of MF burst activity around a mean time value (dotted line), the I/O offset can assume negative values at the beginning of discharge. Note that the GrC and GoC PSTH peaks are delayed with respect to the MF PSTH peak due to the integration time of incoming activity in the cells.

is in the range estimated for active granular-layer clusters activated by impulsive sensory stimulation *in vivo* (Diwakar et al.,2011). The GoCs emitted synchronous spike triplets, which caused a powerful GrC inhibition summating along the burst (Mapelli et al., 2009). In this configuration, the network efficiently filtered the second and third spikes that would otherwise be generated in the absence of inhibition. Therefore, the major question was how the weights at the various synapses could regulate and modify these filtering properties.

The input/output (I/O) function of the circuit was analyzed by evaluating the relationship between input and output bursts in GrCs, which represent the output element of the circuit. I/O plots for the first GrC spike showed the GrC response as a function of the time-offset with respect to MF discharge (**Figure 3B**). By counting the spikes emitted by GrCs, this PSTH summarizes the intensity, delay, and duration of the discharge and can be used to analyze the I/O behavior of the circuit.

## **THE EFFECT OF MF-GrC WEIGHTS**

The MF-GrC connection has been shown to express forms of LTP and LTD (Hansel et al., 2001) and has been proposed to significantly influence cerebellar signal processing (Nieus et al., 2006; D'Angelo and De Zeeuw, 2009; Arleo et al., 2010). In our simulations, synaptic weights at MF-GrC connections effectively modified spike transmission. By increasing the MF-GrC weight (LTP condition), the GrC spike response occurred earlier and with higher probability, while the opposite occurred by reducing weights (LTD condition). This caused corresponding changes in the first-spike GrC PSTH (**Figure 4A**), which became smaller and shifted to the right while moving from higher to lower weights. Consistently, the average I/O offset (**Figure 4B**) decreased monotonically by decreasing the MF-GrC synaptic weight, indicating shortening of the GrC response delay. Moreover, average GrC firing probability increased as expected from a stronger excitatory input (**Figure 4B**), but then tended to stabilize (or even slightly

decreased) beyond a certain MF-GrC synaptic weight. The origin of this *plateau* effect is that, as the GrC response increases due to higher MF-GrC weight, the inhibitory feed-back loop is more intensely activated preventing a further increase in GrC firing (this revealed that the feed-back loop exerted a homeostatic feed-back effect, as further considered in the discussion). Finally, MF-GrC synaptic weights had an impact on the spike pattern emitted by GrCs, which changed from a small number of singlets in the LTD state toward a higher amount of singlets and some doublets in the LTP state (**Figure 4C**). Further modulation of this pattern will occur by combining plastic changes at multiple synaptic sites, as reported below.

## **THE EFFECT OF MF-GoC WEIGHTS**

There is currently no evidence for long-term synaptic plasticity between MFs and GoCs, but its potential existence is of relevance. Here we considered that, in the model, a change in MF-GoC weights could have as much the same effect as a change in the number of active MF-GoC synapses, which is a plausible mechanism of GoC regulation. By adjusting the MF-GoC synaptic weights, the model showed remarkable changes in GrC activation (**Figure 5A**). Not unexpectedly, higher weights at the GoC input anticipated GoC firing and GoC inhibition on the GrCs. The final effect was to reduce the time window for GrC firing and therefore the GrC firing probability. This is clearly reflected into the GrC PSTH, which shows a shortening of the time window as the MF-GoC weights move from LTD to LTP. In parallel the GrC *E*/*I g* balance showed

(Bottom) PSTH of the GrC response after the first spike in the burst. MF-GoC LTD produces greater and more protracted responses, while the opposite occurs with LTP. **(B)** Average input/output offset (top) and GrC firing probability (bottom) with respect to MF-GoC weights. The changes observed in conjunction with three different MF-GrC weights are shown for comparison (MF-GrC LTD dot dash line; MF-GrC control solid line; MF-GrC LTP dashed line). **(C)** Relative number of GrCs generating zero, one, two, or three spikes in response to the stimulation burst. These are reported for three different MF-GrC weights, as specified in **(B)**.

an increase in inhibitory strength (**Figure 5B**). The impact of MF-GoC synaptic weights on the spike pattern emitted from GrCs was to change it from low number of singlets in the LTP state toward higher number of singlets and a considerable amount of doublets in the LTD state (**Figure 5C**). These effects were modulated by plasticity at the MF-GrC synapse, in a way that the amount of doublets markedly increased with combined increase of MF-GrC and decrease of MF-GoC weights. These observations suggest that distribution of plasticity at multiple sites can effectively regulate the GrC firing pattern, as further considered below.

#### **THE EFFECT OF GoC-GrC WEIGHTS**

There is currently no evidence for LTP or LTD at the GoC-GrC synapse, although protracted forms of regulation with a potential homeostatic significance have been reported (Rossi et al., 2006; Mapelli et al., 2009; Brandalise et al., 2012). Our simulations showed that higher weights reduced GrC firing and lower weights enhanced GrC firing, but these changes were poorly effective in controlling the initiation of GrC activity. Thus, the GrC PSTH showed no remarkable changes concerning the first-spike probability and timing (**Figure 6A**). Nonetheless, the GrC I/O *g* balance showed remarkable changes after emission of the first spike, bringing about remarkable consequences for the generation of subsequent spikes. Accordingly, no remarkable changes were observed in I/O offset and firing probability plots for the first GrC spike (**Figure 6B**). The impact of GoC-GrC synaptic weights on the spike pattern emitted from GrCs was to change it from singlets in the high-weight state toward doublets and triplets in the low-weight state (**Figure 6C**). These effects were modulated by plasticity at other synapses in the feed-forward loop, in a way that the amount of doublets and triplets markedly increased when a decrease in GoC-GrC weights was combined with an increase in MF-GrC weights and a decrease in MF-GoC weights.

## **THE EFFECT OF GrC-GoC WEIGHTS**

Recently, the existence of forms of plasticity at the GrC–GoC connection has been reported suggesting that this synapse can undergo persistent transmission changes (Robberechts et al., 2010). Our simulations showed that higher weights reduced GrC firing and lower weights enhanced GrC firing, but these changes were poorly effective in controlling the initiation of GrC activity. Indeed, this inhibition reached the GrCs only at the final part of the excitatory window due to accumulation of delays in the feed-back inhibitory loop. Thus, the GrC PSTH showed no remarkable changes concerning the first-spike probability and timing (**Figure 7A**). Indeed, the GrC conductance balance showed no remarkable differences compared to control (cf. also **Figure 4A**). The inhibitory strength was similar with low GoC-GrC weights and with high GoC-GrC weights. Accordingly, no remarkable changes were observed in I/O offset and firing probability plots for the first GrC spike, unless when the MF-GrC weight were set at medium-high values (**Figure 7B**). Nonetheless, the impact of GrC-GoC synaptic weights was to control the emission of spikes when the activity of the GrC was high (**Figure 7C**). Thus, the combination of high weights at MF-GrC and GrC-GoC connections effectively shifted the average firing time in the GrC while keeping the firing rate at the same range of activity. These results show that the

feed-back loop effectively behaves as a homeostatic mechanism of the granular activity.

## **THE EFFECT OF INHIBITION ONTO GoCs**

Recent results have provided evidence for two modalities of GoC inhibition, namely through GoC-GoC (Hull and Regehr, 2012) and SC-GoC (Casado et al., 2000) connections. In this model, both the GoC-GoC and SC-GoC synapses proved able to regulate the generation of GrC spikes in response to MF burst stimulation. With either GoC-GoC or SC-GoC synaptic connections, the GoCs tend to generate action potentials every second spike of the MF burst, while keeping the GrCs at a reduced level of inhibition in correspondence of other spikes (**Figures 8A** and **9A**). Thus, higher GoC-GoC or SC-GoC weights enhanced GrC firing, while lower weight reduced GrC firing without affecting the initiation of GrC activity. Consistently, the GrC PSTH showed no remarkable changes concerning first-spike probability and timing (**Figures 8A** and **9A**). Nonetheless, in the I/O GrC conductance balance, the inhibitory strength for subsequent spikes was reduced with high GoC-GoC weights and increased with low GoC-GoC weights (**Figure 8A**). Accordingly, no remarkable changes were observed in I/O offset and firing probability plots for the first GrC spike (figure not shown), but the impact of GoC-GoC synaptic weights was observed on the emission of late spikes. Eventually, the GoC-GoC connection could markedly increase the emission of doublets and even some triplets in the high-weight state (**Figure 8B**).

A similar behavior was observed by regulating the weights of SC-GoC loop (**Figure 9**), although we could not take into account the complex regulatory mechanisms of the molecular layer interneuron network, which could substantially modify the impact of this pathway.

## **DISCUSSION**

This paper shows that *distributed synaptic plasticity* allows simultaneously regulating multiple processing features of the cerebellum granular-layer network. By adjusting synaptic weights at the MF-GrC synapse, in feed-back and feed-forward inhibitory loops and in the interneuron inhibitory network, the probability, positioning, and number of spikes emitted by the GrCs changed generating quasi-digital spike patterns. The relevance of these effects for cerebellar regulatory mechanisms and network computation is discussed.

#### **CONTROL OF SPIKE TIMING BY DISTRIBUTED SYNAPTIC PLASTICITY**

Simulations showed that MF bursts caused the emission of GrC spikes through a permissive time-window limited by inhibition, which controlled the evolution of the response (D'Angelo and De Zeeuw, 2009). The precise timing of the *first spike* was mostly regulated by theMF-GrC connection strength. Then, the granular-layer inhibitory loops regulated GrC activity in response to the second and subsequent spikes in the MF burst. In particular, simulations showed that LTP at MF-GrC synapse reduced the GrC reaction times and LTP at MF-GoC shortened the excitatory time-window generating more precise (i.e., less time-dispersed) PF responses. The synaptic weights at GoC-GrC connections influenced the strength and duration of the inhibitory window. In the feed-back inhibitory loop, regulation of weights at the GrC-GoC synapses

**FIGURE 6 | Effect of GoC-GrC synaptic weight on GrC activity.** In these simulations the following weight configuration was used: MF-GrC 0.61 nS (LTD), 0.87 nS (control), or 1.13 nS (LTP) (AMPA-receptor weight), MF-GoC LTD (0.5 nS), Control (1 nS) and LTP (2 nS), GrC-GoC 3 nS, GoC-GrC 0-3 nS, SC-GoC 0 nS, GoC-GoC 0 nS. **(A)** GrC response with MF-GrC and MF-GoC control configurations and three different GoC-GrC weights: LTD (0.75 nS, green), Control (1.5 nS, red), and LTP (3 nS, blue). The theoretical distribution of the MF stimulation burst is shown with a dashed line. (Top) Membrane potential in a single GrC and its firing threshold (dotted straight line at −40 mV). (Middle) E/I g balance of the GrC population. (Bottom) PSTH of the GrC

response after the first spike in the burst. GoC-GrC did not influence the GrC response during the initial part of the burst. **(B)** Average input/output offset (top) and GrC firing probability (bottom) with respect to GoC-GrC weights. The changes observed in conjunction with three different MF-GrC weights (LTD dot dash line; control solid line; LTP dashed line) are shown for comparison. **(C)** Relative number of GrCs generating zero, one, two, or three spikes in response to the stimulation burst. These are reported for three different MF-GrC weights and for three MF-GoC weights. GoC-GrC LTD weight noticeably increased the proportion of GrCs firing doublets and even triplets (especially with MF-GrC LTP and with MF-GoC LTD].

transformed the increasing activity at the PFs in shorter, and more precise excitatory windows. The inhibitory connections impinging onto GoCs effectively controlled generation of *late spikes*. The control of synaptic weights in the feed-forward and feed-back inhibitory loops and in the interneuron inhibitory network indeed allowed generating a large variety of patterns in GrCs determining the number and timing of emitted spikes. While long-term

synaptic plasticity has been demonstrated at the MF-GrC synapse LTD may indeed occur at the PF – GoC synapse (Robberechts et al., 2010), it is currently unknown if and how the MF-GoC, GoC-GrC, SC-GoC, and GoC-GoC connections undergo plastic changes, whose investigation is therefore of interest. In general, the concept of plasticity should refer to any kind of changes in the number and strength of connections between these neurons

occurring during ontogenesis or as a consequence of modulation and learning.

line. (Top) Membrane potential in a single GrC and its firing threshold (dotted straight line at −40 mV). (Middle) E/I g balance of the GrC population. (Bottom) PSTH of the GrC response after the first spike in the

The present model was made of LIF neurons and was not endowed with realistic ionic channels properties of the kind characterizing neuronal membranes and synaptic connections. Therefore,it revealedfundamental network-dependent properties, which could then be compared with intrinsic properties of neurons and synapses. Interestingly, the most relevant properties of neurons and synapses reported so far are congruent with the network properties reported here. (i) The nature of GrC synaptic receptors

LTP). **(C)** Relative number of GrCs generating zero, one, two, or three spikes in response to the stimulation burst. These are reported for three

different MF-GrC configurations.

simulations the following weight configuration was used: MF-GrC 0.61 nS (LTD), 0.87 nS (control), or 1.13 nS (LTP) (AMPA-receptor weight), MF-GoC 1 nS, GrC-GoC 3 nS, GoC-GrC 1.5 nS, SC-GoC 0 nS, GoC-GoC 0 nS (LTD/disabled, green), 1 nS (control, red), or 3 nS (LTP, blue). **(A)** GrC response with MF-GrC control configuration and the three GoC-GoC configurations. The theoretical distribution of the MF stimulation burst is shown with a dashed line. (Top left corner) Membrane potential in a single GrC and its firing

is such that they can precisely control both first-spike timing on the sub-millisecond scale through AMPA receptors (Silver et al., 1992; D'Angelo et al., 1995; Cathala et al., 2003) and the continuation of burst discharge through NMDA-receptors (D'Angelo et al., 1990). (ii) Presynaptic regulation of release probability during LTP and LTD (Nieus et al., 2006; D'Errico et al., 2009) efficiently regulates first-spike timing. (iii) GrC and GoC discharge properties revealed by electrophysiological investigations (D'Angelo et al., 1998; Forti et al., 2006; Solinas et al., 2007b; Kanichay and Silver, 2008) are consistent with network behaviors emerging in our experiments. In aggregate, the cellular and synaptic properties of the granularlayer match fundamental regulatory properties embedded into the network structure and tuned by distributed synaptic plasticity.

## **THE EFFECT OF COMBINING SYNAPTIC CHANGES AT MULTIPLE SITES**

The present simulations show that there are combinations of synaptic weight changes, which can achieve differential control over network processing, alternatively increasing or filtering spike transmission, maximizing first-spike precision or bursting (**Table 4**; **Figure 10**).

GrC population. (Top right corner) PSTH of the GrC response after the first spike in the burst. GoC-GoC did not influence the GrC responsiveness during the initial part of the burst. **(B)** Relative number of GrCs generating zero, one, two, or three spikes in response to the stimulation burst. These are reported for three different MF-GrC configurations. Note that GoC-GoC LTP configuration noticeably increased the proportion of GrCs that fire doublets in MF-GrC control and LTP configurations.

## **Increasing transmission**

Spike transmission through the MF-GrC relay can be increased by lowering activity in the feed-forward and feed-back inhibitory loops and by strengthening inhibition of GoCs. This occurs at the expense of first-spike precision and bursting. It is not clear when this condition could be exploited, as there is normally a strong inhibitory activity in the granular layer (Roggeri et al., 2008). However, there could be activity states in which maximizing transmission might be useful to improve subsequent pattern emergence through learning. Indeed, LTP generation is strongly dependent on removal of inhibition, which could be controlled by local release of specific neuromodulators (Prestori et al., 2013). Plasticity would then turn network weight settings in favor of first-spike precision or bursting.

## **Filtering**

Filtering requires LTD at the MF-GrC synapse and high activity in the feed-forward inhibitory loop. LTD at the MF-GrC synapse may reflect protracted uncorrelated low-frequency activity in MFs. In this state, the network can filter spurious spikes allowing

transmission of just highly synchronous spikes. Interestingly, various mechanisms based on NMDA and GABA-A receptors, make this filtering function of the granular-layer frequency-dependent (Mapelli et al., 2010).

## **Maximize time precision**

The precision of first-spike emission can be maximized by raising all the weights. This state may emerge at the end of a learning process and is likely to be modified in accordance with the needing for spike filtering and patterning. Traditional pattern recognition models are based on fast responses to known patterns and on the absence of responses to non-correlated activity (Masquelier et al., 2008). Accordingly, the GrCs fire quickly in response to the earliest spikes of the MF burst and the inhibitory loops silence the GrC response to the latest spikes of such burst. The GoC inhibitory connections maintain a high GrC precision by anticipating the onset of inhibition and preventing multiple GrC spikes. The plausibility of this mechanism is evident when considering the numerous mechanisms implementing high-precision first-spike timing in GrCs (Silver et al., 1992; Cathala et al., 2003). Moreover, the information

transmitted through the MF-GrC relay is largely due (about 50%) to millisecond precision first-spike timing (Arleo et al., 2010).

## **Maximize bursting**

The bursts emitted by GrCs, which are composed of spike doublets/triples, in response to MF input bursts, can be optimized by combining LTP at the MF-GRC synapse with low activity in GoCs (obtained by weakening the inhibitory loops and by strengthening GoC inhibition). The composition of the output burst is criticalfor cerebellar network computation, as shown by the powerful regulation exerted by NMDA receptors (D'Angelo et al., 1995) and by the motor dysfunction that emerges when this control system is disrupted (Andreescu et al., 2011). The generation of doublets/triples could be critical to control activation of Purkinje cells (PCs) and molecular layer interneurons, which are highly sensitive to temporal summation through short-term synaptic plasticity (Dittman et al., 2000) and may also control long-term synaptic plasticity at the same synapses (Casado et al., 2000).

As long as spike bursts are regulated to achieve specific computational effects, the granular-layer circuit needs to maintain a



The main transmission pathway is indicated in red, the feed-back and feed-forward GrC inhibitory loops are indicated in blue, and the GoC inhibitory loops are indicated in green. The specific synaptic weights, which define either LTD or control or LTP states in every single synapsis, are shown in**Table 3**.

homeostatic balance in order to prevent saturation of PF activity (Marr, 1969). The granular-layer circuit is intrinsically homeostatic in that GrC activity can be depressed by raising activity in the feed-back inhibitory loop. It is probable that homeostasis occurs, together with the various optimization processes, in order to balance network activity. Homeostasis may extend over space, for example balancing LTP and LTD over neighboring granularlayer areas (Mapelli and D'Angelo, 2007; Diwakar et al., 2011). There is also evidence that homeostasis may exploit specific mechanisms raising GrC responsiveness when inhibition is persistently increased (Rossi et al., 2006; Mapelli et al., 2009; Brandalise et al., 2012). Finally, it should be noted that during protracted bursts GrC firing is maintained higher than expected from a pure timewindow mechanism (Kanichay and Silver, 2008; Mapelli et al., 2010), suggesting that additional mechanisms are indeed at work. Further physiological experiments and larger scale models should be developed to investigate the issue.

## **MILLISECOND-PRECISE QUASI-DIGITAL SPIKE PATTERN**

By fully implementing the *time-window* mechanism (D'Angelo and De Zeeuw, 2009; Solinas et al., 2010), distributed synaptic plasticity can fine-tune the initiation of first-spike emission as well as the burst spike pattern (**Figure 10**). Regulation of firstspike delay is almost fully determined at the MF-GrC synapse with millisecond precision. Low MF-GrC weights and high activity in the inhibitory loops favor generation of singlets, while high MF-GrC weights and low activity in the inhibitory loops favors generation of doublets and triplets. The exact number of spikes emitted in specific functional contexts is not fully clear. In response to a single MF impulse, local field potentials, cell-attached and whole-cell recordings in brain slices reveal mostly singlets, while doublets and triplets become common after blocking synaptic inhibition and generating MF-GrC LTP (Mapelli and D'Angelo, 2007; Andreescu et al., 2011). In anesthetized rats *in vivo*, sensory stimulation generates short spike bursts and whole-cell recordings from GrCs show generation of a new burst (Chadderton et al., 2004; Rancz and Hausser, 2006; Duguid et al., 2012). Spike patterns reported *in vivo* in response to MF bursts show on average five EPSCs and one to three spikes in GrCs (e.g., see Duguid et al., 2012), and the composition of GrC bursts is regulated by

the inhibitory circuit. In these papers, emphasis has been put on tonic inhibition (a form of inhibition caused by ambient GABA in the cerebellar glomerulus), but the impact of the inhibitory circuit on spike timing or the duration of the discharge has not been analyzed. However, in local field potential recordings *in vivo*, an apparent reduction of response duration was observed when the inhibitory circuit was blocked and a clear anticipation of the response emerged when LTP was induced at the MF-GrC synapse (Roggeri et al., 2008; Diwakar et al., 2011), as much as it was observed in the PSTHs reported in this paper (see **Figures 4**, **5**, and **10**). Therefore, the onset and duration of spike bursts in GrCs can be regulated *in vivo* in a way consistent with that predicted here.

The importance of this quasi-digital GrC spike pattern becomes evident when considering that PC responses are differentially sensitive to the number of spikes transmitted by GrCs along the PFs. First of all, the PF-PC synapse shows a pronounced short-term facilitation, so that single PF spikes are not transmitted but transmission becomes effective with two or more spikes (Casado et al., 2000; Dittman et al., 2000). A more puzzling effect is that presynaptic LTD at the PF-PC synapse is induced by spike triplets,which are needed to unblock presynaptic NMDA receptors (Casado et al., 2000). Importantly, a precise timing is implied by the millisecond precision of PC responses in relation to movements (Timmann et al., 1999; Osborne et al., 2007).

## **GENERAL CONCLUSIONS AND IMPLICATIONS FOR CEREBELLAR NETWORK COMPUTATION**

These simulations show that distributed synaptic plasticity fully implements the *time-window* mechanism (D'Angelo and De Zeeuw, 2009) causing the emission of quasi-digital spike patterns, with differential regulation of the precision and probability of the first spike compared to that of late spikes. This, in turn, gives a specific significance to distributed plasticity,which is shown to control spike transmission much better than plasticity at a single synapse. Moreover, distributed synaptic plasticity can determine multiple activity states of the network, alternatively increasing or filtering transmitted spikes or maximizing first-spike precision and bursting. These states may be inter-converted, modified, or stabilized by exploiting biological properties of plasticity, like reversibility and

**FIGURE 10 | Effect of suggested weight configurations in the GrC response.** In these simulations the weights were set according to the four configurations reported in**Table 4**: increasing transmission (green), filtering (blue), maximize time precision (red), and maximize bursting (black). **(A)** Raster plots of the network responses to the same MF stimulation (left) with each weight configuration. Raster plots of activity recorded in the GrC (center) and GoC (right) populations with the hypothesized weight configurations (one per row), respectively. **(B)** PSTH of the GrC response after the first spike in the burst. **(C)** Relative number of GrCs generating zero, one,

two, or three spikes in response to the stimulation burst. **(D)** (Left) Relative number of GrCs generating one spike after the first spike in the stimulation burst. Average (center) and offset variance (right) of the spikes elicited by the GrCs in response to the first spike in the burst. Maximize time precision configuration noticeably reduces and anticipate the activity of the GrCs. Increasing transmission configuration enhances the activity in response to the first and second spikes in the burst. Maximize bursting configuration increases the number of doublets and triplets elicited. Finally, filtering configuration nearly avoided the GrCs firing.

consolidation. In this flexible scenario, the specific asset of synaptic weights at a given time could be strictly dependent on ontogenetic factors and on gating processes controlling network plasticity in relation to brain states (Schweighofer et al., 2004). Most of the information available on the potential mechanisms controlling the formation of plasticity in the cerebellar granular-layer network concerns the MF-GrC relay. For example, acetylcholine (Prestori et al., 2013) can gate MF-GrC LTP raising precision and bursting of GrC spiking. In response to MF bursts, nitric oxide (NO) can favor MF-GrC LTP and orchestrate the LTP/LTD balance in the surrounding circuit area (Maffei et al., 2003). A powerful organizing mechanism for LTP and LTD could also be provided by theta-frequency oscillation and resonance (Gandolfi et al., 2013). Finally, mechanisms intrinsic to the cerebellar glomerulus can raise GrC excitability under conditions of weak MF transmission or strong GoC activity (Mitchell and Silver, 2003; Rossi et al., 2006; Brandalise et al., 2012) and other mechanisms located on GoC dendrites can silence the Golgi cell following intense GrC – GoC transmission (Watanabe and Nakanishi, 2003). This latter set of synaptic and non-synaptic mechanisms could implement a homeostatic balance preventing neuronal activity from exceeding the functional limits of the network. The identification

## **REFERENCES**


of the biochemical and physiological mechanisms orchestrating this multitude of network operations and determining network learning represents a challenge for future research.

These simulations also address one major issue in cerebellar modeling. Theoretical models of the cerebellum assume that learning is driven by some optimization factors related to gain, signal-to-noise ratio, and mutual information transfer. Although some models consider only learning at the PF-PC synapse (e.g., Schweighofer et al., 1998; Medina and Mauk, 2000; Kawato et al., 2011), others implement learning (Schweighofer et al., 2001) or non-recurrent state-generation in the granular layer (Yamazaki and Tanaka, 2007). Here we show that timing and bursting could also be important parameters, which can be optimized by distributed plasticity in the granular layer. Future theoretical modeling should consider this issue.

## **ACKNOWLEDGMENTS**

This work was supported by grants of European Union to Egidio D'Angelo (CEREBNET FP7-ITN238686, REALNET FP7- ICT270434) and by grants of the Italian Ministry of Health to ED (RF-2009-1475845). We thank G. Ferrari and M. Rossin for technical support.


Prestori, F. (2013). The cerebellar Golgi cell and spatiotemporal organization of granular layer activity. *Front. Neural Circuits* 7:93. doi:10.3389/fncir.2013.00093


conductances predicted by singlechannel kinetics. *J. Neurosci.* 10, 3178.


glomerulus. *J. Neurophysiol.* 101, 3089–3099.


(2010). Long-term depression at parallel fiber to Golgi cell synapses. *J. Neurophysiol.* 104, 3413.


synaptic input. *Neuron* 67, 435–451.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 15 March 2013; accepted: 02 May 2013; published online: 22 May 2013.*

*Citation: Garrido JA, Ros E and D'Angelo E (2013) Spike timing regulation on the millisecond scale by distributed synaptic plasticity at the cerebellum input stage: a simulation study. Front. Comput. Neurosci. 7:64. doi: 10.3389/fncom.2013.00064*

*Copyright © 2013 Garrido, Ros and D'Angelo. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

## Toggling between gamma-frequency activity and suppression of cell assemblies

## *Christoph Börgers\* and Bryan Walker*

*Department of Mathematics, Tufts University, Medford, MA, USA*

#### *Edited by:*

*Julie Wall, Queen Mary, University of London, UK*

#### *Reviewed by:*

*Carmen Canavier, LSU Health Sciences Center, USA G. B. Ermentrout, University of Pittsburgh, USA*

#### *\*Correspondence:*

*Christoph Börgers, Department of Mathematics, Tufts University, Bromfield-Pearson Hall, Medford, MA 02155, USA. e-mail: cborgers@tufts.edu*

Gamma (30–80 Hz) rhythms in hippocampus and neocortex resulting from the interaction of excitatory and inhibitory cells (E- and I-cells), called Pyramidal-Interneuronal Network Gamma (PING), require that the I-cells respond to the E-cells, but don't fire on their own. In idealized models, there is a sharp boundary between a parameter regime where the I-cells have weak-enough drive for PING, and one where they have so much drive that they fire without being prompted by the E-cells. In the latter regime, they often de-synchronize and suppress the E-cells; the boundary was therefore called the "suppression boundary" by Börgers and Kopell (2005). The model I-cells used in the earlier work by Börgers and Kopell have a "type 1" phase response, i.e., excitatory input always advances them. However, fast-spiking inhibitory basket cells often have a "type 2" phase response: Excitatory input arriving soon after they fire delays them. We study the effect of the phase response type on the suppression transition, under the additional assumption that the I-cells are kept synchronous by gap junctions. When many E-cells participate on a given cycle, the resulting excitation advances the I-cells on the next cycle if their phase response is of type 1, and this can result in suppression of more E-cells on the next cycle. Therefore, strong E-cell spike volleys tend to be followed by weaker ones, and vice versa. This often results in erratic fluctuations in the strengths of the E-cell spike volleys. When the phase response of the I-cells is of type 2, the opposite happens: strong E-cell spike volleys delay the inhibition on the next cycle, therefore tend to be followed by yet stronger ones. The strengths of the E-cell spike volleys don't oscillate, and there is a nearly abrupt transition from PING to ING (a rhythm involving I-cells only).

**Keywords: gamma oscillation, feedback inhibtion, cell assembly, attentional selection, type 2 neuron**

## **1. INTRODUCTION**

Gamma-frequency (30–80 Hz) oscillations in hippocampus and neocortex are known to result, in many instances, from the interaction of excitatory pyramidal cells (E-cells) and fastspiking inhibitory interneurons (I-cells) (Whittington et al., 2000; Börgers and Kopell, 2003; Bartos et al., 2007; Traub and Whittington, 2010). Rhythms arising in this way are called Pyramidal-Interneuronal Network Gamma (PING) rhythms. The PING mechanism requires that the I-cells respond to the E-cells, but do not fire on their own; thus the drive to the I-cells must be sufficiently weak, in comparison with the drive to the E-cells. In idealized model networks, there can be a sharp boundary in parameter space between a regime in which the I-cells have weakenough drive for PING, and a regime in which they have so much drive that they fire without being prompted by the E-cells. In the latter regime, they often de-synchronize, and suppress the E-cells altogether; the boundary in parameter space was therefore called the "suppression boundary" in Börgers and Kopell (2005). [The loss of synchrony among the I-cells is the result of heterogeneity in drives (White et al., 1998), and would not be expected in a homogeneous network (Achuthan and Canavier, 2009)]. However, the transition from PING to suppression is truly discontinuous only under very idealized circumstances. We therefore replace the term "suppression boundary" by "suppression transition" in this paper. Even in networks with heterogeneous neuronal properties, this transition can be narrow (Börgers et al., 2008). Thus, a small amount of modulation of the excitability of the neurons can result in crossing from the PING regime to the suppression regime or vice versa, and therefore cause a dramatic change in network dynamics. In Börgers et al. (2005), it was explained how this mechanism could be exploited in attentional processing, turning on or off the processing of certain stimuli.

There was no gap-junctional coupling among I-cells in Börgers and Kopell (2005) and Börgers et al. (2005), even though such coupling is known to be present among fast-spiking interneurons in neocortex (Galarreta and Hestrin, 2002) and hippocampus (Fukuda and Kosaka, 2000). Furthermore, the model I-cells used in Börgers and Kopell (2005) and Börgers et al. (2005) have a type 1 phase response, i.e., excitatory input always advances them. However, fast-spiking inhibitory basket cells often have a type 2 phase response: they are delayed by excitatory input arriving soon after they fire (Tateno and Robinson, 2007, Figure 5). We study the effect on the suppression transition of introducing I-cells with type 2 phase response, and coupling them with gap junctions strong enough to keep them synchronous (Kopell and Ermentrout, 2004; Ostojic et al., 2009).

The type of the phase response is not the only notion of neuronal type in mathematical neuroscience. The bifurcation from rest to spiking is said to be of type 1 if it is a saddle-node bifurcation on an invariant circle, and of type 2 if it is a Hopf bifurcation (Rinzel and Ermentrout, 1998). The frequency-current (*f*-*I*) relation is said to be of type 1 if it has no discontinuity, i.e., if arbitrarily slow spiking is possible for drives sufficiently close to (but above) threshold, and of type 2 if it has a discontinuity at spike onset. [This last notion of neuronal type was described by Hodgkin (1948)]. The three kinds of neuronal type often coincide. For instance, the classical Hodgkin–Huxley model is of type 2 according to all three definitions, and for each of the three model neurons used in this paper, the three notions of type coincide. However, there is reason to be cautious about identifying the types of bifurcations, phase response curves, and *f*-*I* relations: Ermentrout et al. (2012) recently gave an example showing that a type 1 bifurcation can be associated with a type 2 phase response. What matters to us in this paper is the type of the phase response. When we call a model neuron "of type 1," we mean that weak excitatory inputs always accelerate it. When we call it "of type 2," we mean that weak excitatory inputs arriving early in the cycle hold it back.

If type 2 I-cells are introduced in the models of Börgers and Kopell (2005) and Börgers et al. (2005), but without gapjunctional coupling, or if the type 1 I-cells are kept, but coupled by synchronizing gap junctions, we find that the suppression transition becomes considerably less tight. However, a sharp suppression transition is restored when I-cells of type 2 and synchronizing gap-junctional coupling among them are introduced at the same time; crossing it causes a nearly abrupt transition from PING to ING (Whittington et al., 2000), i.e., to a gamma rhythm involving the rhythmic firing of I-cells only, with the E-cells suppressed. We give an analysis explaining why in the presence of synchronizing gap junctions among the I-cells, the suppression transition is much tighter with I-cells of type 2 than with I-cells of type 1.

In summary, the idea that the suppression transition may play a central role in attentional processing remains intact when the I-cells are of type 2, connected by gap junctions.

## **2. MODELS**

## **2.1. A VARIATION OF THE ERISIR INTERNEURON MODEL**

Erisir et al. (1999) proposed a model of inhibitory interneurons in mouse somatosensory cortex. We use it here because it is the simplest Hodgkin–Huxley-like interneuron model of type 2 that we know of. Because several variants of the Erisir model have appeared in the literature, and because we use a variant slightly different from any of those in the literature, we will state our equations here. The form of the differential equations is

$$C\frac{d\nu}{dt} = \mathcal{g}\_{\rm Na} m\_{\infty}(\nu)^3 h(\nu\_{\rm Na} - \nu) + \mathcal{g}\_{\rm K} n^2(\nu\_{\rm K} - \nu) + \mathcal{g}\_{\rm L}(\nu\_{\rm L} - \nu) + I,\quad(1)$$

$$\frac{d\mathbf{x}}{dt} = \frac{\mathbf{x}\_{\infty}(\nu) - \mathbf{x}}{\mathbf{t}\_{\mathbf{x}}(\nu)}, \quad \mathbf{x} = h, n. \tag{2}$$

Deviating from Erisir et al. (1999), we take the activation variable *m* of the sodium current to be a direct function of *v*. Following Erisir et al. (1999), the second power of *n* appears in the delayed rectifier potassium current, even though in the original Hodgkin– Huxley model (Hodgkin and Huxley, 1952) and almost all similar models, the fourth power appears there. The original model of Erisir et al. (1999) also included a weak, slow, depolarizationinduced potassium current, which plays no role in our discussion, and will be omitted here.

The letters *C*, *v*, *t*, *g*, and *I* in Equations (1) and (2) denote capacitance density, voltage (membrane potential), time, conductance density, and current density, respectively, measured in μF/cm2, mV, ms, mS/cm2, and μA/cm2; we will usually omit units from here on. The reversal potentials are, following (Erisir et al., 1999), *vNa* = 60, *vK* = −90, *vL* = −70. Erisir et al. specified conductances and currents; to translate to conductance and current *densities*, we assume, following Erisir et al., that the neuron is a sphere of radius 8μm. The parameter choices of (Erisir et al., 1999) then become, using the units specified above and rounding to three significant digits, *C* = 1, *gNa* = 112, *gK* = 224, and *gL* = 1*.*24. Gouwens et al. (2010) reduced the leak conductance, using a value which translates into a conductance density of approximately 0*.*5 mS/cm2; this is the value that we use here. The lowest possible firing frequency of the Erisir neuron with *gL* = 1*.*24 is quite high, about 65 Hz; with *gL* = 0*.*5, it is significantly lower, approximately 37 Hz. The gating variables *m*, *h*, and *n* are non-dimensional quantities varying between 0 and 1. The equations for *x*<sup>∞</sup> (*x* = *m, h, n*) and τ*<sup>x</sup>* (*x* = *h, n*) are

$$\begin{aligned} \alpha\_{\infty} &= \frac{\alpha\_{\mathbf{x}}}{\alpha\_{\mathbf{x}} + \beta\_{\mathbf{x}}}, \quad \mathbf{x} = m, h, n, \\ \tau\_{\mathbf{x}} &= \frac{1}{\alpha\_{\mathbf{x}} + \beta\_{\mathbf{x}}}, \quad \mathbf{x} = h, n, \\ \alpha\_{m}(\boldsymbol{\nu}) &= 40(75.5 - \boldsymbol{\nu})/(\exp((75.5 - \boldsymbol{\nu})/13.5) - 1), \\ \beta\_{m}(\boldsymbol{\nu}) &= 1.2262/\exp(\boldsymbol{\nu}/42.248), \\ \alpha\_{\boldsymbol{\nu}}(\boldsymbol{\nu}) &= 0.0035/\exp(\boldsymbol{\nu}/24.186)), \\ \beta\_{\boldsymbol{\nu}}(\boldsymbol{\nu}) &= -0.017(\boldsymbol{\nu} + 51.25)/(\exp(-(\boldsymbol{\nu} + 51.25)/5.2 - 1), \\ \alpha\_{\boldsymbol{\nu}}(\boldsymbol{\nu}) &= (95 - \boldsymbol{\nu})/(\exp((95 - \boldsymbol{\nu})/11.8) - 1), \\ \beta\_{\boldsymbol{\nu}}(\boldsymbol{\nu}) &= 0.025/\exp(\boldsymbol{\nu}/22.222). \end{aligned}$$

We have corrected a typographical error in the formula for α*<sup>m</sup>* in Erisir et al. (1999) [pointed out by (Gouwens et al., 2010)], and made a slight correction in the formula for β*h*: Erisir et al. wrote 0*.*8712 + 0*.*017*v* instead of 0*.*017*(v* + 51*.*25*)*. Up to rounding, these two expressions are equal, but if one writes 0*.*8712 + 0*.*017*v*, then β*<sup>h</sup>* has a singularity, since the denominator vanishes at *v* = −51*.*25, whereas the numerator vanishes at *v* = −0*.*8712/0*.*017 ≈ 51*.*247; we have found that this can in fact have adverse effects during simulations.

We define the "firing times" of the neurons as times at which *v* = −20 and *dv/dt <* 0. If the firing period is *T >* 0, the frequency is *f* = 1000*/T*. The factor of 1000 arises because we measure time in ms, but frequencies not in reciprocal ms, but in reciprocal s, namely, in Hz.

We compute the frequency-current (*f*-*I*) relation of the Erisir model neuron as follows. We begin with a simulation for *I* = 6, starting at *(v, h, n)* = *(*−20*,* 1*,* 0*)*. The computed trajectory converges to a fixed point which appears to be globally attracting. We then raise *I*, in steps of 0.05, from 6.0 to 7.5, starting each new simulation at the point in phase space at which the previous simulation ended. As soon as *I* rises above 7.0 (approximately), periodic spiking begins, at an onset frequency of approximately 60 Hz. The dots in **Figure 1A** indicate the spiking frequency *f* as a function of *I*. We then lower*I*, in steps of 0.05, from 7.5 to 6, again starting each new simulation at the point in phase space at which the previous simulation ended. Periodic spiking continues as *I* falls below 7, and ceases only when *I* falls below 6.5. The circles in **Figure 1A** indicate *f* as a function of *I*, as *I* is gradually lowered. For *I* approximately between 6.5 and 7, there is bi-stability: both rest and periodic spiking are possible and stable in this range.

The *f*-*I*-relation shown in **Figure 1A** is typical of a subcritical Hopf bifurcation (Strogatz, 1994). Specifically, the figure suggests that the resting state loses its stability in a subcritical Hopf bifurcation as *I* rises above approximately 7, and the stable limit cycle corresponding to periodic spiking is annihilated, likely in a saddle-node bifurcation of cycles (Strogatz, 1994), as *I* falls below approximately 6.5.

We also compute a phase response curve for the Erisir interneuron, defined as follows. Assume that *I* is large enough to allow periodic firing (above 6.5, approximately). Denote the firing period by *T*. Suppose that *(v, h, n)* = *(*−20*, h*0*, n*0*)* is the uniquely determined point on the limit cycle with *v* = −20 and *dv/dt <* 0. At time *t* = 0, we start a simulation at this point. At time ϕ*T*, with 0 *<* ϕ *<* 1, we abruptly increase *v* by 1mV; this corresponds to an instantaneous charge injection at time ϕ*T*. Denote by *T*˜ the next time when *v* = −20, *dv/dt <* 0. The phase advance

**FIGURE 1 | (A)** *f*-*I* relation for the Erisir interneuron. Dots show *f* as a function of *I* as *I* is slowly increased. Open circles show *f* as a function of *I* as *I* is slowly decreased. **(B)** Phase response of the Erisir interneuron (*I* = 7*.*2) to an instantaneous increase in the membrane potential by 1 mV. **(C)** Phase response of the Erisir interneuron (*I* = 7*.*2) with an inhibitory autapse (modeled as described in Section 2.4, with maximal conductance equal to 0*.*2) to an instantaneous increase in the membrane potential by 1 mV.

produced by the input is

$$\lg(\varphi) = \frac{T - \tilde{T}}{T}.$$

**Figure 1B** shows the phase response curve, that is, the graph of the function *g*, for *I* = 7*.*2. It is of type 2: Excitatory input received early in the cycle delays the next spike instead of advancing it. Since in our network simulations, the I-cells inhibit each other, we also compute the phase response curve for the Erisir interneuron with an inhibitory autapse, modeled as described in Section 2.4, with maximal conductance equal to 0*.*2. The selfinhibition makes the type 2 character of the phase response more pronounced; see **Figure 1C**.

**Figure 2** presents a closer look at the transition from rest to firing in the Erisir interneuron, and in particular provides strong evidence for a subcritical Hopf bifurcation. The figure shows the range 6*.*2 ≤ *I* ≤ 7*.*4. For *I* = 6*.*2, there is a single stable fixed point. **Figure 2A** tracks this fixed point as*I* rises.1**Figure 2A** shows the membrane potential at the fixed point, as a function of *I*, with blue indicating stability, and red instability. At a value of *I* near 6.5, a stable limit cycle arises; **Figure 2A** indicates the maximum and minimum membrane potentials along the limit cycle in black. The fixed point becomes unstable at a value of *I* very slightly above 7.

To confirm that the fixed point loses its stability in a Hopf bifurcation, we plot, in **Figure 2B**, the non-real eigenvalues of the Jacobi matrix at the fixed point, for the range of drives *I* in **Figure 2A**. There is indeed a complex-conjugate pair of eigenvalues that crosses the imaginary axis. The parameter value at which the crossing occurs is ≈7*.*03.

#### **2.2. THE WANG–BUZSÁKI (WB) INTERNEURON MODEL**

We compare networks in which the I-cells are the Erisir neurons presented in the previous section with networks in which they are

1Two additional unstable fixed points come into existence as *I* rises above a critical value very close to 6.3, but that bifurcation is not our interest here, and is not shown in **Figure 2**.

**FIGURE 2 | (A)** Membrane potential *v* of fixed point of Erisir interneuron, as a function of *I* (blue and red indicate stability and instability, respectively), and maximum and minimum membrane potentials along the stable limit cycle (black). **(B)** Non-real eigenvalues of Jacobian at fixed point, for values of *I* as in panel **(A)**. The crossing of the imaginary axis from the left half plane to the right occurs for the critical value of *I* at which the fixed point in A loses its stability.

Wang–Buzsáki (WB) neurons (Wang and Buzsáki, 1996), modeling fast-firing interneurons in rat hippocampus. We use the equations without any change from Wang and Buzsáki (1996). **Figure 3** shows the *f*-*I* relation and the phase response curve, without self-inhibition (panel **B**) and with self-inhibition (panel **C**). The *f*-*I* relation now indicates no region of bi-stability, and is of type 1, with a square-root-like appearance. The phase response is of type 1 as well, positive throughout.

## **2.3. REDUCED TRAUB-MILES (RTM) MODEL OF PYRAMIDAL NEURONS**

The pyramidal cell model used in this article is that of Kopell et al. (2010). We refer to Kopell et al. (2010, Appendix 1) for the details. The model is a slight variation of that of Olufsen et al. (2003), which in turn is a slight variation of that of Ermentrout and Kopell (1998), a one-compartment reduction of a model of a rat hippocampal pyramidal neuron due to Traub and Miles (1991). **Figure 4** shows the *f*-*I* relation and the phase response curve. The *f*-*I* relation and the phase response curve are of type 1.

## **2.4. NETWORKS**

We use the network model described in Kopell et al. (2010, Appendix 1). For clarity and completeness, we briefly recapitulate this model here, and we state the specific parameters used in our simulations.

We denote by *NE* the number of E-cells, and by *NI* the number of I-cells. For our larger networks (**Figures 5**, **6**, and **9**), *NE* = 160 and *NI* = 40. We also report on numerical experiments with networks of one E- and one I-cell (**Figure 7**). The drive to each cell of the network is constant in time. We denote the drive to the *i*-th E-cell by *IE,i*, and the drive to the *j*-the I-cell by *II,j*. The values of these drives will be varied; see Results.

**FIGURE 3 | (A)** *f*-*I* relation for the Wang–Buzsáki (WB) interneuron. Dots show *f* as a function of *I* as *I* is slowly increased. Open circles show *f* as a function of *I* as *I* is slowly decreased. The dots and the circles coincide for this model. **(B)** Phase response of the WB interneuron (*I* = 1*.*0) to an instantaneous increase in the membrane potential by 1 mV. **(C)** Phase response of the self-inhibiting WB interneuron (*I* = 1*.*0, maximal inhibitory conductance = 0.2) to an instantaneous increase in the membrane potential by 1 mV.

Each synapse is characterized by a synaptic gating variable *s* associated with the presynaptic neuron, 0 ≤ *s* ≤ 1, with

$$\frac{ds}{dt} = \rho(\nu) \frac{1-s}{\mathfrak{r}\_R} - \frac{s}{\mathfrak{r}\_D},$$

where ρ denotes a smoothed Heaviside function, ρ*(v)* = *(*1 + tanh*(v/*4*))/*2, *v* denotes the presynaptic membrane potential, and τ*<sup>R</sup>* and τ*<sup>D</sup>* are the rise and decay time constants, respectively. To model the synaptic input from neuron *i* to neuron *j*, we add to the right-hand side of the equation governing the membrane potential *vj* of neuron *j* a term of the form *gijsi(t)(vrev* − *vj)* where *gij* denotes the maximal conductance associated with the synapse, *si* denotes the gating variable associated with neuron *i*, and *vrev* denotes the synaptic reversal potential. For excitatory, AMPA-receptor-mediated synapses, we use τ*<sup>R</sup>* = 0*.*1, τ*<sup>D</sup>* = 3, and *vrev* = 0; for inhibitory, GABAA-receptor-mediated synapses, τ*<sup>R</sup>* = 0*.*3, τ*<sup>D</sup>* = 9, and *vrev* = −80.

In comparison with many of the values reported in the literature, our choice of *vrev* is low. Hyperpolarizing reversal potentials of GABAA-receptor-mediated inhibition have been reported, for instance, in Wang and Buzsáki (1996), Connors et al. (1988, Table 1), Sanchez-Vives and McCormick (1997), and Traub et al. (1996). Higher (sometimes much higher) reversal potentials have been reported by others; see for instance McCormick (1989); Vida et al. (2006); Bartos et al. (2007); Gouwens et al. (2010), but also Bregestovski and Bernard (2012). We have not systematically investigated the effect of a higher reversal potential on our conclusions. Powerful inhibition often seems to be needed for an abrupt suppression transition. However, inhibition can be sufficiently powerful for several reasons: low reversal potential, strong inhibitory conductance, or relatively low excitability of the post-synaptic cells.

Connectivity is all-to-all in our networks. (Sparse, random connectivity would not yield substantially different results. The randomness would add heterogeneity, with effects similar to those of heterogeneity in external drives.) The value of *gij* depends only on the *types* (E or I) of neurons *i* and *j*. For instance, we denote by *gEI* the value of *gij* when the *i*-th neuron is an E-cell, and the *j*-th neuron is an I-cell. Parameters *gIE*, *gII*, and *gEE* are defined similarly. We scale these parameters with network size: *gEI* = ˆ*gEI/NE*, *gIE* = ˆ*gIE/NI*, *gII* = ˆ*gII/NI*, *gEE* = ˆ*gEE/NE*. We choose parameters similar to those in Börgers and Kopell (2005): *g*ˆ*EI* is so strong that a population spike volley of the E-cells promptly triggers one of the I-cells (but not much stronger), and *g*ˆ*IE* is significantly stronger: *g*ˆ*EI* = 0*.*2, *g*ˆ*IE* = 0*.*8, *g*ˆ*II* = 0*.*2, *g*ˆ*EE* = 0. The value of *g*ˆ*IE* is varied in **Figure 8**.

In some of our networks, the I-cells are also gap-junctionally coupled. The *i*-th and *j*-th I-cells are gap-junctionally connected with probability 1*/*5. If *vi* and *vj* are the membrane potentials of the two cells, we add to the right-hand side of the equation for *vi* the term *ggap(vj* − *vi)*, and to the right-hand side of the equation for *vj* the term *ggap(vi* − *vj)*, with *ggap* = 0*.*8; this ensures that the I-cells remain synchronous when there are gap junctions.

Each figure in this paper was generated by a stand-alone Matlab code, available from the first author upon request.

## **3. RESULTS**

#### **3.1. SIMULATION RESULTS FOR LARGE NETWORKS**

**Figure 5** shows spike rastergrams resulting from simulations in which the mean drive to the I-cells, *II*, increases linearly with time. The drive to the I-cells is heterogeneous, here and in all of our large network simulations; actual drive to the *j*-th I-cell is

$$I\_{I,j} = \left(0.85 + \frac{j - 1/2}{40} \times 0.30\right) I\_I, \quad 1 \le j \le 40.4$$

The simulations are 1000 ms long, and *II* varies from 0 to 2 for WB neurons, and from 6 to 8 for Erisir neurons. The horizontal axis shows *II*. Panel **(A)** of the figure illustrates the "suppression boundary" as described in Börgers and Kopell (2005). The I-cells are WB neurons here [in Börgers and Kopell (2005), they were theta neurons, which are also of type 1], and there is [as in Börgers and Kopell (2005)] no gap-junctional coupling among them. The abrupt cessation of gamma oscillations and suppression of the E-cells when the mean drive to the I-cells exceeds (approximately) 0.9 indicates the crossing of the suppression boundary.

Panels **(B)** through **(D)** of **Figure 5** illustrate what happens when the WB interneurons are replaced by Erisir interneurons, or gap-junctional coupling among I-cells is introduced, or both. When only one of those two changes is made, the suppression transition broadens considerably, with a fairly large intermediate regime of cycle-skipping emerging (panels **B** and **C**). When both changes are made at the same time, however, one returns to an abrupt suppression transition (panel **D**). In the presence of gap junctions (panels **C** and **D** of **Figure 5**), the I-cells do not de-synchronize after they suppress the E-cells. The gap junctions together with the I→I-synapses synchronize the I-cells in spite

junctions. **(B)** I-cells are Erisir neurons, and there are no gap junctions. **(C)** I-cells are WB neurons, with gap junctions. **(D)** I-cells are Erisir neurons, with gap junctions.

of the fact that the external drive to the I-cells is heterogeneous (Kopell and Ermentrout, 2004).

In **Figure 5**, all E-cells receive the same constant drive, *IE* = 2. This, of course, is not realistic. When the network is less perfect, for instance when different E-cells receive different amounts of drive, the suppression transition becomes broader. This is illustrated by **Figure 6**, which is analogous to **5**, but with drive to the E-cells uniformly distributed between 1.85 and 2.15: The *i*-the E-cell receives drive

$$I\_{E,i} = 1.85 + \frac{i - 1/2}{160} \times 0.30, \quad 1 \le i \le 160.5$$

Panels **(A)** through **(C)** of **Figure 6** show a transition regime in which stronger and weaker E-cell population spike volleys alternate somewhat erratically. Panel **(D)** of **Figure 6** shows results of a simulation in which the I-cells were gap-junctionally coupled Erisir neurons. In this case, the transition regime is much narrower. It is also more orderly: the number of suppressed E-cells increases monotonically with *II*. We discuss these two points in greater detail in sections 3.2 and 3.3.

## **3.2. WITH GAP-JUNCTIONS, I-CELLS OF TYPE 2 PRODUCE A TIGHTER SUPPRESSION TRANSITION THAN I-CELLS OF TYPE 1**

Our main goal in this section is to explain why the suppression transition is gradual in **Figure 5C**, but sudden in **Figure 5D**. In these simulations, the E-cell population is tightly synchronous because there is no heterogeneity in drive to the E-cells, and the I-cell population is tightly synchronous because of gapjunctional coupling. Much about these simulations can therefore be understood by thinking about networks consisting of just one E-cell and one I-cell.

If one couples an RTM neuron with a WB neuron, there is typically a fairly broad range of drives, *II*, to the I-cell for which the E-cells skip every second cycle; see **Figure 7A** for an example. **Figure 7B** shows the inter-spike intervals of the I-cell in this example: they alternate between longer and shorter intervals. On one cycle of the I-cell, the E-cell fires immediately prior to the I-cell. Because the excitatory synaptic currents have a positive decay time constant, the excitation resulting from the E-cell spike lingers slightly beyond the I-cell spike and into the next I-cell cycle. Recall that the WB neuron has a type 1 phase response curve; see **Figure 3**. The next I-cell spike is thereby advanced, and this can result in suppression of the E-cell on the next cycle.

**Figures 7C,D** show a similar experiment for a two-cell network consisting of an RTM neuron and an Erisir neuron. For the value of *II* = 7*.*27 shown in the figure, there is no cycleskipping, and the network settles into an oscillation with a period of approximately 26 ms. If *II* is raised from 7*.*27 to 7*.*28 (not shown in the figure), the E-cell is suppressed altogether, and the period of the I-cell falls to approximately 23 ms. Because the I-cell is of type 2, the input from the E-cell, by lasting for a few milliseconds beyond the spike of the I-cell, does not advance the next I-cell spike; it delays it. Thus if the I-cell is unable to suppress the E-cell on a given cycle, it will be *delayed* on the next cycle, and is therefore even less able to suppress the E-cell on the next cycle. As a result, the E-cell is either suppressed on all cycles, or on none.

In **Figure 8**, we show further results of simulations for twocell networks, illustrating the transition from gamma frequency firing to suppression of the E-cell, and in particular the effect of

*IE* = 2 and *II* = 1. **(B)** Inter-spike intervals of the I-cell in panel **(A)**, alternating between short ones (following a spike of the E-cell) and long ones (following

neuron (blue), with *IE* = 2 and *II* = 7*.*27. The E-cell would be entirely suppressed if *II* were 7*.*28. **(D)** Inter-spike intervals of the I-cell in panel **(C)**.

varying the strength of inhibition. Here we plot the number, *fE*, of E-cell spikes in 1000 ms (that is, the E-cell frequency in Hz) as a function of *II*, with *gIE* = 0*.*5 (blue), 0.8 (black), and 1.1 (red). The figure confirms that there is an abrupt suppression transition when the I-cell is an Erisir neuron (panel **B**), but not when it is a WB neuron (panel **A**).

In **Figure 8**, note that an increase in *II* typically causes either an abrupt drop in *fE*, or a slight increase. The reason for the increase in *fE* with increasing *II* is that the enhanced drive to the I-cells makes them fire earlier on each cycle, thereby allowing the E-cell to fire earlier on the next cycle, unless there is a change in entrainment pattern, and the E-cell is suppressed on more cycles than before. When there is a change in entrainment pattern as a result of an increase in *II*, the frequency *fE* drops abruptly.

## **3.3. WITH GAP-JUNCTIONALLY COUPLED I-CELLS OF TYPE 2, THE DYNAMICS IN THE TRANSITION REGIME ARE MORE REGULAR THAN WITH I-CELLS OF TYPE 1**

There are seemingly irregular sequences of strong and weak E-cell spike volleys in **Figure 6C**, while much more regular behavior is seen in **Figure 6D**. We now discuss this difference. The behavior in **Figure 6C** becomes clearer when one fixes the mean drive, *II*, to the I-cells in the transition regime. For illustration, **Figure 9** shows a simulation similar to that of panel **(C)** of **Figure 6**—WB interneurons, gap junctions, heterogeneous drive to the E-cells but with a fixed drive of 0.9 to the I-cells. We see a rather irregular sequence of stronger and weaker E-cell population spike volleys.

To understand this irregularity, we reduce the network dynamics to a one-dimensional map, examining how the strength of a given E-cell spike volley depends on the strength of the previous volley. We denote by *sk* the number of E-cells that fire on the *k*-th gamma cycle in **Figure 9**. In **Figure 10A**, we have plotted the pairs *(sk, sk*<sup>+</sup>1*)* for the simulation of **Figure 9**, run over 10,000 ms. If *sk <* 160, then *sk*<sup>+</sup><sup>1</sup> can be deduced from *sk*, approximately at least:

$$s\_{k+1} = \mathfrak{g}(s\_k),$$

and the figure suggests that *g* has exactly one fixed point *s* <sup>∗</sup>, which is unstable: *g (x*∗*) <* −1. If *sk* = 160, then *sk*<sup>+</sup><sup>1</sup> is not determined by *sk*. Instead, *sk*<sup>+</sup><sup>1</sup> then depends on the precise placement of the inhibitory spike volley in the *k*-th cycle, relatively to the excitatory one. If the inhibitory spike volley comes early, then the following I-cell spike volley comes so early that it largely or completely suppresses the E-cell spike volley; see **Figure 10B**. If the inhibitory spike volley comes late, then the following I-cell spike volley comes so late that it is ineffective at suppressing the E-cell spike volley on the next cycle; see **Figure 10C**.

To construct a figure analogous to **Figure 10A** for Erisir interneurons is more difficult. There is no transition regime in which the behavior is irregular, and therefore simply running a long simulation does not produce many pairs *(sk,sk*<sup>+</sup>1*)*. However, we describe now a method for constructing an idealized version of **Figure 10A**, and this method generalizes to the case of Erisir interneurons, leading to a better understanding of the difference between **Figures 6C** and **6D**.

We start the simulation of the network, at time *t* = 0, in a point, *X*0, in phase space chosen so that a full population spike volley of the E-cells is imminent within a few milliseconds. Exactly how *X*<sup>0</sup> is defined is largely irrelevant. We obtain it from the simulation of **Figure 9** by recording all dependent variables 4 ms prior to the 5th population spike volley of the E-cells. (All E-cells happen to participate in that volley.) Within a few milliseconds, at time *t* = *t*<sup>0</sup> *>* 0, we re-set the phase space variables associated with the I-cell population to a point *Y*<sup>0</sup> to force an immediate population spike volley of the I-cells. Again, it is largely irrelevant exactly how *Y*<sup>0</sup> is defined. We obtain it from the simulation of **Figure 9** by recording all dependent variables associated with the I-cells at the onset of the 4th population spike volley of the I-cells.

Depending on the choice of *t*0, the first E-cell spike volley may be suppressed partially or completely. The number, *s*1, of E-cells

firing soon after time *t* = 0, 0 ≤ *s*<sup>1</sup> ≤ 160, is a function of *t*<sup>0</sup> ≥ 0; see **Figure 11A**. We then record the number, *s*2, of E-cells firing in the second E-cell spike volley, and plot it as a function of *t*<sup>0</sup> (**Figure 11B**), and also *s*<sup>2</sup> as a function of *s*<sup>1</sup> (**Figure 11C)**. Note that **Figure 11C** is strikingly similar to **Figure 10A**.

The procedure used to generate **Figures 11A–C** can be applied to generate analogous figures for a network in which the I-cells are Erisir interneurons; see **Figures 11D–F**. The mean external drive to the I-cells in **Figures 11D–F** is *II* = 7 (this is near the lower end of the transition regime, see **Figure 6D**). Since the simulation with *II* = 7 does not produce any full E-cell population spike volleys, it cannot be used to initialize the network so that a full Ecell population spike volley is imminent; we therefore obtain the points *X*<sup>0</sup> and *Y*<sup>0</sup> just as described earlier, but based on a preliminary calculation with the reduced value *II* = 6*.*5, for which the E-cells do fire full spike volleys.

Comparing **Figures 11C,F**, we see that *s*<sup>2</sup> is a decreasing function of *s*<sup>1</sup> when the I-cells are WB neurons, but an increasing function of *s*<sup>1</sup> when the I-cells are Erisir neurons. The difference in monotonicity is the crucial point here. It is a reflection of the difference in the types of the phase response curves: for type 1 interneurons, the firing of a greater number of E-cells accelerates the inhibitory response, reducing the number of E-cells firing on the next cycle; for type 2 interneurons, the opposite is true.

An iteration of the form *sk*<sup>+</sup><sup>1</sup> = *g(sk)*, where *g* = *g(s)* is a smooth function defined for 0 ≤ *s* ≤ 160 with 0 ≤ *g(s)* ≤ 160, may not have a stable fixed point when *g* is decreasing, but it does have at least one stable fixed point when *g* is increasing. So the mapping from *s*<sup>1</sup> to *s*<sup>2</sup> has an attracting fixed point when the interneurons are of type 2, but not necessarily, and in particular not in the example shown in **Figure 11C**, when they are of type 1. This is why type 2 interneurons yield periodic behavior (E-cell spike volleys of a steady size) after a transient, whereas type 1

**FIGURE 10 | (A)** The strength, *sk*+1, of the *(k* + 1*)*-st spike volley as a function of the strength, *sk* , of the *k*-th, for the simulation of **Figure 9**, run over 10,000 ms. (**B** and **C**): Closeups of **Figure 9**: What happens after a full E-cell spike volley depends sensitively on when, relative to the E-cell spike volley, the I-cell spike volley occurs.

neuronal index.

interneurons may produce more complicated, possibly chaotic dynamics in the transition regime.

## **3.4. NETWORKS WITHOUT GAP-JUNCTIONAL COUPLING AMONG THE I-CELLS**

Simulations similar to that of **Figure 5A** were studied, for networks of theta neurons, in Börgers and Kopell (2005). Recall that the drive to the I-cells is heterogeneous; *II* denotes the mean drive to the I-cells. With increasing *II*, increasingly many I-cells have enough drive to fire without being prompted by the E-cells. Once enough I-cells are in this group, the E-cells are suppressed altogether. In a network of cells coupled by inhibitory chemical synapses, but not by gap junctions, heterogeneity of drives typically prevents synchronization (White et al., 1998); this is why the I-cells in **Figure 5A** become asynchronous as soon as they begin firing without being prompted by E-cell spike volleys.

This discussion suggests that even in **Figure 5A**, considering that different I-cells receive different drives, the transition from PING to suppression of the E-cells should not be abrupt. Rather, one would expect that in an intermediate regime, some I-cells (the more strongly driven ones) are asynchronously active without being prompted by the E-cells, while others (the less strongly driven ones) only fire in synchronous volleys immediately following E-cell spike volleys. This is in fact so, but not visible in **Figure 5A** because the intermediate regime is quite narrow, and the rising *II* passes through it rapidly.

**Figure 5B** is in some regards similar to **Figure 5A**. The transition regime in which some I-cells fire without being prompted by the E-cells, but the E-cells still fire population spike volleys occasionally, is now much broader, and clearly visible. The most striking difference between **Figures 5A,B** is that in **Figure 5B**, some I-cells are completely suppressed even when the mean drive to the I-cells gets strong, whereas the same is not true in **Figure 5A**. We have not attempted a theoretical explanation of this difference; it seems natural to hypothesize that it is due to the difference in the types of the *f*-*I* relations of WB and Erisir neurons.

**Figures 6A,B** are similar. The suppression transitions are softened considerably because of heterogeneity in drive to the E-cells. With Erisir interneurons, one sees a *broader* suppression transition than with WB interneurons. This is opposite to what is seen in the presence of gap junctions. We have no explanation of this effect. In particular, our arguments based on the phase response of the I-cells do not apply when the I-cells are not kept synchronous by gap junctions: once the I-cells de-synchronize, they receive excitatory input from the E-cells at all phases, not just at the early phases at which type 2 I-cells are delayed by such inputs.

## **4. DISCUSSION**

The fast-spiking inhibitory basket cells believed to be central in the formation of gamma rhythms are in fact gap-junctionally connected (Beierlein et al., 2000). There is direct experimental evidence that they have type 2 phase response curves (Tateno and Robinson, 2007, Figure 5). In addition, there are reports that they also have type 2 frequency-current relations, i.e., that their firing starts at a non-zero frequency (Beierlein et al., 2003; Tateno et al., 2004; Tateno and Robinson, 2006, 2009). Further, there are several papers documenting resonance properties of fast-spiking interneurons; e.g., (Pike et al., 2000). These are properties usually associated with a Hopf bifurcation, and a type 2 phase response curve. It therefore appears that the biologically most relevant case is that of gap-junctionally connected I-cells with a type 2 phase response curve—precisely the case that yields the narrowest and most orderly suppression transition.

In this paper, we have discussed the loss of gamma rhythms due to too much excitation of the I-cells. By contrast, a recent paper of Börgers et al. (2012) investigated the loss of gamma rhythms due to too little excitation of the I-cells. Moca et al. (2012) discussed a different way in which the type of the interneurons may be important in gamma oscillations: they found that resonance properties of the interneurons, associated with bifurcation type 2, may contribute to stabilizing the gamma frequency.

All numerical experiments in this paper have been for "strong PING," that is, for PING oscillations in which participating E-cells fire at or near gamma frequency. By contrast, "weak PING" oscillations are noise-driven, and individual participating E-cells fire on a small, randomly selected fraction of cycles only. In Börgers et al. (2005), it was suggested that weak PING might be associated with general alertness or vigilance, while strong PING might be a model of a cell assembly in a state of actively processing a specific item. We would expect our main conclusion, that I-cells with type 2 phase response curves result in a tighter suppression transition, to hold for weak PING as well. The reason is simply that for type 2 I-cells, the spiking of a few E-cells promotes the spiking of other E-cells on the next cycle, whereas for type 1 I-cells, it may make it harder for other E-cells to fire on the next cycle.

Our discussion suggests that the competition among E-cells associated with PING oscillations [see for instance (Olufsen et al.,

## **REFERENCES**


model. *Proc. Natl. Acad. Sci. U.S.A.* 105, 18023–18028.


2003; Börgers et al., 2008)] is less fierce when the I-cells have a type 2 phase response. We think that this is true only in a narrow time window: an E-cell that lags behind others by just a millisecond or two can fire more easily when the I-cells are type 2 than when they are type 1, since the firing of the E-cells that are ahead delays the firing of the I-cells on the next gamma cycle. In other words, I-cells of type 2 may allow for less tightly synchronous PING assemblies. However, even with I-cells of type 2, a cell assembly can suppress a less strongly driven competitor if the difference in drive is just slightly greater.

In summary, we have found that in the presence of gap junctions, when the I-cells are of type 2, the suppression transition tends to be both narrower and more orderly than when the Icells are of type 1. It is tempting to speculate that fast-spiking inhibitory basket cells might have evolved to have type 2 phase response curves precisely because that leads to clean suppression transitions, reducing the amount of modulation of local recurrent inhibition needed to turn gamma frequency cell assemblies off or on, an operation that seems likely to be crucial in brain function.

## **ACKNOWLEDGMENTS**

The work of Christoph Börgers is supported in part by the Collaborative Research in Computational Neuroscience (CRCNS) program through NIH grant 1R01 NS067199. Bryan Walker was supported by Tufts University through the 2011 Summer Scholars program, and by NIH grant 1R01 NS067199 through a 2012 summer stipend.

high-frequency firing of fastspiking neocortical interneurons. *J. Neurophysiol.* 82, 2476–2489.


gamma oscillations. *Cereb. Cortex* doi: 10.1093/cercor/bhs293. [Epub ahead of print].


dorsal lateral geniculate nucleus thalamocortical neurons *in vitro*. *J. Neurosci.* 17, 8880–8893.


*the Hippocampus*. Cambridge: Cambridge University Press.


Buhl, E. H. (2000). Inhibitionbased rhythms: experimental and mathematical observations on network dynamics. *Int. J. Psychophysiol.* 38, 315–336.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 19 December 2012; accepted: 25 March 2013; published online: April 2013. 16*

*Citation: Börgers C and Walker B (2013) Toggling between gamma-frequency activity and suppression of cell assemblies. Front. Comput. Neurosci. 7:33. doi: 10.3389/fncom.2013.00033*

*Copyright © 2013 Börgers and Walker. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

## Rapid, parallel path planning by propagating wavefronts of spiking neural activity

## *Filip Ponulak1,2\* and John J. Hopfield2,3*

*<sup>1</sup> Brain Corporation, San Diego, CA, USA*

*<sup>2</sup> Department of Molecular Biology, Princeton University, Princeton, NJ, USA*

*<sup>3</sup> Institute for Advanced Study, Princeton University, Princeton, NJ, USA*

#### *Edited by:*

*Cornelius Glackin, University of Hertfordshire, UK*

#### *Reviewed by:*

*Christian Leibold, Ludwig Maximilians University, Germany Dougal Tervo, Howard Hughes Medical Institute, USA*

#### *\*Correspondence:*

*Filip Ponulak, Brain Corporation, 5665 Morehouse Drive, San Diego, CA 92121, USA e-mail: filip.ponulak@ braincorporation.com*

Efficient path planning and navigation is critical for animals, robotics, logistics and transportation. We study a model in which spatial navigation problems can rapidly be solved in the brain by parallel mental exploration of alternative routes using propagating waves of neural activity. A wave of spiking activity propagates through a hippocampus-like network, altering the synaptic connectivity. The resulting vector field of synaptic change then guides a simulated animal to the appropriate selected target locations. We demonstrate that the navigation problem can be solved using realistic, local synaptic plasticity rules during a single passage of a wavefront. Our model can find optimal solutions for competing possible targets or learn and navigate in multiple environments. The model provides a hypothesis on the possible computational mechanisms for optimal path planning in the brain, at the same time it is useful for neuromorphic implementations, where the parallelism of information processing proposed here can fully be harnessed in hardware.

**Keywords: path planning, navigation, parallel processing, mental exploration, wave propagation, spike timing dependent plasticity, hippocampus, neuromorphic systems**

## **AUTHOR SUMMARY**

Humans and animals can quickly and reliably solve spatial navigation and path planning tasks. However, neural mechanisms underlying these processes are not completely understood. Discovery of, so called, place cells—the hippocampal cells getting activated whenever an animal enters a certain spatial location gave rise to the idea that the hippocampus contributes to the creation of internal, neural representations of the environment. Here we demonstrate that spatial navigation can rapidly be solved in the hippocampus-like neural network by parallel mental exploration of alternative routes. A possible biological mechanism to implement parallel exploration is through propagating waves of neural activity spreading across the entire network representing a given environment. We present a model, where such waves of spiking activity alter synaptic connectivity through spike-timingdependent plasticity and create a vector field, which can guide an animal through the environment to selected target locations. In a set of computational experiments we demonstrate that planning can be solved during a single wavefront passage through the network. Moreover, the model is capable of suggesting an optimal solution for multiple competing targets, and it can embed multiple environments for trajectory planning.

## **INTRODUCTION**

One of the central problems for neurobiology is to understand the computational effectiveness of the brains of higher animals. Brains rapidly carry out extraordinary feats of visual scene analysis or problem solving through thinking on "wetware" that is tens of millions times slower than modern digital hardware. Part of the explanation is brute-force *anatomical* parallelism.

In this paper we develop a model of parallel computational processing in the context of path planning and spatial navigation. We propose that spatial navigation can be solved through simultaneous mental exploration of multiple possible routes. A typical mental exploration task for an animal might involve knowing an extensive terrain containing a few water sources, being motivated (being thirsty) to seek the nearest water source. Hopfield (2010) recently described a way that serial mental search for a useful route could be done by a moving clump of activity and synapse modification in a hippocampus-like neural network <sup>1</sup> . We show here that a best path can rapidly be found by *parallel* search in the same kind of network, but by a propagating wave of spiking activity. The process of path planning and navigation, as proposed in our model, consists of the following steps: (1) expanding waves of neural activity are initiated from the place cells corresponding to selected target location(s); (2) the propagating waves alter synaptic connectivity within the network through spike-timingdependent plasticity and create a directed synaptic vector field (SVF) converging on the goal locations; (3) this vector field is used by an animal to navigate toward targets; (4) whenever a new planning process is necessary, all synapses are reset to the baseline state and waves of activity can be initiated from the new target locations.

Can animals employ such parallel mental exploration to solve novel problems? Indeed can humans do so? Recent electrophysiology experiments demonstrated existence of expanding, traveling waves of neural activity in the hippocampus, associated with

<sup>1</sup>We use the term "clump of activity" to refer to a localized neural activity of a cluster of place cells representing nearby spatial locations.

theta-oscillations (Lubenov and Siapas, 2009; Patel et al., 2012), as well as with much faster sharp wave ripples (Ellender et al., 2010), yet, no link between such waves and spatial planning has been shown so far.

One of the major roles of theory is to elucidate interesting consequences and possibilities inherent in our incomplete experimental knowledge of a system. The fact that hippocampuslike neural substrate can support parallel mental exploration, as explored here, is such a possibility. New experimental paradigms could easily test for parallel mental exploration in rats. These ideas also form the basis for novel neuromorphic circuits in engineering, which could be used to implement effectively certain Artificial Intelligence algorithms such as those based on the idea of a wave-front propagation (Dorst and Trovato, 1988; Dorst et al., 1991; LaValle, 2006) by taking advantage of the true parallelism of the neuromorphic hardware systems (Boahen, 2005; Misra and Saha, 2010).

## **RESULTS**

We consider like (Hopfield, 2010) a network of excitatory "place" cells for a very simple model animal. Through experience in an environment, each cell has learned synaptic connections from a sensory system (not specified here) that make it respond strongly only when the model animal is near a particular spatial location. These response place fields are our modeling equivalent of the response place fields observed in the rodent hippocampus (O'Keefe and Dostrovsky, 1971). For display purposes, the activity of each place cell can be plotted at the spatial location of the center of the receptive field corresponding to that place cell. In such a display there is a localized activity clump surrounding the actual spatial position of the model animal. When the animal moves, this activity region follows the location of the animal. If an animal wanders throughout an environment over an extended time, the synaptic plasticity will result in excitatory synaptic connections being made only between cells that are almost simultaneously active (Hebb, 1949). If the exploration process is not systematically directional and is extensive, connections will on average not have directionality. The CA3 region of the hippocampus has such intra-area excitatory connections with the requisite spike-timing-dependent plasticity, or STDP (Amaral and Lavenex, 2006).

The fundamental neural network to be studied is thus a sheet of place cells, each having excitatory connections to the others with centers within its receptive field footprint, but not to distant neurons. Experimental support for the existence of such connections (direct or indirect) comes from the coordinated phase-change-like response of place cells, trained in two environments, experiencing a visual environment that mixes the two environments (Wills et al., 2005).

The model neurons considered in our study are of the integrate-and-fire type with a short dead-time and spikefrequency adaptation (implementation details are provided in the Methods section at the end of the paper).

We investigate whether and how the described setup can implement parallel search for optimal pathways in the environment represented by the neural network. Because we rely on simulations of a system whose mathematics we cannot fully analyze, it is sensible to present a line of argument that develops insight about expected behaviors. Consider a simplified model comprising of a line of neurons, each reciprocally connected to its two nearest neighbors (cf. **Figures 1A**,**B**). With specific parameter settings, a single spike can initiate an activity pattern that consists of a pair of spikes marching from the initiation site toward the ends of the line at constant speed, one in each direction (Aertsen et al., 1996). In a system with intrinsic neuronal adaptation, there is a dead time before another pair can be propagated in this same region.

A similar phenomenon can be observed also in a twodimensional sheet of neurons with recurrent local connections over a small but extended region. In an example presented in **Figures 2A**,**B**, the synaptic connection strengths are chosen so that a few pre-synaptic cells must spike almost simultaneously to fire the post-synaptic cell. Seeded with a few approximately synchronized firings of nearby cluster of neurons, a propagating circular wavefront of activity is observed in which each neuron fires only once (Kumar et al., 2008). A second wavefront cannot be initiated in a region that the initial wavefront has traversed until the adaptation has decayed (cf. **Figures 2C**,**D**). Note, that although in our model we consider a single-spike activity, the basic activity events propagated through the network may in principle also consist of short bursts of spikes, which is biologically more realistic in the context of the hippocampal cell activity.

Propagating wavefronts can have profound effects on synaptic modifications through STDP. Consider again a one-dimensional network as illustrated in **Figure 1**. Any non-symmetric STDP rule will produce, in one dimension, synaptic change patterns that display whether the "front" of activity that went by was going toward the left or toward the right. Normal or "forward" STDP which enhances synapses at which the pre-synaptic spike comes before the post-synaptic spike will result in rightwardgoing synapses being stronger than leftward-going synapses if the wavefront passes moving to the right (**Figure 1A**). "Reverse" or "anti-" STDP which enhances synapses at which the pre-synaptic spike comes after the post-synaptic spike (Bell et al., 1997; Kampa et al., 2007; Roberts and Leen, 2010) will result in leftward-going synapses being stronger than rightward-going synapses if the wavefront passes moving to the right (**Figure 1B**). The same basic idea intuitively extends to two dimensions, where STDP results in synaptic change that can be interpreted as a vector field (in the following we shall call it a synapse vector field or SVF), showing the orientation of the propagating wavefront that caused the synaptic change. In all our simulations we use reverse STDP induced by propagating spike wavefronts that creates an SVF pointing toward the center (initial point) of the waves. Our use of reverse STDP is motivated by certain conceptual and technical advantages of this approach over regular STDP, as it will be described later in the paper.

## **SIMPLE PATH PLANNING PROBLEM**

Consider for definiteness the "T" shape environment shown in **Figure 2A**. We presume that by exploring the environment, each neuron has acquired a place field such that it is driven strongly only when the simulated animal is near the place field center and the drive to the cell falls off smoothly away from that

location. For display purposes, in all figures the cells are arranged so that whatever property of the cell is being plotted, its (*x,y*) plot location is the location of its place field center. The receptive fields considered in our experiments are assumed to have Gaussian shapes and to cover 25–50 cells in their footprints in a simulation using a network with 2000 place cells. In such a setup, if an animal explores an environment, synapses with simple STDP will form strong connections between neurons with similar place fields, i.e., between neurons that are close together. To this point, the general approach is like that previously used in Hopfield (2010).

Imagine that the simulated animal, in exploring an environment, finds a target T, such as a source of water, to which it may later want to return. Let the dendrites of the place cells in the vicinity of T become connected to axons from an "exciter" which, when activated, can briefly drive these place cells to fire. Such activation will result in an outgoing wave of single spike activity emanating from T as center as illustrated in **Figure 2A** (where the cyan field represents the T location). This wave will spread until every neuron has fired an action potential. As noted before, the next wavefront is possible only after the neural adaptation fades away. Also, to prevent runaway, we use a global inhibitory mechanism, where inhibition is proportional to the network activity, resulting in a balanced excitation-inhibition (for more details we refer to the Methods section).

The propagating wave and the asymmetric synaptic plasticity implicitly define a vector field, which represents the local direction of the wavefront, i.e., the vector is normal to the wavefront and points in the direction of propagation. We should define what is meant by "shortest path" or more generally "optimal path" for present purposes. While the synapse vector field is defined only at the discrete locations of place cell centers, the synapses themselves will be used to control the continuous motion of an animal in real space. The discreteness of the place cell representation will contribute fine-scale noise in the actual physical path. The optimality we are interested in is macroscopic optimality for example, choosing the right way to go around an obstacle. The physical pathlength contribution introduced by jitter from the discreteness of the neural representation is not of interest. Before the single-spike activity wave was initiated, all directions were equivalent, and the SVF was zero everywhere. Afterward there is a local directionality, because the timing of pre-post spike pairs depends on the spatial separation of the pairs projected on the direction of wavefront propagation. Sample SVFs that result from the anti-STDP rule are shown in **Figures 3A**–**D**. Here the vector fields are illustrated using directed arrows originating from the preferred locations of each place cell in the network. The direction and the length of each arrow represent, respectively, the direction and the strength of the vector field in a given location (see Methods for details).

The SVF can be used for finding the shortest pathway to the location being the source of the propagated wave. Intuitively, since the first wave to arrive at your position comes via the fastest path, if you simply backtrack, always going backward along this vector field, you will reach the target by the shortest path. In either case, the synapse vector field contains the information necessary to find the shortest route to the target. It is merely a question of following the vector field forward (anti-STDP) or backward (STDP).

## **OBTAINING MOTOR COMMANDS FOR FOLLOWING THE SYNAPTIC VECTOR FIELD**

For illustrating an idea of how the SVF can be used for guiding an agent (a simulated animal or a robot) movement, we return to the one-dimensional case (**Figure 1C**). In 1 dimension, if the propagating wavefront has passed by locations *k* through *(k-n)* while moving leftward, and the anti-STDP rule has been applied, rightward-directed synapses [e.g., *(k-1)* → *k*] are strengthened more than leftward ones (*k* → (*k-*1)). Before this process, if the animal was located at a particular location in space, a bump of place cells would be active, symmetrically located around that location. In the presence of the asymmetric synapse modification, the bump of activity is biased and no longer centered on the actual physical location (**Figure 1C**; cf. Levy, 1989; Blum and Abbott, 1996). This bias can be converted into a motor command proportional to the bias and pointing toward the direction of a wavefront passage.

Precisely the same problem occurs in earlier work on a hippocampus-like model of actions based on "thinking" (Hopfield, 2010). In that model there were two clumps of activity, one representing the present position of the animal and the other representing where the animal thought it should be a short time later. The difference between the locations of these two clumps was used to produce motor commands that moved the animal toward the desired future location. The model was fully implemented with spiking neurons.

Since the task of generation of motor command is not the major focus of our paper, here we use a simplified approach. Namely, we assume that a receptive field corresponding to the present animal location is activated by applying tonic excitation to the corresponding place cells and then any place cell firing a spike causes a pulse of force moving an agent toward the preferred location of that cell. The asymmetry in the weight configuration around the receptive field results in a higher probability of firing of those adjacent place cells that are located along a direction of a vector field. As a consequence an agent moves to a spatial new location along the optimal pathway. The details of the algorithm are provided in the Methods section at the end of the paper.

Sample movement trajectories resulting from applying the described procedure to a simulated animal are shown in **Figures 3E**,**F** (see also **Movies S1** and **S2** in Supplementary materials). These trajectories result from the SVFs illustrated in **Figures 3A**,**C** and **Figures 3B**,**D**, respectively. In particular, **Figure 3F** illustrates the shortest path aspect of the available information—because the target is located above the midline, the wavefront arrives at the branch containing the animal at S from above before the wavefront from below (cf. **Figures 2B**, **3B**). Neural adaptation prevents the wavefront arriving from below from penetrating this region. Thus, the SVF leads to a route from S to T going upward.

Notwithstanding the fact that the algorithm used here is not providing details on the possible neural implementation of action execution, it is important to emphasize that the actions are triggered by individual spikes and hence each spike contributes to the agent behavior change. The average population activity pattern determines the mean movement trajectory along the vector field, whereas the particular spikes add some stochasticity to the behavior (reflected, e.g., in a small trial-to-trial variability of the movement pathways observed in **Figures 3E**,**F**). Such stochasticity has some advantage in certain situations. For example, it may

show details of the vector fields around the bifurcations in the simulated mazes. **(E,F)** Typical movement trajectories observed in the considered models resulting from the vector fields from **(A,B)**, respectively. The trajectories begin in points "S" and end in the target locations "T." For additional results see also **Movie S1** in supplementary materials.

be useful for avoiding local minima, or for selecting one choice when several alternatives have equal probability.

## **NAVIGATION IN AN ENVIRONMENT WITH MULTIPLE TARGETS AND VALUES**

Several different relevant targets might be simultaneously available in an environment. For simplicity, the case when all targets have the same intrinsic value is first considered. **Figure 4A** shows the SVF that results when single spike propagating circular waves simultaneously originated at three targets. Because the singlespike wavefront cannot propagate into a region that another wavefront has recently traversed, any subregion is therefore traversed by only a single wavefront, the one that arrives first, and is thus closest to its source. Within that subregion, the vector field is the same as it would have been if only the source responsible for the traversing wave had been present. The three subregions of the three possible targets of **Figure 4A** are shown in **Figure 4B** (compare to **Movie S2** in the Supplementary materials). Which target is nearest, and thus should be navigated to, depends on the current location of the agent. The same figure illustrates the paths followed for three possible initial agent locations. Note that the SVF is defined everywhere, independent of the location of the agent when the wavefront is generated.

When multiple targets are present, an optimal choice will involve balancing the cost due to the length of a path and the reward that will result if that path is followed. For a single target, the net reward due to following a path of length *L* is *R–CL*, where *C* is the cost per unit length of following any path, and optimizing net reward simply minimizes *L* over the set of possible paths. When multiple targets of equal value are present, the same net reward expression applies, but the set of relevant paths over which a minimum is sought includes paths to each possible target. Accordingly, if the targets all have equal value, the described procedure selects the target that can be reached by the shortest possible route.

**FIGURE 4 | Navigation in a system with multiple targets. (A)** Synaptic vector field created in the network with targets in locations Tl, T2, T3. **(B)** Typical movement trajectories observed in the system for the initial agent locations as indicated by spots Sl, S2, S3. The path selection and the path shapes are determined by the shape of the vector field and by the initial agent location's. The vector field has three basins of attraction corresponding to the particular targets—the bounds of the basins of attraction are indicated by the gray dotted lines. For additional results see also **Movie S2** in supplementary materials.

Now suppose that different possible targets *Tk*, *k* = 1, have different rewards *Rk*. When all wavefronts propagate with the same velocity, it is useful to think in terms of times rather than lengths. *Rk* then can be seen as an effective shortening of the time to navigate to a reward. A simple way to implement it is to initiate wavefronts first at the target locations corresponding to greater rewards and later at the locations with lower rewards. The introduction of these differential delays represents the value differentials between the various targets. These delays shift the boundaries of the regions such as those of **Figure 4B** in a way that represents the differing values of the target. The optimal relative initiation times can be learned on the basis of maximizing the term (*Rk*–CL). For any winning target, the path followed to that target is the same as would have been used if that target alone were present.

#### **DEALING WITH NOISE**

Noise can adversely affect the ability of the network to propagate a wavefront in the ideal fashion to set up the desired synaptic field. **Figure 5** illustrates what can happen when noise is severe. Spurious single spikes are generated, and spikes can fail to occur. When spurious spikes cluster, they can serve as initiation sites for new circular waves centered at locations where there is no target. In addition, spurious and absent spikes cause irregular wavefront propagation or even wavefront extinction.

The major noise issues concern setting up the SVF. Once it is set up, the motor control system effectively averages over the vector field in a small region, and noise in following the SVF is not a major issue.

Having a large system is the first defense against noise. As the system size grows, the number of neurons making synapses onto a particular cell, which must be simultaneously active to initiate a spike in that cell, can be increased, and the likelihood of spurious single spikes decreases. The likelihood of a spatial cluster of spurious spikes being large enough to trigger a new wavefront is also reduced. There is considerable latitude for exploiting the large number of cells available in real neurobiology.

There are also cellular means to suppress the effect of noise. Set the threshold for spike generation at some particular level, and consider the ability of *N* cells connected to this one to trigger it to spike when a passing wavefront goes by. There will never be exact synchrony in the firing of the presynaptic cells, so while *N* cells firing may typically be required to fire the cell, less than *N* may also sometimes do so, and more than *N* may fail to fire it. Reliable wavefront propagation is enhanced by any biophysical effect that sharpens this threshold on *N*. One way to sharpen this threshold is by determining whether a particular neuron in a network is excited by spikes coming from a small number of neurons being unusually effective (for example because of noise), or by a larger number of neurons with typical effectiveness. A method of making this distinction can be implemented in a biologically realistic way by using supra-linear spatial summation, a phenomenon observed in biological neural circuits (Nettleton and Spain, 2000; Urakubo et al., 2004). In our work we use a simple phenomenological model of such a supra-linear integration that favors weak excitation from multiple inputs over strong excitation from a few inputs. This is achieved through a non-linear summation of synaptic input currents to the neuron, such that the effectiveness of presynaptic spikes is increasing with a number of simultaneously active inputs to the neuron (see Methods section for details). Although, in this algorithm the appropriate setting of the neuron activation threshold is still important, it is no longer a critical factor for the problem at hand. With this approach more emphasis is put on how many presynaptic neurons are active simultaneously, rather than how strong the particular connections are. In this way the algorithm works better than the threshold algorithm for networks with greater heterogeneity of synaptic connection strengths.

#### **NAVIGATION IN MULTIPLE ENVIRONMENTS**

When a rat is familiar with multiple environments, a particular hippocampal neuron can have place fields in more than one environment, with no apparent coordination between them (Bostock et al., 1991; Wilson and McNaughton, 1993). We also therefore investigate whether our network model can learn and effectively perform navigation in multiple environments when each neuron has a place field in each environment. When the place cells in one-environment and place cells for a second environment are

target. In addition, spurious and absent spikes cause irregular wavefront propagation or even wavefront extinction as illustrated in **(B,C)**. Network activity shown at times as indicated. The noise is modeled by injecting spike currents to randomly selected neurons at random time steps.

uncorrelated, the synaptic connections needed in both environments can be simultaneously present. If the number of neurons is sufficiently large, when the sensory signals come from one environment there is little crosstalk between the representations of both environments, and the presence of the second set of synapses simply inserts a modest level of noise. One can similarly anticipate that single spike wavefronts can be initiated and will propagate in any particular environment when multiple environments are known. The wavefronts will produce a vector field that can later be used to guide the animal in this particular environment. This is a significant extension, for without it each neuron needs to be specific to a single environment, which would be both inefficient and not in correspondence with biology.

Consider a network that is supposed to operate on two different environments as illustrated in **Figure 6**. Due to their shapes we call these environments "A" and "∞." While in the rat many place cells would be specific to one environment, such specificity reduces the crosstalk between the environments, and de-emphasizes the crosstalk effect we wish to evaluate. Here, however, we assume that each place cell represents the animal's locations in both environments.

A spike generated in any cell will produce excitatory postsynaptic potentials in all its neighbor cells in one environment and all its neighbor cells in another environment. As in the previous experiment, the model parameters are set such a single spike cannot cause action potentials in the postsynaptic neurons. As before, supra-linear summation helps to promote stable propagation of the existing wavefronts, and to prevent single, isolated spikes from producing new wavefronts.

Consider a network activity caused by the simultaneous excitation of a certain set of the topologically nearby cells in the environment "A." When a plot is made with each cell located at the preferred location it represents in environment "A," the dynamics of this neural activity will be seen as a wave propagated through the network (**Figure 6A**, left). The same activity observed from the perspective of the "∞" environment (that is by reorganizing the network by putting place cells at the locations they represent in the "∞" environment) would appear as a random network activity (**Figure 6A**, right). Since the spikes observed in the "∞" environment appear sparse, they are unlikely to initiate a wavefront in this representation. Similarly, at any particular moment while a wavefront in the "A" environment is propagating, the synaptic connections representing the "∞" environment introduce drive to neurons that should not be driven at that moment. Occasionally such neurons can produce crosstalk-induced spurious spikes (cf. solitary spikes in the left panel in **Figure 6A**, occurring far away from the wavefront).

**Figures 6B**–**D** illustrates that at the level of two environments and around 2000 place cells, there is little effect of crosstalk on the ability to function in each environment as though the other did not exist. **Figure 6B** (left) shows that the SVF induced by a wavefront initiated at T (cf. **Figure 6A**, left) develop as expected, representing a flow back toward the target from all points in the "A" environment. **Figure 6B** (right) shows the SVF for the same synaptic changes, but calculated for the place cell locations in the "∞" environment. Here the vectors point in random directions because there is no spatial organization to the

(**B**, left) Synaptic vector field resulting from the propagation of a wavefront illustrated in (**A**, left). Note a single attractor corresponding to the location T, that is the center of the wavefront. (**B**, right) The synapse vector field due to the same synapse changes as in (**B**, left), but calculated using the positions of the neurons in the "∞" environment. **(C,D)** The same plots as for **(A,B)** except that the wavefront has been initiated at target T in the "∞" environment. All results are qualitatively like those in **(A,B)**, except that the roles of the two environments are reversed. Synaptic vector fields in plots **(B,D)** are visualized using the same normalization factor (arrow scale) for both environments.

synapse change in this representation. The same kind of result is obtained when the wavefront is initiated in the "∞" environment as in **Figures 6C**,**D** with the roles of the two environments reversing. In each case the vector field created by the single-spike wavefront successfully navigates an animal from a starting point in the given environment to the target as illustrated in **Figure 7**.

## **DISCUSSION**

The problem of planning and executing a complex motion over a protracted time-period which will optimally take an autonomous agent from its present location and configuration to a desired target location and configuration is common to both animal behavior and robotics. In its simplest manifestation there is only a single target, a single known environment, and a short or fast path is preferred over a longer or slower one. The trajectory planning must accommodate the physical constraints posed by the environment. Additional complexities might include the simultaneous presence of multiple targets, possibly of different intrinsic values, terrain which affects the value of trajectories in a non-trivial fashion, and multiple environments.

The neurally-inspired network presented in our work has been shown to solve the planning problem in several steps. First, in an exploratory phase it learns an environment by developing a set of "place" cells whose locations reflect all possible trajectory boundaries due to kinematic constraints or constraints in the behavior arena. It develops in this exploration process interconnections between all pairs of places that can be visited in temporal contiguity, and thus can be possible candidates for a section of a trajectory. Second, given the expected set of synaptic connections, the excitation of a target location (or locations) initiates a wavefront of single spike- or single burst activity that propagates outward from the initiation site(s). The wave propagation process is terminated when a wavefront reaches the present location of the agent. The passage of such a wavefront produces synapse modification pattern that can be described as a vector field. The desired trajectory is simply the path along the SVF from the present location to the (or a) target. Since the SVF lines are produced by an expanding circular wavefront, they converge when followed backward toward a source, and thus provide stable guidance for going to a target location.

The full extent of the parallelism available in our concept is perhaps best illustrated in **Figure 4**. The system simultaneously selects the closest target and the best route to that target from

respectively. Three different trials for each environment are illustrated. The trajectories start in the S locations and end in the T locations.

a single propagation of the exploration wave. Conventionally, a best path would be found for each target sequentially, using a serial algorithm to rate possible paths, and a choice of target then made between these optimal single-target paths. The conceptualization of the parallel search method and the demonstration by simulation that best trajectories can be followed in neuromorphic simulation are the major accomplishments of this paper.

## **NETWORK ANALYSIS**

As mentioned before, the goal of our paper was to present a concept of parallel exploration through propagating waves of neural activity and STDP-altered SVFs. We have illustrated our concept in a set of simulations, but we have not attempted to quantify our results. An interesting extension of our work thus would be to perform an analysis of the properties of our system. Interestingly, such an analysis has recently been offered for the network proposed in (Hopfield, 2010), which is of the same type and topology as the one considered in our work. Indeed, Monasson and Rosay (2013) provided an indepth theoretical analysis of the dynamics and storage capacity of that network as a function of such parameters as: network size, level of neural activity, level of noise, or size of place cells. Specifically, using the statistical mechanics tools, the authors analysed conditions necessary for the network to learn multiple maps (environments). The storage of a map manifests itself through the fact that the neural activity is localized, and acquires a clump-like shape in the corresponding environment. Remarkably, according to the analysis performed by the authors, a moderate level of noise can slightly increase the capacity storage with respect to the noiseless case. However, when the number of environments or the noise are too high the neural activity cannot be localized any longer in any one of the environments. For high noise, the activity, averaged over time, becomes uniform over space. For high loads the activity is not uniform, but is delocalized with spatial heterogeneities controlled by the cross-talks between the maps. The paper provides quantitative results for the transition between these states. The authors also analyse storage capacity of the network, that is a maximum number of environments for which a stable representation of a given environment can still be retrieved, as a function of network size and topology. For the network of the type considered in (Hopfield, 2010), and so also in our work, the storage capacity is proportional to the network size and is estimated to be of the order of 10−<sup>3</sup> bits per synapse (for the 2 dimensional space representation and under the optimal conditions). Interestingly, these results are consistent with an earlier analysis for a network with a similar topology but with a different neuron type given in Battaglia and Treves (1998).

#### **RELATED MODELS**

The wave-propagation concept has first been introduced by Dorst and Trovato as an efficient parallel method for path planning (Dorst and Trovato, 1988; Dorst et al., 1991) and since then has widely been used in robotics and computer science (LaValle, 2006). The wave-front methods are essentially the same as exhaustive or heuristic versions of a classical A<sup>∗</sup> search algorithm (Dijkstra, 1959; Hart et al., 1968) of whose optimality is proven. Several neural models for spatial navigation using the concept of propagating waves have been proposed so far (for reviews see, e.g., Lebedev et al., 2005; Qu et al., 2009). However, only a few models addressed a question on how the propagating neural activity can be transformed into an appropriate configuration of synaptic connectivity able to later guide an agent to a target location (Roth et al., 1997; Gorchetchnikov and Hasselmo, 2005; Qu et al., 2009; Ivey et al., 2011). To the best of our knowledge, our model is the first one to demonstrate that biologically plausible, temporally asymmetric synaptic plasticity rules can achieve this goal. Also, most of the previous models assumed multiple trials for learning a complete set of optimal paths for every new selected target location. In contrast, in our model, once an agent becomes familiar with an environment, a single passage of an activity wavefront through the network is sufficient to create a SVF guiding an animal from any possible location in the experienced environment to a target location. Interesting enough, such an ability of animals to rapidly replan routes if the starting and goal points are changed to new, random locations within a known environment has recently also been observed experimentally (Pfeiffer and Foster, 2013).

## **BIOLOGICAL RELEVANCE**

Parallel exploration as proposed in our model requires mechanisms that support stable propagation of expanding waves of neural activity throughout the network. Conditions for such stable propagation of spiking activity in biological neural circuits have been examined both theoretically (Diesmann et al., 1999; Kumar et al., 2008, 2010) and experimentally (Reyes, 2003; Wu et al., 2008; Nauhaus et al., 2012). Recent electrophysiological results suggest existence of expanding waves of neural activity in the hippocampus during, so called, sharp wave ripple (SWR) episodes (Ellender et al., 2010). Sharp wave ripples are brief highfrequency bursts of neural activity observed during sleep or at awake rest (Buzsaki, 1986). Hippocampal SWRs are frequently accompanied by sequential reactivation of place cells occuring in the same- or reverse temporal order as previously experienced during behavior, but replayed at a compressed time scale (Pavlides and Winson, 1989; Wilson and McNaughton, 1994; Foster and Wilson, 2006). Interestingly, reactivation patterns observed in the awake animals are not always just a simple function of experience (Gupta et al., 2010), and have also been reported to represent trajectories never directly or fully experienced by an animal, suggesting a possible role of the awake SWRs in planning, navigation or decision making (Pastalkova et al., 2008; Buhry et al., 2011; Foster and Knierim, 2012; Singer et al., 2013). These results point to the awake-state SWRs as a possible biological candidate process for parallel mental exploration as required in our model. Moreover, it has been suggested that the SWRs provide optimal conditions for the activation of synaptic plasticity processes, such as STDP (Sadowski et al., 2011)—which, again, is consistent with our assumption that a propagating wave of neural activity should be able to modify connectivity within the network in order to create structured SVFs.

The SVFs are in turn used in our model to guide behavior. Indeed we assume that the movement of an agent (an animal) is guided by the activity of places cells surrounding the present agent location. Therefore, the problem is to generate motor forces which will bring into better alignment two "bumps" of neural activity, one coming from the sensory system representing the actual location of the agent, and the other clump of neural activity having a location biased by the modified synapses. In our paper, this problem is solved by a mathematical algorithm (cf. Methods). However, neurophysiological experiments suggest that the same problem can also be solved by a biological neural network, for it is isomorphic to the problem of moving the two eyes so that the image of one bright spot is centered on both fovea (Ohzawa et al., 1990, 1997). A relatively inefficient but fully neural solution to this two-bump problem was given in (Hopfield, 2010).

As mentioned already, generation of directed connections for SVFs requires asymmetric STDP rules. Such asymmetry in the STDP learning windows has been found in the synaptic connections between hippocampal cells, first in cultured cells (Bi and Poo, 1998) and more recently also in slice preparations (Aihara et al., 2007; Campanac and Debanne, 2008).

"Anti-" or "reverse-" STDP, in which a pairing of a presynaptic spike that precedes a post-synaptic spike *decreases* the strength of a synapse (Bell et al., 1997; Kampa et al., 2007), was used in our model to produce the SVF. There are two important reasons for why "normal" (or "pro") STDP cannot be used in the model. If parameters are set in the fashion of (Hopfield, 2010) so that a clump of activity, once initiated by sensory input, is stable when sensory input is removed, that clump of activity will move, following the vector field. Thus, when the "anti" sign is used, the agent can rehearse mentally the chosen trajectory from its present location to the chosen goal. It could even, with slight elaboration, communicate a sequential list of way points. Such a natural behavior of mental rehearsal in sequential order from the starting point is not available with "pro" STDP, for the clump of activity in this case moves away from the target. Initiating a clump of activity at the target location does not create an equivalent in reverse order because the vector field diverges from that point. Another advantage of using anti-STDP over STDP is apparent for navigation in the presence of neural noise or external perturbation (physical forces pushing the agent away from the original path). When using anti-STDP, flow field lines converge when looking toward the source of the expanding circular wavefront that generated the field. When following in this direction, nearby vector field lines all converge toward the same destination, so noise is attenuated by the following process and has little effect. When following away from a source, as would be the case for normal STDP, vector field lines diverge, the effect of a noise error is amplified, and effects of noise accumulate.

Our model assumes that whenever a new planning process is necessary, all synapses are reset to the baseline state and waves of activity can be initiated from the present target locations to create new SVFs. There are several candidate phenomena observed in the nervous system that could potentially realize the necessary resetting mechanism. One hypothesis, that seems to have both theoretical and experimental support, is that the population bursts during sharp wave ripples could serve this task by desynchronizing neurons through STDP (Mehta, 2007; Lubenov and Siapas, 2008). If this is the case indeed, the SWR episodes in our model would need to serve both tasks: memory erasing (hypothetically during the synchronious activation of populations of neurons) and formation of new memories (during the reactivation). To the best of our knowledge though, no such double-function of the SWR has been reported in the experimental literature so far. Another hypothetic mechanism for resetting synaptic connectivity in the hippocampus is through the neuromodulators. For example Bouret and Sara (2005) point to the role of noradrenaline in reorganizing the network structure in a way necessary for memory erasing.

We recognize that not all mechanisms proposed in our work have experimental support from the studies on hippocampus. Hence, biological relevance of our model remains hypothetical. Nevertheless, we believe our approach is useful as a conceptual model, laying grounds for efficient parallel neural computation for navigation and path planning.

#### **OUTLOOK**

Our model can be usefully expanded in many ways. As mentioned before, different costs can be associated with the particular pathways or spatial locations through the uneven distribution of place cells and/or uneven distribution of strength of synaptic connections. This will affect the speed and the shape of the particular wavefronts, and consequently will determine the boundaries of the basins of attraction and best path within each basin.

Giving an animal the ability to actively control the speed of the wavefront propagation through the different regions of the network would provide a way to encode certain features of the environment in the path planning algorithm. Imagine that there is a cost associated with a certain path, e.g., an animal has to go through a "hazardous" area. This cost can be represented in the network through relatively weaker or "shorter" connections between neurons along this path. As a consequence, a wavefront will have a lower velocity when propagating through the place cells associated with this path, making the choice of this pathway less likely. Another possible way to dynamically control the local speed of the wavefront propagation as a function of environmental features, is by enabling interactions of the mental map considered in our present model with other mental maps, each one encoding for different features of the same environment. In this case, mental selection of particular path planning criteria (for example, "find the shortest/the fastest/the safest path") would activate interactions between the "path planning map" and the appropriate feature maps. These interactions could be implemented through the local excitatory or inhibitory feedback loops between the "path planning" map and the selected "feature maps," triggered by the propagating wavefront and resulting in the local changes of neuronal excitability, and so of the wavefront propagation speed in the "path planning map."

In our model we use place cells distributed uniformly, having a single spatial scale, and a simple place field in each of several separate environments. None of these are literally true in the hippocampus. However, by being an oversimplified idealization, it has allowed an exploration of rapid computational possibilities in a network that perhaps over-represents space, and seems a profligate use of neurons. An interesting extension of our work could be a hierarchical model, where space (or more generally memories) would be represented by different groups of neurons at different levels of abstraction.

Several recent studies suggest that the hippocampus can encode memories at multiple levels of "resolution," from a detailed rendition of specific places or events within a single experience, to a broad generalization across multiple environments or experiences (Steinmetz et al., 2011; Komorowski et al., 2013). Indeed, when we think about our own experience, we seem to be using a context-dependent switching between different representations of space. For example, when we plan to drive from our present location to another place in a town, we typically only focus on specific points in space when decisions about further route need to be taken (e.g., "turn left or turn right")—at this point we typically don't think about the details of a highway we drive on, but rather on "when and where to turn or what exit to take." To the contrary, when we need to change a lane on a highway, we quickly switch to the "high-resolution" local map and we use a spatial map of our surround to navigate between other cars and objects. A similar mechanism could be used in an extension of our model to increase efficiency of the implementation and to reduce the demand on resources (number of neurons), without compromising performance and robustness of computation.

From the application point of view our neural model can be extended to the path planning problems in systems with more than two dimensions or in tasks with extra constraints, such as, e.g., non-holonomic navigation, arm movement planning. Our model, as a particular implementation of the wavefront expansion algorithm, can also be used for solving variety of optimality problems from other domains than motor control (Dorst et al., 1991; LaValle, 2006).

#### **METHODS**

The place cell models considered in the paper have been simulated using adapting leaky integrate and fire neurons. The dynamics of the neuron models between spikes are defined by the following formula:

$$\begin{split} \pi\_m \frac{d\boldsymbol{u}\_m(t)}{dt} &= -\left(\boldsymbol{u}\_m(t) - \boldsymbol{u}\_r\right) \\ &+ R\_m(\dot{\boldsymbol{i}}\_{\text{sens}}(t) + \dot{\boldsymbol{i}}\_{\text{syn}}(t) + \dot{\boldsymbol{i}}\_{\text{ns}}(t) - \dot{\boldsymbol{i}}\_{\text{inh}}(t) - \dot{\boldsymbol{i}}\_{\text{Ca}}(t)), \end{split} \tag{1}$$

$$
\pi\_{\rm Ca} \frac{d i\_{\rm Ca}(t)}{dt} = -|i\_{\rm Ca}(t), \tag{2}
$$

where *um(t)* is the membrane potential, τ*<sup>m</sup>* = *Cm*R*<sup>m</sup>* is the membrane time constant, *Cm* = 1 nF and *Rm* = 20 M are the membrane conductance and resistance, respectively, *ur* = 0 mV is the membrane potential at rest, *i*sens(*t*) is the sensory input, *i*syn(*t*) is a sum of the currents supplied by the particular excitatory synapses entering the given neuron, *i*ns(*t*) is the non-specific background current modeled as a gaussian process with zero mean and variance 5 nA , *i*inh(*t*) is the global inhibitory current, *i*Ca(*t*) represents a neuron-specific inhibitory current that could be caused by calcium-activated potassium channels in real neurons.

The neuron produces an instantaneous action potential when *um*(*t*) reaches a threshold of 10 mV, and then *um*(*t*) is reset to 0 and held at that value for 2 ms to produce an absolute refractory period. Each action potential produced by the neuron allows for a momentary burst of calcium (Ca2<sup>+</sup>) ions to flow into the cell (through high-potential Ca2+ channels) and increments *i*Ca(*t*) upward. Calcium ions also leak out, with a characteristic time τ*Ca* usually set at 1–5 s. Because *i*Ca(*t*) and the internal Ca2<sup>+</sup> ion concentration of the neuron are proportional, the adaptive effect can be written in terms of the variables *i*Ca(*t*), and the cellular internal Ca2<sup>+</sup> concentration is needed only to understand a possible mechanism of spike-frequency adaptation. The timescale of adaptation is set by the size of increment to *i*Ca(*t*) that occurs when a neuron spikes.

For the calculation of the total synaptic currents *i*syn(*t*) injected into the particular neurons we use a supra-linear spatial summation model (Nettleton and Spain, 2000; Urakubo et al., 2004). The model favors a near simultaneous activation of a neuron from multiple presynaptic neurons over the activation from a single neuron. This approach is supposed to decrease the probability of initiating random wavefronts arising from isolated spikes in the noisy network. The model for supralinear summation used in our simulations is described by the following equation:

$$i\_{\rm syn}(t) = a\_{\rm syn} = \tanh\left(b\_{\rm syn} \sum\_{j} H\left(i\_{j}(t)\right)\right) \sum\_{j} w\_{j}(t) i\_{j}(t), \quad (3)$$

where *ij*(*t*) is the synaptic current of the *j*-th input; *wj*(*t*) is the synaptic strength of the *j*-th input; *H(x)* is the step function [*H(x)* = 1 for *x >* 0 and *H(x)* = 0 for *x* = 0]; *a*syn and *b*syn are the positive constants. The particular synaptic currents *ij*(*t*) rise instantaneously and decay exponentially with a 25 ms time constant. The supralinear summation function given by Equation 3 is illustrated in **Figure 8**.

Sensory currents *i*sens(*t*) for each place cell are modeled as having an isotropic Gaussian form around the center of the receptive field for that cell, with the same width and strength for each neuron. When modeling multiple environments, each cell has a receptive field in each environment, assigned randomly.

It is assumed that the modeled network contains a set of inhibitory interneurons whose function is to limit the total activity of the network. Because the inhibitory feedback is assumed to be global, and because this essential function is computationally trivial, its effect is modeled in a continuous fashion and using global variables rather than by using spiking interneurons. Hence the dynamics of inhibitory population are given by the following equations:

$$\text{tr}\_{\varepsilon} \frac{d i\_{\varepsilon}(t)}{dt} = -i\_{\varepsilon}(t) + a\_{\varepsilon} \sum\_{j} \sum\_{f} \\$\left(t - t\_{j}^{f}\right). \tag{4}$$

$$\begin{cases} A\_{\text{inh}}(t) \propto (i\_{\text{\textell}}(t) - I\_{\text{\textell}}) & \text{if} \quad i\_{\text{\textell}}(t) > I\_{\text{\textell}}, \\ A\_{\text{inh}}(t) = 0, & \text{otherwise.} \end{cases} \tag{5}$$

The variable *ie*(*t*) represents the input current to the inhibitory population from all excitatory cells in the network, whereas *A*inh(*t*) reflects the activity of the inhibitory population. According to (4) the current *ie*(*t*) decays with a time constant τ*<sup>e</sup>* and is incremented by *ae* by each individual spike fired at time

is defined by: *i*syn(*t*) = *<sup>j</sup>* w*<sup>j</sup>* i*<sup>j</sup>* (*t*). Here, for the supralinear function we took *a*syn = 10, *b*syn = 0*.*05, and for both functions we assumed *wj* i*<sup>j</sup>* (*t*) = 1 for all *j*.

*t f <sup>j</sup>* (with *f*-being the label of the spike) by any excitatory neuron *j* in the network. The parameters τ*<sup>e</sup>* and *ae* are positive and constant; a Dirac function δ(.) is defined as: δ(*t*) = 0 for *t* = 0 and  δ(*t*)*dt* = 1. According to (5) the population activity *A*inh(*t*) is proportional to the current *ie*(*t*) with a firing threshold *Ie*0. Given the activity *A*inh(*t*), the global inhibitory feedback *i*inh(*t*) to every excitatory neuron in the network is assumed:

$$i\_{\rm inh}(t) = a\_{\rm inh} A\_{\rm inh}(t),\tag{6}$$

where *a*inh is a binary gating variable. The gating variable *a*inh is set to 1, and accordingly the inhibition is active, during the network exploration or during the navigation task; whereas *a*inh = 0 and the inhibition is deactivated during the wavefront propagation.

A fully connected network with excitatory connections has been assumed in all simulations, with all network connections being initially silent. A typical size of the simulated networks varied from 2000 to 4000 place cells in the particular experiments. The simulations were carried out using an Euler integration of the differential equations and a 0.2-ms time step.

#### **SYNAPTIC PLASTICITY**

Synaptic connections have been altered according to the STDP model described by the following equation [cf. Kempter et al. (1999)]:

$$\frac{d\boldsymbol{w}\_{ji}(t)}{dt} = \boldsymbol{a} + \boldsymbol{d} \left[ \mathbf{S}\_i(t) \int\_0^\infty \boldsymbol{a}\_{ij}(s)\mathbf{S}\_j(t-s)ds + \mathbf{S}\_j(t) \int\_0^\infty \boldsymbol{a}\_{ji}(s)\mathbf{S}\_i(t-s)ds \right],\tag{7}$$

where *wji*(*t*) is the synaptic coupling from neuron *i* to neuron *j*, *a <* 0 is the activity-independent weight decay, *Si*(*t*) and *Sj*(*t*) are the pre- and postsynaptic spike trains, respectively. A spike train is defined as: *S(t)* = *<sup>f</sup>* δ *t <sup>f</sup>* − *t* , where *t <sup>f</sup>* is the *f*-th firing time. The terms *aij*(*s*) and *aji*(*s*) are the integral kernels, with *s* being the delay between the pre-and post-synaptic firing times *s* = *t f <sup>i</sup>* − *t f j* . The kernels *aij*(*s*) and *aji*(*s*) determine the shape of the STDP learning window. In our model we use exponential functions given by (8) to describe the STDP curve, however, other shapes are also possible.

$$\begin{cases} a\_{\vec{\mu}}(-s) = +A\_{\vec{\mu}} \cdot \exp\left(s/\mathfrak{r}\_{\vec{\mu}}\right) & \text{if} \quad s \le 0, \\\ a\_{\vec{\eta}}(s) = -A\vec{\mu} \cdot \exp\left(-s/\mathfrak{r}\_{\vec{\eta}}\right) & \text{if} \quad s > 0, \end{cases} \tag{8}$$

Here, *Aji*, *Aij* are the amplitudes and τ*ji*, τ*ij* are the time constants of the learning window. In our model we assume that *Aji > Aij >*0 and τ*ji* = τ*ij >* 0. The parameter *d* in (7) controls the polarity of the STDP process and can be linked to the concentration of specific neuromodulators known to be able to change the polarity of the synaptic plasticity in biological synapses (Seol et al., 2007). For simplicity, in our model *d* = {−1*,* 0*,* 1}. We assume that during the environment exploration phase *d* = 1, and consequently the synaptic connections undergo STDP with a positive net effect (because *Aji > Aij*). During the wavefront propagaton phase: *d* = −1 and accordingly the synaptic connections are altered by the reversed STDP rule. No synaptic plasticity is assumed during the movement execution phase (*d* = 0).

#### **SYNAPTIC VECTOR FIELD ILLUSTRATION**

In **Figures 4**, **5**, **8** we present sample SVFs created by the propagating activity wavefronts. These vector fields are illustrated using directed arrows originating from the preferred locations of each place cell in the network. The direction and the length of each arrow represent, respectively, the direction and the strength of the vector field in a given location. Here we describe an algorithm used to illustrate the vector field.

For each neuron *ni* in the network consider a set *Nji* of all neurons *nj* on which *ni* makes direct synaptic projections. Now for the neuron *ni* we define a vector *ri*(*t*):

$$r\_{\bar{i}}(t) = \sum\_{j} w\_{\bar{j}i}(t) \left(\chi\_{\bar{j}} - \chi\_{\bar{i}}\right) / \sum\_{j} w\_{\bar{j}i}(t),\tag{9}$$

We assume that the vector *ri*(*t*) begins in the preferred location *xi* of place cell *ni* and ends in a center of gravity of the preferred locations *xj* of the neighboring place cells *nj* ∈ N*ji*, weighted by the corresponding connection strengths *wji*(*t*).

#### **EXPLORATION ALGORITHM**

An exploration procedure was used to establish a set of synaptic connections appropriate to the topology of a particular environment, based on earlier work (Hopfield, 2010). The trajectory followed was a noisy straight line with constant speed, with a directional persistence length of the same scale as the largest dimension of an environment. The trajectory made a specular bounce when it encountered a wall. During this exploration the place cells had sensory inputs according to their spatial receptive fields. Place field centers were assigned on a regular grid, with Gaussian noise around those locations. Prepost synaptic spike pairs were accumulated for each intra-place cell synapse during the exploration. The potential for synapse change was evaluated over these spike pairs with a weighting function *dwji*(*t*)*/dt* = exp −|*ti* − *tj*|*/*τ*<sup>e</sup>* and used to select which synapses should be established. In the equation, *wji*(*t*) is the strength of the synaptic equation from a presynaptic neuron *i* to a postsynaptic neuron *j*; *ti* and *tj* are the firing times of the pre- and postsynaptic neuron, respectively; τ*<sup>e</sup>* is the learning time constant. When the exploration is finished, each place cell *j* was given incoming synapses of the same size to the set of *m* neurons with the largest values of weights *wji*.

This procedure is insensitive to the details. Since any trajectory could be traversed in either direction, it will yield virtually the same set of synapses over a large range of parameters and variations in the form of *S*, as long as there is a net positive area under the curve *S*, and the exploration is extensive. The resulting connection matrix is similar to that which would be achieved by connecting each place cells to its *m* nearest neighbors.

#### **NAVIGATION ALGORITHM**

Once a vector field is created, a simple motor control algorithm is applied for the animal navigation. The algorithm is performed in the following steps:


$$F(t) = a\_F \sum\_{j} \sum\_{f} \delta \left(t\_j^f - t\right) \left(\mathbf{x}\_j(t) - \mathbf{x}\_a(t)\right) \tag{10}$$

$$H(\mathbf{x}\_a)\ddot{\mathbf{x}}\_a + c(\mathbf{x}\_a, \dot{\mathbf{x}}\_a, F\_{\text{ext}}) - F = 0. \tag{11}$$

Equation 10 defines the force vector *F*(*t*) caused by spikes generated by place cells active at time *t*. Equation 11 describes the dynamics of the animals movement in the physical world. Here *xa*(*t*), *x <sup>a</sup>*(*t*) and *<sup>x</sup> <sup>a</sup>* (*t*) are, respectively, the location, velocity and acceleration of the animal's center of mass (for clarity we omitted the symbol *t* in Equation 11); *xj*—is the preferred location of the place cell *nj*; as before, *t f <sup>j</sup>* is the firing time of the *f*-th spike in neuron *nj*; δ(.) is the Dirac function; *aF* is the constant gain, *F*ext denotes all possible external forces acting on the animal, *H* is the inertia matrix and *c* is a bias force (Craig, 2004).

#### **ACKNOWLEDGMENTS**

Most of this work was done by the authors during a stay of Filip Ponulak at Princeton University sponsored by Dr. Carlos Brody, whose generosity we gratefully acknowledge. The authors would also like to thank Dr. Piotr Skrzypczynski for useful discussions on vector-field based algorithms for robot navigation.

## **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at: http://www.frontiersin.org/Computational\_Neuroscience/ 10.3389/fncom.2013.00098/abstract

**Movie S1 | Path planning and navigation in a network of place cells using a wavefront expansion concept.** This movie is related to **Figure 3** and illustrates two processes: (1) path planning and (2) navigation. In the first phase—path planning—a wavefront is initiated at the place cells representing the navigational target B. The wavefront propagating through the network modifies synaptic connections and creates a SVF with a single attractor at the location B. Red dots are the action potentials. A simulated animal is initially located at point A. Activation of the place cells at A through the passing wavefront triggers the second phase of the process—the navigation. In this phase, the place cells with receptive fields covering the current animal location receive strong excitatory currents from sensory inputs. These cells are indicated in the movie by green dots. The current animal location is denoted by the yellow circle. The stimulated cells fire and in turn excite neighboring cells. Due to the

## **REFERENCES**


SVF the active cells excite most strongly these neighbors that are located along an optimal pathway toward the target. A simple motor control algorithm (Equations 10, 11) is used to move the animal toward the locations represented by the firing cells, up to the target location.

#### **Movie S2 | Path planning and navigation in a system with multiple**

**targets.** This movie is related to **Figure 4**. Three wavefronts are initiated simultaneously at the place cells representing target locations B1, B2, B3. The expanding waves create SVFs with centers corresponding to the points of the wavefront initiation. The wavefronts inhibit each other effectively. The points where the wavefronts meet define borders of the basins of attractions of the particular SVFs. Red dots flashing on the screen represent action potentials. A simulated animal is initially located at point A. Activation of the place cells at A through the passing wavefront triggers the second phase of the process—the navigation. In this phase, the place cells with receptive fields covering the current animal location receive strong excitatory currents from sensory inputs. These cells are indicated in the movie by green dots. The current animal location is denoted by the yellow circle. The stimulated cells fire and in turn excite neighboring cells. Due to the SVF the active cells excite most strongly these neighbors that are located along an optimal pathway toward the nearest target. A simple motor control algorithm (Equations 10, 11) is used to move the animal toward the locations represented by the firing cells, up to the target location.

*Brain Res*. 398, 242–252. doi: 10.1016/0006-8993(86)91483-6


perisomatic-targeting interneurons. *J. Neurosci*. 30, 5979–5991. doi:


*Systems*, eds R. D. Hawkins, and G. H. Bower (Orlando, FL: Academic Press), 243–305. doi: 10.1016/S0079-7421(08)60113-9


hippocampus. *Neuron* 75, 410–417. doi: 10.1016/j.neuron.2012.07.015


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 01 April 2013; paper pending published: 19 April 2013; accepted: 26 June 2013; published online: 18 July 2013.*

*Citation: Ponulak F and Hopfield JJ (2013) Rapid, parallel path planning by propagating wavefronts of spiking neural activity. Front. Comput. Neurosci. 7:98. doi: 10.3389/fncom.2013.00098*

*Copyright © 2013 Ponulak and Hopfield. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

## Perspectives for computational modeling of cell replacement for neurological disorders

## *James B. Aimone1 and Jason P. Weick2\**

*<sup>1</sup> Cognitive Modeling Group, Sandia National Laboratories, Albuquerque, NM, USA*

*<sup>2</sup> Department of Neurosciences, University of New Mexico, Albuquerque, NM, USA*

#### *Edited by:*

*Julie Wall, Queen Mary, University of London, UK*

#### *Reviewed by:*

*Egidio D'Angelo, University of Pavia, Italy*

*Ma.de Guadalupe Garcia-Hernandez, Universidad de Guanajuato, Mexico*

#### *\*Correspondence:*

*Jason P. Weick, Department of Neurosciences, University of New Mexico HSC, 915 Camino de Salud NE, BMSB 145, Albuquerque, NM 87131, USA e-mail: JPWeick@salud.unm.edu*

Mathematical modeling of anatomically-constrained neural networks has provided significant insights regarding the response of networks to neurological disorders or injury. A logical extension of these models is to incorporate treatment regimens to investigate network responses to intervention. The addition of nascent neurons from stem cell precursors into damaged or diseased tissue has been used as a successful therapeutic tool in recent decades. Interestingly, models have been developed to examine the incorporation of new neurons into intact adult structures, particularly the dentate granule neurons of the hippocampus. These studies suggest that the unique properties of maturing neurons, can impact circuit behavior in unanticipated ways. In this perspective, we review the current status of models used to examine damaged CNS structures with particular focus on cortical damage due to stroke. Secondly, we suggest that computational modeling of cell replacement therapies can be made feasible by implementing approaches taken by current models of adult neurogenesis. The development of these models is critical for generating hypotheses regarding transplant therapies and improving outcomes by tailoring transplants to desired effects.

**Keywords: neurogenesis, functional integration, stroke, embryonic stem cells, induced pluripotent stem cells, dentate gyrus, cerebral cortex**

## **INTRODUCTION**

In addition to describing normal neural function, new computational modeling paradigms have successfully recapitulated various aspects of neurodegeneration and injury. For instance, models of Parkinson's Disease (PD) have demonstrated overt motor deficits as well as subtle cognitive symptoms due to loss of striatal dopamine and suggested new hypotheses regarding PD as a disorder of altered synaptic plasticity and not simply of motor function (Wiecki and Frank, 2010). Models of ischemic stroke have successfully recapitulated the reorganization of cortical receptive fields (RFs) after lesion, lending credence to a number of hypothesized mechanisms underlying cortical network dynamics (Duch, 2007). In addition, stroke models employing behavioral metrics have also simulated use-dependent recovery of movement strength that closely mimic clinical observations in stroke patients (Reinkensmeyer et al., 2012).

While a major goal of computational neuroscience is to improve therapeutics for neurological disorders, an as-yet overlooked avenue for treatment modeling has been the incorporation of new cells into an impaired network. Cell replacement therapies have shown significant promise in pre-clinical and clinical trials (Lindvall et al., 2012), where multiple sources of stem cell-derived neurons are effective at ameliorating behavioral deficits of disease models including PD, Huntington's disease, age-related dementia and stroke (Bjorklund and Lindvall, 2000; Koch et al., 2009). Interestingly, while many mechanisms may cooperate to produce transplant-mediated recovery, evidence suggests that functional integration of transplanted cells with existing circuitries is critical for the long-term benefits of cell replacement.

Notably, modeling therapeutics presents an additional challenge beyond simply reversing the effects of the neurological perturbation. Impaired neural circuits are often moving targets, changing themselves continuously in response to their altered state. Likewise, the effects of proposed therapies have their own temporal and spatial dynamics (**Figures 2C, D**) . This perspective will focus on the current status of cell replacement for neurological disorders, and will utilize the framework from the adult neurogenesis field to provide a template for understanding how the addition of neurons to adult networks can affect overall network function.

We have developed a set of criteria that we believe will be required for an accurate predictive modeling resource (**Table 1**). We advocate the use of anatomically accurate models of a neural system (I); while such models are considerably more challenging, the value of an abstract model to examine therapies is likely limited. Related, the model should have the capacity to respond to simulated injury or disease in a biologically realistic way, and should be capable of exhibiting response/recovery processes observed *in vivo* (II). Next, the model should provide a readout that maps to a behavioral metric to examine clinical efficacy of a treatment regimen (III). As for the therapy, it is essential that a mechanistic representation of the therapy itself be incorporated (IV), including temporal and spatial dynamics to the extent possible (V). To illustrate how this approach will apply to an exist-

#### **Table 1 | Criteria for computational models of cell replacement**.


ing paradigm we will discuss the computational modeling implications of using cells derived from human pluripotent stem cells (hPSCs) in the context of stroke, as this area has proven a significant target for both modeling and cell replacement.

## **COMPUTATIONAL MODELING OF CORTICAL REORGANIZATION AND STROKE**

Stroke lesions in humans typically result from damage to, or occlusion of the middle cerebral artery (MCA), which supplies blood flow to the basal ganglia and nearly the entire dorsolateral surface of the temporal and parietal cortices (Mohr et al., 1998; Ng, 2007). Following mild ischemic injury the brain demonstrates a remarkable ability to compensate for lost or damaged tissue by reorganizing peri-lesional regions of cortex (Nudo, 1997). Recovery is correlated with increases in axonal and dendritic sprouting in peri-infarct regions as well as alterations in synaptic strengths between surviving thalamic and cortical neurons (Stroemer et al., 1995; Brown and Murphy, 2008). Recent *in vivo* imaging studies in mice have revealed that peri-infarct sensory regions can "remap" their RFs after stroke, responding to peripheral stimulations that were previously restricted to now-damaged cortices (Brown et al., 2009; Sigler et al., 2009).

Due to the hierarchical nature of the cerebral cortex, its development and reorganization following injury have been significant targets for modeling studies (Willshaw and von der Malsburg, 1976; Armentrout et al., 1994; Reggia et al., 1996). Many early models focused on the formation of topographic maps of sensory cortex through experience-dependent remodeling. Models that incorporated Hebbian learning paradigms could account for the experimental observations of the "inverse magnification" rule, where small areas of peripheral sensory organs had large cortical representations due to innervation density. Interestingly, even these early abstract models based on self-organizing maps (SOMs) successfully predicted the remapping of somatosensory or visual cortices following lesions (Reggia et al., 1996), as well as changes to RFs based on stimulation and deafferentation (Pearson et al., 1987). However, while SOM models can demonstrate comparable recovery, they are likely too abstract to represent subtle aspects of diseases and therapies such as connectivity dynamics following lesion (Butz et al., 2009).

More anatomically-relevant models have developed hypotheses as to the mechanisms underlying alterations in RFs. For instance, models of sensory cortex that incorporate laterally projecting excitatory and inhibitory units demonstrate immediate expansion of RFs near to the lesion site due to disinhibition; units near the lesion site no longer receive lateral inhibitory connections from ablated cells, unmasking weaker afferent synaptic connections (Sober et al., 1997). Increasingly complex models have incorporated neural spiking dynamics, thalamic relay neurons and neurotransmitter receptors to uncover more subtle changes. A report using this model demonstrated two temporal phases during RF reorganization due to peripheral amputation: a fast (millisecond) phase of "dynamic plasticity" based on simple electrical properties after loss of input, and a slower (hours to days) phase in which NMDA receptor-dependent synaptic plasticity allowed for the reorganization of network dynamics to incorporate surviving cortical cells into remaining circuits (Mazza et al., 2004).

One of the key aspects for modeling stroke recovery *in vivo* and *in silico* is an effective measure of behavioral output (Lytton et al., 1999; Rohrer et al., 2002). Using a simple virtual arm simulation with three pairs of abductors and adductors, Goodall and colleagues successfully demonstrated muscle "weakness" in response to acute lesions of somatosensory cortex (Goodall et al., 1997). Similar models have been used to examine the observation of decreased "smoothness" of movement in stroke patients (Rohrer et al., 2002). A recent report incorporated a measure of wrist flexion force as a function of firing rates of efferent motor neurons along with a stochastic local search algorithm to feedback to the circuit after cortical lesion. In this way the authors successfully modeled motor recovery (increased flexion force) as a function of plasticity within residual, fixed pathways, without alterations in structural dynamics (Reinkensmeyer et al., 2012). Thus, existing models that incorporate multi-layered input and output pathways with lateral connectivity, spiking behavior, and behavioral metrics satisfy our first three criteria for assessing cellreplacement interventions (**Table 1**) and can likely be extended to examine cell replacement relatively quickly.

## **FUNCTIONAL INTEGRATION OF STEM CELL-DERIVED NEURONS AFTER TRANSPLANTATION**

While physical rehabilitation can assist stroke patients with regaining motor function, neither spontaneous recovery nor current intervention strategies provide complete symptom amelioration (Kalra, 2010), and no therapies exist to recover lost tissue following ischemic insult. In recent years, cell replacement therapies have become an attractive option in pre-clinical studies of ischemic injury, demonstrating transplant-mediated behavioral recovery using multiple cell types and animal models (Bliss et al., 2007). hPSC-derived neurons (hPSNs) can ameliorate limb asymmetries and amphetamine-induced rotational behavior in animals with unilateral MCA lesions, the effects of which can be maintained for months after transplantation. Successful preclinical studies have prompted a number of phase I clinical studies which verified safety and feasibility for such therapies in stroke patients (Kondziolka et al., 2000, 2005; Bang et al., 2005). While these studies were not designed to demonstrate efficacy, notable improvements were observed in some patients (Bliss et al., 2010).

While multiple mechanisms may underlie transplantmediated recovery (Lee et al., 2008; Ohtaki et al., 2008; Horie et al., 2011; Oki et al., 2012; Polentes et al., 2012) incorporation of transplanted cells into host circuitry is thought to be critical for long-term benefits (Bjorklund and Lindvall, 2000; Dunnett et al., 2001). PSNs display all basic physiological capabilities of neurons *in vivo*, including voltage-gated currents, spiking, synaptic activity (Muotri et al., 2005; Johnson et al., 2007; Wu et al., 2007), and integration with existing circuitries. Benninger et al. (2003) showed that stimulation of intact perforant path fibers could elicit post-synaptic responses in PSC-derived neurons grown on rat dentate gyrus (DG) within hippocampal slice cultures. Furthermore, optical stimulation of hPSNs expressing Channelrhodopsin-2 (Boyden et al., 2005; Weick et al., 2010) caused rapid alterations in whole network activity of established mouse networks both *in vitro* and *in vivo* (Weick et al., 2011; Pina-Crespo et al., 2012), confirming a reciprocal interaction between graft and host.

With respect to stroke, transplanted cells improve behavioral outcomes, extend processes into brain parenchyma, and express synaptic proteins (Ishibashi et al., 2004; Daadi et al., 2008; Dihne et al., 2011). Importantly, a temporal correlation exists between the maturation of hPSNs and the recovery of lost contralesional motor function (Gomi et al., 2012; Polentes et al., 2012). Lastly, electrical stimulation of endogenous cortical neurons in perilesional regions was shown to trigger immediate post-synaptic responses in transplanted neurons in animals with MCA lesions (Oki et al., 2012). Thus, stem cell-derived neurons are capable of reciprocally integrating with host brain tissue either in normal or diseased animals, and are capable of altering network function via synaptic activity.

## **CELL TYPE AND NETWORK FUNCTION OF CORTICAL-LIKE STEM CELL-DERIVED NEURONS**

As mentioned, a major consideration for modeling therapeutics is the incorporation of an accurate mechanistic and temporal representation of the therapy itself (**Table 1**; criteria IV and V). Critical features include the proportion and connectivity of neurons with various spiking phenotypes, as well as the temporal maturation of developing neurons. In the case of stroke this means recapitulating spiking phenotypes of afferent sensory, excitatory projection, and inhibitory interneurons (Xing and Gerstein, 1996; Mazza et al., 2004). At least four major classes of neurons exist according to intrinsic spiking patterns in the cerebral cortex (Gupta et al., 2000; Contreras, 2004) including Regularly spiking (RS), Irregularly spiking (IS), Fast spiking (FS), intrinsically bursting (IB), but also include multiple subcategories such as adapting, nonadapting, delayed, accelerating and stuttering (Ascoli et al., 2008). While most cortical excitatory projection neurons are RS neurons, inhibitory interneurons and excitatory neurons of various subcortical nuclei display a range of spiking phenotypes (Llinas and Jahnsen, 1982), which can have significant consequences to information processing capabilities (Jahnsen and Llinas, 1984; Koch and Segev, 2000; Pissadaki et al., 2010).

Unfortunately, most publications measuring functional properties of hPSNs demonstrate report only RS phenotypes (Wernig et al., 2004; Johnson et al., 2007; Wu et al., 2007). In our hands, hPSNs derived from the WA09 (H9) cell line display primarily RS neurons with a frequency range of 10 to 36 Hz (**Figure 1A**). In approximately 10% of cells however, we observed delayed spiking phenotype (**Figure 1B**), while an even smaller minority displayed IS behavior (**Figure 1C**), with no evidence of IB neurons. However, it is likely that that the dearth of variation is partially due to the immature nature of most hPSNs reported, as many display relatively depolarized resting membrane potentials (RMP) and diminishing action potential (AP) amplitude during current pulses (Johnson et al., 2007).

Interestingly, the development and incorporation of hPSNs into existing neural networks shares many features of adult neurogenesis. Most importantly is the progressive maturation of excitable properties including synaptogenesis (Zhao et al., 2006; Ge et al., 2007), where unitary and network-based synaptic potentials can be observed only after several weeks in the presence of mature cells (Weick et al., 2011; Pina-Crespo et al., 2012), much slower than their rodent counterparts (Johnson et al., 2007). Thus, an additional benefit of computational models may be to examine the incorporation of cells with primate characteristics into a "rodent" network, which will likely affect pre-clinical assessments of cell replacement.

## **LESSONS FROM ADULT STEM CELLS: NEW NEURON INCORPORATION IN THE DENTATE GYRUS**

In order to illustrate the complexity of incorporating new neurons into existing circuits, it is useful to consider systems in which there is ongoing integration of new neurons within the normal brain. Adult neurogenesis in the DG region of the hippocampus provides just such a system (Zhao et al., 2008), with DG neurogenesis producing new excitatory granule cells throughout life. Naturally occurring adult neurogenesis provides the following several important insights into different functional considerations for induced neurogenesis.


A direct interaction between young and old neurons may be suitable for the DG; its primary function is thought to be in facilitating memory encoding, not memory storage. As a result, sharpening RFs over time could be beneficial (O'Reilly and McClelland, 1994; Aimone et al., 2011). In contrast, it is possible that changing otherwise stable RFs in regions involved in motor control or sensory perception would be disruptive. One consideration is that neuronal replacement therapies are generally going to target areas in which neurons have been lost, suggesting that a number of synapses would have been vacated (Lehmann et al., 2005; Butz et al., 2008). Together these possibilities highlight the need to capture the full complexities of the disease and treatment in a theoretical framework (**Figure 2B**).

## **FUTURE PERSPECTIVES FOR MODELING FUNCTIONAL TRANSPLANTATION**

The above summary highlights two major computational considerations that must be accounted for when approaching neuronal replacement therapy. First, new neurons do not mature in isolation, and the progression from a progenitor state to a fully functional neuron is complex. While neurogenesis has been evolutionarily preserved in the DG, networks normally lacking neurogenesis may be more sensitive to the addition of neurons. It would not be unreasonable to expect that the addition of neurons could shift the balance of a network out of normal operating bounds, causing either seizures or depressed overall activity (Schneider-Mizell et al., 2010).

Second, the functional displacement of an impairment, either acute or through degeneration, is not static. Rather, compensatory mechanisms are initiated almost immediately. While such compensation can be clinically beneficial it represents an additional challenge for cell replacement. In the time that it takes for a replacement therapy to begin and the neurons to be incorporated, the network itself will have changed; as a result, "reversing" the effects of disease may no longer be sufficient (**Figure 2C**). Rather, we need a theoretical understanding of the post-disease, steady-state circuit to design an approach to regain a functional equivalence to the pre-disease state (**Figure 2D**).

What should a model capable of addressing these issues look like? Although not used to model stroke, the three-layer GENE-SIS model used by Mazza et al. (2004) is among the most neurobiologically realistic as it uses several thousand conductancebased, multi-compartment neurons and accurately reorganizes in response to peripheral lesions. Accordingly, we recommend that the neurogenesis process be appropriately mapped to the primary somatosensory cortex, which could follow the model used in Aimone et al. (2009), whereby the biophysical properties and connectivity of maturing neurons are dynamic over extended time scales similar to hPSNs. This type of hybrid model would provide insight into how remapping could be altered by synaptic connections between hPSNs and host neurons at various levels (e.g., cortical vs. thalamic relay neurons). Alternatively, motor output could be monitored by the creation of a hybrid model using parameters from Reinkensmeyer et al. (2012) and a two-layer GENESIS model to incorporate new neurons into motor cortex, which could allow the system to "learn" to use transplanted cells.

While large-scale, biophysically realistic models are currently time-consuming to construct and computationally expensive to simulate, we envision that future simulation tools and high performance computing capabilities will facilitate such modeling endeavors. We hope that our outlined vision for how to approach these efforts will provide a roadmap for understanding the computational implications of cell replacement therapy. The extent to which insights from adult neurogenesis apply to stem cell therapies remains to be seen, and future models will have to consider not only the proposed therapy but also the unique aspects of the brain region affected, its function, and the disease. Nonetheless, improved modeling tools and approaches in recent years provide computational neuroscience with a unique opportunity to influence the development of an exciting area of therapeutic development.

## **ACKNOWLEDGMENTS**

James B. Aimone is supported by the Laboratory Directed Research and Development program at Sandia National Laboratories.

Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-AC04-94AL85000.

Jason P. Weick is supported by the Department of Neurosciences at the University of New Mexico and a grant from NINDS RO3 RNS074281A and NIAAA P20-AA017068.

## **REFERENCES**


596. doi: 10.1016/j.neuron.2011.05. 010


ence of adult neurogenesis on memory encoding. *Neuron* 61, 187–202. doi: 10.1016/j.neuron.2008.11.026


mesenchymal stem cell transplantation in stroke patients. *Ann. Neurol.* 57, 874–882. doi: 10.1002/ana. 20501

Benninger, F., Beck, H., Wernig, M., Tucker, K. L., Brüstle, O., and Scheffler, B. (2003). Functional integration of embryonic stem cellderived neurons in hippocampal slice cultures. *J. Neurosci.* 23, 7075– 7083.


tion after stroke by cell replacement: anatomic and functional considerations. *Stroke* 42, 2342–2350. doi: 10. 1161/strokeaha.111.613422


Ryan, S. D., et al. (2012). Highfrequency hippocampal oscillations activated by optogenetic stimulation of transplanted human ESC-derived neurons. *J. Neurosci.* 32, 15837– 15842. doi: 10.1523/jneurosci.3735- 12.2012


and Sander, L. M. (2010). From network structure to network reorganization: implications for adult neurogenesis. *Phys. Biol.* 7:046008. doi: 10. 1088/1478-3975/7/4/046008


netic targeting. *Stem Cells* 28, 2008– 2016. doi: 10.1002/stem.514


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 02 April 2013; accepted: 10 October 2013; published online: 06 November 2013.*

*Citation: Aimone JB and Weick JP (2013) Perspectives for computational modeling of cell replacement for neurological disorders. Front. Comput. Neurosci. 7:150. doi: 10.3389/fncom.2013.00150*

*This article was submitted to the journal Frontiers in Computational Neuroscience.*

*Copyright © 2013 Aimone and Weick. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Generalized activity equations for spiking neural network dynamics

#### *Michael A. Buice1 and Carson C. Chow2 \**

*<sup>1</sup> Modeling, Analysis and Theory Team, Allen Institute for Brain Science, Seattle, WA, USA*

*<sup>2</sup> Laboratory of Biological Modeling, NIDDK, National Institutes of Health, Bethesda, MD, USA*

#### *Edited by:*

*Julie Wall, Queen Mary, University of London, UK*

#### *Reviewed by:*

*Brent Doiron, University of Pittsburgh, USA G. Bard Ermentrout, University of Pittsburgh, USA*

#### *\*Correspondence:*

*Carson C. Chow, Laboratory of Biological Modeling, NIDDK, National Institutes of Health, Building 12A Room 4007, 12 South Drive, Bethesda, MD 20814, USA e-mail: carsonc@mail.nih.gov*

Much progress has been made in uncovering the computational capabilities of spiking neural networks. However, spiking neurons will always be more expensive to simulate compared to rate neurons because of the inherent disparity in time scales—the spike duration time is much shorter than the inter-spike time, which is much shorter than any learning time scale. In numerical analysis, this is a classic stiff problem. Spiking neurons are also much more difficult to study analytically. One possible approach to making spiking networks more tractable is to augment mean field activity models with some information about spiking correlations. For example, such a generalized activity model could carry information about spiking rates and correlations between spikes self-consistently. Here, we will show how this can be accomplished by constructing a complete formal probabilistic description of the network and then expanding around a small parameter such as the inverse of the number of neurons in the network. The mean field theory of the system gives a rate-like description. The first order terms in the perturbation expansion keep track of covariances.

**Keywords: mean field theory, theta model, fokker-planck, correlations, finite size networks, wilson-cowan model, population rate, fluctuations**

## **INTRODUCTION**

Even with the rapid increase in computing power due to Moore's law and proposals to simulate the entire human brain notwithstanding Ailamaki et al. (2012), a realistic simulation of a functioning human brain performing non-trivial tasks is remote. While it is plausible that a network the size of the human brain could be simulated in real time Izhikevich and Edelman (2008); Eliasmith et al. (2012) there are no systematic ways to explore the parameter space. Technology to experimentally determine all the parameters in a single brain simultaneously does not exist and any attempt to infer parameters by fitting to data would require exponentially more computing power than a single simulation. We also have no idea how much detail is required. Is it sufficient to simulate a large number of single compartment neurons or do we need multiple-compartments? How much molecular detail is required? Do we even know all the important biochemical and biophysical mechanisms? There are an exponential number of ways a simulation would not work and figuring out which remains computationally intractable. Hence, an alternative means to provide appropriate prior distributions for parameter values and model detail is desirable. Current theoretical explorations of the brain utilize either abstract mean field models or small numbers of more biophysical spiking models. The regime of large but finite numbers of spiking neurons remains largely unexplored. It is not fully known what role spike time correlations play in the brain. It would thus be very useful if mean field models could be augmented with some spike correlation information.

This paper outlines a scheme to derive generalized activity equations for the mean and correlation dynamics of a fully deterministic system of coupled spiking neurons. It synthesizes methods we have developed to solve two different types of problems. The first problem was how to compute finite system size effects in a network of coupled oscillators. We adapted the methods of the kinetic theory of gases and plasmas Ichimaru (1973); Nicholson (1993) to solve this problem. The method exploits the exchange symmetry of the oscillators and characterizes the phases of all the oscillators in terms of a phase density function η*(*θ*, t)*, where each oscillator is represented as a point mass in this density. We then write down a formal flux conservation equation of this density, called the Klimontovich equation, which completely characterizes the system. However, because the density is not differentiable, the Klimontovich equation only exists in the weak or distributional sense. Previously, e.g., Desai and Zwanzig (1978); Strogatz and Mirollo (1991); Abbott and van Vreeswijk (1993); Treves (1993) the equations were made usable by taking the "mean field limit" of *N* → ∞ and assuming that the density is differentiable in that limit, resulting in what is called the Vlasov equation. Instead of immediately taking the mean field limit, we regularize the density by averaging over initial conditions and parameters and then expand in the inverse system size *N*−<sup>1</sup> around the mean field limit. This results in a system of coupled moment equations known as the BBGKY moment hierarchy. In Hildebrand et al. (2007), we solved the moment equations for the Kuramoto model perturbatively to compute the pair correlation function between oscillators. However, the procedure was somewhat *ad-hoc* and complicated. We then subsequently showed in Buice and Chow (2007) that the BBGKY moment hierarchy could be recast in terms of a density functional of the phase density. This density functional could be written down explicitly as an integral over all possible phase histories, i.e., a Feynman-Kac path integral. The advantage of using this density functional formalism is that the moments to arbitrary order in 1*/N* could be computed as a steepest-descent expansion of the path integral, which can be expressed in terms of Feynman diagrams. This made the calculation more systematic and mechanical. We later applied the same formalism to synaptically coupled spiking models Buice and Chow (2013b).

Concurrently with this line of research, we also explored the question of how to generalize population activity equations, such as the Wilson-Cowan equations, to include the effects of correlations. The motivation for this question is that the Wilson-Cowan equations are mean field equations and do not capture the effects of spike-time correlations. For example, the gain in the Wilson-Cowan equations is fixed, (which is a valid approximation when the neurons fire asynchronously), but correlations in the firing times can change the gain Salinas and Sejnowski (2000). Thus, it would be useful to develop a systematic procedure to augment population activity equations to include spike correlation effects. The approach we took was to posit plausible microscopic stochastic dynamics, dubbed the spike model, that reduced to the Wilson-Cowan equations in the mean field limit and compute the self-consistent moment equations from that microscopic theory. Buice and Cowan (2009) showed that the solution of the master equation of the spike model could be expressed formally in terms of a path integral over all possible spiking histories. The random variable in the path integral is a spike count whereas in the path integral for the deterministic phase model we described above, the random variable is a phase density. To generate a system of moment equations for the microscopic stochastic system, we transformed the random spike count variable in the path integral into moment variables Buice et al. (2010). This is accomplished using the effective action approach of field theory, where the exponent of the cumulant generating functional, called the action, which is a function of the random variable is Legendre transformed into an effective action of the cumulants. The desired generalized Wilson-Cowan activity equations are then the equations of motion of the effective action. This is analogous to the transformation from Lagrangian variables of position and velocity to Hamiltonian variables of position and momentum. Here, we show how to apply the effective action approach to a deterministic system of synaptically coupled spiking neurons to derive a set of moment equations.

#### **APPROACH**

Consider a network of single compartment conductance-based neurons

$$\begin{aligned} C\frac{dV\_i}{dt} &= -\sum\_{r=1}^n g\_r \left(\mathbf{x}\_i^r\right) (V\_i - \nu\_r) + \sum\_{j=1}^N g\_{\bar{j}} s\_j(t) \\ \mathbf{r}\_i^r \frac{d\mathbf{x}\_i^r}{dt} &= f(V\_i, \mathbf{x}\_i) \\ \mathbf{r}\_j \frac{d\mathbf{s}\_j}{dt} &= h(V\_j, \mathbf{s}\_j) \\ \mathbf{r}\_{\mathcal{S}} \frac{d\mathbf{g}\_{ij}}{dt} &= \phi(\mathbf{g}\_{ij}, V) \end{aligned}$$

The equations are remarkably stiff with time scales spanning orders of magnitude from milliseconds for ion channels, to seconds for adaptation, and from hours to years for changes in synaptic weights and connections. Parameter values must be assigned for 10<sup>11</sup> neurons with 104 connections each. Here, we present a formalism to derive a set of reduced activity equations directly from a network of deterministic spiking neurons that capture the spike rate and spike correlation dynamics. The formalism first constructs a density functional for the firing dynamics of all the neurons in a network. It then systematically marginalizes the unwanted degrees of freedom to isolate a set of self-consistent equations for the desired quantities. For heuristic reasons, we derive an example set of generalized activity equations for the first and second cumulants of the firing dynamics of a simple spiking model but the method can be applied to any spiking model.

A convenient form to express spiking dynamics is with a phase oscillator. Consider the quadratic integrate-and-fire neuron

$$\frac{dV\_i}{dt} = I\_i + V\_i^2 + \alpha\_i \mu(t) \tag{1}$$

where *I* is an external current and *u(t)* are the synaptic currents with some weight α*i*. The spike is said to occur when *V* goes to infinity whereupon it is reset to minus infinity. The quadratic non-linearity ensures that this transit will occur in a finite amount of time. The substitution *V* = tan*(*θ*/*2*)* yields the theta model Ermentrout and Kopell (1986):

$$\frac{d\theta\_i}{dt} = 1 - \cos\theta\_i + (1 + \cos\theta\_i) \left(I\_i + \alpha\_i u\right) \tag{2}$$

which is the normal form of a Type I neuron near the bifurcation to firing Ermentrout (1996). The phase neuron is an adequate approximation to spiking dynamics provided the inputs are not overly strong as to disturb the limit cycle. The phase neuron also includes realistic dynamics such as not firing when the input is below threshold. Coupled phase models arise naturally in weakly coupled neural networks Ermentrout and Kopell (1991); Hoppensteadt and Izhikevich (1997); Golomb and Hansel (2000). They include the Kuramoto model Kuramoto (1984), which we have previously analyzed Buice and Chow (2007); Hildebrand et al. (2007).

Here, we consider the phase dynamics of a set of *N* coupled phase neurons obeying

$$\dot{\theta}\_i = F(\theta, \chi\_i, u(t)) \tag{3}$$

$$
\dot{u}(t) = -\beta u(t) + \beta \nu(t) \tag{4}
$$

$$\psi(t) = \frac{1}{N} \sum\_{j=1}^{N} \sum\_{l} \delta \left( t - t\_j^l \right) \tag{5}$$

where each neuron has a phase θ*<sup>i</sup>* that is indexed by *i*, *u* is a global synaptic drive, *F(*θ*,* γ*, u)* is the phase and synaptic drive dependent frequency, γ*<sup>i</sup>* represents all the parameters for neuron *i* drawn from a distribution with density *g(*γ*)*, ν is the population firing rate of the network,*t l <sup>j</sup>* is the *l*th firing time of neuron *j* and a neuron fires when its phase crosses π. In the present paper, we consider all-to-all or global coupling through a synaptic drive variable *u(t)*. However, our basic approach is not restricted to global coupling.

We can encapsulate the phase information of all the neurons into a neuron density function Buice and Chow (2007, 2011, 2013a,b); Hildebrand et al. (2007).

$$\eta(\theta,\chi,t) = \frac{1}{N} \sum\_{i=1}^{N} \delta(\theta - \theta\_i(t)) \left\{ \chi - \chi\_i \right\} \tag{6}$$

where δ*(*·*)* is the Dirac delta functional, and θ*i(t)* is a solution to system (3–5). The neuron density gives a count of the number of neurons with phase θ and synaptic strength γ at time *t*. Using the fact that the Dirac delta functional in (5) can be expressed as *<sup>l</sup>* δ*(t* − *t l j )* = θ˙ *<sup>j</sup>*δ*(*π − θ*j)*, the population firing rate can be rewritten as

$$\nu(t) = \int d\boldsymbol{\chi} \, F(\boldsymbol{\pi}, \boldsymbol{\chi}, \boldsymbol{\mu}(t)) \boldsymbol{\eta}(\boldsymbol{\pi}, \boldsymbol{\chi}, t) \tag{7}$$

The neuron density formally obeys the conservation equation

$$\frac{\partial}{\partial t}\eta(\theta,\,\,\nu,t) + \frac{\partial}{\partial \theta}[F\eta(\theta,\,\,\nu,t)] = 0\tag{8}$$

with initial condition η*(*θ*,* γ*, t*0*)* = η0*(*θ*,* γ*)* and *u(t*0*)* = *u*0. Equation (8) is known as the Klimontovich equation Ichimaru (1973); Liboff (2003). The Klimontovich equation, the equation for the synaptic drive (4), and the firing rate expressed in terms of the neuron density (7), fully define the system. The system is still fully deterministic but is now in a form where various sets of reduced descriptions can be derived. Here, we will produce an example of a set of reduced equations or generalized activity equations that capture some aspects of the spiking dynamics. The path we take toward the end will require the introduction of some formal machinery that may obscure the intuition around the approximations. However, we feel that it is useful because it provides a systematic and controlled way of generating averaged quantities that can be easily generalized.

For finite *N*, (8) is only valid in the weak or distributional sense since η is not differentiable. In the *N* → ∞ limit, it has been argued that η will approach a smooth density *ρ* that evolves according to the Vlasov equation that has the same form as (8) but with η replaced by *ρ* Ichimaru (1973); Desai and Zwanzig (1978); Strogatz and Mirollo (1991); Nicholson (1993); Hildebrand et al. (2007). This has been proved rigorously in the case where noise is added using the theory of coupled diffusions McKean Jr (1966); Faugeras et al. (2009); Baladron et al. (2012); Touboul (2012). This *N* → ∞ limit is called mean field theory. In mean field theory, the original microscopic many body neuronal network is represented by a smooth macroscopic density function. In other words, the ensemble of networks prepared with different microscopic initial conditions is sharply peaked at the mean field solution. For large but finite *N*, there will be deviations away from mean field Buice and Chow (2007); Hildebrand et al. (2007); Buice and Chow (2013a,b). These deviations can be characterized in terms of a distribution over an ensemble of coupled networks that are all prepared with different initial conditions and parameter values. Here, we show how a perturbation theory in *N*−<sup>1</sup> can be developed to expand around the mean field solution. This requires the construction of the probability density functional over the ensemble of spiking neural networks. We adapt the tools of statistical field theory to perform such a construction.

#### **FORMALISM**

The complete description of the system given by equations (4, 7, 8) can be written as

$$\dot{u}(t) + \beta u(t) - \beta \int d\gamma F(\pi, \chi, u(t))\eta(\pi, \chi, t) = 0 \qquad (9)$$
 
$$\text{a}$$

$$\frac{\partial}{\partial t}\eta(\theta,\chi,t) + \frac{\partial}{\partial \theta}[F(\theta,\chi,u(t))\eta(\theta,\chi,t)] \equiv \mathcal{L}\eta = 0 \tag{10}$$

The probability density functional governing the system specified by the synaptic drive and Klimontovich equations (9) and (10) given initial conditions *(*η0*, u*0*)* can be written as

$$P[\eta, u] = \int \mathcal{D}u\_0(t) \, \mathcal{D}\eta\_0(\theta, \gamma) \, P[\eta, u|\eta\_0, u\_0] \, \, P\_0[\eta\_0, u\_0, \gamma] \, (11)$$

where *P* [η*, u*|η0*, u*0] is the conditional probability density functional of the functions *(*η*, u)*, and *P*<sup>0</sup> [η0*, u*0] is the density functional over initial conditions of the system. The integral is a Feynman-Kac path integral over all allowed initial condition functions. Formally we can write *P* [η*, u*|η0*, u*0] as a point mass (Dirac delta) located at the solutions of (9) and (10) given the initial conditions:

$$\begin{aligned} & \left[ \mathcal{S} \left[ \mathcal{L} \eta - \eta\_0 \mathcal{S} (t - t\_0) \right] \right] \\ & \mathcal{S} \left[ \dot{\mu} + \mathfrak{B} \mu - \mathfrak{B} \int d\chi F(\pi, \chi, \mu(t)) \eta(\pi, \chi, t) - \mu\_0 \mathfrak{S} (t - t\_0) \right] \end{aligned}$$

The probability density functional (11) is then

$$P[\eta, u] = \int \mathcal{D}u\_0(t) \, \mathcal{D}\eta\_0(\theta, \chi) \, \delta \left[\mathcal{L}\eta - \eta\_0 \delta(t - t\_0)\right]$$

$$\times \delta \left[\dot{u} + \beta u - \beta \int d\chi \, F(\pi, \chi, u(t))\eta(\pi, \chi, t)\right]$$

$$- \left. u\_0 \delta(t - t\_0) \right] P\_0\left[\eta\_0, u\_0, \chi\right] \tag{12}$$

Equation (12) can be made useful by noting that the Fourier representation of a Dirac delta is given by δ*(x)* ∝ *dk eikx*. Using the infinite dimensional Fourier functional transform then gives

$$P[\eta,\mu] = \int \mathcal{D}\tilde{\eta}\mathcal{D}\tilde{\mu}\,e^{-\mathcal{NS}\{\eta,\tilde{\eta},\mu,\tilde{\mu}\}}.$$

The exponent *S*[η*, u*] in the probability density functional is called the *action* and has the form

$$S = S\_{\mathfrak{u}} + S\_{\mathfrak{q}} + S\_0 \tag{13}$$

$$S\_{\Psi} = \int d\theta d\gamma dt \,\tilde{\varphi}(\mathbf{x}) \left[ \partial\_{\mathbf{l}} \varphi(\mathbf{x}) + \partial\_{\theta} F(\theta, \mathbf{y}, u(t)) \varphi(\mathbf{x}) \right] \quad (14)$$

represents the contribution of the transformed neuron density to the action,

$$\begin{aligned} S\_{\mathsf{u}} &= \frac{1}{N} \int dt \,\tilde{u}(t) \Big( \dot{\boldsymbol{u}}(t) + \beta \boldsymbol{u}(t) \\\\ &- \beta \int d\boldsymbol{\upgamma} F(\boldsymbol{\pi}, \boldsymbol{\upgamma}, \boldsymbol{u}(t)) [\tilde{\boldsymbol{\uprho}}(\boldsymbol{\pi}, \boldsymbol{\upgamma}, t) + 1] \boldsymbol{\uprho}(\boldsymbol{\pi}, \boldsymbol{\upgamma}, t) \Big) \end{aligned} (15)$$

represents the global synaptic drive, *S*0[ ˜ϕ0*(x*0*), u*0*(t*0*)*] represents the initial conditions, and *x* = *(*θ*,* γ*, t)*. For the case where the neurons are considered to be independent in the initial state, we have

$$\left[\text{S}\_{\text{0}}\left[\tilde{\text{q}}\text{o}\left(\text{x}\right),\mu\_{\text{0}}\left(\text{t}\right)\right]\right] = -\frac{1}{N}\tilde{\mu}\left(\text{t}\_{0}\right)\mu\_{\text{0}}\tag{16}$$

$$-\ln\left(1+\int d\theta d\gamma \tilde{\wp}\_{0}\left(\theta,\,\chi,t\_{0}\right)\,\rho\_{0}\left(\theta,\,\chi,t\_{0}\right)\right)$$

where *u*<sup>0</sup> is the initial value of the coupling variable and *ρ*0*(*θ*,* γ*, t)* is the distribution from which the initial configuration is drawn for each neuron. The action includes two imaginary auxiliary response fields (indicated with a tilde), which are the infinite dimensional Fourier transform variables. The factor of 1*/N* appears to ensure correct scaling between the *u* and ϕ variables since *u* applies to a single neuron while ϕ applies to the entire population. The full derivation is given in Buice and Chow (2013b) and a review of path integral methods applied to differential equations is given in Buice and Chow (2010). In the course of the derivation we have made a Doi-Peliti-Jannsen transformation Janssen and Täuber (2005); Buice and Chow (2013b), given by

$$
\varphi(\mathfrak{x}) = \mathfrak{n}(\mathfrak{x})e^{-\mathfrak{h}(\mathfrak{x})}
$$

$$
\tilde{\varphi}(\mathfrak{x}) = e^{\tilde{\mathfrak{h}}(\mathfrak{x})} - 1
$$

In deriving the action, we have explicitly chosen the Ito convention so that the auxiliary variables only depend on variables in the past. The action (13) contains all the information about the statistics of the network.

The moments for this distribution can be obtained by taking functional derivatives of a moment generating functional. Generally, the moment generating function for a random variable is given by the expectation value of the exponential of that variable with a single parameter. Because our goal is to transform to new variables for the first and second cumulants, we form a "two-field" moment generating functional, which includes a second parameter for pairs of random variables,

$$\begin{aligned} \exp(N\,W[I,K]) &= \\ \int \mathcal{D}\xi \, \exp\left[-N\xi[\xi] + N \int d\mathbf{x} \, J^i(\mathbf{x})\xi\_i(\mathbf{x}) + \frac{N}{2} \right] \\ \int d\mathbf{x} d\mathbf{x}' \xi\_i(\mathbf{x}) K^{\vec{\eta}}(\mathbf{x}, \mathbf{x}') \xi\_j(\mathbf{x}') \end{aligned} \tag{17}$$

where *J* and *K* are moment generating fields, ξ1*(x)* = *u(t)*, ξ2*(x)* = *u*˜*(t)*, ξ3*(x)* = ϕ*(x)*, ξ4*(x)* = ˜ϕ*(x)*, and *x* = *(*θ*,* γ*, t)*. Einstein summation convention is observed beween upper and lower indices. Unindexed variables represent vectors. The integration measure *dx* is assumed to be *dt* when involving indices 1 and 2. Covariances between an odd and even index corresponds to a covariance between a field and an auxiliary field. Based on the structure of the action *S* and (17) we see that this represents a linear propagator and by causality and the choice of the Ito convention is only non-zero if the time of the auxiliary field is evaluated at an earlier time than the field. Covariances between two even indices correspond to that between two auxiliary fields and are always zero because of the Ito convention.

The mean and covariances of ξ can be obtained by taking derivatives of the action *W*[*J, K*] in (17), with respect to *J* and *K* and setting *J* and *K* to zero:

$$\frac{\delta W}{\delta j^i} = \left. \langle \xi\_i \rangle \right|\_{J, K=0}$$

$$\frac{\delta W}{\delta K^{\vec{\eta}}} = \left. \frac{1}{2} \langle \xi\_i \xi\_j \rangle \right|\_{J, K=0}$$

Expressions for these moments can be computed by expanding the path integral in (17) perturbatively around some mean field solution. However, this can be unwieldy if closed form expressions for the mean field equations do not exist. Alternatively, the moments at any order can be expressed as self-consistent dynamical equations that can be analyzed or simulated numerically. Such equations form a set of generalized activity equations for the means *ai* = ξ*i*, and covariances *Cij* = *N*[ξ*i*ξ*j* − *aiaj*].

We derive the generalized activity equations by Legendre transforming the action *W*, which is a function of *J* and *K*, to an effective action that is a function of *a* and *C*. Just as a Fourier transform expresses a function in terms of its frequencies, a Legendre transform expresses a convex function in terms of its derivatives. This is appropriate for our case because the moments are derivatives of the action. The Legendre transform of *W*[*J, K*] is

$$\Gamma[a,C] = -\mathcal{W}[I,K] + \int d\mathbf{x} f^i a\_i + \frac{1}{2} \int d\mathbf{x} d\mathbf{x}' \left[ a\_i a\_j + \frac{1}{N} C\_{ij} \right] K^{ij} \tag{18}$$

which must obey the constraints

$$\begin{aligned} \frac{\ $W}{\$ j^i} &= a\_i\\ \frac{\ $W}{\$ K^{ij}} &= \frac{1}{2} \left[ a\_i a\_j + \frac{1}{N} C\_{ij} \right] \end{aligned}$$

and

$$\frac{\delta\Gamma}{8a\_i} \equiv \Gamma^{i, \, 00} = J^i + \frac{1}{2}a\_j \left[ K^{\dot{\jmath}} + K^{\dot{\jmath}i} \right]$$

$$\frac{\delta\Gamma}{8C\_{\dot{\jmath}}} \equiv \Gamma^{0, \, \dot{\jmath}} = \frac{1}{2N} K^{\dot{\jmath}} \tag{19}$$

The generalized activity equations are given by the equations of motion of the effective action, in direct analogy to the Euler-Lagrange equations of classical mechanics, and are obtained by setting *J<sup>i</sup>* = 0 and *Kij* = 0 in (19).

In essence, what the effective action does is to take a probabilistic (statistical mechanical) system in the variables ξ with action *S* and transform them to a deterministic (classical mechanical) system with an action . Our approach here follows that used in Buice et al. (2010) to construct generalized activity equations for the Wilson Cowan model. However, there are major differences between that system and this one. In Buice et al. (2010), the microscopic equations were for the spike counts of an inherently probabilistic model so the effective action and ensuing generalized activity equations could be constructed directly from the Markovian spike count dynamics. Here, we start from deterministically firing individual neurons and get to a probabilistic description through the Klimontovich equation. It would be straightforward to include stochastic effects into the spiking dynamics.

Using (18) in (17) gives

$$\begin{split} \exp(-N\,\Gamma[a,C]) &= \int \mathcal{D}\psi \, \exp\left[-NS[\xi] + N \int d\mathbf{x} \, J^i \, (\xi\_i - a\_i) \\ &+ \frac{N}{2} \int d\mathbf{x} d\mathbf{x}' \bigg[ \xi\_i \xi\_j - a\_i a\_j - \frac{1}{N} C\_{ij} \right] K^{ij} \end{split} (20)$$

where *J* and *K* are constrainted by (19). We cannot compute the effective action explicitly but we can compute it perturbatively in *N*<sup>−</sup>1. We first perform a shift ξ*<sup>i</sup>* = *ai* + ψ*i*, expand the action as *S*[*a* + ψ] = *S*[*a*] + *dx(L<sup>i</sup>* [*a*]ψ*<sup>i</sup>* + *(*1*/*2*) dx Lij*[*a*]ψ*i*ψ*j)* +··· and substitute for *J* and *K* with the constraints (19) to obtain

$$\exp(-N\,\Gamma[a,C]) = \exp\left(-\text{NS}[a] - N\text{Tr}\,\Gamma^{0,\,\,\dot{\mathcal{Y}}}C\_{\dot{\mathcal{Y}}}\right)$$

$$\int \mathcal{D}\Psi \, \exp\left[-N\int d\mathbf{x}\Big(L^{i}[a]\Psi\_{i} + \frac{1}{2}\int d\mathbf{x}'L^{\dot{\mathcal{Y}}}[a]\Psi\_{i}\Psi\_{\dot{\mathcal{Y}}}\Big)\right.$$

$$+ N\int d\mathbf{x}\,\Gamma^{i,\,\,\mathcal{W}}\Psi\_{i} + N^{2}\int d\mathbf{x}d\mathbf{x}'\Psi\_{i}\Psi\_{\dot{\mathcal{Y}}}\Gamma^{0,\,\dot{\mathcal{Y}}}\Big]\tag{21}$$

where

$$\text{Tr}\,A^{\dagger j}B\_{i\bar{j}} = \int d\mathbf{x}d\mathbf{x}' A^{\dagger\bar{j}}(\mathbf{x},\mathbf{x}')B\_{i\bar{j}}(\mathbf{x},\mathbf{x}')\tag{22}$$

Our goal is to construct an expansion for by collecting terms in successive orders of *N*−<sup>1</sup> in the path integral of (21). Expanding as [*a, C*] = <sup>0</sup> + *N*−1<sup>1</sup> + *N*−2<sup>2</sup> and equating coefficients of *N* in (21) immediately leads to the conclusion that <sup>0</sup> = *S*[*a*], which gives

$$\exp(-N\,\Gamma[a,C]) = \exp\left(-NS[a] - \text{Tr}\,\Gamma^{0,\,\,ij}\_1 C\_{\vec{\eta}}\right) \int \mathcal{D}\Psi$$

$$\exp\left[-\frac{N}{2} \int d\mathbf{x} L^{\vec{\eta}}[a] \Psi\_i \Psi\_j + N \int d\mathbf{x} \,\Gamma^{0,\,\,ij}\_1 \Psi\_i \Psi\_j\right]$$

where higher order terms in *N*−<sup>1</sup> are not included. To lowest nonzero order 0*, ij* = *N*−10*, ij* <sup>1</sup> since <sup>0</sup> is only a function of *a* and not *C*. If we set

$$
\Gamma^{0,\,\vec{ij}}\_1 = (1/2)L^{\vec{ij}} - (1/2)Q^{\vec{ij}},\tag{23}
$$

we obtain

$$\exp(-N\,\Gamma[a,C]) = \exp\left(-\text{NS}[a] - \frac{1}{2}\text{Tr}\,L^{\vec{\mathcal{Y}}}C\_{\vec{\mathcal{Y}}} + \frac{1}{2}\text{Tr}\,Q^{\vec{\mathcal{Y}}}C\_{\vec{\mathcal{Y}}}\right)$$

$$\times \int \mathcal{D}\Psi \, \exp\left[-\frac{N}{2}\int d\mathbf{x}\,Q^{\vec{\mathcal{Y}}}[a]\Psi\_{i}\Psi\_{j}\right] (24)$$

to order *N*<sup>−</sup>1. *Qij* is an unknown function of *a* and *C*, which we will deduce using self-consistency. The path integral in (24), which is an infinite dimensional Gaussian that can be explicitly integrated, is proportional to 1*/* det *Qij* = exp*(*−*(*1*/*2*)*ln det *Qij)* = exp*(*−*(*1*/*2*)*Tr ln *Qij)*, using properties of matrices. Hence, (24) becomes

$$\begin{aligned} & \exp(-N\,\Gamma[a,C]) \\ &= \exp\left(-\text{NS}[a] - \frac{1}{2}\text{Tr}\,L^{\vec{\eta}}C\_{\vec{\eta}} - \frac{1}{2}\text{Tr}\,Q^{\vec{\eta}}C\_{\vec{\eta}} + \frac{1}{2}\text{Tr}\,\ln Q^{\vec{\eta}}\right), \end{aligned}$$

and

$$\Gamma[a,C] = S[a] + \frac{1}{2N} \text{Tr} \, L^{\vec{y}} C\_{\vec{y}} + \frac{1}{2N} \text{Tr} \, \ln Q^{\vec{y}} - \frac{1}{2N} \text{Tr } Q^{\vec{y}} C\_{\vec{y}}$$

Taking the derivative of with respect to *Cij* yields

$$\Gamma^{0,\,\dot{\eta}} = \frac{1}{2N} \left( L^{\ddot{\eta}} + (Q^{-1})^{kl} \frac{\partial}{\partial C\_{\dot{\eta}}} Q^{lk} - \frac{\partial}{\partial C\_{\dot{\eta}}} (Q^{kl} C\_{lk}) \right)$$

Self consistency with (23) then requires that *Qij* = *(C*−1*)ij* which leads to the effective action

$$\Gamma[a,C] = S[a] + \frac{1}{2N} \text{Tr } \ln(C^{-1})^{\vec{y}} + \frac{1}{2N} \text{Tr } L^{\vec{y}} C\_{\vec{y}} \tag{25}$$

where

$$\int d\mathbf{x}' \, (C^{-1})^{ik}(\mathbf{x}, \mathbf{x}') C\_{kj}(\mathbf{x}', \mathbf{x}\_0) = \delta\_{ij} \delta(\mathbf{x} - \mathbf{x}\_0)$$

and we have dropped the irrelevant constant terms.

The equations of motion to order *N*−<sup>1</sup> are obtained from (19) with *J<sup>i</sup>* and *Kij* set to zero:

$$\frac{\ $S[a]}{\$ a\_{i}} + \frac{1}{2N} \frac{\ $}{\$ a\_{i}} \text{Tr} \, L^{\dot{\eta}} C\_{\dot{\eta}} = 0 \tag{26}$$

$$\frac{1}{2N}[-(C^{-1})^{\dot{\jmath}} + L^{\dot{\jmath}}] = 0\tag{27}$$

and (27) can be rewritten as

$$\int d\mathbf{x}' L^{\vec{k}}(\mathbf{x}, \mathbf{x}') C\_{\vec{k}\vec{j}}(\mathbf{x}', \mathbf{x}\_0) = \delta\_{\vec{l}\vec{j}} \delta(\mathbf{x} - \mathbf{x}\_0) \tag{28}$$

Hence, given any network of spiking neurons, we can write down a set of generalized activity equations for the mean and covariance functions by (1) constructing a neuron density function, (2) writing down the conservation law (Klimontovich equation), (3) constructing the action and (4) using formulas (26) and (28). We could have constructed these equations directly by multiplying the Klimontovich and synaptic drive equations by various factors of *u* and η and recombining. However, as we saw in Buice et al. (2010) this is not a straightforward calculation. The effective action approach makes this much more systematic and mechanical.

#### **PHASE MODEL EXAMPLE**

We now present a simple example to demonstrate the concepts and approximations involved in our expansion. Our goal is not to analyze the system *per se* but only to demonstrate the application of our method in a heuristic setting. We begin with a simple non-leaky integrate-and-fire neuron model, which responds to a global coupling variable. This is a special case of the dynamics given above, with *F* given by

$$F[\theta, \chi, u] = I(t) + \chi u \tag{29}$$

The action from (14) and (15) is

$$S[a] = \int d\theta d\gamma dt \, a\_4(\mathbf{x}) \left[\partial\_t a\_3(\mathbf{x}) + \partial\_\theta (I + \gamma a\_1(t)) \, a\_3(\mathbf{x})\right]$$

$$+ \frac{1}{N} \int dt \, a\_2(t) \left(\dot{a}\_1(t) + \beta a\_1(t) - \mathfrak{H} \int d\gamma \, (I + \gamma a\_1(t))\right)$$

$$\left[a\_4(\pi, \gamma, t) + 1\right] a\_3(\pi, \gamma, t)\tag{30}$$

and we ignore initial conditions for now.

In order to construct the generalized activity equations we need to compute the first and second derivatives of the action *L<sup>i</sup>* and *Lij*. Taking the first derivative of (30) gives

*L*1[*a*] *x, x* <sup>=</sup> <sup>δ</sup>*S*[*a(x)*] δ*a*<sup>1</sup> *(t )* = *d*θ*d*γ *dt*γ*a*4*(x)∂*θ*a*3*(x)*δ *t* − *t* + 1 *N dt a*2*(t) d dt* δ *t* − *t* + β*a*<sup>2</sup> *t* − *a*<sup>2</sup> *t* β *d*γ γ *a*4*(*π*,* γ*, t )* + 1 *a*3*(*π*,* γ*, t ) L*2[*a*]*(x, x )* <sup>=</sup> <sup>δ</sup>*S*[*a(x)*] δ*a*2*(t )* <sup>=</sup> <sup>1</sup> *N da*<sup>1</sup> *dt* <sup>+</sup> <sup>β</sup>*a*1*(<sup>t</sup> )* − β *d*γ *I* + γ*a*1*(t ) a*4*(*π*,* γ*, t )* + 1 *a*3*(*π*,* γ*, t ) L*3[*a*]*(x, x )* <sup>=</sup> <sup>δ</sup>*S*[*a(x)*] δ*a*3*(x )* <sup>=</sup> *dt a*4*(*θ *,* γ *, t)∂t*δ*(t* − *t )* + *d*θ*a*4*(*θ*,* γ *, t )∂*θ*(I* + γ *a*1*(t ))*δ*(*θ − θ *)* <sup>−</sup> <sup>β</sup> *<sup>N</sup> <sup>a</sup>*2*(<sup>t</sup> ) I* + γ *a*1*(t ) a*4*(*π*,* γ *, t )* + 1 × δ*(*π − θ *) L*4[*a*]*(x, x )* <sup>=</sup> <sup>δ</sup>*S*[*a(x)*] δ*a*4*(x )* <sup>=</sup> *<sup>∂</sup>ta*3*(x )* + *∂*θ *I* + γ *a*1*(t ) a*3*(x )* <sup>−</sup> <sup>β</sup> *<sup>N</sup> <sup>a</sup>*2*(<sup>t</sup> ) I* + γ *a*1*(t ) a*3*(*π*,* γ *, t )*δ*(*π − θ *)* (31)

The mean field equations are obtained by solving *L<sup>i</sup>* = 0 using (31). We immediately see that *a*<sup>2</sup> = *a*<sup>4</sup> = 0 are solutions, which leaves us with

$$\left(\dot{a\_1} + \beta a\_1 - \beta \int d\gamma (I + \gamma a\_1) \, a\_3(\pi, \gamma, t) = 0\right) \tag{32}$$

$$
\partial\_t a\_3 + (I + \gamma a\_1) \,\partial\_\theta a\_3 = 0 \qquad \text{(33)}
$$

The mean field equations should be compared to those of the spike response model Gerstner (1995, 2000). We can also solve (33) directly to obtain

$$a\_{\Omega}(\mathbf{x}, t) = \rho\_0 \left( \theta - \int\_{t\_0}^t dt' \left[ I\_{\Omega}(t') + \gamma a\_1(t') \right], \,\chi, \,\Omega \right),$$

where *ρ*<sup>0</sup> is the initial distribution. If the neurons are initially distributed uniformly in phase, then *ρ*<sup>0</sup> = *g(*γ*)/*2π and the mean field equations reduce to

$$
\dot{a\_1}(t) + \beta a\_1(t) - \frac{\beta}{2\pi} \left( I + \ddot{\mathbb{Y}} a\_1(t) \right) = 0 \tag{34}
$$

which has the form of the Wilson-Cowan equation, with *(*β*/*2π*)(I* + γ¯*a*1*)* acting as a gain function. Hence, the Wilson-Cowan equation is a full description of the infinitely large system limit of a network of globally coupled simple phase oscillators in the asynchronous state. For all other initial conditions, the one-neuron conservation equation (called the Vlasov equation in kinetic theory) must be included in mean field theory.

To go beyond mean field theory we need to compute *Lij(x, x , x)* = δ*Li (x, x )/*δ*aj(x)*:

*L*11[*a*] = 0 *L*12[*a*] = 1 *N* <sup>−</sup> *<sup>d</sup> dt* <sup>+</sup> <sup>β</sup> <sup>−</sup> <sup>β</sup> *d*γ γ *a*4*(*π*,* γ*, t )* + 1 *a*3*(*π*,* γ*, t )* δ*(t* − *t ) <sup>L</sup>*13[*a*] = <sup>γ</sup> *d*θ *a*4*(x)*δ*(*γ − γ*)∂*θδ*(*θ − θ*)* <sup>−</sup> <sup>β</sup> *<sup>N</sup>* <sup>γ</sup>*a*2*(<sup>t</sup> ) a*4*(*π*,* γ*, t )* + 1 δ*(*π − θ*)* δ*(t* − *t ) <sup>L</sup>*14[*a*] = <sup>γ</sup>*∂*θ*a*3*(x)* <sup>−</sup> <sup>β</sup> *<sup>N</sup>* <sup>γ</sup>*a*2*(<sup>t</sup> )a*3*(*π*,* γ*, t )*δ*(*π − θ*)* δ*(t* − *t ) L*21[*a*] = 1 *N d dt* <sup>+</sup> <sup>β</sup> <sup>−</sup> <sup>β</sup> *d*γ γ *a*4*(*π*,* γ*, t )* + 1 *a*3*(*π*,* γ*, t )* δ*(t* − *t ) L*22[*a*] = 0 *<sup>L</sup>*23[*a*]=− <sup>β</sup> *<sup>N</sup> (<sup>I</sup>* <sup>+</sup> <sup>γ</sup>*a*1*(<sup>t</sup> )) a*4*(*π*,* γ*, t ))* + 1 δ*(*π − θ *)* δ*(t* − *t ) <sup>L</sup>*24[*a*]=− <sup>β</sup> *<sup>N</sup> (<sup>I</sup>* <sup>+</sup> <sup>γ</sup>*a*1*(<sup>t</sup> ))a*3*(*π*,* γ*, t )*]δ*(*π − θ*)*δ*(t* − *t ) <sup>L</sup>*31[*a*] = *d*θ *a*4*(*θ*,* γ *, t )*γ *∂*θδ*(*θ − θ *)* <sup>−</sup> <sup>β</sup> *<sup>N</sup> <sup>a</sup>*2*(<sup>t</sup> )*γ *a*4*(*π*,* γ *, t )* + 1 δ*(*π − θ *)* δ*(t* − *t )*

$$L^{32}[a] = -\frac{\beta}{N} \left( I + \chi' a\_1(t') \right) \left( a\_4(\pi, \chi', t') + 1 \right) \delta(\pi - \theta')$$
 
$$\delta(t' - t')$$

$$\begin{aligned} L^{33}[a] &= 0 \\ L^{34}[a] &= \left[ \delta(\theta'-\theta')\partial\_{t''} - \partial\_{\theta''} \left( I + \chi' a\_1(t') \right) \delta(\theta''-\theta') \right] \\ &- \frac{\beta}{N} a\_2(t') (I + \chi' a\_1(t')) \delta(\pi - \theta') \delta(\pi - \theta'') \end{aligned}$$

$$\begin{aligned} \delta(\chi'-\chi')\delta(t''-t') \end{aligned}$$

$$\begin{aligned} L^{41}[a] &= \left[ \partial\_{\theta'} \chi' a\_3(\chi') - \frac{\beta}{N} a\_2(t') \chi' a\_3(\pi, \chi', t') \delta(\pi - \theta') \right] \\ \delta(t'-t'') \end{aligned}$$

$$\begin{aligned} L^{42}[a] &= -\frac{\beta}{N} (I + \chi' a\_1(t')) a\_3(\pi, \chi', t') \delta(\pi - \theta') \delta(t'-t'') \end{aligned}$$

$$\begin{aligned} \mathbf{^N} \\ L^{43} [a] &= \partial\_{t'} \delta(\mathbf{x'} - \mathbf{x''}) + \partial\_{\theta'} \left( I + \mathbf{y'} a\_1(t') \right) \delta(\mathbf{x'} - \mathbf{x''}) \\ &- \frac{\partial}{N} a\_2(t') \left( I + \chi a\_1(t') \right) \delta(\pi - \theta') \delta(\pi - \theta'') \end{aligned}$$

$$
\mathbb{S}(\lambda'-\lambda'')\mathbb{S}(t'-t'')\mathbb{S}
$$

*L*44[*a*] = 0

The activity equations for the means to order *N*−<sup>1</sup> are given by (26). The only non-zero contributions are given by *L*<sup>13</sup> and *L*<sup>31</sup> resulting in

$$L^2 + \frac{1}{2N} \frac{\delta}{\delta a\_2} \int d\mathbf{x} d\mathbf{x}' (L^{13} \mathbf{C}\_{13} + L^{31} \mathbf{C}\_{31}) = 0$$

$$L^4 + \frac{1}{2N} \frac{\delta}{\delta a\_4} \int d\mathbf{x} d\mathbf{x}' (L^{13} \mathbf{C}\_{13} + L^{31} \mathbf{C}\_{31}) = 0$$

since *a*<sup>2</sup> = *a*<sup>4</sup> = 0 and correlations involving response variables (even indices) will be zero for equal times. The full activity equations for the means are thus

$$
\dot{a}\_1 + \beta a\_1 - \beta \int d\gamma (I + \gamma a\_1) \, a\_3(\pi, \gamma, t)
$$

$$
$$

$$\left(\partial\_t a\_3 + (I + \chi a\_1)\,\partial\_\theta a\_3 + \frac{1}{N} \chi \partial\_\theta C(\theta, \chi, t) = 0\right) \qquad (36)$$

where *C(*θ*,* γ*, t)* = *C*13*(t*; θ*,* γ*, t)* = *C*31*(*θ*,* γ*, t*;*t)*.

We can now use the *Lij* in (28) to obtain activity equations for *Cij*. There will be sixteen coupled equations in total but the applicable non-zero ones are

 *d dt* <sup>+</sup> <sup>β</sup> <sup>−</sup> <sup>β</sup> *d*γ γ*a*3*(*π*,* γ*, t) C*<sup>11</sup> *(t*;*t*0*)* − β *d*γ *(I* + γ*a*1*) C*<sup>31</sup> *(*π*,* γ*, t*;*t*0*)* − β *d*γ *(I* + γ*a*1*(t)) a*3*(*π*,* γ*, t)C*41*(*π*,* γ*, t*;*t*0*)* = 0 (37) *d dt* <sup>+</sup> <sup>β</sup> <sup>−</sup> <sup>β</sup> *d*γ γ*a*3*(*π*,* γ*, t) C*13*(t*; *x*0*)* − β *d*γ *(I* + γ*a*1*) C*<sup>33</sup> *(*π*,* γ*, t*; *x*0*)* − β *d*γ *(I* + γ*a*1*(t)) a*3*(*π*,* γ*, t)C*<sup>43</sup> *(*π*,* γ*, t*; *x*0*)* = 0 (38) γ*∂*θ*a*3*(x)C*<sup>11</sup> *(t*;*t*0*)* + [*∂<sup>t</sup>* + *(I* + γ*a*1*) ∂*θ] *C*31*(x*;*t*0*)* <sup>−</sup> <sup>β</sup> *<sup>N</sup> (<sup>I</sup>* <sup>+</sup> <sup>γ</sup>*a*1*(t)) <sup>a</sup>*3*(*π*,* <sup>γ</sup>*, <sup>t</sup>)*δ*(*<sup>π</sup> <sup>−</sup> <sup>θ</sup>*)C*21*(t, <sup>t</sup>*0*)* <sup>=</sup> 0 (39) γ*∂*θ*a*3*(x)C*13*(t*; *x*0*)* + [*∂<sup>t</sup>* + *(I* + γ*a*1*(t)) ∂*θ] *C*33*(x, x*0*)* <sup>−</sup> <sup>β</sup> *<sup>N</sup> (<sup>I</sup>* <sup>+</sup> <sup>γ</sup>*a*1*(t)) <sup>a</sup>*3*(*π*,* <sup>γ</sup>*, <sup>t</sup>)*δ*(*<sup>π</sup> <sup>−</sup> <sup>θ</sup>*)C*23*(t, <sup>x</sup>*0*)* <sup>=</sup> 0 (40)

Adding (38) and (39) and taking the limit *t*<sup>0</sup> → *t* and setting θ<sup>0</sup> = θ, γ<sup>0</sup> = γ gives

$$
\partial\_t C(\theta, \chi, t) + \left[ \mathfrak{F} - \mathfrak{F} \int d\gamma' \,\chi' a\_3(\pi, \gamma', t) + (I + \chi a\_1) \,\partial\_{\theta} \right],
$$

$$\begin{aligned} &C(\theta,\chi,t) - \theta \int d\chi' \left(I + \chi' a\_1\right) C\_{33}(\pi,\chi',t;x) \\ &-2\theta \left(I + \chi a\_1(t)\right) a\_3(\pi,\chi,t) \delta(\pi-\theta) + \chi \partial\_\theta a\_3(\chi) C\_{11}(t;t) = 0 \end{aligned}$$

where we use the fact that *C*21*(t, t )* = *N* and *C*43*(x*; *x )* = δ*(*θ − θ *)*δ*(*γ − γ *)* in the limit of *t* approaching *t* from below and equal to zero when approaching from above. Adding (37) and (40) to themselves with *t* and *t*<sup>0</sup> interchanged and taking the limit of *t*<sup>0</sup> approaching *t* gives

$$
\left[\frac{d}{dt} + 2\mathfrak{H} - 2\mathfrak{H} \int d\mathbf{y} \,\gamma a\_3(\mathfrak{m}, \mathbf{y}, t) \right] C\_{11}(t; t)
$$

$$
$$

$$
\left[\mathfrak{d}\_l + \left(I + \chi a\_1(t)\right) \partial\_{\theta}\right] C\_{33}(\mathbf{x}; \mathbf{x}) + 2\chi [\partial\_{\theta} a\_3(\mathbf{x})] C(\mathbf{x}) = 0
$$

because *C*41*(x*;*t)* = 0 and *C*23*(t*; *x)* = 0. Putting this all together, we get the generalized activity equations

$$\frac{da\_1}{dt} + \beta a\_1(t) - \beta \int d\gamma \,(I + \gamma a\_1(t)) \, a\_3(\pi, \gamma, t)$$

$$-\frac{\beta}{N} \int d\gamma \,\chi C(\pi, \gamma, t) = 0\tag{41}$$

*∂ta*3*(*θ*,* γ*, t)* + *(I* + γ*a*1*) ∂*θ*a*3*(*θ*,* γ*, t)*

$$+\frac{1}{N}\chi\partial\_{\theta}C(\theta,\chi,t) = 0\tag{42}$$

$$
\partial\_t C(\theta, \chi, t) + \left[\beta - \beta \int d\gamma' \chi' a\_3(\pi, \chi', t) + (I + \chi a\_1) \, \partial\_\theta \right]
$$

$$
\begin{split}
&C(\theta, \chi, t) - \beta \int d\gamma' \left(I + \chi' a\_1\right) C\_{33}(\pi, \chi', t; \theta, \chi, t) \\
&- 2\beta \left(I + \chi a\_1(t)\right) a\_3(\theta, \chi, t) \delta(\pi - \theta) \\
&+ \gamma \partial\_\theta a\_3(\theta, \chi, t) C\_{11}(t; t) = 0
\end{split}
\tag{43}
$$

$$\left[\frac{d}{dt} + 2\mathfrak{H} - 2\mathfrak{H} \int d\boldsymbol{\chi} \,\boldsymbol{\chi} a\_3(\boldsymbol{\pi}, \boldsymbol{\chi}, t) \right] \mathcal{C}\_{11}(t; t)$$

$$-2\mathfrak{H} \int d\boldsymbol{\chi} \,(\boldsymbol{I} + \boldsymbol{\chi} a\_1) \, \boldsymbol{C}(\boldsymbol{\pi}, \boldsymbol{\chi}, t) = 0 \tag{44}$$

$$\left[\partial\_t + (I + \gamma a\_1(t))\,\partial\_\theta\right]C\_{33}(\theta, \chi, t; \theta, \chi, t)$$

$$+2\gamma\partial\_\theta a\_3(\theta, \chi, t)C(\theta, \chi, t) = 0\tag{45}$$

Initial conditions, which are specified in the action, are required for each of these equations. The derivation of these equations using classical means require careful consideration for each particular model. Our method provides a blanket mechanistic algorithm. We propose that these equations represent a new scheme for studying neural networks.

Equations 41–45 are the complete self-consistent generalized activity equations for the mean and correlations to order *N*<sup>−</sup>1. It is a system of partial differential equations in *t* and θ. These equations can be directly analyzed or numerically simulated. Although the equations seem complicated, one must bear in mind that they represent the dynamics of the system averaged over initial conditions and unknown parameters. Hence, the solution of this PDE system replaces multiple simulations of the original system. In previous work, we required over a million simulations of the original system to obtained adequate statistics Buice and Chow (2013b). There is also a possibility that simplifying approximations can be applied to such systems. The system has complete phase memory because the original system was fully deterministic. However, the inclusion of stochastic effects will shorten the memory and possibly simplify the dynamics. It will pose no problem to include such stochastic effects. In fact, the formalism is actually more suited for stochastic systems Buice et al. (2010).

## **DISCUSSION**

The main goal of this paper was to show how to systematically derive generalized activity equations for the ensemble averaged moments of a deterministically coupled network of spiking neurons. Our method utilizes a path integral formalism that makes the process algorithmic. The resulting equations could be derived using more conventional perturbative methods although possibly with more calculational difficulty as we found before Buice et al. (2010). For example, for the case of the stochastic spike model, Buice et al. (2010) presumed that the Wilson-Cowan activity variable was the rate of a Poisson process and derived a system of generalized activity equations that corresponded to deviations around Poisson firing. Bressloff (2010), on the other hand, assumed that the Wilson-Cowan activity variable was a mean density and used a system-size expansion to derive an alternative set of generalized activity equations for the spike model. The classical derivations of these two interpretations look quite different and the differences and similarities between them are not readily apparent. However, the connections between the two types of expansions are very transparent using the path integral formalism.

Here, we derived equations for the rate and covariances (first and second cumulants) of a deterministic synaptically coupled spiking network as a system size expansion to first order. However, our method is not restricted to these choices. What is particularly advantageous about the path integral formalism is that it is straightforward to generalize to include higher order cumulants, extend to higher orders in the inverse system size, or to expand in other small parameters such as the inverse of a slow time scale. The action fully specifies the system and all questions regarding the system can be addressed with it.

To give a concrete illustration of the method, we derived the self-consistent generalized activity equations for the rates and covariances to order *N*−<sup>1</sup> for a simple phase model. The resulting equations consist of ordinary and partial differential equations. This is to be expected since the original system was fully deterministic and memory cannot be lost. Even mean field theory requires the solution of an advective partial differential equation. The properties of these and similar equations remain to be explored computationally and analytically. The system is possibly simpler near the asynchronous state, which is marginally stable in mean field theory like the Kuramoto model Strogatz and Mirollo (1991) and like the Kuramoto model, we conjecture that the finite size effects will stabilize the asynchronous state Buice and Chow (2007); Hildebrand et al. (2007). The addition of noise will also stabilize the asynchronous state. Near asynchrony could be exploited to generate simplified versions of the asynchronous state.

We considered a globally connected network, which allowed us to assume that networks for different parameter values and initial conditions converge toward a "typical" system in the large *N* limit. However, this property may not hold for more realistic networks. While the formalism describing the ensemble average will hold regardless of this assumption, the utility of the equations as descriptions of a particular network behavior may suffer. For example, heterogeneity in the connectivity (as opposed to the global connectivity we consider here) may threaten this assumption. This is the case with so called "chaotic random networks" Sompolinsky et al. (1988) in which there is a spin-glass transition owing to the variance of the connectivity crossing a critical threshold. This results in the loss of a "typical" system in the large *N* limit requiring an effective stochastic equation which incorporates the noise induced by the network heterogeneity. Whether the expansion we present here is useful without further consideration depends upon whether the network heterogeneity induces this sort of effect. This is an area for future work. A simpler issue arises when there are a small discrete number of "typical" systems (such as with bistable solutions to the continuity equation). In this case, there are noise induced transitions between states. While the formalism has a means of computing this transition Elgart and Kamenev (2004), we do not consider this case here.

An alternative means to incorporate heterogeneous connections is to consider a network of coupled systems. In such a network, a set of generalized activity equations, such as those derived here or simplified versions, would be derived for each local system, together with equations governing the covariances between the local systems. Correlation based learning dynamics could then be imposed on the connections between the local systems. Such a network could serve as a generalization of current rate based neural networks to include the effects of spike correlations with applications to both neuroscience and machine learning.

#### **ACKNOWLEDGMENTS**

This work was supported by the Intramural Research Program of the NIH, NIDDK.

#### **REFERENCES**


Strogatz, S., and Mirollo, R. (1991). Stability of incoherence in a population of coupled oscillators. *J. Stat. Phys.* 63, 613–635. doi: 10.1007/BF01029202


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 29 April 2013; accepted: 23 October 2013; published online: November 2013. 15*

*Citation: Buice MA and Chow CC (2013) Generalized activity equations for spiking neural network dynamics. Front. Comput. Neurosci. 7:162. doi: 10.3389/fncom. 2013.00162*

*This article was submitted to the journal Frontiers in Computational Neuroscience. Copyright © 2013 Buice and Chow. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Stable learning of functional maps in self-organizing spiking neural networks with continuous synaptic plasticity

## *Narayan Srinivasa\* and Qin Jiang*

*Center for Neural and Emergent Systems, HRL Laboratories LLC, Malibu, CA, USA*

#### *Edited by:*

*Cornelius Glackin, University of Hertfordshire, UK*

#### *Reviewed by:*

*James A. Bednar, University of Edinburgh, UK Cees Van Leeuwen, Katholieke Universiteit Leuven, Belgium*

#### *\*Correspondence:*

*Narayan Srinivasa, Center for Neural and Emergent Systems, HRL Laboratories LLC, 3011 Malibu Canyon Road, Malibu, CA 91388, USA.*

*e-mail: nsrinivasa@hrl.com*

This study describes a spiking model that self-organizes for stable formation and maintenance of orientation and ocular dominance maps in the visual cortex (V1). This self-organization process simulates three development phases: an early experience-independent phase, a late experience-independent phase and a subsequent refinement phase during which experience acts to shape the map properties. The ocular dominance maps that emerge accommodate the two sets of monocular inputs that arise from the lateral geniculate nucleus (LGN) to layer 4 of V1. The orientation selectivity maps that emerge feature well-developed iso-orientation domains and fractures. During the last two phases of development the orientation preferences at some locations appear to rotate continuously through ±180◦ along circular paths and referred to as pinwheel-like patterns but without any corresponding point discontinuities in the orientation gradient maps. The formation of these functional maps is driven by balanced excitatory and inhibitory currents that are established via synaptic plasticity based on spike timing for both excitatory and inhibitory synapses. The stability and maintenance of the formed maps with continuous synaptic plasticity is enabled by homeostasis caused by inhibitory plasticity. However, a prolonged exposure to repeated stimuli does alter the formed maps over time due to plasticity. The results from this study suggest that continuous synaptic plasticity in both excitatory neurons and interneurons could play a critical role in the formation, stability, and maintenance of functional maps in the cortex.

**Keywords: spiking networks, STDP, learning, functional maps, orientation selectivity, ocular dominance, stability**

## **INTRODUCTION**

The spatial and temporal properties of a distributed pattern of neural activity in V1 with differentially tuned responses of individual neurons to features of visual space such as orientation, spatial frequency, and direction of motion were first recognized by Hubel and Wiesel (1962, 1963, 1968, 2005). Using microelectrodes and neuroanatomical tracers, they established that the neural activity of a population of neurons with such differentiation represented *functional maps*. This seminal work has inspired a large body of subsequent research to understand the properties of these functional maps and its relation to cortical function as well as to understand the mechanisms by which these maps are formed during development (Blakemore and Cooper, 1970; Miller, 1996; Rao et al., 1997; Buonamano and Merzenich, 1998; Miller et al., 1999; Yuste and Sur, 1999; Basole et al., 2003, 2006; Hensch, 2005; Schummers et al., 2005; Yu et al., 2005; Shapley et al., 2007; White and Fitzpatrick, 2007; Huberman et al., 2008; Xing et al., 2011).

Two types of functional maps have been particularly well explored. The first map called the ocular dominance maps, or ODM, is based on interactions between axons of the neurons in the lateral geniculate nucleus (LGN) and neurons in layer 4 of V1. Here clusters of thalamocortical axon terminals that serve the left or right eye are organized in layer 4 via the topological relations established in the LGN to form ODMs. The second functional map called the orientation selectivity map, or OSM, is a map of orientation preference that is elaborated with a high degree of selectivity in V1 but not in the LGN. A key mechanism implicated in the formation of these functional maps during development is activity-dependent plasticity (Purves and Lichtman, 1985; Katz and Shatz, 1996; Ruthazer and Stryker, 1996; Crair et al., 1998; Crowley and Katz, 1999).

A number of *in vitro* experimental studies (Levy and Steward, 1983; Magee and Johnston, 1997; Markram et al., 1997; Bi and Poo, 1998; Debbane et al., 1998; Caporale and Dan, 2008) suggest that repeated pairing of pre- and postsynaptic activity in the form of action potentials, or spikes, can lead to long-term changes in synaptic efficacy. The sign and magnitude of the change in synaptic efficacy depends upon on the relative timing between the preand postsynaptic spikes and is known as spike-timing-dependent plasticity (STDP). STDP is now a well-established physiological mechanism of activity-driven synaptic regulation *in vivo* as well as observed in the Xenopus visual system (Mu and Poo, 2006; Vislay-Meltzer et al., 2006), the locust mushroom body (Cassenaer and Laurent, 2007), and rat visual (Meliza and Dan, 2006) and barrel (Jacob et al., 2007) cortex. STDP has also shown to have better explanatory power than more conventional Hebbian correlationbased plasticity at explaining both cortical reorganization in cat primary visual cortex (Young et al., 2007) and connectivity in locust olfactory system (Finelli et al., 2008). The STDP is a local learning rule that forces synapses to compete such that the spiking activity of a post-synaptic neuron becomes selective to a small subset of pre-synaptic input spikes. This feature was exploited in some spiking models to demonstrate map development (Bartsch and van Hemmen, 2000; Song and Abbott, 2001; Billings and van Rossum, 2009).

In the present study a spiking model is described that provides a plausible set of mechanisms based on STDP for the formation and maintenance of ODMs and OSMs. This developmental model simulates functional map formation during three phases: an early experience-independent phase, a late experienceindependent phase, and a subsequent refinement phase during which experience acts to shape map properties. There are other models that have also modeled developmental phases of functional maps in the visual cortex (Sirosh and Miikkulainen, 1995; Bauer et al., 2000; Swindale, 2000; Bednar and Miikkulainen, 2004; Yang et al., 2012). However, we present a spiking model that employs STDP as the basis to form and stabilize functional maps across the three phases of development. The resulting ODMs represent V1 neurons that show eye selectivity in response to two sets of monocular inputs from the LGN. The OSMs on the other hand feature pin wheel-like patterns to represent orientation preference in a smooth and continuous fashion. However, point discontinuities that appear at the center of these pin wheels in animal data (Maldonado et al., 1997) is not found in our model simulations. It also contains other forms of discontinuities such as fractures and breaks as will be defined in the next section.

## **MATERIALS AND METHODS**

For the purpose of simplicity and clarity, the focus of this study will be on the dynamics of interaction between the neurons in LGN and layer 4 in V1. The model is developed in three phases where the first and second phases correspond to a *pre-critical* period while the third phase corresponds to a *critical* period (Hensch, 2005). In the early experience-independent phase, spontaneously generated neuronal activity in the cortex and LGN facilitates activity-dependent plasticity and formation of OSMs and ODMs (Wiesel and Hubel, 1974; Chapman et al., 1996; Crair et al., 1997; Ferster and Miller, 2000; Trachtenberg et al., 2000; Chiu and Weliky, 2001; Huberman et al., 2006). This is followed by a late-experience-independent phase where interactions between LGN and layer 4 are driven by the influence of retinal waves (Godfrey and Swindale, 2007) to enable the refinement of ODMs and OSMs (Crair et al., 1997; Crowley and Katz, 1999, 2002; Butts, 2002; Katz and Crowley, 2002; Huberman et al., 2006; Feller, 2009). In the final experience-dependent phase, neuronal activity is driven by natural visual stimuli from the environment that drives the maturation of the already formed ODMs and OSMs (Crair et al., 1998; Sengpiel et al., 1999; White et al., 2001; Coppola and White, 2004; Smith and Trachtenberg, 2007; White and Fitzpatrick, 2007). It should be noted that the model results are meant to show that the maturation of the formed OSMs and ODMs are driven by activity-dependent plasticity while qualitatively simulating some of the process constraints (such as the influence of retinal waves) during various stages of development. These maps have some similar qualitative properties as those found in animals but this does not imply actual adherence to the maps or process in any particular species.

## **MODEL ARCHITECTURE**

The spiking model architecture in this study assumes that the initial structure of connections despite being random and local in nature is nevertheless present from the beginning. It is known that this initial formation of the map depends upon molecular gradients that serve as guides for axons to topologically appropriate portions of the map (Yuste and Sur, 1999; Crowley and Katz, 2002; Hensch, 2005; Taha and Stryker, 2005; Huberman et al., 2006; White and Fitzpatrick, 2007). The model also assumes that the neurons are mature unlike in reality where neurons are immature during very early stages of development and are characterized by a high concentration of Cl<sup>−</sup> ions as a result of which all neurons are depolarizing (Hensch, 2005).

The model is designed to address the thalamocortical circuit where thalamic afferents from the LGN activate the principal cells of layer 4 of V1 via geniculocortical synapses (Antonini and Stryker, 1978; Ursey et al., 1999; Yuste and Sur, 1999; Bartsch and van Hemmen, 2000). For convenience, the principal cells within layer 4 will be referred to as *E* neurons while all the inhibitory interneurons will be referred to as *I* neurons throughout the article. The *E* neurons are connected to other local *E* neurons and *I* neurons to form a dense local recurrent network. In our model, the *E* and *I* neurons will make up two sub-layers within layer 4. Similarly the LGN in the full model architecture (**Figure 1**) is also modeled as an Excitatory-Inhibitory (*E-I*) network. The *E-I* network model is a commonly used design to simulate models of thalamocortical areas (Binzegger et al., 2004; Kremkow et al., 2007; Kumar et al., 2008). The synapses in the model are plastic throughout all phases of development (Hensch, 2005) and the self-organization process refers to the change in the synaptic conductance during development.

In general, neurons in the cerebral cortex are densely connected to neurons close to it and sparsely connected to neurons far away from it (Schummers et al., 2004; Song et al., 2005; Perin et al., 2011; Voges and Perrinet, 2012). In particular, models of cortical function often assume that cortical circuitry acts in a center–surround fashion, whereas separated pairs of cells have a mutually suppressive influence. Further, to make the enhanced cortical patterns congruent with the sensory representation of the system, the cortical interactions must depend on the functional distance between the cells, determined by the features coded by them. This functional circuitry, known as "Mexican hat" organization, has been adopted in network models of orientation selectivity (Somers et al., 1995; Kang et al., 2003). In our model, we make a similar assumption where the Gaussian distribution of synaptic connections for the excitatory or glutamatergic synapses is dense and narrow in its spatial extent (**Figure 2A**) compared to the inhibitory or GABAergic synapses which are more broadly distributed.

The *E* neurons in layer 4 (**Figure 2B**) are connected to its neighboring *E* neurons and *I* neurons in *I* layer of layer 4. Similarly, *I* neurons in layer 4 are connected to its neighboring *I* neurons as well as the *E* neurons in layer 4. Thus there are four types of synapses depending on the pre- and post-synaptic neuron: *E* → *E, E* → *I, I* → *E*, and *I* → *I*. The first two types of synapses are excitatory in nature and obey E-STDP rule while the last two synapses are inhibitory in nature and obey the I-STDP

**where thalamic afferents from the LGN activate the principal cells of layer 4 of V1 via geniculocortical synapses.** The layer 4 and LGN are both modeled as an *E-I* network. There are two inhibitory populations in layer 4: the feedback inhibitory population *I*1, which does not receive any inputs from LGN but only from the *E* neurons of layer 4 and the feedforward inhibitory population *I*2, which does. The LGN receives spikes from retinal ganglion cells (RGC). The LGN and layer 4 neurons in the model are separated by the dashed line in the figure. This complete model is used for simulating the

rule for plasticity (Woodin et al., 2003; Caporale and Dan, 2008; Hartmann et al., 2008). The synaptic connections are initialized with random synaptic strengths and obey the connection density as prescribed by the Gaussian distribution shown in **Figure 2A** for all four types of synapses. An example of a 10 × 10 set of *E* neurons in layer 4 with *E* → *E* synaptic connectivity is provided in **Figure 2D**. Without loss of generality, the *E* and *I* layers in layer 4 in all our simulations will consist of a 2-D sheet of 128 × 128 neurons. The LGN in the full model (**Figure 1**) is also composed of an *E-I* network similar to layer 4. The *E* and *I* layers within the LGN consist of 48 × 48 neurons and there are two such *E*-*I* networks in the LGN corresponding to each eye. Each LGN *E* neuron from both the eyes makes sparse and random synaptic connections to *E* neurons in layer 4 (**Figure 2C**). In addition to these LGN → *E* synaptic connections, the LGN neurons are also connected to a feedforward inhibitory population of neurons which provide feedforward inhibition to the same *E* neighborhood in layer 4. Feedforward inhibition is known to play a role in input normalization and expansion of cortical dynamic range (Pouille et al., 2009). The ultimate distribution of synaptic strengths in model synapses is dictated by E-STDP and I-STDP during the developmental process.

The spiking model simulations were performed using the HRLSim (Minkovich et al., under revision) which is a multiple graphical processing unit (GPU) based spiking simulator in C++. This simulator is an extension of a single GPU developed previously (Nageswaran et al., 2009) and uses an MPI interface and other optimizations to enable scalable and real-time simulation of large scale spiking neural networks. The computations to estimate the ODM and OSM were performed in MATLAB. The details experience-dependent phase of development. For each network layer, 60% of randomly chosen neurons are injected with background noise in form of currents (*I*inj) for 30 ms. A new of set of 60% randomly chosen neurons at all layers are selected again after that and are injected with background noise. This process is repeated throughout all three phases of development. For simulating early experience-independent phase (Phase 1) the complete network is purely driven by the background noise at both LGN and layer 4. For simulating late experience-independent phase (Phase 2) the LGN is activated by spikes due to retinal waves from RGC.

of the neuronal and synaptic mechanisms and the performance metrics and measures used for the various experiments are now provided.

## **NEURON MODEL**

In this study the leaky integrate and fire neuron model (Vogels et al., 2005) is used where each neuron receives multiple input current signals and the dynamics of its membrane potential *V* can be determined as:

$$\pi\_m \frac{dV}{dt} = (V\_{\text{rest}} - V) + \sum \mathbf{w}(t)(E\_{\text{ex}} - V) - \sum \mathbf{z}(t)(E\_m - V) + I\_{\text{inj}} \tag{1}$$

when *V* reaches a threshold voltage *VT*, the neuron fires a spike, and *V* is reset to *V*reset. This basic model provides several control variables for the membrane voltage including synaptic conductance *w* (excitatory) and *z* (inhibitory), membrane time constant τ*m*, reversal potentials *E*ex and *E*in, and resting voltage *V*rest. The parameter *I*inj represents the current injected into the neuron. Synaptic inputs to the neuron are modeled as conductance changes where a single pre-synaptic spike at time *t* generates a synaptic conductance for excitatory and inhibitory synapses as follows:

$$\omega(t) = \omega e^{\frac{-t}{t\_{\text{AMPA}}}} \tag{2}$$

$$z(t) = z e^{\frac{-t}{\text{r}\_{\text{CABA}}}} \tag{3}$$

where the time constants τAMPA and τGABA are used to model the kinetics of AMPA and GABA receptors. The value of the excitatory and inhibitory synaptic conductance *w* and *z* in Equation (1)

**FIGURE 2 | The synaptic connectivity density distribution for the network. (A)** From any *E*-neuron to any other neuron (*E* or *I*) in shown in green and from any *I*-neuron to any other neuron (*E* or *I*) is shown in red. **(B)** There are four types of synapses depending on the pre- and post-synaptic neuron: *E* → *E, E* → *I, I* → *E*, and *I* → *I*. It should be noted that the network has periodic boundary conditions such that topmost and bottommost neurons are regarded as neighbors, as are the leftmost and rightmost columns within each layer. The neighborhood around each *E* or *I* neuron is shown as a dotted circle. **(C)** The LGN network is also an *E-I* network as shown here with mutually inhibiting connections between neurons that receive inputs from the RGCs (not shown) from the left and right eye. Each LGN neuron from both eyes project to a neuron and its neighborhood in

layer 4. For convenience, only one such projection is shown here. The LGN network (2 × 48 × 48) is smaller than the layer 4 network (128 × 128). In addition, the LGN inputs from the left and right eye populations project to the *I*<sup>2</sup> population in layer 4 which consists of 48 × 48 inhibitory neurons as well. **(D)** The synaptic connections for a set of 10 × 10*E* neurons in layer 4 are shown here. The red square shows a single *E* neuron with a 19 × 19 neighborhood. The white pixels within each such square indicate synaptic connections with maximum synaptic strength while black pixels indicate synaptic connections with zero synaptic strength. A closer look at the 19 × 19 neighborhood for one of *E*-neuron shows the initial strengths of synapses from the *E*-neuron in the center to its neighboring *E* neurons. These synaptic strengths are randomly distributed.

**Table 1 | Neuron parameters.**


is determined by STDP. **Table 1** provides a list of values for all the constants used to simulate the neuron model. During model simulations, the continuous membrane equation is updated using Euler integration with a time step of 1 ms.

#### **EXCITATORY STDP**

The E-STDP function modulates the excitatory synaptic conductance *w* based on the timing difference (*t*pre − *t*post), or *t*, between the spike times of pre- and post-synaptic neuron (**Figure 3A**). The control parameters (*A*+, *A*−, τ+, τ−) can be used to modify the amount of potentiation and depression (**Table 2**). The synaptic conductance is computed as:

$$
\omega\_{\text{new}} = \omega\_{\text{old}} + \Delta \boldsymbol{w} \tag{4}
$$

where

$$
\Delta \mathbf{w} = \mathbf{g}\_{\text{max}} \times F(\Delta t) \tag{5}
$$

and

$$F(\Delta t) = \begin{cases} A\_+ \times \exp\left(\frac{\Delta t}{\mathfrak{t}\_+}\right), & \text{if } \Delta t < 0 \\ -A\_- \times \exp\left(\frac{\Delta t}{\mathfrak{t}\_-}\right), & \text{if } \Delta t \ge 0 \end{cases} \tag{6}$$

If *w*new *> g<sup>E</sup>* max, then *w*max = *g<sup>E</sup>* max. On the other hand if *w*new *<* 0, then *w*new = 0. The factor β = |*A*−τ−|*/*|*A*+τ+| controls the relative amounts of depression to potentiation during learning.

#### **INHIBITORY STDP**

The I-STDP function modulates the inhibitory synaptic conductance *z* based on the timing difference  *t* between the spike times of pre- and post-synaptic neuron (**Figure 3B**). The synaptic conductance is computed as:

$$z\_{\text{new}} = z\_{\text{old}} + \Delta z \tag{7}$$

The change  *z* = *B*<sup>+</sup> is more inhibitory when −λ ≤ *t* ≤ λ. The change *z* = *B*<sup>−</sup> is less inhibitory when −λ − δ ≤  *t <* −λ or λ *< t* ≤ λ + δ. There is no change in inhibition if either *t* ≤ −λ − δ or *t* ≥ λ + δ . If *z*new *<* 0 then *z*new = 0. On the other hand, if *z*new *> g<sup>I</sup>* max then *z*new = *g<sup>I</sup>* max. The control parameters (B+, B−, λ, and δ) (see **Table 2**) can be used to vary the relative amounts of potentiation and depression during learning.

#### **MODEL SYNAPTIC CONNECTIVITY**

The synaptic connectivity in layer 4 and LGN is initialized using a Gaussian density function as:

$$p(\mathbf{x}, \boldsymbol{y}) = \boldsymbol{a} \times \boldsymbol{e}^{\frac{-\|(\mathbf{x} - \boldsymbol{x}\_0)^2 + (\boldsymbol{y} - \boldsymbol{y}\_0)^2\|}{2 \times \boldsymbol{e}}} \tag{8}$$

The point (*x*0*, y*0) represents the center neuron and (*x, y*) represents the position of neurons within its neighborhood. The constants α and σ (**Table 3**) control the maximal distribution probability and distribution variation.

#### **BACKGROUND SIGNAL AND NOISE**

A random voltage injection is used to mimic cortical input spikes generated from brain regions (or background) as well as noise. The injected voltage *V*inj*(t)* is modeled as:

$$V\_{\text{inj}}(t) = Au(r), \quad t \in [t\_1, t\_1 + \Delta t] \tag{9}$$

$$\text{with}$$

$$\mu(r) = \begin{cases} 1 & r \ge \mu \\ 0 & \text{otherwise} \end{cases} \tag{10}$$

The constant *A* determines the amplitude of voltage injection. The variable *r* is a random variable uniformly distributed in the range (0, 1.0). The parameter *t* (**Table 3**) is the voltage injection duration while the parameter μ represents the percentage of neurons that receive voltage injection randomly. Thus, the injection time duration *t* is used to set an injection frequency. In our model, before the eyes open, the spiking activity of neurons is driven primarily by random current injection *I*inj (in Equation 1) by multiplying *V*inj*(t)* into currents using a fixed synaptic conductance constant of 0.00125 nS.

#### **RETINAL WAVE MODEL**

Before the onset of stimulus driven activity, which helps refine neural organization in later developmental stages, neural circuits generate spontaneous patterns of activity which guide early development (Katz and Shatz, 1996). In the retina, spontaneous activity takes the form of coordinated bursts of spikes in the neighboring retinal ganglion cells (RGC) that slowly spread across the retina. They can initiate at any retinal location and cover the entire retina in minutes (Schiller, 1992; Butts, 2002; Godfrey and Swindale, 2007; Feller, 2009). The retinal waves were generated in our model as follows. The retina is assumed to have 48 × 48 neurons or RGCs. For each eye, a randomly chosen set of *N* RGCs are selected for retinal wave initiation. For each initiation site, a direction *d* is selected for wave propagation from a set of eight possible directions (i. e., 0◦, 45◦, 90◦, 135◦, 180◦, 225◦, 270◦, and 315◦). At each initiation site, RGCs within a neighborhood of size 10 × 10 are then activated at the first time step (*t* = 1 ms). The strength of the stimulation is varied in a Gaussian fashion where neighbors at closer distance are stimulated more strongly than neighbors that are further apart. In the next time step (*t* = 2 ms) the next RGC to be stimulated is selected after moving by two steps (i.e., velocity is 2 steps/ms) in the direction *d*. The newly stimulated RGC is used to activate

**FIGURE 3 | (A)** The E-STDP function modulates the excitatory synaptic conductance *w* based on the timing difference (*t* pre *<sup>i</sup>* − *t* post *<sup>j</sup>* ) between the action potentials of pre-synaptic neuron *i* and post-synaptic neuron *j*. The control parameters (*A*+, *A*−, τ+, and τ−) can be used to modify the amount of potentiation and depression (see "Materials and Methods"). **(B)** The I-STDP function modulates the inhibitory synaptic

conductance *z* based on the timing difference (*t <sup>i</sup>* − *t <sup>j</sup>* ) between the action potentials of pre-synaptic neuron *i* and post-synaptic neuron *j* (see "Materials and Methods"). If the timing difference is *>* λ, then the synapses become less inhibitory and the change itself is of a smaller magnitude while it is the opposite case when the timing difference is *<* λ.

RGCs within a 10 × 10 neighborhood and the process is continued. The total duration for each retinal wave for each eye is set to 10 ms. The activity of the RGCs is converted into spike trains as described below. The process is continued to generate several retinal waves during the late activity-independent developmental phase.

## **SPIKE ENCODING OF RETINAL WAVES AND NATURAL IMAGES**

The input visual images are in the form of either retinal waves or other natural images. For natural images, we use the Caltech 101 image dataset (Fei et al., 2006). A total of two thousand 48 × 48 images were cropped from the database as follows. A subset of images was selected from the database such that the central portion of the image had some texture and contrast in them. These original images of size 320 × 200 were down-sampled to 128 × 128 images and then a 48 × 48 portion from the center of these images (see **Figure 11**) was extracted. These 48 × 48 images were provided as RGC inputs to the LGN after spike encoding the images as described below. While the Caltech 101 dataset was developed for object recognition purposes, the statistics of the images extracted is still representative of natural images in general.

We created stereo pairs from every image in the dataset by shifting the right eye image by a maximum of 10 pixels to the left or right and 10 pixels to the top or bottom with respect to the left image. This simulates disparity in the two images. In addition to disparities, we also scale the right image by a small scaling factor between 1 and 1.05 to simulate small perspective changes. Such

**Table 2 | STDP parameters.**


#### **Table 3 | Other model parameters.**


affine transforms are commonly used in computer vision (Zhang and Xu, 1998).

The images are converted into spike sequences by an encoding process, and the sequences serve as input to neurons in the LGN layer. The encoding process is generated based on Poisson statistics where the Poisson distribution is used to generate the interspike interval (ISI) for each pixel. The mean value of pixel intensities in the image serves as the mean value of the Poisson distribution. The Poisson distribution is given as:

$$P(\mathbf{x}) = \frac{\lambda^x e^{-\lambda}}{\mathbf{x}!} \tag{11}$$

where λ is the mean value of pixel intensities and *x* is the ISI. To generate the spikes for a given pixel at a given time, we randomly select a probability value *P*(*x*) at each time step and then compute *x* using Equation (11). This sets the time step at which the next spike is generated as part of the encoding process.

#### **RECURRENT CORTICAL MAP COMPUTATION**

The recurrent cortical map (RCM) computation is based on the synaptic conductances between *E* → *E* neurons in layer 4. The purpose of RCM computation is to identify evolution of local synaptic connectivity within the *E* sub-layer of layer 4 during the various phases of development. The RCM is estimated by using a Gaussian bar function as an orientation template (Bartsch and van Hemmen, 2000) to search for the best orientation match within its neighborhood. The *E* neurons are color coded based on the best match score. The resulting image of the color coded *E* neurons constitutes a RCM.

Assuming that an *E* neuron is located at *p*, the Gaussian bar function is given by:

$$G\_{\rm xy}(\phi, p) = \exp\left\{\frac{-\|\mathbf{x}\cos\phi + \mathbf{y}\sin\phi - \mathbf{y}\|^2}{2 \times \sigma\_a^2}\right\}$$

$$\times \exp\left\{\frac{-\|\mathbf{y}\cos\phi - \mathbf{x}\sin\phi\|^2}{2 \times \sigma\_b^2}\right\} - G\_0(\phi, p) \quad (12)$$

where the variables *x* and *y* varies within *E*-neuron's locally connected neighborhood and  *xy Gxy(*φ*,* γ*)* = 0. For each of the four orientations, φ ∈ {0◦*,* 45◦*,* 90◦*,* 135◦}, the parameter γ is varied to determine the maximal orientation match as:

$$R\left(\phi\right) = \max\_{\mathcal{P}} \left[ \sum\_{\mathcal{X}} \,\,\!\!\nu\left(\mathbf{x},\boldsymbol{\mathcal{Y}}\right) \,\!\!G\_{\mathcal{X}}\left(\phi,\,\,\boldsymbol{\mathcal{Y}}\right) \right] \tag{13}$$

With the maximal orientation match, a direction vector <sup>→</sup> *d (*φ*)* = *(R(*φ*),* 2φ*)* is constructed for each orientation. Then, the four direction vectors of the four orientations are combined into a final direction vector as:

$$\stackrel{\rightarrow}{S} = (\mathbb{R}\_{\circ}, \,\,\phi\_{\circ}) = \stackrel{\rightarrow}{d}(0^{\circ}) + \stackrel{\rightarrow}{d}(45^{\circ}) + \stackrel{\rightarrow}{d}(90^{\circ}) + \stackrel{\rightarrow}{d}(135^{\circ}) \quad (14)$$

The orientation of synaptic weights for the *E*-neuron is determined by

$$
\phi\_{\rm or} = \phi\_{\rm s}/2\tag{15}
$$

The RCM is obtained by color coding each *E* neuron in layer 4 using φor. The RCMs does not reflect orientation selectivity of OSMs since they are not measured based on inputs from the LGN and only show anisotropies in the local pattern of connectivity within layer 4.

#### **ODM COMPUTATION**

The ODM was computed using the following procedure. Each *E* neuron in layer 4 receives inputs from *E* neurons in the LGN. The initial synaptic conductance values for all input synapses from LGN are set to the same value. Thus, the *E* neurons initially start off by responding equally to inputs from both the eyes and are thus labeled as binocular. These geniculocortical synapses are tuned using STDP during the three phases based on random activity, retinal waves and with natural stimuli. The number of afferent synapses into an *E* neuron of layer 4 after each phase of development is tracked from each eye and the *E* neuron is assigned a membership to one of the two eyes based on the eye for which the sum of synaptic conductances from LGN → *E* is greater. It should be noted that this eye selectivity naturally emerges without the need for setting any arbitrary thresholds due to two factors. The first factor is due to mutual inhibition between the LGN neurons corresponding to the left and right eye (see **Figure 2C**). I-STDP in these inhibitory synapses forces the LGN neurons to compete and this competition affects the ability of *E* neurons in LGN to influence the *E* neurons in layer 4. The second factor is that E-STDP in the synapses from LGN → *E* pathway also creates a competition between synapses from the two eyes (Miller et al., 1989). These two competitive factors combine to break the symmetry in the geniculocortical synapses from the two eyes by increasing the synaptic conductances for inputs from the eye that attains a positive bias and vice versa.

Over time, if there is some spatial structure in the inputs (as it happens after Phase 1), the synaptic conductances in the LGN → *E* pathway become more separated in magnitude based on the eye due to aforementioned competition until the difference becomes stable. This causes the inputs from one eye in the LGN to eventually dominate the response of *E* neurons in layer 4. The reason for the appearance of contiguous patches of eye preference is due to the stable formation of the RCM. This map indicates local structure in the lateral excitation of neighboring *E* neurons (see **Figure 2B**) in layer 4 to similar stimuli. Thus, as a given *E* neuron develops a preference for one eye due to bottom–up competition as described above, the neighboring *E* neurons in layer 4 also tend to receive excitation from this *E* neuron and thus can influence their preference as well over time such that a well-developed and stable ODM emerges.

In order to measure the emergence of eye selectivity during various phases of development, we compute the sum of synaptic conductances for the geniculocortical synapses from the left and right eye separately for each *E* neuron. We then compute the average and standard deviation of the difference between the two sums for all the *E* neurons in layer 4. If the ODMs are truly mature, then this average should be substantial in terms of the total dynamic range of the synaptic conductances while the standard deviation must be very small. Furthermore, these values must also stabilize over time indicating that the ODMs are both mature and stable. It should be noted that ODM maps generated in all phases are smoothed using a median filter of size 3 × 3 to remove small speckle noise due to some neurons developing opposite eye preference on occasion in a neighborhood of *E* neurons that have the same eye preference.

## **OSM COMPUTATION**

For clarity, we follow the definition of orientation preference and orientation selectivity provided in Blasdel (1992). Orientation preference of an *E* neuron in layer 4 is the orientation that yields the strongest response while orientation selectivity is the rate at which responses fall to zero with increasing displacement from the preferred orientation. The orientation selectivity of *E* neurons in layer 4 cannot be computed using RCMs since that computation does not involve input stimuli from LGN to the *E* neurons of layer 4. In order to derive OSMs, we stimulate the LGN at a given location *p* within it (on both eyes) using an oriented rectangular

bar with a length of 15 pixels and a width of 5 pixels. At the end of each phase of development, the oriented bar in a given orientation α is moved bilaterally by different amounts (each amount being less than *<*7 pixels) during odd and even trials within the LGN layers and the resulting spikes from the stimuli (Equation 1) are used to stimulate the *E* neurons that receive these spikes in layer 4. The firing rate of spikes generated by each *E* neuron in layer 4 in response to the LGN inputs is calculated by counting the number of spikes in a 10 s (or 10,000 steps) time window. The resulting spike activity provides a sense for which *E* neurons respond more strongly to the input spikes from LGN for a given bar orientation. In other words, it provides an estimate of orientation selectivity of the *E* neurons in layer 4 to that particular oriented bar in the LGN. This process is repeated for four different orientations of the bar (i. e., α = 0◦, 45◦, 90◦, and 135◦).

Using the firing rates of *E* neurons we then apply the algorithm described in Blasdel (1992) to compute OSM as follows. First, we compute the differential between positive and negative images corresponding to complimentary stimuli—for example, α = 0◦ and α = 90◦ would correspond to a positive and negative image. The negative image is subtracted from the positive and this reveals the change in the response to the stimulation due to the horizontal and vertical bar in the LGN. The differential image is then transformed at every location, in every image, into vectors displayed either as cosine and sine pairs, in Cartesian coordinates, and as magnitudes and angles in polar coordinates with angles corresponding to twice the positive stimulus orientation, and lengths corresponding to net intensities. The resulting vectors from the transformed images are added to reflect orientation-weighted contributions from each image. Since stimulus orientations are multiplied by two, contributions generated by similar orientations reinforce one another, while those generated by orthogonal orientations cancel. The polar coordinates (angles and magnitudes) that are computed from the summed image reflect the orientation preference and selectivity of the *E* neurons in layer 4 (Blasdel, 1992; Miikkulainen et al., 2005).

The orientation preferences are analyzed with respect to their gradient that measures the rate of change at every point in two dimensions. Following Blasdel, we compute the gradient in *x* and *y* and then convert the result to Polar coordinates. The magnitude of the gradient calculated from the polar coordinates indicates the steepest rate of change at any point irrespective of direction. The gradient magnitude is used to find discontinuities which appear as short lines or dots running across a region of continuous tone (Blasdel, 1992). These discontinuities can signify either a *fracture* if it appears as a short line between two contiguous regions with a gradient of 90◦ or more, or signify *pinwheel formation*, if the gradient magnitude plot contains dots. This entire process described above was repeated after each developmental phase to assess the formation of OSM.

In order to get robust orientation gradients as well as orientation selectivity and preferences, the Blasdel approach averages the raw data from the summed Cartesian images (see **Figure 13** in Blasdel, 1992). The purpose of this averaging process was to smooth out noise and improve the signal-to-noise ratio in these images. We smooth the summed *x* and *y* Cartesian images from our model using a median filter and then convert them into polar representation to obtain the orientation preference and selectivity of the *E* neurons. In order to estimate the correct filter size for smoothing, we adopt an iterative convergence process as follows. We first compute the orientation gradient responses for the raw OSMs. The raw OSM data in our model has small patches of noisy neuron orientation preference responses amidst regions of uniform orientation preference. The orientation gradients computed with such noise yields spurious discontinuities in the form of extraneous dots and lines. To mitigate this, we gradually increase the filter size of the median filter and repeat the above process to compute the orientation gradients until there are no spurious discontinuities due to those noisy neurons. This process does not change the resulting orientation preference maps in any qualitative fashion but filters out the noisy neuron preferences. We find that a median filter of 9 × 9 was sufficient to avoid any orientation gradient artifacts due to noisy orientation preferences. This approach was used to compute orientation selectivity, orientation preference and orientation gradients as well as perform analysis on the resulting functional maps after each phase of development.

While the smoothing operation during OSM computation based on the Cartesian images (as described above) helps remove the salt and pepper noise in the orientation gradient maps to produce robust fractures, they also remove any point singularities as well. However, visual inspection of the OSMs shows that the orientation preferences rotate continuously through ±180◦ along circular paths. This feature is characteristic of a pinwheel (Ohki et al., 2006). A better model of how the signal is smoothed in animal experiments could possibly help mitigate this problem. At this point, since our methodology does not obtain precise point singularities in the orientation gradient maps, we call these formations *pinwheel-like patterns*.

#### **STABILITY ASSESSMENT**

In order to measure the stability of functional map or RCM, a similarity measure between the synaptic conductance maps that reflect that particular structure in one developmental phase is compared against the synaptic conductance maps in another developmental phase based on Kullback–Leibler (KL) distance (Kullback, 1987). For example, the stability of the RCMs is determined by the least change in the KL distance. Mathematically, let *h*1*(w)* and *h*2*(w)* represent the two synaptic conductance histograms for synaptic conductance maps from two different developmental phases, *W*<sup>1</sup> and *W*2. By normalizing the histograms, the synaptic conductance distribution functions are obtained as:

$$p\_i(\boldsymbol{\nu}) = \frac{h\_i(\boldsymbol{\nu})}{\max[h\_i(\boldsymbol{\nu})]}, \quad i = 1, \ 2. \tag{16}$$

The similarity between synaptic conductance maps is then computed as:

$$S(W\_1, W\_2) = \frac{1}{2} \left[ \sum\_{\mathbf{w}} p\_1(\mathbf{w}) \log \left( \frac{p\_1(\mathbf{w})}{p\_2(\mathbf{w})} \right) \right]$$

$$+ \sum\_{\mathbf{w}} p\_2(\mathbf{w}) \log \left( \frac{p\_2(\mathbf{w})}{p\_1(\mathbf{w})} \right) \bigg| \tag{17}$$

This KL measure provides a way to track changes in synaptic conductance during development and thus provides a measure of stability in the evolving functional maps or RCMs.

## **RESULTS**

## **PHASE 1: EARLY EXPERIENCE-INDEPENDENT OSM AND ODM FORMATION**

The first phase of development begins with an *E-I* network that models the local recurrent microcircuit in layer 4 (**Figure 1**). This network consists of a 2-D lattice of 128 × 128 neurons with a 19 × 19 local neighborhood of synaptic connections to other *E* neurons (see **Figures 2A,B**). Neurons on the borders were assumed to have periodic boundary conditions such that topmost and bottommost neurons are regarded as neighbors, as are the leftmost and rightmost columns within each layer. The neurons exhibit spontaneous spiking activity (Chiu and Weliky, 2001; Huberman et al., 2006, 2008). This condition is simulated by injecting a constant small background current into a sub-population of both *E* and *I* neurons in layer 4 and LGN neurons of the network (see "Materials and Methods"). The conductance based synapses along with the recurrent connections within layer 4 ensures that the network is able to maintain this spontaneous activity (Kumar et al., 2008). The key aspect of the early experience-independent phase in our model is the absence of any retinal inputs (**Figure 2B**).

As the neurons in the network begin to spike due to spontaneous activity, STDP alters the synaptic strengths *w* and *z* (Equations 6, 7) and thus the connectivity between various neurons. Using our initial network model architecture (see "Materials and Methods") combined with STDP in both excitatory and inhibitory synapses helps to achieve a good balance of excitation and inhibition. It is becoming more apparent that the ongoing balance of cortical excitation and inhibition plays a role in early development (Xing et al., 2011). The interesting aspect of our model is that inhibitory plasticity helps maintain this balance in the cortex that is qualitatively similar to observations in some recent experimental studies (Akerman et al., 2002)

This aspect of development was tested by performing three types of experiments. In the first experiment, both *w* and *z* synapses obeyed STDP and in this case the network is able to operate at much lower firing rates (average of ∼10 Hz). As *w* strengthens and create an imbalance in synaptic currents, *z* get rapidly potentiated due to an order-independent I-STDP where inhibition increases irrespective of the order of occurrence of pre- and post-synaptic spikes (Caporale and Dan, 2008) for small timing differences between pre- and post-synaptic spikes (Equation 7). This results in a rapid compensatory increase in inhibitory currents into the neurons in a self-organized manner thus effectively preventing the neurons from exceeding *VT* more often (Vogels et al., 2011). This enables a good balance between excitation and inhibition and results in the emergence of RCM that have a locally smooth structure (**Figure 4A**).

In the second experiment all inhibitory synapses are turned off (i. e., *z* = 0). It was observed that firing rate of the *E* neurons was high on average (∼100 Hz) but there was no emergent structure (**Figure 4B**). A high firing rate of neurons due to lack of inhibition forces the synapses to compete at a faster rate due

**recurrent cortical map (RCM) formation for three different cases are illustrated here.** The patterns of the RCM are aligned systematically with the Caretsian grid on which the population is laid out but the emergence of smooth organization is due to synaptic plasticity that changes the connectivity of neighbors to become similar to each other. **(A)** When STDP in both excitatory and inhibitory synapses are active, RCMs are formed with a clearly developing structure. The RCM is created using the *E* → *E*

rate of the *E* neurons of layer 4 is ∼10 Hz. Low firing rates indicate a good balance between excitation and inhibition. **(B)** When there is no inhibition, RCMs fail to emerge and the average firing rate of the network is ∼100 Hz. **(C)** In the case of fixed inhibition in the network, RCMs do emerge for some but not for all fixed inhibition settings. The average firing rate of the network for one such example setting of fixed inhibition for which no RCM emerges is ∼75 Hz.

to the asymmetric E-STDP rule (Equation 6) that results in a rapid rise in synaptic conductances for some synapses and a rapid fall in synaptic conductances for most others (due to a bias toward depression—see "Materials and Methods"). These rapid changes in synaptic conductance appear to be detrimental to the emergence of structure.

In third experiment excitatory synapses obeyed the STDP rule while plasticity was turned off for all the inhibitory synapses. Instead the inhibitory synapses were fixed (i. e., *z* = *const*) in synaptic strength. The exact setting for the fixed inhibitory synaptic conductance values was critical in order to see any emergence of structure. The firing rate was lower on average (∼75 Hz) and was not conducive for the emergence of structure in most cases (**Figure 4C**).

The emergence of RCM during the first phase of development is captured in **Figure 5**. The time taken for the emergence is just an artifact of the model parameter settings and reflects upon the fact that activity-dependent plasticity caused by STDP enables the emergence of RCMs (see "Materials and Methods"). The distribution of *E* → *E* synapse conductances (**Figure 5A**) shows smoothly varying structure in synaptic conductances within a neighborhood (**Figure 5B**). The distribution of synaptic conductances (**Figure 5C**) shows that the initial bimodal distribution becomes more separated and sparse in strong connections that is due to the competitive nature of STDP (Song and Abbott, 2001) and this competition between synapses that are pre-synaptic to neurons in layer 4 results in the modification of their synaptic connectivity and results in the emergence of RCMs.

In order to test the orientation selectivity of *E* neurons in layer 4, the following experiment was performed (see "Materials and Methods"). The LGN neurons from both eyes were stimulated with oriented bar stimuli and the spiking activity of *E* neurons in layer 4 were measured in time windows of 10 ms and then averaged to compute their firing rates. The goal was to observe if there was preferential firing of V1 neurons to certain orientation of the bar stimuli presented in the LGN. A strong selectivity response feature required the adaptation of both the geniculocortical (LGN → *E*) and cortico-cortical (*E* → *E*) synapses such that the firing response of *E* neurons in layer 4 is strong to particular orientations and not others. The results show that the *E* neurons in layer 4 develop preferential firing to certain orientations and not to others (**Figure 6**). This illustrates that there was an emergence of OSMs with characteristic iso-orientation domains and fractures in the orientation gradient maps. However, there were no point singularities indicating the absence of any pinwheel-like formations. There were also no linear zones formed.

The feedforward inputs triggered by spontaneous background spiking activity in the LGN results in the excitation of layer 4 neurons via the geniculocortical synapses (i.e., LGN → *E* synapses). The synaptic inputs originating from LGN can be partitioned into two groups based on the origin of LGN neurons: left and right

of simulation shows the emergence of structure in local synaptic connectivity between the *E* neurons in layer 4 (see "Materials and Methods"). The RCMs are stable after 1 million steps. **(B)** The synaptic conductances within the pin wheel for a 40 × 40 neighborhood (black square) is shown where there is a clear lack of appearance of structure after the first stage. After a 1 million steps of simulation, the RCM is well

to competition between *E* → *E* synapses that is caused by STDP. This competition creates a sparse network with a majority of the synapses becoming zero while a fewer set of *E* → *E* synapses are fully potentiated to 1.0.

eyes (**Figure 7A**). Initially the synapses are all of equal strength (i.e., no bias) such that the *E* neurons in layer 4 respond equally to spikes from LGN neurons corresponding to both the eyes (**Figure 7C**). However, noisy background spiking activity in LGN causes STDP to select some synapses to potentiate while others to depress depending on temporal correlations among the spikes that impinge on the *E* neurons in layer 4. This process results in introducing a bias in the synaptic strengths from LGN → *E* neurons in layer 4 due to STDP induced competition among all geniculocortical synapses at any given *E* neuron. As a result, the strength of geniculocortical synapses from one eye ends up being more than from the other eye (**Figure 7A**) and thus *E* neurons in layer 4 begins to develop eye selectivity. The eye selectivity of *E* neurons across layer 4 manifests as an ODM (**Figure 7D**). This early experience-independent formation of ODM while balanced (i. e., number of *E* neurons that are selective to the left and right are equal) is still fragmented without any contiguous patches of neurons as found in the visual cortex of several species. This is because in our model, the LGN neurons are primarily stimulated via background activity that has no temporal or spatial contiguity during Phase 1. The mean value of the difference in synaptic conductance between the geniculocortical synapses from both eyes increases slowly (**Figure 7B**). However, the standard deviation of the difference is higher than the mean implying that many of the *E* neurons have a very small difference in synaptic conductances while a few have a much larger difference. This measure shows that the ODMs are not really stable during the Phase 1.

## **PHASE 2: LATE EXPERIENCE-INDEPENDENT ODM AND OSM REFINEMENT**

In the second phase of development, in addition to the basic thalamocortical circuit of Phase 1, the LGN is now connected to two groups of RGC corresponding to the two eyes (**Figure 1**). The RGC inputs to LGN represent internally generated spikes in the form of retinal waves ("Materials and Methods") that provide a robust signal to drive activity in both the LGN and in layer 4 of V1. Such spontaneous activity has been implicated in the development of ODMs and retinotopy in V1 (Mooney et al., 1996; Butts, 2002; Godfrey and Swindale, 2007; White and Fitzpatrick, 2007; Huberman et al., 2008; Feller, 2009). An example of a retinal wave generated for a given eye with *N* = 10 is shown in **Figure 8A**. The retinal waves after 300 and 200,000 waves are shown in **Figures 8B,C**, respectively. It can be noted that by the end of 200,000 waves, all the RGCs have been selected as initiation sites and the distribution of RGC activity resembles a Gaussian random field.

right provide the cyclic color code ranging from 0◦ to 180◦. **(C)** Orientation selectivity of each *E* neuron is indicated using a grayscale map. The brighter colors indicate high selectivity where those *E* neurons in layer 4 respond sharply to a very narrow range of orientations and vice versa. The color scale shows the magnitudes of responses in a relative fashion. For example, neurons in the neighborhood of neuron at (110, 100) show strong selectivity have high gradients in a manner qualitatively consistent with the data from the Blasdel paper. The discontinuous changes either occur alone (singularities), or they group together along lines (fractures). While there are many lines of fracture in this phase, there are no singularities, linear zones or pinwheels.

In this model, the spikes generated by retinal waves are transmitted via geniculocortical synapses to activate the *E* neurons in layer 4. It should be noted that our model assumes that in this phase of development, there is no separation of RGCs into ON and OFF ganglion cells (Myhr et al., 2001; Huberman et al., 2008). The distribution of the geniculocortical synapses to *E* neurons is sharpened further due to STDP and the resulting ODM shows more distinct patches selective to a given eye compared to ODMs in Phase 1 (**Figure 9A**).

The emergence of ODMs was also analyzed based on lesion studies. There are animal studies that suggest, for example, that retinal wave disruption caused by intraocular injection of epibatidine reduces firing of the RGC thereby affecting the development of eye-specific retinogeniculate projections and eventually the development of functional maps as well (Wong, 1999; Feller, 2009). In our model, we qualitatively simulate the disruption of retinal waves by cutting off neural activity in a percentage of neurons in the right eye of the LGN. The resulting RCMs,

constructing the ODMs is outlined here (see "Materials and Methods"). The synaptic conductance changes induced due to STDP at the geniculocortical synapses from the LGN neurons corresponding to each eye are tracked over time. If the *E* neuron in layer 4 has a stronger set of afferents from the LGN from the left eye compared to the right eye, then the *E* neuron is labeled as "left" and color coded as green in the ODM. The exact opposite scenario results in the *E* neuron labeled as "right" and color coded as red in the ODM. If there is a tie (as in the beginning), then the *E* neuron is labeled as "binocular" and color coded as blue in the ODM. **(B)** The sum of the synaptic conductances from the LGN neurons corresponding to the left eye is compared against the sum of the synaptic conductances from the LGN neurons corresponding to the right eye at each *E* neuron. Each of these neurons are labeled as "left" or "eye" as described above. Then the mean and standard deviation of the difference between left and right eye afferent synapses for the neurons (*nleft*) labeled "left" is computed and plotted in a semilog format with the ordinate plotted in log scale while the abscissa data is plotted in regular scale. It can be seen that the mean value of the difference increases slowly.

implying that many of the *E* neurons have a very small difference in synaptic conductances while a few have a much larger difference. Similar behavior was observed for the right eye as well. This measure shows that the ODMs are not really stable during the Phase 1 in our model. **(C)** Early ODM appears to have several binocular *E* neurons since all the geniculocortical synapses are initialized with the same synaptic strength and there have not been sufficient inputs to alter the synaptic strengths via STDP. **(D)** At the end of Phase 1, the *E* neurons in the ODM shows eye selectivity with an even split of neurons becoming selective to one of the two eyes. There are no more binocular neurons. The ODM, however, appears fragmented with no large contiguous areas of neurons showing preference to one eye and not the other. Instead it has lots of small contiguous areas of various sizes. This is due to random stimulation of the LGN neurons with background activity with no temporal or spatial contiguity during Phase 1. The STD and mean data in **(B)** also provides further support to this basic phenomenon during this phase. However, this improves during the second and third phases of development as shown in **Figures 10**, **12** when there is more structure in the input data.

OSMs, and ODMs (**Figures 9B–D**) are compared against the case of normal development (**Figure 9A**). In our model, this bias emerges due to the competitive nature of the STDP rule. When more inputs are received from the left eye, there is a higher probability for the geniculocortical synapses from the left eye to cause a post-synaptic spike. This in turn implies that the geniculocortical synapses from the left eye are going to potentiate a lot more than those from the right eye and thus the *E*

neurons in layer 4 become more selective to spikes from the left eye.

RCM development is only affected in a minor fashion due to lesions to the LGN cells because RCM formation is primarily dictated by the *E* → *E* synaptic conductance maps while ODM and OSM formation is dictated by both LGN → *E* and *E* → *E* synapses. Since the changes in inputs from retinal waves drive the geniculocortical synapses, the spikes in the *E* neurons in layer 4 are primarily caused by the LGN. So, the synaptic changes in geniculocortical synapses (which affects ODM and OSM) are more dramatic than in the cortico-cortical synapses (which affects OSM and RCM).

The ODMs formed in Phase 2 show more contiguous patches of eye selectivity (**Figure 10A**) compared with Phase 1. The mean and standard deviation of difference in synaptic conductances (**Figure 10B**) show a more clear separation on the semi-log plot indicating that the ODM formed in Phase 2 is more stable than in Phase 1. The orientation tuning of *E* neurons in layer 4 during Phase 2 was evaluated by stimulating the LGN neurons for both eyes using the same oriented bar stimuli (**Figure 6**) during Phase 1. The results show that the *E* neurons in layer 4 develop more well-defined iso-orientation domains (**Figure 10E**) and fractures (**Figure 10C**) in Phase 2 compared to Phase 1. This change is caused by STDP due to a spike inputs from the geniculocortical synapses in Phase 2 due to the retinal waves that have more spatial and temporal contiguity compared to noisy background level activity in Phase 1. The peak firing response of *E* neurons in layer 4 are on the average larger than in Phase 1 as indicated by the larger dynamic range in orientation selectivity maps (**Figure 10D**). This shows that the *E* neurons in Phase 2 have developed sharper orientation selectivity and the OSM is actively evolving due to activity-dependent plasticity. There are also three clear pinwheel-like patterns that emerge in this stage as depicted in black circles (**Figure 10E**) where the orientation preferences rotate continuously through ±180◦ along circular paths. However, there were no corresponding point singularities that could be extracted from the orientation gradient due to the limitations of the methodology used (see "Materials and Methods").

## **PHASE 3: EXPERIENCE-DEPENDENT OSM AND ODM REFINEMENT AND MAINTENANCE**

For the third phase of the developmental process, the model is exposed to sensory experience in the form of images from a database of real-world images from the Caltech 101 database (see "Materials and Methods") to study the effects of experiencedependent plasticity (Yao and Dan, 2001) on the refinement of OSMs and ODMs. In our model, the RGCs in this phase are assumed to be segregated into ON and OFF neurons as evident in animals after eye opening (Koehler et al., 2011). The ON and OFF regions each contain 48 × 24 neurons. The ON neuron responses are encoded as spikes (see "Materials and Methods"). The OFF

patches of neurons that respond to only one of the two eyes. The OSM is developed with more patches of iso-orientation domains. **(B)** The development of ODMs is affected with more *E* neurons tuned to the left eye

balanced between the two eyes. The ODM appears to have more contiguous

compared to **(C)** when 75% of RGCs in the right eye are prevented from stimulating the right LGN neurons. The RCMs on the other hand do not appear to be that dramatically affected. The OSMs still show some differentiation in its orientation preferences albeit undergoing more re-organization compared to the 50% lesion case.

neuron responses are computed by taking the inverse of the ON image and then rectifying the image so that there are no negative values (**Figure 11**). The resulting image is encoded as spikes (Equation 11).

The ODMs formed in Phase 3 show contiguous patches of eye selectivity (**Figure 12A**) in a manner similar compared with Phase 2. The mean and standard deviation of difference in synaptic conductances (**Figure 12B**) show an even more clear separation on the semi-log plot indicating that the ODM formed in Phase 3 is more stable than in Phase 2. The results also show that *E* neurons in layer 4 are more finely tuned to particular orientations compared to Phase 2 (**Figure 12E**). The dynamic range of the orientation selectivity map (**Figure 12D**) is larger than in Phase 2. There are four distinct pinwheel-like patterns that emerge after learning in Phase 3 (**Figure 12B**) but do not have any point discontinuities in the orientation gradient maps at the centers of these pin wheel patterns. The ODM maps are also more refined and develop noticeable contiguous regions of eye selectivity. The interesting aspect here is that the RCMs seem to have stabilized in Phase 2 and appear qualitatively similar between Phase 2 and Phase 3 while the ODM and OSM continue to undergo refinements after exposure to the images from the Caltech 101 database for over 6 million steps. This is qualitatively consistent with some experimental observations (Chapman et al., 1996; Crair et al., 1998) where the orientation tuning responses change with environmental stimuli.

#### **STABILITY OF FUNCTIONAL MAPS**

The network was analyzed for stability across all three phases of development. In Phase 1, the balance in synaptic currents driving neurons of layer 4 for this phase of development was measured by computing the difference between excitatory and inhibitory currents at each *E* neuron by averaging across a time window

divergence is very clear unlike in **Figure 7**. The mean and standard deviation also seem to stabilize in the later parts of Phase 2. **(C)** The absolute magnitude of the orientation gradients at each *E* neuron shows singularities and fractures. Here white indicates high gradient values while black indicates no gradient at all. The gradients image show several fractures in the data. The average fraction of the total synaptic drive at each *E* neuron selective to a given eye was also calculated. For example, for the "left" *E* neurons, ( *<sup>i</sup> w*left −  *<sup>i</sup> w*right*)/( <sup>i</sup> w*left +  *<sup>i</sup> w*right*)*, was ∼71%. Similarly, the fraction was ∼70% for the "right" *E* neurons at the end of Phase 2. **(D)** Orientation selectivity shows brighter colors that indicate high selectivity with those *E* neurons in layer 4 respond to a very narrow range of orientations and vice versa. The color scale shows the

iso-orientation domains and three weakly formed pinwheels marked by the three black circles. We call these weakly formed pin wheels since they are not corroborated by singularities in the orientation gradient maps. This is because the smoothing operation on the Cartesian images removes the spurious edges created by noisy neuron responses while also removing any trace of the singularities as well. However, close inspection shows that there are three locations marked with black circles where the orientation preferences rotate continuously through ±180◦ along circular paths. We refer to these patterns as a pinwheel-like pattern. It should be noted that there are no clear appearance of point discontinuities in the orientation gradient maps to corroborate the pin-wheel centers within these pinwheel-like-patterns that clearly appear in animal data.

of 200 ms. Simulations show that the net current during this phase moves from an initial bias toward excitation to a more negative bias toward inhibition that slowly reaches a steady state value (see red line in **Figure 13A**). The negative bias shows that on average the influence of inhibition is stronger than excitation so as to compensate for the imbalance in the number of *E* to *I* neurons in layer 4. This dynamic ensures that the firing rates of the neurons are low and conducive for learning the RCMs and functional maps.

The distribution of synaptic conductances was also tracked dynamically throughout the learning process by computing the difference between the normalized distribution of synaptic conductances (*E* → *E*, *E* → *I*, *I* → *E*, and *I* → *I* as shown in **Figure 14**) using the KL measure (Equation 17) once every 5000 simulation steps (or 5 s). This dynamic captures a measure of stability in the learning process during development. The results show that all the four types of conductances slowly reach a steady state at the end of Phase 1 with the KL divergence showing very small fluctuations.

The network exhibits a better balance between excitation and inhibition progressively as a function of development (Shapley et al., 2003; Okun and Lampl, 2008) (see **Figure 13**). The stability

of the formed RCMs in the second phase of development is measured using KL divergence in synaptic conductances (**Figure 14**). The results show that the network is progressively more stable during development since the KL divergence is lower progres-

applying a scaling factor of 1.025 on the left image. The resulting stereo pair

sively for all four types of synaptic conductances in layer 4. The OSMs and ODMs undergo refinement throughout the three phases of development. The primary cause for the refinement is the change in the nature of LGN inputs from random activity in Phase 1 to retinal waves in Phase 2 to natural stimuli in Phase 3. At the end of Phase 3 (i. e., after over 5.5 million steps of natural image stimuli) the OSMs and ODMs appear to stabilize such that the orientation responses of neurons in layer 4 no longer shift as observed earlier (compare **Figures 6**, **10**, and **12**).

It is known that OSMs and ODMs continue to undergo noticeable refinements if there is substantial change in the input environment (Blakemore and Cooper, 1970; Sengpiel et al., 1999; Krelle et al., 2011). We conducted an experiment to verify if this occurs in our model. We created a set of new flag patterns primarily consisting of horizontal and vertical bars (**Figure 15A**). This stimulus is considerably different from the Caltech 101 database images since they do not provide contrast information in any other direction except 0◦ or 90◦. These inputs stimuli were provided as stereo inputs to the LGN ("Materials and Methods"). Despite repeated presentations of such stimuli ranging from short (*TF <* 1 s) to longer durations (*TF* = 10 s), the OSMs and ODMs are not affected and remain stable (**Figure 15B**). This is due to an exquisite balance in currents created by the E-STDP and I-STDP plasticity mechanisms where any instantaneous imbalance in currents is rapidly compensated by plasticity to restore the balance (**Figure 13**).

terms of the contrast and oriented edges.

If there is a volley of high-frequency spikes due to stimuli during such short durations that cause the *E* neurons to spike more rapidly, the inhibitory plasticity for both feedforward and recurrent synaptic connections enables the system to increase the synaptic conductance *z* of the inhibitory synapses. This results in increased inhibitory currents to the active *E* neurons thereby removing the imbalance (Vogels et al., 2011). Thus E-STDP and I-STDP enables the stable maintenance of OSMs and ODMs despite continuous plasticity.

We also applied the same flag patterns with horizontal and vertical bars for a very long duration (*TF* = 5000 s). The OSMs and ODMs do see changes (**Figure 15C**) after this long exposure reflecting that only prolonged and consistent presentations

**FIGURE 12 | Functional maps summary and orientation selectivity of neurons in OSMs during Phase 3. (A)** Comparison of functional map development during all three phases shows the progressive refinement of the ODM, RCM, and OSM. The ODM is stable and shows well-defined and contiguous patches of eye selectivity. **(B)** The mean and standard deviation parameters as described in **Figures 7**, **10** show more divergence and stability compared to Phase 2 The average fraction of the total synaptic drive for the

of stimuli can change the orientation tuning response of the *E* neurons in layer 4. This change is again possible due to activity-dependent STDP that slowly changes the tuning properties of *E* neurons during the small windows of opportunity where *E* neurons in layer 4 spike despite operating in a well-balanced regime between excitation and inhibition. Since these spikes are sparse, our model requires prolonged periods of stimuli exposure to effect changes in the stabilized ODM and OSM. These results qualitatively agree with experimental results (Blakemore and Cooper, 1970; Sengpiel et al., 1999; Krelle et al., 2011) that suggest that influence due to environmental stimuli can affect functional maps in V1. It also shows that the number of *E* neurons that respond to 90◦ (blue regions) or 0◦ (red regions) increases by 25% or more compared to OSM at the end of 10 million steps (**Figure 15C**). This is also qualitatively consistent with observations in the visual cortex (Sengpiel et al., 1999). It is interesting that RCMs are not affected much because geniculocortical synapses are affected more by new inputs from LGN compared to cortico-cortical

## **DISCUSSION**

The proposed spiking model is the first to cover the three developmental phases during the formation of OSMs and ODMs with continuous synaptic plasticity in the form of STDP. The model offers a biologically plausible explanation for this formation in which E-STDP and I-STDP at the excitatory and inhibitory synapses, respectively combine to enable the development and maintenance of these functional maps. This is consistent with recent models that suggest that cortical reorganization is reliant on spike timing (Song et al., 2000; Young et al., 2007). It is also consistent with a recent model that suggests that inhibitory plasticity could play a key role in the formation and maintenance of functional cortical circuitry (Vogels et al., 2011).

"left" *E* neurons was ∼123% and ∼124% for the "right" *E* neurons at the end of Phase 3. **(C)** The resulting orientation gradient maps show well-defined fractures but no singularities. **(D)** The orientation selectivity has similar characteristic to that of Phase 2 except that the peak magnitude of selectivity showing more sharpness (i.e., higher magnitude). **(E)** The orientation preference maps show clearly defined iso-orientations and four

weakly formed pinwheels marked by four black circles.

It is possible for our model to demonstrate OSM and ODM formation directly via external simulations (i.e., experience) and skipping the two experience-independent phases. However, our simulations demonstrate that STDP based learning driven by spontaneous intra-cortical spiking activity can result in the formation of orientation maps and ocular dominance maps as

synapses.

suggested by prior research (Huberman et al., 2006; White and Fitzpatrick, 2007). In this early experience-independent phase ODMs and OSMs appear due to spontaneous activity between LGN neurons and the *E* neurons in layer 4. In the late activity-independent phase where the spontaneous activity in the LGN is generated via retinal waves (Butts, 2002; Godfrey and Swindale, 2007), the ODMs appear to become selectively tuned to one of the two eyes due to competition induced by E-STDP between geniculocortical synapses (Feller, 2009) thus dividing the cortical layer 4 into ocular dominance patches. The important point here is that the model qualitatively shows that there is no need for external input to enable the development of ODMs or OSMs as long as there is STDP driven plasticity in both *E* and *I* synapses. The influence of external inputs, however, does improve the sharpness of the tuning responses in V1.

However, the model in this study should be considered a simple model that is biologically incomplete in its complexity. It does not consider the development of retinogeniculate synapses (Chen and Regehr, 2000; Feller, 2009) nor does it model the differential sensitivities of the ON and OFF cells in their response to variations in contrast relative to mean luminance (Zaghloul et al., 2003). It also does not consider other types of plasticity found in biology such as short term plasticity (Tsodyks and Markram, 1997; Tsodyks et al., 1998) that affects population dynamics during different functional states (Mark and Tsodyks, 2012) and homeostatic plasticity (Turrigiano and Nelson, 2004; Watt and Desai, 2010). It also does not model the complexity in structure and function of neurons including multiple compartments (Hodgkin and Huxley, 1952; Izhikevich, 2004) and other forms of plasticity such as dendritic plasticity and its relation to STDP (Williams et al., 2007; Sjostrom

phase indicating stability in the formed OSM and ODMs. **(B)** The *E* → *I* change shows a similar trend as in **(A)**. **(C)** The *I* → *E* change also shows gradual stabilization as a function of development. **(D)** The

The model is currently being extended to account for the development of direction selectivity. It is well known that some neurons in the visual cortex are selective to direction of motion of visual stimuli (Weliky et al., 1996; White and Fitzpatrick, 2007). Furthermore, direction selectivity map (DSM) formation appears to lag behind OSM in its formation (Li et al., 2006). Recent physiological evidence suggests that the DSMs in the cortex are nested geometrically within OSMs such that an iso-orientation domain is subdivided into a pair of smaller domains that represent opposite directions of stimulus motion (Kisvarday et al., 2001; White and Fitzpatrick, 2007). Finally, there is also mounting evidence that activitydependent plasticity such as STDP enables the formation of

indicating that the synaptic conductances between inhibitory interneurons are stabilized rapidly compared to the other three types of synapses. In all the plots, the instantaneous change in synaptic conductance distributions at each time step is shown in blue while the average change is shown in red.

DSMs (Fiser et al., 2004; Carver et al., 2008; Markram et al., 2011) and recent models show the possibility of forming DSMs using STDP (Buchs and Senn, 2002; Wenisch et al., 2005). Thus a natural extension of the proposed model is to account for the formation of DSMs using STDP within the context of development of all other functional maps such as OSMs and ODMs.

In summary, the present study developed a simple model of a thalamocortical circuit with an initial unstructured map topology that is refined using continuous plasticity in a selforganized fashion to form RCMs, OSMs, and ODMs based on neural activity during three phases of development: endogenously generated cortical activity, followed by activity that arises endogenously in the form of retinal waves and finally activity evoked during sensory experience. Continuous plasticity based on STDP in both excitatory and inhibitory synapses serves as the key mechanism for the development, refinement, and stable maintenance of the formed maps and could also serve as a basis for the development of other functional maps such as DSMs.

## **ACKNOWLEDGMENTS**

The authors acknowledge the support for this work by Defense Advanced Research Projects Agency (DARPA) SyNAPSE contract HRL0011-09-C-001. The views, opinions and/or findings contained in this paper are those of the authors and should

maps are stable (compared to **Figure 12**) despite constant variations

## **REFERENCES**


manuscript.

reflects the dominance of vertical and horizontal bars in the stimuli.

not be interpreted as representing the official views or policies, either expressed or implied, of the DARPA or the Department of Defense. We are very grateful for the insightful comments of the reviewers that helped in improving the overall quality of the

*Nature* 423, 986–990. Bauer, C., Burger, T., Stetter, M., and Lang, E. W. (2000). A neural network model for the selforganization of cortical grating cells. *Z. Naturforsch. C* 55, 282–291. Bednar, J., and Miikkulainen, R. (2004). Prenatal and postnatal development of laterally connected orientation maps. *Neurocomputing* 58–60, 985–992.


Srinivasa and Jiang Stable learning of functional maps

plasticity. *J. Neurophysiol.* 101, 2755–2788.


orientation maps in visual cortex of monocularly deprived cats. *Neuron* 19, 307–318.


application to conduction and excitation in nerve. *J. Physiol.* 117, 500–544.


of ocular dominance and orientation columns," in *Models of Neural Networks, III*, eds E. Domany, J. van Hemmen, and K. Schulten (New York, NY: Springer-Verlag), 55–78.


cameras. *J. Math. Imaging Vis.* 9, 213–229.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 05 August 2012; accepted: 09 February 2013; published online: 27 February 2013.*

*Citation: Srinivasa N and Jiang Q (2013) Stable learning of functional maps in self-organizing spiking neural networks with continuous synaptic plasticity. Front. Comput. Neurosci. 7:10. doi: 10.3389/fncom.2013.00010*

*Copyright © 2013 Srinivasa and Jiang. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*