NEURAL CIRCUITS UNDERLYING EMOTION AND MOTIVATION: INSIGHTS FROM OPTOGENETICS AND PHARMACOGENETICS

EDITED BY: Anton Ilango and Mary Kay Lobo PUBLISHED IN: Frontiers in Behavioral Neuroscience

#### *Frontiers Copyright Statement*

*© Copyright 2007-2015 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.*

*The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.*

*Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.*

*Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.*

*As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.*

*All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.*

ISSN 1664-8714 ISBN 978-2-88919-534-3 DOI 10.3389/978-2-88919-534-3

# About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

# Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

# Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

# What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# **NEURAL CIRCUITS UNDERLYING EMOTION AND MOTIVATION: INSIGHTS FROM OPTOGENETICS AND PHARMACOGENETICS**

Topic Editors: **Anton Ilango,** Leibniz Institute for Neurobiology, Germany **Mary Kay Lobo,** University of Maryland School of Medicine, USA

Inception of false fear memory

Application of optogenetic and pharmacogenetic tools to study the neural circuits underlying emotional valence, feeding, arousal and motivated behaviors has provided crucial insights into brain function. Expression of light sensitive proteins into specific neurons and subsequent stimulation by light (optogenetics) to control neuronal activity or expression of designer receptors exclusively activated by designer drugs (DREADD) in specific neuronal populations with subsequent activation or suppression of neuronal activity by an otherwise inert ligand (pharmacogenetics) provides control over defined elements of neural circuits. These novel tools have provided a more in depth understanding into several questions about brain function. These include:


• Opposing influences on reinforcement by activation of D1 and D2 expressing medium spiny neurons of dorsal striatum and nucleus accumbens

The list still grows...

From cell type specific manipulations to signaling properties in the cell (Dietz et al 2012) with unprecedented temporal resolution, these tools revolutionize the exploration of pathways/ connectivity. Recent years also witnessed the extension of applying these tools from studying emotional valence and motivated behavior to reactivation of memory. c-fos based genetic approaches allowed us to integrate light sensitive opsins or DREADD receptor into specific neurons that are activated by certain learning events (for example fear) (Garner et al 2012; Liu et al 2012).

In this Research Topic, we welcome researchers to contribute original research articles, review articles, methods and commentary on topics utilizing optogenetic and pharmacogenetic tools to study the neural circuits underlying emotional valence, motivation, reinforcement and memory.

We believe the Research Topic will shine light on various questions we have about brain function by using novel optogenetic and pharmacogenetic tools and will hopefully inspire ongoing research to overcome the hurdles of using these tools to advance clinical applications.

**Citation:** Ilango, A., Lobo, M. K., eds. (2015). Neural Circuits Underlying Emotion and Motivation: Insights From Optogenetics and Pharmacogenetics. Lausanne: Frontiers Media. doi: 10.3389/978-2-88919-534-3

# Table of Contents


Anton Ilango, Andrew J. Kesner, Carl J. Broker, Dong V. Wang and Satoshi Ikemoto


Cora Hübner, Daniel Bosch, Andrea Gall, Andreas Lüthi and Ingrid Ehrlich

*108 Identification and optogenetic manipulation of memory engrams in the hippocampus*

Steve Ramirez, Susumu Tonegawa and Xu Liu


Michelle M. Sidor and Colleen A. McClung

*154 Recombineering strategies for developing next generation BAC transgenic tools for optogenetics and beyond*

Jonathan T. Ting and Guoping Feng

*167 Optogenetic inhibition of neurons by internal light production* Benjamin B. Land, Catherine E. Brayton, Kara E. Furman, Zoe LaPalombara and Ralph J. DiLeone

# Shining light on motivation, emotion, and memory processes

#### *Anton Ilango1 \* and Mary K. Lobo2 \**

*<sup>1</sup> Department of Systems Physiology of Learning, Leibniz Institute for Neurobiology, Magdeburg, Germany*

*<sup>2</sup> Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, MD, USA \*Correspondence: antonilango@hotmail.com; mklobo@umaryland.edu*

#### *Edited and reviewed by:*

*Nuno Sousa, University of Minho, Portugal*

**Keywords: emotion, reward, motivation, addiction, habits, obesity, optogenetics**

Over the last 10 years, methods that combine genetics and optics have neuroscientists glowing about the possibilities to study movement, motivation, reinforcement, emotion and abnormal reward seeking processes such as addiction and obesity. From the fruit fly, rodents, to primates, optogenetics sheds light on the previously unanswered questions. With its specificity and high temporal precision, we gained several insights about the brain processes and we were the witnesses of those changes in the field.

Search with the key word "Optogenetic" retrieved close to 1500 articles in PubMed, almost more than 90% of them published in the past 5 years with tremendous progress in several areas of Neuroscience research. This includes dissection of neural circuitries combined with a vast array of behavioral procedures while simultaneously obtaining neurotransmitter measurement and electrophysiological signatures. These developments led us to deconstruct a detailed view of circuits underlying neuropsychiatric disorders. From studying reward, aversion, anxiety, associative learning, habit formation, the tools also applied to study the long lasting and persistent memories, which are bad for the healthy brain (for example, fear and addiction). The scope of the present research topic "*Neural circuits underlying emotion and motivation: Insights from optogenetics and pharmacogenetics*" is to provide a panoramic view of recent progress in this direction, with special emphasis on circuit level exploration of emotion, motivation and memory. A number of topics are covered which are of great interest to a broad readership. These include reinforcement, motivation under the influence of drugs compulsive consumption of food reward, and formation of habits and reactivation memory. Some of the reviews explicitly addressed the problems and the necessity of developing the next generation tools. We believe, leveraging the knowledge obtained from this research topic in Behavioral Neuroscience will help us to gain deeper insights in the future.

## **FROM ELICITING REINFORCEMENT TO ADDICTIVE PROCESSES**

Dopamine (DA) has been implicated in functions ranging from motor control, motivated behavior, reward, associative learning, and mood regulation. Considering the enormous number of articles and the recent surge in exploring the dopamine system utilizing optogenetics, it is important to provide technical and critical analyses of behavioral function using different light stimulation parameters. In this research topic, two such studies appeared. Bass et al. (2013) demonstrated that distinct patterns of DA cell activation were able to disrupt the alcohol-drinking behavior of rats. Specifically applying the precise pattern of photo stimulation, resulting in low but prolonged levels of DA release (tonic), could prevent the rats from alcohol drinking. Ilango et al. (2014) studied role of phasic DA stimulation in supporting reinforcement by addressing the effect of various stimulation parameters on induction of intracranial self-stimulation. Subsequently, effects of interval and ratio schedules of reinforcement on phasic photo stimulation reinforcement were studied, which was also contrasted with food reinforcement. They found that phasic DA signal briefly enhance the initiation of approach behavior without long-term motivational regulation.

Activation of ventral tegmental area (VTA) DA neurons elicits reward and GABA induces aversion. Creed et al. (2014) discussed about the role of VTA GABA neurons to elicit aversion and their role in modulating associative learning based on their local control on DA neurons and striatal cholinergic interneurons. Another review article by, Cachope and Cheer (2014) discuss about the local control (i.e., the endogenous glutamatergic and cholinergic activity) of striatal DA release and relate their implications on potential mechanisms to modify impaired control of DA release in the diseased brain.

Environmental cues are powerful to reinstate a behavior after extinction. Stimuli previously associated with drugs of abuse can trigger drug-seeking behavior. Stefanik and Kalivas (2013) study describes the specific role of basolateral amygdala (BLA) projection either directly into the nucleus accumbens (NAc), or indirectly via the prelimbic prefrontal cortex (PL), on cue induced reinstatement of drug seeking. Also, long-lasting, drug-induced plasticity within the NAc have been proposed to contribute to drug-mediated addictive behaviors. Song et al. (2014) studied the role of NAc medium spiny neurons (MSNs) expressing dopamine D2 receptors (D2Rs) in cocaine-induced behavioral sensitization. The photo stimulation parameters did not affect the initiation nor the expression of cocaine-induced behavioral sensitization but attenuated the expression when applied during the withdrawal period.

## **PAIN, EMOTION, AND MEMORY**

Pain elicits emotional responses and several studies suggests that these emotive events persist long term in the memory compared to events lacking an emotional component.

Carr and Zachariou (2014) reviewed studies using optogenetic stimulation of nociceptors in *C. elegans* and mice, which provided insights into the nociceptive circuitry and behavior, in both acute nociception and chronic pain states. In their conclusion, they stressed the importance of optopharmacology and novel optoreceptors in pain research with the added advantage of moving this into clinical research.

The amygdala has been linked to the regulation of emotion and affective behavior. Here, LaLumiere (2014) reviewed the recent discoveries on amygdala functioning on emotion, memory and addiction based on recent optogenetic experiments.

In another research article, Hübner et al. (2014) dissected the pathways implicated in fear and emotional memory *ex vivo*. They characterized cellular and synaptic responses of BLA neurons that where optogenetically activated by either medial prefrontal cortex (mPFC) or ventral hippocampus (vHC) inputs. Notably, the authors report that both mPFC and vHC inputs recruit feed forward inhibition in BLA, yet inputs from these structures differ in their presynaptic properties.

Ramirez et al. (2014) elegantly reviewed the concept of "memory engram" using fear conditioning experimental procedures. They narrated the history of the biological conceptualization of memory and then further to describe the biological locus and molecular signatures of memory. From their work, they described the process of identifying, activating and optogenetically incepting the false memories.

It is known that modulating serotonin (5-HT) levels in humans and animals affects perception and response to social threats, but there was little knowledge on the circuit mechanisms that control 5-HT dorsal raphe nucleus (DRN) output during social interaction. Here, Challis et al. (2014) studied the top-down control of 5-HT neurons by examining the vmPFC that project to the DRN in social approach-avoidance decisions.

#### **FOOD ADDICTION AND FORMATION OF HABITS**

Krashes and Kravitz (2014) focus on reviewing the advances in understanding the feeding circuitry using optogenetics and pharmacogenetics (a.k.a. chemogenetics). After discussing the circuitry, which mediates homeostatic feeding, they sufficiently theorized and narrated the relevance between drug craving literature with compulsive consumption of food reward.

Our everyday life is influenced by habits but still the mechanism underlying the regulation of habits (the repetitive, chunking response patterns) is insufficiently understood. Here, Smith and Graybiel (2014), discuss multiple distinct components observed in the contrasting dynamics of neural activity from infralimbic cortex and dorsolateral striatum during different stages of habit learning.

## **NEED FOR THE NEXT GENERATION TOOLS AND NEW PROCEDURES**

In their opinion article, Kravitz and Bonci (2013) discuss about the state of the neural activity during optogenetic stimulation and discuss the firing parameters evoked by the stimulation outside of physiological ranges.

Sidor and McClung (2014) minireview article, focused on the diurnal specific control of neural activity and the necessity of chronic stimulation paradigms and technological advances.

Ting and Feng (2014) provided insights on the development and application of diverse BAC transgenic rodent lines and discuss about the under-appreciated issue of "extra" genes contained within the large BAC DNA sequences. In their research article, they propose simple and reproducible strategies for developing the next generation of BAC transgenic lines that are devoid of extra genes which are very much helpful across several disciplines.

Despite the common practice of photo stimulating the circuitry to activate the opsins, the study by Land et al. (2014) reported a new method using the internal generated light source based on firefly luciferase. They co-infected striatal neurons with halorhodopsin and luciferase-expressing viruses in the striatum followed by treatment with luciferin. Through reduced Fos activation, reduced neuronal firing, and blunted amphetamine induced locomotion; they demonstrated a feasible method to optogenetically inhibit neuronal activity through luciferase, without the need for optic fiber implants.

Given the thorough exploration and insights accomplished in this Research Topic, we hope that you will enjoy these new stages of exciting intersections between the neurobiology of emotion, motivation and memory. We appreciate the reviewers' insights during the review process and we greatly acknowledge their effort. It was for us a great pleasure to collect, read, and edit these manuscripts. We hope, you will profoundly enjoy reading this research topic as well.

## **REFERENCES**


Ting, J. T., and Feng, G. (2014). Recombineering strategies for developing next generation BAC transgenic tools for optogenetics and beyond. *Front. Behav. Neurosci*. 8:111. doi: 10.3389/fnbeh.2014.00111

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 01 January 2015; accepted: 02 January 2015; published online: 20 January 2015.*

*Citation: Ilango A and Lobo MK (2015) Shining light on motivation, emotion, and memory processes. Front. Behav. Neurosci. 9:1. doi: 10.3389/fnbeh.2015.00001 This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2015 Ilango and Lobo. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Optogenetic stimulation of VTA dopamine neurons reveals that tonic but not phasic patterns of dopamine transmission reduce ethanol self-administration

#### *Caroline E. Bass 1, Valentina P. Grinevich2, Dominic Gioia3, Jonathan D. Day-Brown2, Keith D. Bonin4, Garret D. Stuber 5, Jeff L. Weiner <sup>3</sup> and Evgeny A. Budygin2 \**

*<sup>1</sup> Department of Pharmacology and Toxicology, School of Medicine and Biomedical Sciences, University at Buffalo, Buffalo, NY, USA*

*<sup>2</sup> Department of Neurobiology and Anatomy, Wake Forest School of Medicine, Winston-Salem, NC, USA*

*<sup>3</sup> Department of Physiology and Pharmacology, Wake Forest School of Medicine, Winston-Salem, NC, USA*

*<sup>4</sup> Department of Physics, Wake Forest University, Winston-Salem, NC, USA*

*<sup>5</sup> Departments of Psychiatry and Cell Biology and Physiology, Neuroscience Center and Bowles Center for Alcohol Studies, The University of North Carolina at Chapel Hill, Chapel Hill, NC, USA*

#### *Edited by:*

*Anton Ilango, National Institutes of Health, USA*

#### *Reviewed by:*

*Alicia Izquierdo, University of California, Los Angeles, USA Raul Gainetdinov, Italian Institute of Technology, Italy*

#### *\*Correspondence:*

*Evgeny A. Budygin, Department of Neurobiology and Anatomy, Wake Forest School of Medicine, Medical Center Blvd., Winston-Salem, NC 27157, USA*

*e-mail: ebudygin@wakehealth.edu*

There is compelling evidence that acute ethanol exposure stimulates ventral tegmental area (VTA) dopamine cell activity and that VTA-dependent dopamine release in terminal fields within the nucleus accumbens plays an integral role in the regulation of ethanol drinking behaviors. Unfortunately, due to technical limitations, the specific temporal dynamics linking VTA dopamine cell activation and ethanol self-administration are not known. In fact, establishing a causal link between specific patterns of dopamine transmission and ethanol drinking behaviors has proven elusive. Here, we sought to address these gaps in our knowledge using a newly developed viral-mediated gene delivery strategy to selectively express Channelrhodopsin-2 (ChR2) on dopamine cells in the VTA of wild-type rats. We then used this approach to precisely control VTA dopamine transmission during voluntary ethanol drinking sessions. The results confirmed that ChR2 was selectively expressed on VTA dopamine cells and delivery of blue light pulses to the VTA induced dopamine release in accumbal terminal fields with very high temporal and spatial precision. Brief high frequency VTA stimulation induced phasic patterns of dopamine release in the nucleus accumbens. Lower frequency stimulation, applied for longer periods mimicked tonic increases in accumbal dopamine. Notably, using this optogenetic approach in rats engaged in an intermittent ethanol drinking procedure, we found that tonic, but not phasic, stimulation of VTA dopamine cells selectively attenuated ethanol drinking behaviors. Collectively, these data demonstrate the effectiveness of a novel viral targeting strategy that can be used to restrict opsin expression to dopamine cells in standard outbred animals and provide the first causal evidence demonstrating that tonic activation of VTA dopamine neurons selectively decreases ethanol self-administration behaviors.

#### **Keywords: dopamine, VTA, nucleus accumbens, optogenetics, ethanol self-administration**

## **INTRODUCTION**

Alcoholism is a devastating socio-economic problem estimated to account for 4% of the global burden of disease. Despite the widespread use and abuse of alcohol, little is known about the neurocircuitry responsible for the development and progression of alcoholism. Ethanol affects brain function by modulating numerous neurotransmitter and neuromodulator systems, including, but not limited to, GABA (Koob, 2004; Siggins et al., 2005; Weiner and Valenzuela, 2006), glutamate (Woodward, 2000), serotonin (Grant, 1995), norepinephrine (Weinshenker et al., 2000), neuropeptide Y (Thiele et al., 1998), vasopressin (Edwards et al., 2012) adenosine (Nam et al., 2013a,b) and dopamine (Weiss and Porrino, 2002; Gonzales et al., 2004; Koob, 2013). Over the past 20 years, much attention has focused on the mesolimbic dopamine system, which is thought to play an integral role in mediating the positive reinforcing effects of ethanol and other drugs of abuse (Berke and Hyman, 2000; Grace, 2000; Weiss and Porrino, 2002; Gonzales et al., 2004; Stuber et al., 2012). This circuit is comprised of the dopaminergic neurons in the ventral tegmental area (VTA) and their projections to the nucleus accumbens and several other brain regions. A large and growing body of evidence suggests that ethanol acutely enhances mesolimbic dopamine release without significant changes in dopamine transporter function (Budygin et al., 2001; Jones et al., 2006) and that ethanol stimulation of VTA dopamine signaling may contribute to ethanol-drinking behaviors. For example, electrophysiological studies demonstrate that ethanol can directly increase the firing rate of VTA dopamine neurons (Brodie et al., 1999) and elegant microdialysis studies have shown that increases in extracellular dopamine concentrations in the nucleus accumbens are tightly linked to the initial phases of ethanol selfadministration (Weiss et al., 1993; Howard et al., 2009; Carrillo and Gonzales, 2011). Pharmacological studies have demonstrated that blockade of dopamine receptors in the nucleus accumbens can significantly decrease operant ethanol self-administration (Czachowski et al., 2001; Samson and Chappell, 2004). However, D2 dopamine receptor antagonists have two distinct actions within the striatum, first they diminish dopamine signaling at the postsynaptic level and secondly these compounds enhance dopamine release through several different presynaptic mechanisms (Wu et al., 2002; Garris et al., 2003; Kita et al., 2007; Park et al., 2010; Belle et al., 2013). Due to these contrasting actions of dopamine antagonists on dopamine dynamics during the time course of ethanol self-administration, it is difficult to relate specific patterns of dopamine transmission to behavioral changes. Furthermore, direct electrical stimulation of the nucleus accumbens in animal models, or deep-brain stimulation (DBS) of the accumbens of alcoholics, reduces ethanol drinking (Kuhn et al., 2007, 2011; Knapp et al., 2009; Muller et al., 2009; Henderson et al., 2010). However, it is possible that DBS, which induces the release of numerous neurotransmitters in the stimulated brain region, accomplishes its effect via non-dopaminergic mechanisms. Therefore, despite many compelling findings, several questions regarding the role of dopamine signaling in the regulation of ethanol drinking remain unanswered. For example, can DBS-induced changes in mesolimbic dopamine transmission alone be responsible for these alterations in alcohol drinking behaviors? If so, which patterns of VTA-accumbal dopamine transmission are responsible for the inhibition and triggering of ethanol seeking and drinking behaviors?

Until recently it has been impossible to parse the causal role of specific neurotransmitter systems in the regulation of ethanol drinking behaviors. Optogenetics provides a means to experimentally control the activation of specific neuronal sub-populations in heterogeneous brain regions where multiple neuronal subtypes exist and to do so with exquisite temporal precision. While several transgenic strategies have been developed to target opsins to dopaminergic neurons in rats and mice, we have developed a novel viral vector approach that can theoretically be used in any rodent species or strain. In this approach, the expression of channelrhodopsin-2 (ChR2) is driven by a tyrosine hydroxylase (TH) promoter, which has previously been shown to restrict expression primarily to dopaminergic neurons (Oh et al., 2009).

Here, we injected an adeno-associated virus (AAV) construct, in which ChR2 expression is restricted to TH-positive neurons into the rat VTA and characterized the effect of tonic and phasic activation of VTA dopaminergic neurons on dopamine release in the nucleus accumbens and ethanol drinking behaviors. Our data confirm that the viral strategy selectively targets ChR2 expression to dopamine neurons within the VTA and that low (5 Hz) and high (50 Hz) frequency stimulation of VTA dopamine neurons results in distinct patterns of accumbal dopamine release. Moreover, ethanol drinking behaviors were selectively reduced by tonic, but not phasic, VTA dopamine release. Collectively, these data demonstrate a novel strategy that successfully targets opsins to VTA dopamine cells in standard outbred rats and provide new insight in the specific temporal dynamics of dopamine signaling that regulate ethanol drinking behaviors.

# **MATERIALS AND METHODS**

## **VIRAL PACKAGING**

Viruses were packaged using a standard triple transfection packaging protocol to generate pseudotyped AAV2/10 (Xiao et al., 1998). The three plasmids consisted of an AAV2 plasmid containing the transgene to be packaged, pHelper (Stratagene, La Jolla, CA) which contributed the necessary adenoviral helper functions, and an AAV2/10 rep/cap plasmid that provides the AAV2 replicase and AAV10 capsid genes (Gao et al., 2002; De et al., 2006). Dr. K. Deisseroth (Stanford University) kindly provided the ChR2- EYFP construct while the TH promoter (Oh et al., 2009) was obtained from Dr. K-S. Kim (McLean Hospital). Virus packaging and titering has been described previously (Bass et al., 2010).

## **STEREOTAXIC VIRUS INJECTION**

Naive male Long-Evans rats were anesthetized with ketamine hydrochloride (100 mg/kg, i.p.) and xylazine hydrochloride (20 mg/kg, i.p.) and placed in a stereotaxic frame. The scalp was shaved, swabbed with iodine and a central incision made to expose the skull. Two small holes were drilled and 2 skull screws were placed in to secure a cement cap. A third hole was drilled above the right VTA (from bregma: anterior-posterior, 5.8 mm; lateral, 0.7 mm) and an optic-fluid cannula (OFC) (Doric Lenses, Canada) was implanted (dorsal-ventral, 7.3 mm). Finally, 1.3µl of virus that expresses ChR2-EYFP with expression driven by a TH promoter was slowly injected into the VTA (dorsal-ventral, 7.3 mm) over 3 min via a Hamilton syringe connected to the OFC. The exposed skull was coated with dental cement secured by skull screws and upon drying the animals were returned to their home cages for recovery.

## **ETHANOL SELF-ADMINISTRATION**

Ethanol drinking was assessed using an intermittent home-cage drinking procedure which we and others have previously shown to engender relatively high levels of ethanol intake in male Long-Evans rats (Wise, 1973; Simms et al., 2008). Briefly, subjects were given access to 20% ethanol (v/v) and water for 24 h periods every Monday, Wednesday, and Friday with only water available on the remaining days. Water and ethanol were given in graduated drinking tubes (MED Associates, St. Albans, VT, USA), and the position of the bottles was alternated on each drinking day to control for potential side preferences. During ethanol sessions, water and ethanol consumption were measured daily after 30 min and 24 h (daily) access to ethanol. Prior data from our lab have shown that brain ethanol concentrations at the 30 min time point in this model (in rats showing similar intake levels) approximate 40 mg/dl (Chappell et al., 2013). The latency to the first lick and total number of licks during the first 30 min of each drinking session were also measured using custom-made lickometers.

## **VOLTAMMETRIC RECORDINGS**

For the evaluation of changes in extracellular dopamine concentrations in response to optical stimulation of the VTA, rats were anesthetized with urethane (1.5 g/kg, i.p.) and placed in a stereotaxic frame. The cement above the striatum was removed and a hole for a carbon fiber electrode (∼100µm in length, 6µm in diameter) insertion was drilled (from bregma: anteriorposterior, 1.2 mm; lateral, 2 mm). An Ag/AgCl reference electrode was implanted in the contralateral hemisphere and a carbon fiber electrode was positioned in the striatum (dorsal-ventral, 4.2–7.2 mm). An optic-fluid cannula (Doric Lenses, Canada) with fiber 200µm in diameter (dorsal-ventral, 7.3 mm) was connected to the laser (Viasho, China). The reference and carbon fiber electrodes were connected to a voltammetric amplifier (UNC Electronics Design Facility, Chapel Hill, NC) and voltammetric recordings were made at the carbon fiber electrode every 100 ms by applying a triangular waveform (−0.4 to +1.3 V, 300 V/s). Data were digitized (National Instruments, Austin, TX) and stored on a computer. Light evoked dopamine release was identified by the background-subtracted cyclic voltammogram. Carbon fiber microelectrodes were calibrated *in vitro* with known concentrations of dopamine (2–5µM). Dopamine uptake was determined from the clearance rate of dopamine and was assumed to follow Michaelis-Menten kinetics. The changes in dopamine during and after optical or electrical stimulation were fit using the equation:

$$d[DA]/dt = (f)[DA]\_p - (V\_{\text{max}}/\{(K\_m/[DA]) + 1\})$$

where *f* is the stimulation frequency (Hz), [*DA*]*<sup>p</sup>* is the concentration of dopamine released per stimulus pulse, and *V*max is the maximal rate of dopamine uptake, which is proportional to the number of available DAT proteins. The baseline value of *Km* was calculated to be between 0.16–0.2µM, a value determined in rat brain synaptosomes and commonly used in the analysis of voltammetric data (Near et al., 1988; Garris and Rebec, 2002). The derivative form of the above equation was used to simulate the dopamine response (Garris and Rebec, 2002; Phillips et al., 2003a; Oleson et al., 2009; Pattison et al., 2011, 2012).

## **OPTICAL DELIVERY**

The optical setup consisted of a laser at wavelength 473 nm (Beijing Viasho Technology Co., Ltd, Beijing, China) with a maximum power output of 100 mW. The laser was modulated using the TTL input control port on the laser power supply with the control signals provided by a programmable function generator (Hewlett-Packard model 8116A). The start of the whole optical pulse procedure was initiated by manually firing a pulse generator (Systron Donner Model 100C) that then triggered a digital delay generator (SRS Model DG535) that was used to gate the function generator. The main purpose of the digital delay generator was to make sure that the function generator was properly gated to only select out a finite number of pulses from the continuous waveforms normally produced by the function generator. The function generator would produce a series of either 5 Hz or 50 Hz square pulses, and the total number of pulses in one data stream (50 and 250 respectively) would be controlled by the digital delay generator, since the temporal length of a gate pulse sent to the function generator would determine the number of square pulses the function generator would produce for each trigger. Each pulse within a series of pulses had a temporal width of 4 ms. The laser power output was measured using a commercial power meter (Newport Model 1815C). Synchronization of the laser to the voltammetric recording was achieved by manually triggering the laser 5 s after initiating the voltammetric recording that was controlled through an electronic controller instrument that was built by the Electronics Design Facility at the University of North Carolina at Chapel Hill. The light paradigm was 50 light pulses at 50 Hz with 30 s intervals for phasic stimulation and 250 light pulses at 5 Hz with 1 s intervals for tonic stimulation.

## **IMMUNOHISTOCHEMISTRY**

Rats were deeply anesthetized with a mixture of ketamine (100 mg/kg) and xylazine (100 mg/kg) and transcardially perfused with 10% normal buffered formalin. Brains were removed and incubated overnight with fix at 4◦C and then transferred and incubated again overnight in 25% sucrose until the brains sank. They were sectioned at 50 microns on a dry ice chilled stage using an American Optical 860 sliding microtome. Floating coronal sections containing the midbrain were processed for immunohistochemistry. Briefly, sections were rinsed in PBS for 5 min, and again in PBS + 0.5% triton X-100 three times for 10 min. Sections were incubated with primary antibody overnight at 4◦C while shaking. Primary antibodies consisted of a mouse antityrosine hydroxylase (ImmunoStar #22941) at a 1:4000 dilution and a rabbit anti-GFP (*Invitrogen* #A6455) at 1:2000. All antibodies were diluted in PBS + 0.3% triton X-100. The next day the sections were rinsed 3 times for 10 min each in PBS and then incubated with secondary antibodies consisting of Alexa 555 donkey anti-mouse (*Invitrogen* #A31570, 1:4000) and Alexa 488 goat antirabbit (*Invitrogen* #A11034, 1:2000) for 2 h at room temperature while shaking. Sections were rinsed again in PBS three times for 10 min, then mounted on slides and coverslipped using Prolong Gold media. Slides were visualized on a Zeiss Axio Observer Confocal microscope.

#### **STATISTICAL ANALYSIS**

Data were analyzed in GraphPad Prism (GraphPad Software, San Diego, CA). A two-way ANOVA was performed for the neurochemical studies characterizing differences between the effect of tonic and phasic light activation on accumbal dopamine efflux and for the effect of these stimulation patterns on the number of ethanol and water licks. The effects of optical stimulations on other ethanol drinking parameters (e.g., intake, latency to first lick) were analyzed by one-way analysis of variance. Bonferroni's tests were used for all *post-hoc* comparisons. Pearson correlation analysis was used to evaluate the relationship between the number of ethanol licks and the amount of ethanol consumed (g/kg). The data are presented as mean ± SEM and the criterion for significance was set at *P <* 0*.*05.

## **RESULTS**

## **TARGETING ChR2 TO VTA DOPAMINERGIC NEURONS IN WILD-TYPE RATS**

To determine the contribution of VTA dopaminergic neurotransmission to alcohol seeking behavior, we generated AAVs that target dopaminergic neurons in the VTA. A TH promoter was used to restrict expression of ChR2 to TH-positive neurons. When injected directly into the VTA, expression was restricted to TH-positive neurons as revealed by immunohistochemical staining (**Figure 1**). Robust TH expression was observed throughout the VTA and substantia nigra, however, ChR2 expression was found only on cell bodies in VTA in sections containing the entire midbrain. Co-staining revealed many TH-positive cells (**Figure 1B**) that also expressed ChR2 (**Figures 1C,D**). We did not observe ChR2 expression on VTA cells that did not express TH (**Figures 1C,D**). However, there were some apparent TH-positive cells that did not express observable levels of ChR2. This pattern of expression is expected from viral delivery in that not all neurons in a given virus infusion site will be transduced, or receive equivalent DNA payloads.

## **VOLTAMMETRIC ASSESSMENT OF OPTICALLY-EVOKED DOPAMINE RELEASE IN RAT NUCLEUS ACCUMBENS**

We first employed *in vivo* fast-scan cyclic voltammetry (FSCV) to confirm that the level of ChR2 expression achieved in the VTA is sufficient for optical stimulation of dopamine release in the rat nucleus accumbens. **Figure 2A** demonstrates accumbal dopamine transients, triggered by blue-light stimulation of the VTA (30 Hz, 30 flashes with 4-ms flash length, 5.7 mW) in a single animal. The rising fractions of dopamine effluxes were time-locked to the 1 s blue pulse, whereas dopamine concentrations rapidly declined immediately at the end of the pulse. Background-subtracted voltammograms taken at the peak of stimulation confirmed that the signal detected is dopamine. No other electrochemical activity changes were observed during the period of detection. Notably, the viral targeting strategy we employed resulted in expression of ChR2 on dopaminergic terminals in VTA projection areas, such as the nucleus accumbens. As demonstrated in **Figure 2B**, robust light-activated dopamine signals could be consistently recorded in the nucleus accumbens, however when the recording electrode was moved to regions within the dorsal striatum, which receives rich dopaminergic innervation from the substantia nigra, no measurable dopamine responses were evoked. To calculate the parameters of dopamine release and uptake, light-evoked changes in dopamine concentration were modeled as a balance between release and uptake (Wightman et al., 1988; Wu et al., 2002; Phillips et al., 2003a). The magnitudes of dopamine uptake parameters, including maximal velocity of dopamine reuptake rate or *V*max and apparent *Km* (Michaelis-Menten constant) were indistinguishable from those previously reported for the rat nucleus accumbens using FSCV *in vitro* and *in vivo* (Phillips et al., 2003a; Jones et al., 2006; Oleson et al., 2009; Pattison et al., 2011, 2012). *V*max and *Km* were 2420 ± 314 nM/s and 176 ± 10 nM, respectively. The average magnitude of dopamine efflux per bluelight pulse or per flash (*DAp*f) was 24 ± 6 nM, which is less than the amount dopamine released by an electrical pulse of the same length (4 ms).

Finally, we examined various parameters of optical stimulation of the VTA, which allows us to induce phasic (a relatively large but transient rise of extracellular dopamine concentration) and tonic (relatively low but sustained dopamine increase) release patterns in the nucleus accumbens. We found that high frequency (50 Hz), brief (1s) stimulation could mimic phasic dopamine responses, while low frequency (5 Hz), long lasting (50 s) stimulation resulted in a tonic elevation in dopamine concentration (**Figure 3**). As expected, there was a significant difference between these two contrasting dopamine responses [*F(*1*,* <sup>782</sup>*)* = 71*.*21; *p <* 0*.*0001]. Changes in dopamine concentration over time were also significantly different [*F(*98*,* <sup>782</sup>*)* = 14*.*38; *p <* 0*.*0001].

## **EFFECT OF TONIC AND PHASIC DOPAMINE RELEASE ON ETHANOL DRINKING BEHAVIORS**

The overall goal of this study was to determine the relationship between accumbal dopamine release patterns and ethanol drinking behaviors. To that end, we first exposed eight rats to an intermittent drinking procedure for 7 weeks. This protocol engenders relatively high levels of ethanol intake and importantly, subjects consume approximately 25% of their daily ethanol intake in the first 30 min of each drinking session, consistently achieving blood ethanol levels between 40–80 mg % (Simms et al., 2008). Rats that demonstrated high and relatively stable levels of ethanol intake during 7 weeks of baseline drinking were then injected with the viral construct (*n* = 5). High drinking rats were

**FIGURE 1 | ChR2-EYFP expression is targeted to dopaminergic neurons in the VTA. Coronal section containing the midbrain of a representative rat injected with TH restricted ChR2 AAVs were co-stained with EYFP and TH antibodies.** TH staining was apparent throughout the VTA and substantia nigra. Robust EYFP expression was observed in the VTA only. EYFP immunohistochemical staining revealed multiple ChR2 positive cell

bodies present **(A)**. Tyrosine hydroxylase co-staining **(B)** demonstrated many TH positive neurons in the VTA, which colocalized with the ChR2 signal **(C)**. A portion of the merged image is magnified in **(D)**. Neurons that express both TH and ChR2 are marked with arrows while neurons that are TH positive only (no apparent ChR2) are marked with asterisks. ChR2 was not observed in non-TH positive neurons. Scale bar equals 20 microns.

**FIGURE 2 | (A)** Light-induced dopamine release in the ventral striatum. The left panel demonstrates a three dimensional color plot topographically depicting voltammetric data collected with FSCV in the nucleus accumbens of a single anesthetized rat before, during and after optical stimulation of the VTA. Green spikes represent dopamine transients triggered by 30 Hz, 30 pulses light stimulation with 5 s intervals between pulse-train groups. Inset: Background-subtracted cyclic voltammograms taken at the end of the light stimulation indicate that the change in current is due to dopamine oxidation (solid line—for the first dopamine spike; dotted line—for the second spike).

**(B)** Optically-evoked dopamine concentrations in different regions of the striatum from one representative animal are presented. The fiber optic was fixed in the VTA (anterior-posterior, 5.8 mm; lateral, 0.7 mm; dorsal-ventral, 7.3 mm), while the voltammetric carbon fiber electrode was lowered to various depths throughout the dorsal and ventral striatum to determine level of dopamine release (anterior-posterior, 1.2 mm; lateral, 2.0 mm and dorsal-ventral varied with 1: 4.2 mm; 2: 5.2 mm; 3: 6.2 mm; 4: 7.2 mm). Red bar indicates the time of light stimulation. Background-subtracted cyclic voltammograms from 1 and 4 are also presented.

selected for these initial studies to explore the effects of optogenetic manipulation of the VTA on the voluntary consumption of pharmacologically meaningful doses of ethanol (Samson and Czachowski, 2003).

Ten days following the injection, ethanol intake returned to baseline levels. **Figure 4A** shows licking patterns of ethanol and water during the first 30 min of 10 drinking sessions. As observed in previous studies, subjects robustly preferred ethanol to water as indicated by a significant difference in the number of licks between ethanol and water [*F(*1*,* <sup>72</sup>*)* = 55*.*29; *p <* 0*.*0001]. Taking advantage of the stable drinking patterns, we explored the effects of tonic and phasic optical stimulation of dopaminergic neurons in the VTA on drinking behaviors (5 Hz and 50 Hz stimulations during the first 10 min of the 5 and 7th session, respectively). No significant change (*p >* 0*.*05) in the number of water licks was found, while a noticeable trend (*t* = 2*.*4) toward a decrease in ethanol licking following 5 Hz stimulation was evident (**Figure 4A**). As expected, there was a strong, positive correlation between the number of ethanol licks and the amount of ethanol consumed (Pearson *r* = 0*.*9; *p <* 0*.*0001, **Figure 4B**). We further explored the effects of optogenetic stimulation of VTA TH-positive neurons on drinking, using a larger group of animals (*n* = 8) in which optical stimulation was performed in a randomized order. This design allowed us to avoid possible confounds such as a habituation. In this larger cohort of animals, tonic activation of VTA dopamine cell bodies significantly affects ethanol drinking behaviors. **Figure 5** demonstrates representative cumulative records of licking for a single rat on different drinking days. We observed a decrease in ethanol consumption measured as the number of licks (**Figure 6A**, 60 ± 16 % decrease from control, no stimulation, *t* = 3*.*45; *p <* 0*.*01) and the amount of ethanol consumed (**Figure 6B**, 53 ± 13 %, *t* = 4*.*42; *p <* 0*.*001), when rats were stimulated at 5 Hz in the drinking cage. In marked contrast, phasic stimulation of VTA dopamine cells during the first 10 min of the drinking session had no effect on any measures of ethanol intake. Moreover, no significant changes in ethanol intake measures were observed when subjects received tonic stimulation in their home cages just prior to the drinking sessions (*p >* 0*.*05). Finally, we found a significant two-fold delay for the first ethanol lick when rats were stimulated with the frequency that induces tonic dopamine release (**Figure 6C**, *t* = 3*.*08; *p <* 0*.*05). No significant changes in this parameter were evident during other stimulating protocols.

#### **DISCUSSION**

It is increasingly clear that revealing the precise contribution of dopamine neurons to specific behaviors requires more sophisticated control over their activity than traditional techniques can

measures were taking from multiple sessions.

provide (Tsai et al., 2009; Steinberg and Janak, 2012; Tye et al., 2012; van Zessen et al., 2012; Steinberg et al., 2013). Here, we employed newly developed optogenetic tools to address critical questions regarding the role of dopamine in alcohol drinkingrelated behaviors. A viral-mediated gene delivery strategy was used to selectively express ChR2 on dopamine cells in the VTA of wild-type rats. We then used this approach to manipulate mesolimbic dopamine transmission during ethanol drinking sessions. The results confirmed that optogenetic activation of VTA dopamine neurons can induce dopamine release in accumbal terminal fields with very high temporal and spatial precision. Brief (1 s) high-frequency (50 Hz) VTA stimulation induced phasic patterns of dopamine release, which were characterized by a relatively large (∼100 nM) and transient rise of extracellular dopamine concentration in the nucleus accumbens. Lower frequency stimulation (5 Hz) for longer periods (50 s), mimicked tonic dopamine increases, characterized by relatively low (∼40 nM), but longer-lasting elevations in accumbal dopamine levels. Finally, combining this approach with an intermittent ethanol drinking procedure, we found that forcing mesolimbic dopamine transmission into a tonic but not phasic mode significantly affects ethanol drinking measures while having no effect on water intake.

Although several elegant methods have been developed that achieve selective expression of opsins in dopamine cells, all of these methods are restricted to specific lines of genetically engineered rats and mice. A major goal of this study was to develop an alternative targeting strategy that could theoretically work in any animal. To that end, we used a previously characterized rat TH promoter to restrict the expression of ChR2. By infusing the virus directly into the VTA, we restrict the expression to dopaminergic neurons in this region only. Our *in vivo* neurochemical data demonstrate the effectiveness of this approach in a standard outbred wild-type rat strain. Optical stimulation of the VTA induced reliable dopamine efflux in the nucleus accumbens core with release dynamics consistent with previously published uptake and release values for this brain region. In addition, exquisite temporal and spatial control of dopamine release was achieved. High frequency, phasic stimulation resulted in large, rapid dopamine transients whereas longer duration, tonic stimulation evoked smaller but more sustained patterns of dopamine release. Importantly, dopamine release was restricted to VTA terminal fields in the nucleus accumbens. No detectable dopamine transients were observed when the recording electrode was moved to the dorsal striatum, which does not receive appreciable dopamine innervations from the VTA. In a previous study, where a non-selective AAV was used to express ChR2 in the substantia nigra, which preferentially innervates the dorsal, but not ventral striatum, similar spatial precision was achieved (Bass et al., 2010). Optical activation of the SN evoked dopamine transients in the dorsal striatum but not the nucleus accumbens. Dopamine neurons in the VTA and SN have distinct roles in various behavioral phenomena including motivation, reinforcement and learning (Steinberg and Janak, 2012; Stuber et al., 2012; Watabe-Uchida et al., 2012). Therefore, the ability to control dopamine release specifically, without induction of other neurotransmitters, in anatomically distinct terminal fields is crucial. The activation of both of these dopamine circuits using traditional electrical or pharmacological approaches may result in different (and more complex) behavioral consequences, and precludes our ability to establish a causal relationship between dopamine release in distinct brain regions and behavior changes. Importantly, although the VTA sends a robust dopamine projection to the nucleus accumbens, it also projects to the amygdala, prefrontal cortex and hippocampus (Beckstead et al., 1979; Swanson, 1982), which are also likely involved in the control of ethanol drinking. Optical stimulation of the VTA undoubtedly triggers dopamine efflux in these other brain areas, although the present study did not assess dopamine efflux in these regions. Thus, although these data provide causal evidence that VTA dopamine neurons directly modulate voluntary ethanol drinking behaviors, further studies will be needed to establish whether VTA dopamine terminal fields in the nucleus accumbens alone play a causal role in ethanol drinking behaviors.

solution and water are available. The top panel demonstrates ethanol and water drinking patterns of a single rat during 3 separate sessions that were performed 2 days apart: **(A,C)** two sessions were with no stimulation; **(B)**

session with VTA stimulation at 5 Hz frequency. Optogenetic activation of VTA dopamine neurons at a low (5 Hz) but not high (50 Hz) frequency affects ethanol drinking behavior. The blue bar indicates a time of the stimulation.

It is well known that dopamine neurons exhibit two main firing modes: single spikes at a frequency of ∼5 Hz and bursts of several action potentials at higher frequencies (≥20 Hz) (Grace and Bunney, 1983; Schultz et al., 1993; Anstrom et al., 2009). These firing patterns generate distinct tonic and phasic patterns of dopamine release, resulting in dopamine concentration fluctuations on a time scale of several minutes and seconds, respectively (Wightman and Robinson, 2002; Owesson-White et al., 2012). These patterns appear to subserve different functions that result in distinct behavioral consequences. Our results clearly demonstrate that both phasic and tonic patterns of dopamine release can be reproduced in the rat nucleus accumbens by optical stimulation of the VTA. These dopamine increases were timelocked to the light stimulation and their amplitudes were within the range seen with natural dopamine fluctuations as measured by voltammetry.

Dopamine neurotransmission in the nucleus accumbens contributes considerably to many vital behaviors, yet the nature of this contribution is a matter of debate (Salamone et al., 1997; Phillips et al., 2003b; Robbins and Everitt, 2007; Bromberg-Martin et al., 2010; Nicola, 2010; Steinberg et al., 2013). One prominent idea is that increases in dopamine release in this brain region are essential for the activation of reward-seeking for both natural rewards as well as abused substances, like alcohol. Although extensive and compelling studies have demonstrated increases in accumbal extracellular dopamine during ethanol self-administration and during the anticipatory period preceding ethanol availability (Weiss et al., 1993; Howard et al., 2009; Carrillo and Gonzales, 2011), it has been a challenge to establish whether specific patterns of dopamine transmission play a causal role in the initiation or suppression of ethanol seeking and consumption. Here, we provide clear evidence that ethanol drinking can be selectively influenced by distinct patterns of VTA dopamine cell activation. We found a significant delay to the first ethanol lick and a decrease in the total amount of ethanol consumed when VTA dopamine cell bodies were activated at a low frequency (5 Hz). Notably, high-frequency stimulation had no effect on these drinking measures. Low-frequency stimulation of VTA dopamine cells shifts accumbal dopamine release activity to tonic mode, characterized by a small but prolonged increase in extracellular dopamine levels. These changes have been suggested to reduce phasic dopamine release through a D2 dopamine autoreceptor–mediated feedback mechanism (Grace, 2000; Phillips et al., 2003b; Oleson et al., 2009). In fact, it has been proposed that phasic dopamine is required for the ability of reward predictive stimuli to motivate behavioral responding directed toward obtaining the reward (Steinberg et al., 2013).

Notably, the intermittent-access drinking procedure employed in this study promotes binge-like ethanol drinking during the first 5–10 min of each session where ethanol is available, coincident with the placement of animals in the drinking chamber along with the presentation of drinking-associated cues. We hypothesize that optogenetically forcing dopamine release into a tonic mode may prevent phasic dopamine signals normally triggered by the contextual cues associated with the drinking chamber, therefore decreasing ethanol intake during the stimulation period. Notably, ethanol remains available for the last 20 min of each session, providing subjects with the opportunity to compensate for the amount of ethanol that was not consumed during the first 10 min. However, when optical stimulation ceased, rats did not attempt to compensate for their loss. This may be due the fact that the tonic increase in accumbal dopamine concentration may mimic the pharmacological effect of alcohol. In this case, rats would require less ethanol to perceive the normal intoxicating effects of this drug. However, tonic optical stimulation delivered in the home cage, during the 10 min preceding ethanol drinking

sessions, had no effect on ethanol consumption or the pattern of intake. Therefore, it seems more likely that optogenetic suppression of cue-induced phasic dopamine release associated with the onset of a drinking session may diminish drinking behaviors even after the optogenetic stimulation ceases.

High-frequency (50 Hz) optogenetic activation of VTA dopamine cells can induce measurable phasic dopamine response in the mouse nucleus accumbens and is sufficient to induce conditional place preference (Tsai et al., 2009). Moreover, high-frequency (60 Hz) electrical stimulation of the rat VTA also induces phasic dopamine release and this pattern of dopamine cell activation promotes the initiation of seeking behavior in rats self-administering cocaine (Phillips et al., 2003b). In addition, phasic activation of VTA dopamine neurons in mice has been reported to completely reverse chronic mild stress-induced anhedonia, significantly increasing sucrose intake (Tye et al., 2012). Therefore, we expected that phasic stimulation would increase ethanol drinking measures. Surprisingly, in our study no significant changes in the latency to the first ethanol drink or the total amount of ethanol consumed were seen. One possible reason for this finding is that we employed rats that were already habituated to the intermittent drinking procedure and also exhibited relatively high levels of ethanol intake at the time when optogenetic stimulation was delivered. Therefore, the inability of phasic stimulation to enhance ethanol drinking measures may have resulted from a ceiling effect. In other words, since these rats were likely already highly motivated to consume ethanol, perhaps additional optogenetic enhancement of phasic dopamine release could not further increase ethanol intake.

Previous studies have found that appetitive (or motivational) and consummatory measures of ethanol drinking are not correlated (Samson and Czachowski, 2003) and are likely mediated by distinct, albeit related, neural circuits. For example, blockade of accumbal D2 dopamine receptors selectively inhibits leverpressing for ethanol (motivational behavior) but has no effect on ethanol consumption if the lever-press response requirement is removed (Samson and Chappell, 2004). Therefore, shifting dopamine transmission into a phasic mode may primarily promote ethanol seeking behaviors without significantly altering the amount of ethanol consumed. Unfortunately, the intermittent drinking self-administration protocol employed in our present study does not provide a clear separation of seeking and consummatory drinking behaviors. Future experiments employing an operant drinking paradigm that procedurally separates these drinking behaviors will be needed to more rigorously evaluate the role of phasic dopamine release on motivational aspects of ethanol seeking.

# **ACKNOWLEDGMENTS**

We are grateful to the WFU Biology Microscopic Core Imaging Facility under direction of Dr. Glen Marrs for use of the Zeiss LSM 710 confocal microscope. This research was supported by the National Institutes of Health T32 AA007565 to Dominic Gioia, AA020564 to Evgeny A. Budygin, AA021099, AA017531, and AA010422 to Jeff L. Weiner, DA024763 to Caroline E. Bass and WFU Cross-Campus Collaborative Fund Award (Evgeny A. Budygin and Keith D. Bonin).

## **REFERENCES**

Anstrom, K. K., Miczek, K. A., and Budygin, E. A. (2009). Increased phasic dopamine signaling in the mesolimbic pathway during social defeat in rats. *Neuroscience* 161, 3–12. doi: 10.1016/j.neuroscience.2009.03.023


core and shell during operant ethanol self-administration. *Alcohol. Clin. Exp. Res.* 33, 1355–1365. doi: 10.1111/j.1530-0277.2009.00965.x


prenatally exposed to cocaine. *Brain Res.* 961, 63–72. doi: 10.1016/S0006- 8993(02)03840-4


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 08 October 2013; accepted: 05 November 2013; published online: 26 November 2013.*

*Weiner JL and Budygin EA (2013) Optogenetic stimulation of VTA dopamine neurons reveals that tonic but not phasic patterns of dopamine transmission reduce ethanol self-administration. Front. Behav. Neurosci. 7:173. doi: 10.3389/fnbeh.2013.00173 This article was submitted to the journal Frontiers in Behavioral Neuroscience. Citation: Bass CE, Grinevich VP, Gioia D, Day-Brown JD, Bonin KD, Stuber GD,*

*Copyright © 2013 Bass, Grinevich, Gioia, Day-Brown, Bonin, Stuber, Weiner and Budygin. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Phasic excitation of ventral tegmental dopamine neurons potentiates the initiation of conditioned approach behavior: parametric and reinforcement-schedule analyses

# *Anton Ilango , Andrew J. Kesner , Carl J. Broker , Dong V. Wang and Satoshi Ikemoto\**

*Behavioral Neuroscience Branch, National Institute on Drug Abuse, National Institutes of Health, Baltimore, MD, USA*

#### *Edited by:*

*Mary Kay Lobo, University of Maryland School of Medicine, USA*

#### *Reviewed by:*

*Dipesh Chaudhury, Mount Sinai School of Medicine, USA Evgeny Budygin, Wake Forest School of Medicine, USA Caroline E. Bass, University at Buffalo, USA*

#### *\*Correspondence:*

*Satoshi Ikemoto, National Institute on Drug Abuse, 251 Bayview Boulevard, Suite 200, Baltimore, MD 21224, USA e-mail: satoshi.ikemoto@nih.gov*

Midbrain dopamine neurons are implicated in motivation and learning. However, it is unclear how phasic excitation of dopamine neurons, which is implicated in learning, is involved in motivation. Here we used a self-stimulation procedure to examine how mice seek for optogenetically-induced phasic excitation of dopamine neurons, with an emphasis on the temporal dimension. TH-Cre transgenic mice received adeno-associated viral vectors encoding channelrhodopsin-2 into the ventral tegmental area, resulting in selective expression of the opsin in dopamine neurons. These mice were trained to press on a lever for photo-pulse trains that phasically excited dopamine neurons. They learned to self-stimulate in a fast, constant manner, and rapidly reduced pressing during extinction. We first determined effective parameters of photo-pulse trains in self-stimulation. Leverpress rates changed as a function of the manipulation of pulse number, duration, intensity, and frequency. We then examined effects of interval and ratio schedules of reinforcement on photo-pulse train reinforcement, which was contrasted with food reinforcement. Reinforcement with food inhibited lever pressing for a few seconds, after which pressing was robustly regulated in a goal-directed manner. In contrast, phasic excitation of dopamine neurons robustly potentiated the initiation of lever pressing; however, this effect did not last more than 1 s and quickly diminished. Indeed, response rates markedly decreased when lever pressing was reinforced with inter-reinforcement interval schedules of 3 or 10 s or ratio schedules requiring multiple responses per reinforcement. Thus, phasic excitation of dopamine neurons briefly potentiates the initiation of approach behavior with apparent lack of long-term motivational regulation.

**Keywords: phasic firing, optogenetics, conditioning, operant, reinforcement schedule, approach motivation, reward**

## **INTRODUCTION**

Midbrain dopamine (DA) neurons play important roles in both learning, which guides behavior, and motivation, which invigorates it (Ikemoto, 2007). DA neurons fire in two notable ways: low-frequency tonic firing and high-frequency phasic firing. Electrophysiological studies have found that the latter occurs upon unexpected presentation of salient stimuli (Schultz et al., 1997; Bromberg-Martin et al., 2010), thus providing incidental information about the external environment. This type of phasic firing is characterized as a "reward prediction errors" (Montague et al., 1996; Schultz et al., 1997) and contributes to associative (or trial-and-error) learning (Steinberg et al., 2013), to maximize the procurement of rewards from the environment.

In addition to participating in associative learning, phasic excitation of DA neurons may have positive motivational effects. Pharmacological studies established dopamine's role in motivation and reward (Wise and Rompre, 1989; Ikemoto and Panksepp, 1999; Ikemoto, 2007); in particular, pharmacological agents that increase DAergic signaling are readily self-administered into the ventral striatum (Ikemoto et al., 1997, 2005; Ikemoto, 2003; Shin et al., 2008), a major projection area of DA neurons localized in the ventral tegmental area (VTA). DA receptor antagonist injections into the ventral striatum appear to disrupt the initiation of, but not ongoing, approach responses (Nicola, 2010). However, such studies employed pharmacological manipulations that modulate DA systems on the order of minutes or longer; therefore, it is unclear how phasic signals of DA neurons, which typically last no more than a half second at a time, play a role in motivation and reward. Optogenetic procedures allow experimenters to selectively stimulate DA neurons in phasic-firing manners. A recent optogenetic study began to shed light on this issue and found that phasic, but not tonic, stimulation of DA neurons induces conditioned place preference (Tsai et al., 2009).

The first aim of the present study was to determine the most effective parameters of photo-pulse trains that phasically excite VTA DA neurons as reinforcer. Specifically, we manipulated the duration, intensity, frequency, and number of pulses of photopulse trains stimulating the opsin channelrhodopsin-2 (ChR2) expressed on VTA DA neurons, and examined their effects on lever-press rates in an operant reinforcement procedure, often referred to as "self-stimulation." Such information has not yet been systematically examined and is important in designing and interpreting behavioral studies on functions of phasic firing of DA neurons.

Phasic excitation of VTA DA neurons appears to be a powerful reinforcer. We and others have observed that phasic stimulation of VTA DA neurons is so rewarding that animals engage in fast, constant self-stimulation behavior while keeping any other activity minimal, resulting in hundreds of presses within several minutes (Witten et al., 2011; Ilango et al., 2014). However, animals show remarkably rapid reductions in response rate when the conditioned response is no longer reinforced (i.e., extinction) (Witten et al., 2011; Ilango et al., 2014). This set of observations is quite distinct from the food-reinforced response, which typically persists for a prolonged period of time during extinction. Therefore, we tested the hypothesis that photo-pulse trains inducing phasic excitation of DA neurons briefly potentiate the initiation of conditioned responses, and that the responses are regulated by little or no long-term motivation. In this light, we contrasted photo-pulse train reinforcement with food reinforcement.

## **MATERIALS AND METHODS**

#### **ANIMALS**

Adult male tyrosine hydroxylase (TH)::IRES-Cre knock-in mice (Lindeberg et al., 2004) crossed with C57BL/6j (The Jackson Laboratory, Bar Harbor, Maine) were used. Mice (weighing 24– 35 g, 2–4 months old at the time of surgery) were group housed until surgery upon which they were individually housed in rooms maintained with a 12:12 light-dark cycle (lights on at 07:00 AM). Mice had free access to food and water outside of daily testing (30–50 min), with the exception of 6 mice (experiment 5 below) that were food-restricted in such a way that they received 1.5– 3 g of rodent chow once a day to maintain 85% of their original weights. All procedures were approved by the Animal Care and Use Committee of the Intramural Research Program at National Institute on Drug Abuse and were in accordance with the Guide for the care and use of laboratory animals (National Research Council, 2011). All efforts were made to minimize suffering and the number of animals for research use.

### **VIRAL VECTORS**

NIDA Optogenetics and Transgenic Technology Core produced the serotype 1 adeno-associated virus (AAV1) encoding ChR2 and enhanced yellow fluorescent protein (EYFP) from plasmids obtained from the Stanford Optogenetics Innovation Lab (pAAV1-Ef1a-DIO-hChR2(H134R)-EYFP-WPRE-pA). The final viral concentration was 6*.*3 × 10<sup>12</sup> viral genomes per ml.

#### **CONSTRUCTION OF OPTICAL FIBERS**

Standard hard cladding multimode fibers with the core size of 200µm and numerical aperture of 0.37 (BFL 37–200, Thorlabs, Newton, NJ) were stripped off at the end and inserted into the 1.25 mm zirconia ferrule (MM-FER 2007C-2300, Precision Fiber Products) (Sparta et al., 2012). The fiber was cut approximately 6 mm in length and the tapered end was polished with different gradients of silicon carbide and aluminum oxide sheets. We used an optical-power meter to measure the intensity of laserlight output. Only fibers emitting concentric circles of light with 75% or higher light transmission were used for implantation. We recorded the percent of light transmission for each fiber and adjusted laser-light output intensities accordingly during experimentation.

## **STEREOTAXIC SURGERY**

Each mouse was anesthetized with a ketamine/xylazine mixture (80/12 mg/kg, i.p.) and received an injection of the AAV vector into the VTA (coordinates: AP 3.5 mm posteriorly from bregma, ML 0.5 mm laterally from midline, DV 4.0 mm ventrally from the dorsal surface of the brain). The AAV vector was microinjected in the volume of 1µl over 10 min using a syringe pump. Then, an optic fiber aiming at 0.2 mm dorsal to the injection site was permanently secured on the skull using dental cement. A dust cap was placed at the tip of the implanted fiber to protect its surface when they were not in test chambers. After the surgery, mice were singly housed, and 2–4 weeks were given to allow gene expression before experiments were started.

#### **APPARATUS**

All behavioral experiments were conducted in standard operant conditioning chambers (ENV-307W: Med Associates, St. Albans, VT). Each chamber was equipped with two levers, two cue lamps and a house lamp. Mice were gently connected to the opticfiber cable that was connected to a 473-nm DPSS laser (OEM Laser Systems, Bluffdale, UT) via an optical commutator. A computer interface system (Med Associates) coordinated pressing on the lever and a stimulus generator (Master 9, A.M.P.I, Jerusalem, Israel) that produced pulses and controlled the laser.

## **EXPERIMENT1: CONFIRMATION OF PHASIC EXCITATION OF DA NEURONS BY PHOTO-PULSE TRAINS**

Electrophysiological tests were performed in two mice. The administration of photo trains and recording of neural activity was accomplished by a screw-driven microdrive coupled with an optic fiber (105µm in diameter) and 7 tetrodes (28 wires). Tetrodes were inserted into 7 individual polyamide cannulas (inner/outer diameters 100/160µm, Ploymicro Technologies), and the polyamide cannulas were then glued on to the optic fiber. The tips of the tetrodes were separated from the tip of optic fiber by 300–500µm. Each tetrode consisted of four 17-µm diameter platinum wires (90% Platinum 10% Iridium, California Fine Wire; with impedances of 1–2 M for each wire). A similar optic-fiber/tetrode microdrive assembly was previously described (Cohen et al., 2012). Two weeks after surgery, electrode signals were screened daily as a function of photo stimulation in two unanesthetized, freely-moving mice. If neural activity responsive to photo stimulation was not detected, the microdrive was lowered by 50–100µm daily. This procedure was repeated until we found neural signals responsive to photo stimulation. Neural signals were recorded using a Digital Lynx recording system (Neuralynx, Bozeman, MT). Spike signals were band-pass-filtered between 300–5000 Hz and digitized at 32 kHz. Spike signals were sorted offline using the Plexon OfflineSorter. Sorted neural spikes were processed and analyzed in NeuroExplorer (Nex Technologies, Madison, AL).

#### **INITIAL SELF-STIMULATION TRAINING**

For behavioral experiments, we used three groups of mice that were trained to respond on the lever reinforced by photo-pulse trains exciting VTA DA neurons. We employed within-subjects designs, and each mouse received both control and experimental manipulations. For all the three groups, a press on the active lever was accompanied by the presentation of a discrete visual cue lasting 1 s during self-stimulation acquisition. The 1-s visual cue was provided to facilitate operant conditioning (Kish, 1966; Shin et al., 2010). However, to eliminate possible confounding effects (Shin et al., 2010; Keller et al., 2014), visual cue was not accompanied with lever-pressing in all other experiments described in the present study. In addition, for the purpose of simplicity, we only described the analyses of active-lever presses in all of the experiments described below, except the initial acquisition test (experiment 2), which will show that inactive-lever presses are low throughout the experiment and do not significantly change as a function of manipulation.

## **EXPERIMENT 2: REPLICATION OF HOW MICE ACQUIRE AND EXTINGUISH SELF-STIMULATION BEHAVIOR REINFORCED WITH PHASIC EXCITATION OF DA NEURONS**

The first group (*N* = 8) was experimentally naïve and was trained as described below with the photo-pulse train (**Figure 1A**) consisting of 1 ms pulse-duration (PD), 10 mW pulse-intensity (PI), 25 Hz pulse-frequency (PF), 15 pulse-number (PN). Their extinction-effect experiment used the photo-pulse train consisting of 10-ms PD, 10-mW PI, 25-Hz PF, 15 PN.

## **EXPERIMENT 3: EFFECTS OF PARAMETRIC MANIPULATIONS OF PHOTO-PULSE TRAINS ON SELF-STIMULATION**

Group 1 mice were then tested with a continuous reinforcement (CR) procedure, where pressing on the lever was immediately reinforced by a photo-pulse train, and the next reinforcement was available as soon as the ongoing train ended. The level of each independent variable (PD: 1, 3, and 10 ms; PI: 0.1, 1, and 10 mW; PF: 8, 20, and 50 Hz) was changed every 15 min, testing 3 levels in ascending order in one session and then in reversed (descending) order in another session. Thus, effects of each parametric variable were tested over two 45-min sessions, and sessions were separated by 1–2 days. These parametric levels were obtained by consulting with previous studies that stimulated VTA DA neurons (Tsai et al., 2009; Adamantidis et al., 2011; Witten et al., 2011; Kim et al., 2012; Ilango et al., 2014).

Mice in the second group (*N* = 7) were initially used in another study described by Ilango et al (2014). These mice extensively received VTA photo-pulse trains (10-ms PD, 20-mW PI, 25-Hz PF, 15 PN) prior to the present experiments, and went through the acquisition of self-stimulation, operant extinction, re-acquisition, and place-preference tests, adding up to about 17– 40 test sessions (30 min per session) before they were used for the experiments described below. The Ilango study (2014) describes the prior test procedures and histological data of group 2 mice. These mice were used to examine effects of pulse-number. They were tested for lever pressing reinforced on a CR schedule with photo-pulse train (PD: 10 ms; PI: 20 mW; PF: 25 Hz) of varying pulse-number: 0, 2, 4, 8, and 16 pulses. Each pulse's effects

were examined in a 10-min period (or 2 bins of 5 min), and the pulses were tested in ascending order over a 50-min session. At the start of each period, mice received 5 "priming" trains (one train per s) consisting of the number of pulse that was tested for that period, to stimulate pressing. The number of presses during the 2nd bin of each 10-min pulse-period was used to represent the effect of the specific pulse-number. These counts were analyzed with a One-Way within-subjects ANOVA.

## **EXPERIMENT 4: EFFECTS OF INTER-REINFORCEMENT INTERVAL ON SELF-STIMULATION**

Group 2 mice were then used to examine effects of interreinforcement intervals. Four intervals were tested over five 10-min blocks in the order of 0, 1, 3, 10, and 0 s. The 0-s interval was repeated at the end, to show that mice were still capable of pressing at similar levels as the first 10-min period after a series of intervals. Effects of these intervals were examined with 4 different pulse numbers (2, 4, 8, and 16) per train (PD: 10 ms; PI: 20 mW; PF: 25 Hz) over 4 sessions, to determine how different values of reinforcer would interact with reinforcement intervals. Technically, the 0-s interval was interval schedules of 50–610 ms depending on how many pulses the reinforcer contained, since additional reinforcement was not provided during photo-pulse trains. For simplicity, we called it 0-s interval or CR schedule. Similarly, 1–10 s intervals were also slightly longer depending on the length of photo-pulse trains. The press count of the 2nd 5-min bin of each 10-min interval was analyzed with a 5interval × 4pulse<sup>−</sup>number within-subjects ANOVA.

#### **EXPERIMENT 5: EFFECTS OF RATIO SCHEDULES ON RESPONSES REINFORCED WITH FOOD PELLETS OR PHOTO-PULSE TRAINS**

The third group (*N* = 6) used for this experiment were initially used in still another study where they were trained to selfstimulate for photo trains (PD: 1 ms; PI: 10 mW; PF: 50 Hz; PN: 8) exciting VTA DA neurons on a CR schedule up to 12 sessions, and then they had received a bundle of electrodes implanted into the ventral striatum for recording, which was not performed in the present experiment. After the surgery, they were tested with VTA photo-pulse trains (PD: 1 ms; PI: 0.1–10 mW; PF: 8–50 Hz; PN: 8) for additional 8 sessions (30–60 min per session). Following these testing experience, they were used to examine effects of ratio schedules of reinforcement as described in the result section.

## **HISTOLOGY**

After completion of the behavioral experiment, all animals were deeply anesthetized with a ketamine/xylazine mixture (80/12 mg/kg, i.p.) and intracardially perfused with ice cold 0.9% saline followed by 10% formalin. Brains were isolated and post-fixed in 10% formalin up to 24 h and cryoprotected in 30% sucrose solution until they sank to the bottom. Brains were coronally sectioned at 40µm, and mounted sections were cover slipped with Mowiol 4–88 (Sigma-Aldrich) plus Vectashield mounting medium with DAPI nuclear counterstain (H-1200, Vector laboratories, Burlingame, CA). Optical fiber placements and EYFP expression were confirmed with fluorescent microscopy.

## **STATISTICAL ANALYSES**

All data pertaining to lever presses were square-root transformed, to minimize heterogeneous variances for parametric statistical tests (McDonald, 2009). Square-root transformed data were analyzed with the repeated measures of ANOVA using Statistica (version 6.1, StatSoft, Inc., Tulsa, OK). Significant effects were further analyzed by the Tukey's Honestly Significant Difference (HSD) *post-hoc* test.

## **RESULTS**

To phasically stimulate VTA DA neurons, we injected a doublefloxed inverted open reading frame (DIO) Cre-dependent AAV1 vector encoding ChR2 fused with EYFP into the VTA of TH-Cre transgenic mice and implanted an optic fiber just dorsal to the VTA for photo-pulse trains. Two to four weeks after the surgery, we examined effects of photo-pulse trains (473-nm wavelength) through the implanted optic fiber. **Figure 1A** displays a histological result showing the expression of EYFP and the placement of optic fiber of a representative mouse. Expression of EYFP and optic fiber placements were confirmed in all the mice we tested in this study.

## **EXPERIMENT1: CONFIRMATION OF PHASIC EXCITATION OF DA NEURONS BY PHOTO-PULSE TRAINS**

We confirmed in two mice that our photo-train administration procedure was capable of inducing phasic firing in ChR2 expressing (i.e., TH-positive, DA) neurons and modulating phasic firing as a function of change in pulse-train property (**Figure 1B**). In one mouse, we compared effects of pulse durations between 0.1 and 0.5 ms on neural response. The 0.5-ms pulse almost always excited the DA neuron, whereas the 0.1-ms pulse led to 10-15 action potentials out of 40 trials (less than 40% fidelity; **Figure 2A**). Three different frequencies of a pulse train were tested in a neuron of another mouse. The administration of an 8-pulse train at 20 or 50 Hz caused reliable excitation for almost every pulse (**Figure 2B**: left and middle panels), whereas 100-Hz trains reliably excited the neuron for the first 4 pulses, but failed to reliably excite it for the last 4 pulses of the train (**Figure 2B**: left panel insert). Previous electrophysiology experiments suggested that the success rate of triggering an action potential in DA neurons decreased markedly when frequency increased greater than 20 Hz, reaching only 50% of spike fidelity with the frequencies of 40–50 Hz (Tsai et al., 2009; Witten et al., 2011). Inconsistently, we found that spike fidelity with the 50-Hz train was as good as that of the 20-Hz in a single neuron. Note also that DA neurons became inactive immediately after the offset of photo-pulse trains, and the length of neural inactivity seems to have progressively increased as the train frequency increased (**Figure 2B**). Although these data in two DA neurons do not allow us to generalize how DA neurons respond to photo-pulse trains, the data confirm that our optogenetic-stimulation procedure induces phasic firing in neurons and seem to be in general agreement with what is known about how ChR2 responds to light (Nagel et al., 2003; Boyden et al., 2005) and how DA neurons fire (Tsai et al., 2009; Wang and Tsien, 2011; Witten et al., 2011).

## **EXPERIMENT 2: REPLICATION OF HOW MICE ACQUIRE AND EXTINGUISH SELF-STIMULATION BEHAVIOR REINFORCED WITH PHASIC EXCITATION OF DA NEURONS**

Experimentally-naïve mice (*n* = 8) were individually placed in an operant conditioning chamber equipped with two levers. During the first two sessions (30 min per session), a press on the "active" lever delivered the presentation of a 1-s visual cue just above the lever, but no VTA photo-pulse train, while pressing on the other "inactive" lever had no programmed consequence throughout this experiment. During the first two sessions, pressing on both levers remained relatively low (**Figure 3A**). In sessions 3–7, a press on the active lever delivered a 1-s visual cue and photo-pulse train (0.57 s in duration), and the next photo-pulse train became available for the taking as soon as the visual cue was turned off.

**FIGURE 2 | Confirmation of phasic excitation of DA neurons by photo-pulse trains in freely-behaving mice. (A)** The perievent spike rasters (top) and histograms (bottom) show photo stimulation-evoked firing of a neuron. Experimenter administered 40 photo trains (PI: 0.2 mW; PF: 20 Hz; PN: 8) of the 0.5 (left) and 0.1 (right) ms pulse-durations with the inter-train intervals varying between 10 and 15 s. Bin = 5 ms. **(B)**. The

perievent spike rasters (top) and histograms (bottom) show photo stimulation-evoked firing of a neuron. Experimenter administered 20 photo trains (PD: 1 ms; PI: 0.2 mW; PN: 8) of the 20- (left), 50- (middle), and 100-Hz (right) frequencies with the inter-train intervals varying between 10 and 15 s. Bin = 5 ms. **(B')** It shows an enlarged histogram for the 100-Hz train. Bin = 2 ms.

**FIGURE 3 | Self-stimulation behavior with phasic excitation of DA neurons during acquisition and extinction. (A)** Mean rates of activeand inactive-lever presses and presses of individual mice (black dots) during acquisition. Active-lever values in sessions 3–7 were significantly greater than those of session 1 or 2 (*Ps <* 0.0005). **(B)** The cumulative presses of a representative mouse that responded at the rate of 243

per 5 min in session 7. When the cumulative lever-press count reaches 250, the count is reset at 0, and the angle of the cumulative-lever-press line indicates the rate of lever-press. Slashes indicate reinforcement incidents. **(C)** Rapid decrease in pressing during extinction. The data are mean with s.e.m. \**P <* 0*.*0005, value significantly lower than that of block 1 or 5.

The mice increased pressing on the active lever over sessions 3–7. A 2lever × 7session ANOVA revealed a significant lever × session interaction [*F*(6*,* 42) = 49*.*12, *P <* 0*.*0001]. While presses on the active lever did not differ between sessions 1 and 2, response counts significantly differ between sessions 2 and 3 (*P <* 0*.*001), 3 and 4 (*P <* 0*.*05), and 4 and 5 (*P <* 0*.*005, Tukey's test). Response rates of the active lever began leveling out by sessions 5; presses in sessions 5–7 were significantly greater than those of sessions 1–4 (*P <* 0*.*0005) or inactive-lever presses of respective sessions (*P <* 0*.*0005, Tukey's test). Inactive-lever presses did not differ between sessions 1–7 (Tukey's test). **Figure 3B** shows the response pattern of a representative mouse that responded at the rate of 243 per 5 min in session 7. The data suggest that pressing on the active lever was so vigorous that the mouse took few breaks from pressing on the lever and maintained a relatively-constant response-rate throughout the 30-min session. By session 7, this response pattern was typical among the mice.

We then showed how quickly mice would change lever-press rates when pressing was no longer reinforced with photo-pulse trains (i.e., extinction). Effects of extinction were examined in a 40-min session consisting of two 20-min phases. In each phase, pressing on the active lever was reinforced with photo-pulse trains during the 1st 5-min block, followed by a 15-min extinction phase. Mice responded at a rate of over 200 presses during the 1st 5-min block and quickly decreased pressing during the extinction phase (blocks 2–4) (**Figure 3C**). After 5 min of no reinforcement, mice responded at rates of lower than 50 per 5-min (blocks 3 and 4). At the onset of the 2nd phase (i.e., block 5 in **Figure 3C**), mice received 5 "priming" trains administered one train per sec, and they quickly reinstated pressing on the lever to a level similar to that of the 1st block. Again response rates markedly decreased during the extinction phase (blocks 6–8) in a similar manner as in the first extinction phase (blocks 2–4). These effects are supported by a significant block effect [*F*(3*,* 18) = 142*.*61, *P <* 0*.*0001 with a 2phase × 4block within-subjects ANOVA]. Although a phase × block interaction was not reliable, the second phase had slightly lower response rates than the first [a significant phase effect: *F*(1*,*6) = 15*.*17, *P <* 0*.*01].

#### **EXPERIMENT 3: EFFECTS OF PARAMETRIC MANIPULATIONS OF PHOTO-PULSE TRAINS ON SELF-STIMULATION**

We examined effects of pulse-duration, pulse-intensity (measured at the tip of the ferrule that connects with the implanted optic fiber), pulse-frequency, and pulse-number per train in reinforcing behavior (**Figure 1B**). While no priming train was provided for descending-order sessions, for ascending-order sessions, mice received, at the onset of each 15-min period, 5 "priming" trains (with the rate of one train/s) whose parameters were assigned for that period, to reinstate pressing. The median lever-press count from 3 bins (5 min each) of each 15-min period was selected and analyzed with 2test<sup>−</sup>order × 3variable<sup>−</sup>level ANOVAs for active-lever presses. The main effect of test order was not significant for any of the variables; therefore, the data obtained with the ascending and descending orders are combined and presented together (**Figures 4A–C**). Lever-press rates were slightly reduced with the 1-ms pulse-duration than the 3- or 10-ms durations, and the 3 ms duration did not differ from the 10-ms duration [duration:

**FIGURE 4 | Effects of parametric manipulations of photo-pulse trains on self-stimulation.** The data are mean with s.e.m. **(A)** Pulse duration was varied. \**P <* 0*.*005, significantly lower than the 3- or 10–ms value. **(B)** Photo intensity was varied. \**P <* 0*.*01, significantly lower than the 1- or 10-mW value; #*P <* 0*.*05, significantly greater than the 0.1-ms value. **(C)** Pulse frequency was varied. \**P <* 0*.*005, significantly lower than the 20- or 50-Hz values; #*P <* 0*.*05, significantly greater than the 8-Hz value. **(D)** Pulse number was varied. \**P <* 0*.*005, significantly greater than the 0-pulse value; #*P <* 0*.*05, significantly greater than the 0- and 2-pulse values. **(E)** Summarized are effects of the factors: duration, intensity, frequency, and number. The maximal values of the factors (10 ms, 10 mW, 50 Hz, and 16 pulses, respectively) are set at 100%, and the rest of the values of each factor are transformed accordingly.

*F*(2*,* 14) = 15*.*66, *P <* 0*.*0005; **Figure 4A**]. All 3 pulse-intensities significantly differed from each other, with the 10-mW intensity being the most effective [intensity: *F*(2*,* 14) = 21*.*09, *P <* 0*.*0001; **Figure 4B**]. The mice markedly reduced press rates with the 8-Hz train compared to the 20- or 50-Hz train [frequency: *F*(2*,* 14) = 100*.*05, *P <* 0*.*0001; **Figure 4C**].

We then examined effects of pulse-number per train (PN: 2, 4, 8, and 16) on lever pressing, using the second group of mice (*n* = 7) described in the method section. The mice maintained low press rates during the 0-pulse (i.e., no stimulation) period and markedly increased pressing when it was reinforced with 2 pulse train or greater [pulse: *F*(4*,* 24) = 33*.*49, *P <* 0*.*0001]. The 8-pulse train was significantly more effective than the 2-pulse, but not the 4-, or 16-pulse train (**Figure 4D**).

**Figure 4E** summarizes how lever-press rates were altered by pulse-duration, intensity, frequency, and number. The manipulations of various properties of photo-pulse trains readily alter their efficacy, leading to changes in lever-press rate. In addition, we should point out that despite of repeated test sessions, photopulse trains of DA neurons produced comparable data when the same parameters of photo-pulse trains was examined between sessions, as we observe in this experiment and in our previous study (Ilango et al., 2014). Therefore, the repeated-subjects design of experiments is generally effective method for the investigation involving stimulation manipulations on self-stimulation.

#### **EXPERIMENT 4A: EFFECTS OF INTER-REINFORCEMENT INTERVAL AND PULSE NUMBER ON SELF-STIMULATION**

The manipulation of reinforcement schedules produces a wide range of change in behavior reflecting emotional and motivational processes (Ferster and Skinner, 1957). We examined effects of inter-reinforcement interval schedules, in which mice had to wait a given interval of time measured from the preceding reinforcement before their lever-pressing was reinforced.

The 1-s interval was sufficient to significantly reduce leverpress rates compared to those of 0-s interval (*P <* 0*.*0005, **Figure 5A**), while the 3- or 10-s interval diminished lever-press rates [*P <* 0*.*0005, Tukey's test after a significant interval effect, *F*(4*,* 24) = 44*.*41, *P <* 0*.*0001]. This result needs to be considered in light of the fact that the number of reinforcements obtainable per unit of time becomes fewer as reinforcement interval increases (**Table 1**). Because mice responded at relatively fast rates, the reduced numbers of pressing can be explained by the reduced availability of reinforcements.

However, this reduced reinforcements explanation is not consistent with a significant interaction between reinforcement interval and pulse number [*F*(12*,* 72) = 4*.*18, *P <* 0*.*0001], which shows opposite effects that this explanation predicts. Lever-press rates maintained with 16-pulse trains did not significantly differ between the 1-s and the 0-s interval schedules; the 8-pulse train significantly decreased press rates during the 1-s interval than the 0-s interval (*P <* 0*.*0005), while maintaining significantly greater press rates during the 1-s than the 3- or 10-s interval (*Ps <* 0.05 and 0.001, respectively); the 4- and 2-pulse train diminished press rates with the 1-s interval, which did not differ from those of the 3- or 10-s interval (**Figure 5A**). Therefore, graded effectiveness of 4-, 8-, and 16-pulse trains was apparent with the 1-s interval, effects that are not consistent with the fewer reinforcements explanation. Similar effects of pulse and interval were found on reinforcement incidents (**Figure 5B**): interval effect on reinforcement incidents is more robust than that of lever presses, while interval-pulse interaction is less striking [interval: *F*(4*,* 24) = 80*.*44, *P <* 0*.*0001; interval × pulse interaction: *F*(12*,* 72) = 2*.*96, *P <* 0*.*005]. In summary, even though more trains were available for the 2- and 4-pulse trains than the 16-pulse train, the 2- and 4-pulse trains did not maintain response rates greater than that of the 16-pulse train. Therefore, the reduction of response rates must be largely caused by the longer inter-reinforcement interval reducing motivation for pulse trains.

#### **EXPERIMENT 4B: EFFECTS OF INTER-REINFORCEMENT INTERVAL AND PULSE NUMBER ON THE FIRST LEVER PRESS AFTER REINFORCEMENT**

One notable feature of photo-pulse train reinforcement is that the mice responded on the lever even during ongoing photo-pulse trains. **Figure 6A** plots the lever-press events of a representative mouse in relation to reinforcements and intervals when tested with 16-pulse trains. Lever pressing during ongoing trains

**FIGURE 5 | Effects of inter-reinforcement interval on self-stimulation. (A)** Mean lever-press rates are shown as a function of inter-reinforcement interval and pulse number. All 0-s interval values were significantly greater than respective 3- or 10-s interval values (*Ps <* 0.001). \**P <* 0*.*001, significantly greater than its 3- and 10-s values, but not from its 0-s value; #*P <* 0*.*05, significantly greater than its 3- and 10-s values, but lower than its 0-s value; <sup>+</sup>, <sup>∧</sup>*P <* 0*.*001, not significantly different from its 3- and 10-s values, but lower than its 0-s value **(B).** Mean reinforcement rates are shown as a function of reinforcement interval and pulse number. All 0-s interval values were significantly greater than respective 1-, 3-, or 10-s interval values (*Ps <* 0.001). \**P <* 0*.*001, significantly greater than its 3- and 10-s values; #*,*+*P <* 0*.*01, significantly greater than its 10-s values.

#### **Table 1 | Dependency of the numbers of photo-pulse trains obtainable per 5 min on photo-pulse trains durations and inter-reinforcement intervals.**


**reinforcement. (A)** Lever-press incidents with the 16-pulse train are plotted in relation to the last reinforcement incident and train duration. When presses shown in blue dots were emitted during the train duration (shown in yellow area) or during inter-reinforcement interval (gray area), they were not reinforced. The number on the right indicates inter-reinforcement intervals for the 10-min periods. **(A')** Mean presses

**(B)** Mean lever-press speeds (1/latencies) of first and second presses after reinforcement incidents as a function of pulse and interval. \**P <* 0*.*005, significantly slower than the 0-, 1-, and 3-s interval values. \$*P <* 0*.*005, significantly slower than its first response speed. **(C)** Mean percent of CR-schedule values. \**P <* 0*.*005, significantly greater than lever-press rate or reinforcement rate value for respective interval.

(610 ms in duration) was not reinforced with additional photopulse trains. Despite this, this mouse and the others responded often during ongoing trains (**Figure 6A** ). This observation is consistent with our hypothesis that photo-pulse trains potentiated the initiation of *conditioned* approach response. However, this observation is also consistent with the notion that mice responded during the photo-pulse trains because mice responded on the lever so fast that responding had its *momentum*, which caused responding even during the non-reinforced period. The results described below are more consistent with the response initiation hypothesis than the momentum explanation.

We compared speeds between the first and the second leverpresses as a function of interval for each pulse, using 2press<sup>−</sup>order × 4interval ANOVAs. A significant press-order × interval interaction was found for the 16-pulse train [*F*(3*,* 18) = 6*.*23, *P <* 0*.*005; **Figure 6B**: right panel]. The 1- and 3-s intervals did not significantly reduced the first press speed, while the 10 s interval did; and the speeds of second press was significantly reduced for all intervals compared to those of the first (**Figure 6B**: left panel). To determine how differently overall response rates, reinforcement rates, and first lever-press speeds were altered by interval schedules, we converted the 1-, 3-, and 10-s interval values of these variables with respect to their 0-s interval values as being 100% (**Figure 6C**). Normalized values were analyzed with a 3variable × 3interval ANOVA. The 16 pulse data revealed a significant variable x interval interaction, [*F*(3*,* 18) = 3*.*56, *P <* 0*.*05]. Lever-press rates and reinforcement rates had lower values than first lever-press speeds for the 1- (*P <* 0*.*05 and *P <* 0*.*0005, respectively) and 3-s (*P <* 0*.*0005 and *P <* 0*.*0005, respectively) intervals (**Figure 6C**: left panel). The lever-press speed was also significantly different from reinforcement rate during the 10-s interval schedule (*P <* 0*.*05). Lever-press rates did not differ from reinforcement rates at any interval (Supplemental Figure 1 provides comparisons for the 8- and 4-pulse trains). Therefore, the clear dissociation between first lever-press speeds and response rates supports the response initiation hypothesis, but not the momentum explanation.

## **EXPERIMENT 5: EFFECTS OF RATIO SCHEDULES ON RESPONSES REINFORCED WITH FOOD PELLETS OR PHOTO-PULSE TRAINS**

The above observation about interval schedules suggest that timely reinforcement with DA signals is critical in maintaining fast, constant lever-pressing, and any schedule that prevents mice from obtaining timely reinforcement will disrupt pressing. To further confirm this notion, we examined how mice would respond for photo-pulse trains when challenged by fixed-ratio (FR) schedules in which a lever-press was reinforced after a completion of a fixed number of presses counted from the last delivery of reinforcer. Effects of FR on self-stimulation were contrasted with those on food-seeking. The third group of mice described in the method section were trained to respond for food pellets on various FR schedules first and then re-trained to respond for photo-pulse trains exciting DA neurons.

Significant sessions effects were found following one-way within-subjects ANOVAs on lever-press and reinforcement rates with 20 selected sessions consisting of 6 food and 14 photopulse train sessions [*F*(19*,* 95) = 25*.*67, *P <* 0*.*0001 and *F*(19*,* 95) = 125*.*83, *P <* 0*.*0001, respectively; **Figure 7A**]. The same lever that had produced VTA photo-pulse trains in the previous study also delivered pellets, and these mice quickly learned to respond for pellets and displayed stable pressing for pellets during the first 3 sessions, although pellet reinforcement on a CR schedule kept lever-press rates low (**Figure 7A**: session 3, left panel). We used FR schedules in the following sessions, to examine how FR challenges alter lever pressing. The mice were initially trained to respond 5 times on the lever for a pellet (i.e., FR5) for 11 sessions, then on a FR10 for 7 sessions and on a FR15 for 4 sessions. The mice learned to respond about 150 times per 5 min on the FR15 schedule in session 25, while earning similar numbers of pellets to those of sessions 3 (CR schedule; **Figure 7A**: left panel). **Figure 7B** depicts that the mice obtained pellets with the mean inter-reinforcement interval of 24 s on the CR schedule, and they increased rates of pressing as response-ratio requirement increased for a pellet in such a way that they maintained similar inter-reinforcement intervals (green arrows) across the different ratio schedules. When they were tested with FR15 combined with fixed-interval (FI) schedules of 3 and 10 s in sessions 26 and 27, respectively, they displayed numbers of lever presses and reinforcements similar to those of session 25 (**Figure 7A**: left panel). These results confirm that mice can regulate lever-press rates in response to ratio challenges to obtain food at a constant rate.

After session 27, the mice were no longer food-restricted and received VTA photo-pulse trains (PD: 10 ms; PI: 20 mW; PF: 25 Hz; PN: 15), instead of pellets, as reinforcer. While lever-press rates in sessions 29 and 30 on a CR schedule were greater than that of pellets in session 25 (**Figure 7A**: left panel), lever-press rates plummeted when the mice had to earn photo-pulse trains on the FR15 schedule in sessions 31–34 (**Figure 7A**: middle panel). Lever-press rates of sessions 31–34 were significantly lower than those of session 25, and repeated testing with the FR15 schedule tended to worsen self-stimulation performance until session 35 when the CR schedule was reinstated (see Supplementary Figure 2). **Figure 7C** depicts pressing patterns of the best selfstimulator (with respect to lever-press rate) and shows that its performance deteriorated over sessions 31–34, while it recovered in session 35 with the CR schedule to a similar level as that in session 30. Lever-press rates of this and other mice during 31–34 were reduced to similar levels as those we observed in other groups on extinction or interval schedules of 3 and 10-s (**Figures 3**, **5**). Therefore, repeated testing with FR 15 did not appear to help improve self-stimulation performance even though the same animals had responded at a high level for pellet reinforcer with the same schedule.

We then examined effects of a gradual increase in response requirement over sessions on lever pressing reinforced by photopulse trains. Their lever-press rates significantly decreased as the lever-press ratio increased over the sessions (**Figure 7A**: right panel). Specifically, as the response-ratio requirement increased, lever-press intervals increased, resulting in longer and longer inter-reinforcement intervals (black arrows in **Figure 7B**). Thus, we did not detect evidence that mice maintained certain reinforcement rates with excitation of DA neurons across different FR schedules like they did with food pellets.

To determine how differently overall response rates, reinforcement rates, and first lever-press speeds were altered by FR schedules, we converted the values of these variables with respect to FR1 values as being 100%. Specifically, the values of sessions 14, 21, 25, 26, and 27 were normalized with respect to those of session 3 for the pellet reinforcement experiment (**Figure 7D**: left panel). A significant variable × session interaction was found with a 3variable × 5session ANOVA for pellet reinforcement [*F*(8*,* 40) = 12*.*02, *P <* 0*.*001]. The response-ratio schedules involving pellets had opposing effects between lever-press and reinforcement rates, and affected them differently for all sessions (14, 21, 25, 26, and 27). While the speeds of the first presses and lever-press rates increased, lever-press rates increased to greater degrees than the first lever-press speeds for sessions 21, 25–27. The responseratio schedules tended to increase lever-press speeds and decrease reinforcement rates for sessions 25–27.

Similarly, the photo-pulse trains data in sessions 30–34 were analyzed by a 3variable × 4session ANOVA with session-30 values being 100% (**Figure 7D**: middle panel). A significant variable effect was found [*F*(2*,* 10) = 26*.*66, *P <* 0*.*005]. The FR15 schedule reduced first lever-press speeds less severely than leverpress rates (*P <* 0*.*005) and reinforcement rates (*P <* 0*.*001), and reduced similarly between lever-press and reinforcement rates (Tukey's test). The photo-pulse trains data in sessions 35–41 were analyzed by a 3variable × 6session ANOVA with session-35 values being 100% (**Figure 7D**: right panel). A significant variable and session effects, but not an interaction, were found [*F*(2*,* 10) = 81*.*54, *P <* 0*.*0001 and *F*(5*,* 25) = 31*.*24, *P <* 0*.*0001, respectively]. The response-ratio schedules reduced reinforcement rates more severely than first lever-press speeds (*P <* 0*.*005) and lever-press rates (*P <* 0*.*005), and first speeds than lever-press rates (*P <* 0*.*05, Tukey's test). Thus, an important difference in this analysis from the one for sessions 30–34 is that speeds were affected more than lever-press rates throughout sessions 35–41. This suggests that repeated experience with high ratio schedules diminished brief potentiating effect of photo-pulse trains on conditioned response.

#### **DISCUSSION**

#### **EFFECTS OF PARAMETRIC MANIPULATIONS ON SELF-STIMULATION RATES**

The evidence that we consider below suggests that the parameters of photo-pulse trains that support fast, constant pressing probably excite DA neurons in such a manner that they significantly increase DA concentrations at projection regions. In addition, efficacy of photo-pulse trains in reward depends on how well photo-pulse trains interact with physical property of ChR2 and that of DA neurons, and whether it recruits a large population of DA neurons. Factors such as the density of ChR2 expression in each neuron should make a significant difference in stimulation efficacy between subjects and studies; because such factors are beyond the scope of the present study, we do not discuss them.

#### **FIGURE 7 | Continued**

During sessions 28–41, pressing was reinforced with VTA photo-pulse trains. <sup>+</sup>*P <* 0*.*05, significantly greater than the session-25 value; \*1*P <* 0*.*005, significantly lower than the session-28 value; #*P <* 0*.*005, significantly lower than the session-25 value; \*2*P <* 0*.*005, significantly lower than the session-35 value. **(B)** Mean lever-press latencies (s) as a function of lever-press count in FR schedules and reinforcers. Red arrowheads indicate the latencies to initiate leverpressing for photostimulation. Green and black arrows indicate the latencies to complete FR requirement for pellets and

Reinforcement efficacy of pulse duration critically depends on how quickly ChR2 responds to light, allowing cations to enter the cell. ChR2 is capable of responding to light for maximal intracellular conductance within 0.5 ms (Bamann et al., 2008). Consistent with this observation, our preliminary investigation in a single neuron showed that the 0.5-ms pulse triggered action potential at almost 100%, while 0.1-ms pulse at about 50% (**Figure 2B**). We did find, however, 3- and 10-ms pulses were more effective in increasing self-stimulation rates than a 1-ms pulse (**Figure 4A**). There should not be any difference between them when the 0.5 ms pulse is sufficient in activating ChR2 at 100%. It is likely that 3- and 10-ms pulses can recruit more neurons by activating ChR2 at distal sites, where the 1-ms pulse is too short to deliver enough light energy for action potentials. That is, the pulse duration factor may interact with the distance factor in activating ChR2.

Photo intensity is known to exponentially decrease as the distance between its source and target increases. Therefore, the number of neurons affected by light is thought to depend on intensity. Consistently, we found that self-stimulation performance became better and better as the photo intensity increased from 0.1 to 1 to 10 mW (**Figure 4B**). Likewise, voltammetry study found that as the intensity increases, DA is released more in projection regions (Bass et al., 2010).

The pulse frequency factor exquisitely affected self-stimulation performance. Pulse frequency is important because it controls phasic firing rates of DA neurons and interacts with the DA uptake process that keeps extracellular DA concentration in check. That is, the higher the frequency that DA neurons fire, the greater the DA concentration becomes, because DA release outperforms the uptake process. However, as frequency increases, each subsequent pulse loses its fidelity to excite neurons. Consistent with our single-unit data, we observed the 50-Hz train was significantly more effective than the 20-Hz in selfstimulation (**Figure 4C**), and our behavioral results are consistent with those reported by a recent self-stimulation study (Rossi et al., 2013). If spike fidelity of a 50-Hz train is 50%, a 50-Hz train would trigger about the same number of action potentials as the 20-Hz train; therefore, it was expected to see a similar selfstimulation rate between 20- and 50-Hz trains. The discrepancies may be resolved by considering the contributions of other factors. A voltammetry study suggests that the efficacy of frequency on spike fidelity depends on pulse duration. A pulse train with 20 ms pulse duration tended to decrease DA release when delivered at higher frequencies than 20 Hz, while a 4-ms pulse-duration train increased DA release up to 50 Hz and decreased it at higher frequencies (Bass et al., 2010). Caveats include recording and stimulation sites. Bass et al performed voltammetry in the dorsal photostimulation, respectively. **(C)** Lever-press incidents during the first 10 min block during sessions 30–35. Notice that they are plotted on a log<sup>2</sup> scale (x-axis). Presses shown in blue dots occurring during the train duration (shown in yellow area) were not reinforced. Every lever-press occurring after each pulse train was reinforced for FR1 session, while for FR15 sessions, only the 15th lever-press (indicated by a maroon circle) occurring after each pulse train was reinforced. **(D)** Mean percent of CR-schedule values. <sup>+</sup>*P <* 0*.*05, significantly greater than reinforcement rate, but not response rate. \**P <* 0*.*05, significantly different from both reinforcement and response rates.

striatum with the stimulation of substantia nigra DA neurons. Another important factor is the recording condition. Our frequency experiment on DA neuron spikes used a pulse duration of 1 ms recorded in an unanesthetized, unrestraint mouse, while the studies (Tsai et al., 2009; Witten et al., 2011) that reported 50% of spike fidelity with the frequencies of 40–50 Hz used 15- and 5-ms pulse durations, respectively, in slice recording conditions.

Pulse number had relatively small effects on lever-press rate. Given the voltammetry finding that as the pulse number increases, extracellular DA concentration increases (Bass et al., 2010; Witten et al., 2011), we expected to see large effects between 2, 4, 8, and 16 pulses. The 4-pulse train was almost as effective as the 8- or 16-pulse train (**Figure 4D**). The lack of difference between these is most likely explained by a ceiling effect in that mice appeared to self-stimulate as fast as they could with stimulation of 4 and greater pulses. Consistently, when challenged by 1-s interval schedule, the 16-pulse train supported greater lever-press rates than lower pulse trains.

## **PHASIC EXCITATION OF DA NEURONS POTENTIATE THE INITIATION OF APPROACH BEHAVIOR**

Previous studies noted that DA concentrations in the ventral striatum phasically increase during ongoing approach response that earns cocaine (Phillips et al., 2003), VTA stimulation (Cheer et al., 2007), or food (Roitman et al., 2004). It is tempting to interpret that DA releases coinciding with approach responses actually help to drive the behavior. Our finding that phasic excitation of VTA DA neurons potentiates the initiation of conditioned approach behavior (**Figure 6C**) provides direct evidence that indeed, phasic excitation of DA neurons, which results in phasic DA release in the ventral striatum (Wightman and Robinson, 2002), potentiates the initiation of conditioned approach behavior. Our data show that potentiated lever-pressing is not consistent with the explanation involving a motor reflex since its effect dissipated as inter-reinforcement interval increased (**Figure 6B**; Supplemental Figure 1A). It is not momentum either, since the 16-pulse train maintained the same initiation speeds between the 0- and 3-s intervals when overall rates of responding with the 3-s interval was reduced to 30% of the 0-s interval (**Figure 6C**). Moreover, it is a conditioned approach response, since mice learned not to respond with vigor after repeated training with large ratio schedules of reinforcement (**Figures 7C,D** and Supplemental Figure 2).

The present study also provides insight on other functional properties of phasic excitation of DA neurons. There appear to be, at least, two distinct mechanisms to control rates of conditioned lever-pressing reinforced by phasic excitation of DA neurons. We initially observed that mice needed many repetitions of responsereinforcer experience to acquire a high response rate (**Figure 3A**). This observation is consistent with the notions that many repetitions are needed to form and refine new skillful responses in an environment-dependent manner (Ferster and Skinner, 1957) and to form long-term memory concerning the associative structure of the environment and its action, i.e., associative learning, to maximize the procurement of reinforcers through "trial-anderror" processes (Hull, 1952; Sutton and Barto, 1998). Thus, response-contingent excitation of DA neurons triggers long-term changes pertaining to learning. In addition, phasic firing of DA neurons appears to have motivational effects. After acquiring self-stimulation, their response rates dropped from the high asymptotic level to a low baseline level during extinction in just several minutes and from the low to high levels during reacquisition in just a few seconds (**Figures 3C**, **6A**). Our observations suggest that DA signals potentiated the initiation of conditioned lever-pressing, an effect that lasted for about 1 s (**Figure 5A**) and lacked long-term motivational control (**Figure 7B**). This property of DA signals most likely plays a critical role in fast, constant self-stimulation on a CR schedule and rapid reduction of selfstimulation upon relatively minor interval- or ratio-schedule challenges.

We have also observed that distinct regulations of behavior reinforced by DA neuron excitation and food pellets. Most notably, pellet reinforcement did not potentiate presses just after reinforcements, and it instead inhibited them for a few seconds (**Figure 7B**). This is not surprising as animals take time to consume earned food before next responses. On the other hand, photo-pulse trains exciting DA neurons did not distinguish between approach and consummatory responses or inhibit approach behavior upon the delivery of the reinforcer; instead, it facilitated pressing in a short-lasting manner (**Figures 6A–C**). Another striking feature of food reinforcement was that it regulates approach behavior in a long-term frame, a regulatory system that behavioral scientists referred to as "primary drive" or "primary motivation." The mice were able to control pressing to obtain pellets at a constant interval, i.e., every 25–35 s when challenged with FR schedules; on the other hand, the same mice did not display such regulated responding with photo-pulse trains when challenged by the same environmental conditions (**Figure 7B**). Similarly, when challenged with interval schedules, mice working for photo-pulse trains fail to show persisting leverpressing on the order of seconds (**Figure 5A**). Therefore, phasic excitation of DA neurons neither inhibited its approach behavior for consummatory processes nor supplied any long-term motivation.

Apparent lack of long-term motivation for phasic excitation of DA neurons raises interesting issues with respect to its role in addiction. DA has been implicated in various addictive behaviors, most strongly with abused drugs. It seems to be difficult to reconcile the role of DA in addiction with the rapid decline in photo-pulse reinforced response during extinction. Future research may address whether mere phasic excitation is sufficient to recruit addictive processes. Indeed, drugs can activate DA transmission for hours at a time before going back to a baseline level (Di Chiara and Imperato, 1988). It may also be important for environmental stimuli to be paired with phasic excitation of DA neurons, so that conditioned stimuli gain control over behavior (Flagel et al., 2011).

In summary, we found that phasic excitation of DA neurons potentiates the initiation of approach behavior, and this motivational effect quickly diminishes. The study provides direct evidence for the role of phasic excitation of DA neurons in motivation.

## **ACKNOWLEDGMENTS**

The present work was supported by the Intramural Research Program of National Institute on Drug Abuse, National Institutes of Health. We thank the NIDA-IRP Optogenetics and Transgenic Technology Core for providing viral vectors.

#### **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at: http://www*.*frontiersin*.*org/journal/10*.*3389/fnbeh*.*2014*.* 00155/abstract

# **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 06 February 2014; accepted: 16 April 2014; published online: 06 May 2014. Citation: Ilango A, Kesner AJ, Broker CJ, Wang DV and Ikemoto S (2014) Phasic excitation of ventral tegmental dopamine neurons potentiates the initiation of conditioned approach behavior: parametric and reinforcement-schedule analyses. Front. Behav. Neurosci. 8:155. doi: 10.3389/fnbeh.2014.00155*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience. Copyright © 2014 Ilango, Kesner, Broker, Wang and Ikemoto. This is an openaccess article distributed under the terms of the Creative Commons Attribution License*

*(CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# VTA GABA neurons modulate specific learning behaviors through the control of dopamine and cholinergic systems

# **Meaghan C. Creed, Niels R. Ntamati and Kelly R. Tan\***

Department of Basic Neurosciences, University of Geneva, Geneva, Switzerland

#### **Edited by:**

Mary K. Lobo, University of Maryland School of Medicine, USA

#### **Reviewed by:**

Joseph F. Cheer, University of Maryland School of Medicine, USA Ming-Hu Han, Mount Sinai School of Medicine, USA

**\*Correspondence:** Kelly R. Tan, Department of Basic Neurosciences, University of Geneva, 1 rue Michel Servet, CH 1211 Geneva, Switzerland e-mail: kelly.tan@unige.ch

The mesolimbic reward system is primarily comprised of the ventral tegmental area (VTA) and the nucleus accumbens (NAc) as well as their afferent and efferent connections. This circuitry is essential for learning about stimuli associated with motivationally-relevant outcomes. Moreover, addictive drugs affect and remodel this system, which may underlie their addictive properties. In addition to dopamine (DA) neurons, the VTA also contains approximately 30% γ -aminobutyric acid (GABA) neurons. The task of signaling both rewarding and aversive events from the VTA to the NAc has mostly been ascribed to DA neurons and the role of GABA neurons has been largely neglected until recently. GABA neurons provide local inhibition of DA neurons and also long-range inhibition of projection regions, including the NAc. Here we review studies using a combination of in vivo and ex vivo electrophysiology, pharmacogenetic and optogenetic manipulations that have characterized the functional neuroanatomy of inhibitory circuits in the mesolimbic system, and describe how GABA neurons of the VTA regulate reward and aversion-related learning. We also discuss pharmacogenetic manipulation of this system with benzodiazepines (BDZs), a class of addictive drugs, which act directly on GABA<sup>A</sup> receptors located on GABA neurons of the VTA. The results gathered with each of these approaches suggest that VTA GABA neurons bi-directionally modulate activity of local DA neurons, underlying reward or aversion at the behavioral level. Conversely, long-range GABA projections from the VTA to the NAc selectively target cholinergic interneurons (CINs) to pause their firing and temporarily reduce cholinergic tone in the NAc, which modulates associative learning. Further characterization of inhibitory circuit function within and beyond the VTA is needed in order to fully understand the function of the mesolimbic system under normal and pathological conditions.

**Keywords: benzodiazepine, dopamine, acetylcholine, NAc, VTA, interneuron, pharmacogenetics, optogenetics**

## **INTRODUCTION**

Learning about motivationally relevant stimuli in the environment is critical for all aspects of survival, from feeding and reproduction to avoiding dangerous or aversive situations. This learning is primarily mediated by the mesolimbic system. Often referred to as the reward system, the mesolimbic system is primarily comprised of the ventral tegmental area (VTA) and the nucleus accumbens (NAc) as well as their afferent and efferent connections (**Figure 1**). Signaling the salience of an event or stimulus from the VTA in the midbrain to the NAc and other forebrain regions has largely been ascribed to dopamine (DA) neurons (Berridge and Robinson, 1998; Smith et al., 2011). Over the past several decades, extensive studies at the cellular and behavioral levels have shown that populations of DA neurons in the VTA increase their behavior in response to rewarding stimuli (Schultz et al., 1993). In response to aversive stimuli, the activity of most DA neurons is silenced (Mileykovskiy and Morales, 2011; Cohen et al., 2012), although a subset of DA neurons show activation (Joshua et al., 2008). Over training with reward experience, this DA activation will occur not only in response to the salient primary stimuli *per se*, but will also occur in response to previously neutral cues in the environment that the animal has associated with these motivationally relevant outcomes (Schultz, 2013). As a consequence of DA activity, DA in the NAc induces a motivational drive, and this DA signal is modulated by past experience of reward and punishment (Oleson et al., 2012; Howe et al., 2013). In this way, mesolimbic DA has been conceptualized as a teaching signal, coding the magnitude of aversive and rewarding environmental stimuli and increasing behavioral vigor related to these salient stimuli (Peciña and Berridge, 2013). The rewarding effect of addictive drugs is also mediated by mesolimbic DA. Following exposure to drugs of abuse, DA is elevated throughout the mesolimbic system (Lüscher and Ungless, 2006). This persistent increase in DA results in pathological salience attributed to drug-associated cues, and compulsive drug use despite negative consequences. In this way, drug addiction is an example of adverse behavioral consequences arising from a malfunction of the mesolimbic system.

As previously discussed, the role of DA in the mesolimbic system has been the subject of intense investigation over the

**FIGURE 1 | Inhibitory and modulatory connections between the VTA and the NAc.** The VTA is composed of two major cell types: GABAergic neurons (green) and DA neurons (grey). Glutamate neurons are not schematized for clarity. GABA interneurons control the activity of DA cells that release DA in the NAc through their long-range projections. In turn, MSNs send projections back to the VTA, preferentially inhibiting GABA cells and thus disinhibiting DA neurons. Some GABA neurons also send long-range projections to the NAc, where they selectively inhibit CINs. Abbreviations: VTA: ventral tegmental area; NAc: nucleus accumbens; GABA: γ -aminobutyric acid; DA: dopamine; CIN: cholinergic interneuron; MSN: medium sized-spiny neuron.

past several decades. However, in addition to DA, the VTA also contains approximately 30% γ -aminobutyric acid (GABA) neurons, and the role of these inhibitory neurons in the mesolimbic system is less well understood. Within the VTA, there are two general populations of GABA neurons: interneurons, which provide local inhibition of DA neurons and projection neurons, which provide long-range inhibition of multiple brain areas including the NAc (**Figure 1**). In general, inhibition is critical for regulating neuronal excitability, and allows flexibility in circuit connectivity. A consequence of this flexibility is plasticity within the mesolimbic system, which permits reward-related learning. Despite their likely functional consequences, the role of VTA GABA neurons in reward- and aversion-related learning is not well understood. Until recently, it has been difficult to disentangle the function of DA neurons and GABA neurons, and between local VTA GABA neurons and GABAergic projection neurons (GPNs).

In this review we discuss studies using optogenetic and pharmacogenetic tools to dissect the precise role of GABAergic neurons in the mesolimbic system. Much of this recent work has characterized the functional neuroanatomy of inhibitory circuits in the mesolimbic system, and has begun to elucidate the precise mechanisms by which these inhibitory circuits regulate activity of DA neurons and other neuromodulatory systems, such as acetylcholine (ACh). It is now evident that these inhibitory circuits critically regulate DA neuron function and reward-related learning. VTA GABA neurons bi-directionally modulate activity of local DA neurons, which underlies reward or aversion at the behavioral level. Conversely, long-range GABA projections from the VTA to the NAc selectively target cholinergic interneurons (CINs; **Figure 1**), regulating local ACh release and modulating associative learning. We also use the example of drug addiction to discuss how inhibitory neurons contribute to dysfunction of the mesolimbic system and consequent maladaptive behaviors. Continuing to clarify the role of inhibitory neurons within the VTA and beyond will be necessary to fully understand the function of the mesolimbic system under normal and pathological conditions.

# **ACTIVATION OF LOCAL GABA NEURONS IN THE VENTRAL TEGMENTAL AREA (VTA)**

As mentioned, the VTA contains a large proportion of GABA neurons (∼30%; Dobi et al., 2010). We now appreciate the intimate role these neurons play in regulating activity of DA neurons, and the behavioral consequences of their activity. GABA released from local VTA neurons profoundly affects the activity of VTA DA neurons. Using Cre mice (GAD-cre (Tan et al., 2012) or VGAT-cre (van Zessen et al., 2012) to express channelrhodopsin 2 (ChR2) selectively in GABAergic neurons), it has been possible to selectively manipulate VTA GABA neurons *in vitro* and *in vivo*, and determine the effects on DA neurons as well as the consequences of this activation on behavior (**Figure 2**). Using *in vivo* extracellular recording in VTA ChR2 injected and anesthetized GAD-cre mice to monitor DA neuron activity, we have shown that driving activity of VTA GABA neurons strongly inhibits DA neuron spontaneous firing rate (Tan et al., 2012; **Figure 2A**). In stark contrast, shutting down the activity of GABA neurons, by expressing the proton pump halorhodpsin in VTA GABA neurons, leads to an increase or disinhibition of DA cells (Bocklisch et al., 2013). These results were confirmed with *ex vivo* patch clamp electrophysiology, showing that local VTA GABA neurons make direct synaptic connections with DA neurons. Blue-light activation of ChR2-expressing GABA neurons induced inhibitory post-synaptic responses in neighboring DA neurons. These lightevoked currents were abolished by the sodium channel blocker tetrodotoxin or by the chloride channel blocker picrotoxin, confirming a monosynaptic connection mediated by GABA<sup>A</sup> receptors (Tan et al., 2012; van Zessen et al., 2012). Through activation of GABA<sup>A</sup> receptors, local GABA cells control their target DA neurons, decreasing their excitability, thereby balancing excitation from glutamatergic inputs (Overton and Clark, 1992).

This inhibitory control has important consequences for DA function and downstream behavior, particularly regarding significance to motivationally relevant outcomes (Schultz et al., 1997; Schultz, 2006; Fields et al., 2007; Bromberg-Martin et al., 2010). Specifically, exposure to salient but aversive stimuli can lead to the inhibition of VTA DA neurons (Mirenowicz and Schultz, 1996; Ungless et al., 2004, 2010; Brischoux et al., 2009; Matsumoto and Hikosaka, 2009; Hong et al., 2011; Zweifel et al., 2011; Tan et al., 2012; van Zessen et al., 2012). Recordings in anesthetized animals have demonstrated that aversive stimuli (such as a footshock) transiently and potently increase the spontaneous firing rate of GABAergic neurons of the VTA and decrease the activity of DA neurons (Ungless et al., 2004; Hong et al., 2011; Tan et al.,

**FIGURE 2 | VTA GABA neurons drive specific behaviors through the control of the DA and cholinergic systems. (A)** Left, VTA GABA neurons are selectively infected to express ChR2-eYFP (green, AAV-ChR2-eYFP-flox in the VTA of GADcre+ mice). Middle, in vivo example trace of single unit recording of a GABA and a DA cell showing that blue-light stimulation induces a time locked increase of firing rate in infected GABA neurons and as a consequence a shut down in the activity of DA cells. Right, dynamic conditioned place aversion (CPA): GADcre+ mice developed an aversion for the blue-light paired chamber compared to GADcre− mice. This is shown on the example tracking trace of a GADcre+ mouse and on the aversive learning curve plotting the relative time spent in the conditioned chamber for both groups of mice. **(B)** Left, the ventral tegmental area (VTA) contains DA neurons expressing α3-containing GABA<sup>A</sup> receptors and GABA cells that express α1-containing GABA<sup>A</sup> receptors. Middle, in vivo example trace of single unit recording of GABA and DA cells before and after midazolam (MDZ) tail vein injection. GABA neurons decrease their basal firing rate, leading to the disinhibiton of DA cells, which increase their firing rate. These effects are abolished in α1-knock-in mice in which GABA<sup>A</sup> receptors containing the α1

subunit isoform are insensitive to BDZs. Right, oral-self administration of MDZ paradigm: WT mice preferred to consume from the MDZ containing solution whereas this preference is not developed in α1-knock-in mice. **(C)** Left, same preparation as in **B**, VTA GABA projection neurons selectively inhibit CIN of the NAc. Middle, in vivo example trace of single unit recording of GABA and CIN cells showing that in response to blue-light stimulation CIN are completely inhibited due to the excitation of VTA GABA cells. Right, activation of these fibers during the association of a conditioned stimulus (CS+, white noise) to an aversive outcome (brief mild footshock) enhances the ability to subsequently discriminate between the conditioned stimulus and the unconditioned stimulus (CS, pure tone) that was not paired with an aversive outcome, as shown on the graph for the percentage of freezing measured on the test day in response to CS+ and CS− for both groups of mice. Abbreviations: VTA: ventral tegmental area; NAc: nucleus accumbens; WT: wild type mouse; α1KI: α1 subunit point mutant knock-in mouse; BDZ: benzodiazepine; GABA: γ-aminobutyric acid; DA: dopamine; CIN: cholinergic interneuron; MSN: medium sized-spiny neuron. Adapted from Tan et al. (2010, 2012) and Brown et al. (2012).

2012). Likewise, *in vivo* recording in awake behaving animals determined that aversive stimuli lead to excitation of the majority of VTA GABA neurons, which were optogenetically identified (Cohen et al., 2012). Conversely, mixed responses were observed in DA neurons, with the predominant response being inhibition (Brischoux et al., 2009; Cohen et al., 2012; Tan et al., 2012). The increase in GABA neuron spontaneous firing rate and decrease in DA neuron activity following the aversive stimuli of a foot-shock were prevented by a GABA<sup>A</sup> receptor antagonist, but were not affected by DA antagonists (Tan et al., 2012), indicating that the effects on DA firing rate were downstream of the increased activity of GABA neurons.

Confirming the regulatory role of VTA GABA neurons on motivationally relevant behaviors, driving local VTA GABA neurons with ChR2 results in strong aversion to a compartment associated with light stimulation (Tan et al., 2012). In this dynamic conditioned place aversion (CPA) paradigm (also named real time CPA task), the VTA of ChR2-infected GAD-cre mice was stimulated with blue light when the animal was exploring one of the two compartments, and the light was turned off as soon as the mice exited this chamber. On the test day, where no blue light is applied to the VTA, an aversion of the light paired-chamber was measured (**Figure 2A**). Interestingly, the CPA produced in response to activating VTA GABA neurons persisted during test sessions, indicating a strong learning effect resulting from DA neuron inhibition. Furthermore, driving VTA GABA neurons also disrupted reward consumption, as measured in a free-sucrose access paradigm (van Zessen et al., 2012). In another study, *in vivo* recordings of the activity of VTA cells were done while mice learned to associate an odor with appetitive rewarding or aversive outcomes. GABA neurons increased their firing during the delay between a conditioned stimulus (odor) and an unconditioned stimulus (reward or aversive air puff), while the firing rate was not changed during the delivery of the reward or punishment. The authors speculated that this ramping up of GABAergic firing may modulate DA activity during reward expectation and may help compute the reward prediction error (Cohen et al., 2012). Such cellular mechanism of action has been correlated with subsecond DA release measurements in the NAc of rat undergoing fearconditioning tasks. These studies showed that a warning signal presentation (Oleson et al., 2012; Olesson and Cheer, 2013) or a fear conditioning cue (Badrinarayan et al., 2012) result in decreased DA levels as measured by fast scan cyclic voltametry.

Taken together, these observations implicate VTA GABA neuron activity in learning motivationally relevant outcomes associated with unconditioned stimuli. Specifically, driving VTA GABA neurons leads to negative modulation of DA neurons and aversion at the behavioral level. This learning signal may have implications for the involvement of VTA GABA neurons in maladaptive learned behaviors, such as those that characterized in drug addiction.

## **INHIBITION OF LOCAL GABA NEURONS IN THE VENTRAL TEGMENTAL AREA (VTA)**

VTA GABA neurons receive inhibitory inputs mainly from medium sized-spiny neuron (MSN) of the NAc (Xia et al., 2011). Activation of MSNs with ChR2, for instance, leads to the inhibition of VTA GABA cells, and consequent disinhibition of DA neurons (Bocklisch et al., 2013). This mechanism of cellular disinhibition is implicated in the rewarding effects of several classes of addictive drugs (Lüscher and Ungless, 2006). Specifically, opioids (Johnson and North, 1992a), gamma-hydroxy butyric acid (GHB) (Cruz et al., 2004), cannabinoids (Oleson and Cheer, 2012) and benzodiazepines (BDZs; Tan et al., 2010, 2011) act on VTA GABA neurons (Johnson and North, 1992b) and inhibit their activity. We and others have examined how decreasing the activity of VTA GABA neurons, contributes to the synaptic and behavioral adaptations induced by addictive drugs.

BDZs are a class of drug used in clinic to treat anxiety, insomnia and muscle spasms. They also induce sedation and anterograde amnesia and unfortunately, they carry the potential for addiction. All these effects are mediated through BDZs action as positive allosteric modulators of GABA<sup>A</sup> receptors; they potentiate inhibition in the brain. *In vivo* administration of BDZs inhibits the activity of VTA GABA neurons and consequently increases DA neuron's firing rate (**Figure 2B**). Along with BDZs, opioids are another class of addictive drugs that act initially on GABA neurons through binding to µ-opioid receptors specifically expressed on GABA cells (Pickel et al., 2002), decreasing their activity thus leading to the disinhibition of VTA DA neurons (Johnson and North, 1992a).

All addictive drugs, such as cocaine, amphetamine, morphine, GHB, nicotine and ethanol induce characteristic forms of synaptic plasticity on VTA DA neurons within 24 h following an acute exposure (Lüscher and Malenka, 2011). Specifically, an increase in excitatory synaptic strength (as measured by the ratio of the amplitude of AMPA to NMDA-mediated currents (Ungless et al., 2001)) is observed, as well as inward rectification of the AMPA current at positive potentials (Bellone and Lüscher, 2006), which results from the insertion of calcium-permeable AMPA receptors. Interestingly, activation of DA neurons is sufficient to induce this plasticity, since optogenetic activation of these neurons alone (using DA transporter-cre mice) induces the same form of plasticity (Brown et al., 2010). BDZs also trigger this critical feature (Heikkinen et al., 2009; Tan et al., 2010) indicating that they share a common mechanism of DA regulation with other classes of addictive drugs.

Using a pharmacogenetic approach, we demonstrated that the effect of BDZs and more specifically midazolam (MDZ, a BDZ potentiating all GABA<sup>A</sup> receptors) on neuronal activity and plasticity are mediated by α1-containing GABA<sup>A</sup> receptors. In the α1(H101R) point mutant mouse line, which carries a single nucleotide mutation that disrupts binding of BDZs to the GABAAR binding site (Rudolph et al., 1999), all effects of MDZ are abolished (Tan et al., 2010). Moreover, unlike wildtype mice, these α1(H101R) mice did not self-administer MDZ in an oral-self administration task where they had the option to drink from two bottles, differing only in that one contains a MDZ (**Figure 2B**).

Inhibition of VTA GABA cells results in the release of the inhibitory break onto DA neurons. Such disinhibition represents a means to trigger the release of DA in target brain regions to modulate synaptic transmission and hence regulate specific behaviors such as the oral-self administration of BDZs, a rewardrelated behavior.

# **LONG-RANGE GABAERGIC PROJECTIONS FROM THE VENTRAL TEGMENTAL AREA (VTA)**

VTA DA neurons are long-range projection neurons, which target the prefrontal cortex and the NAc, among other areas. These same areas are also innervated by a subset of VTA GPNs; these neurons comprise approximately 25% of VTA GABA cells (Margolis et al., 2006; **Figure 1**). In the NAc, GPNs target CINs nearly exclusively, with very sparse innervation of MSNs (Brown et al., 2012). Conversely, VTA DA neurons modulate both MSNs and CINs, either through volume transmission or direct synaptic contacts (Moss and Bolam, 2008).

Using *ex vivo* whole cell recording methods in GAD-cre mice expressing ChR2 in VTA GABA cells, we showed that the connection between VTA GPNs and CINs in the NAc is monosynaptic. Light-evoked currents were recorded in 100% of CINs and blocked by picrotoxin or tetrodotoxin, whereas such currents were rarely detected in MSNs or parvalbumin-expressing interneurons (Brown et al., 2012). *In vivo* recordings confirmed that activating VTA GABA neurons at physiological frequencies potently inhibited CINs for the duration of the stimulation (**Figure 2C**). This anatomical and functional connectivity has important consequences for associative learning. In a fear conditioning task, when GPNs were optogenetically activated during presentation of a specific tone paired with a foot-shock, mice showed increased ability to discriminate this conditioned tone from an unconditioned tone (Brown et al., 2012; **Figure 2C**).

During Pavlovian conditioning, VTA DA neurons increase their firing in response to a stimulus predicting either reward or aversion resulting in increased DA release in the NAc. In parallel, the inhibition of CINs produced by GPNs would modulate information processing by MSNs either directly through the actions of muscarinic receptors (Surmeier et al., 2007) or by increasing DA release from VTA terminal fields in the NAc (Cragg and Rice, 2004). Moreover, CINs express DA D2 receptors and are subject to DA regulation (Alcantara et al., 2003) and tonic activity of CINs elicits DA release (Cachope et al., 2012). In this way, through their actions on MSNs and CINs, VTA GABA and DA neurons interact to coordinate activity and reward processing of the NAc. Interestingly, however, inhibiting GPNs terminating in the NAc was insufficient to disrupt reward consumption. These observations support a dual mechanism of VTA to NAc communication, in which targeting of NAc CINs by VTA GPNs and of MSNs and CINs by DA neurons operates in concert to enable the association of motivationally relevant outcomes to unconditioned stimuli.

## **CONCLUDING REMARKS**

Within the mesolimbic reward system, communication of rewarding and aversive stimuli has largely been ascribed to DA neurons. The firing rate of DA neurons responds to salient stimuli that are both rewarding and aversive in valence, and is involved in computing reward prediction error. These DA neurons project to regions throughout the brain, including the NAc, which is implicated in behavioral responses to rewarding and aversive stimuli. As such, the mesolimbic DA system is critically involved in learning about motivationally-relevant outcomes. However, in addition to DA cells, the VTA contains GABA neurons, which synapse locally onto DA neurons, and send long-range projections to brain areas innervated by VTA DA neurons. Until recently, the role of VTA GABA neurons on the regulation of DA neurons and on reward-learning was not well understood. However, with the advent of selective optogenetic and pharmacogenetic tools, the role of these inhibitory neurons within the mesolimbic reward system has begun to be elucidated.

We have discussed how within the VTA, aversive stimuli drive the activity of GABA neurons, leading to inhibition of DA neuron firing. Conversely, inhibiting the activity of these VTA GABA neurons (for example, by administration of BDZs) increases the activity of DA neurons and is behaviorally rewarding. The contribution of long-range GABA projection neurons to mesolimbic system function, and the interaction of GPNs with DA projection neurons at the level of their projection areas have also begun to be investigated. For example, despite the established role of the NAc in the maintenance of reward-related behavior (From Introduction), optical inhibition of GPN afferents in the NAc had no effect on reward consumption (van Zessen et al., 2012), and produces no aversive state *per se* (Brown et al., 2012). By confirming that disruption of GABAergic signaling from VTA GPNs to the NAc is insufficient to disrupt reward or produce aversion, these observations further underscore the critical role of DA projections in reward-related learning and coding the valence of environmental stimuli. However, driving activity of GPNs from the VTA to NAc enhances associative learning despite this lack of effect on consumatory behavior or aversion. These observations lead us to the question of the heterogeneity of VTA GABA cells. Within the NAc, antidromic optical activation of VTA GABA neurons inhibited approximately 25% of VTA DA neurons (Brown et al., 2012), suggesting that only a subset of VTA GABA neurons project to the NAc and some others have local collaterals. In addition the population of GABA interneurons that do not project outside the VTA, it is likely that a distinct set of GABA neurons project to other brain regions involved in reward processing and behavior, such as the PFC and hippocampus.

To fully appreciate the role of inhibition in the mesolimbic system, a better understanding of upstream inputs onto VTA GABA neurons, and the role of GABA neurons originating in other nodes of the mesolimbic system are needed. Optogenetic manipulations have recently been used to explore both of these issues. Using these tools to map functional connectivity, it was revealed that both bed-nucleus of the stria terminalis (BNST) glutamatergic and GABAergic projections preferentially innervate putative GABA neurons in the VTA (Jennings et al., 2013). *In vivo* photostimulation of BNST glutamatergic projections resulted in aversive and anxiogenic behavioral phenotypes. Conversely, activation of BNST GABAergic projections produced rewarding and anxiolytic phenotypes, similarly to direct activation of VTA GABA cells (Jennings et al., 2013). Another source of input onto VTA GABA cells is through back-projections from the NAc. Dopamine receptor type 1 (D1R)-receptor-expressing MSNs, but not Dopamine receptor type 2 (D2R)-expressing MSNs of the NAc form functional synaptic contacts with GABA neurons in the VTA (Xia et al., 2011; Bocklisch et al., 2013). This connection is potentiated following exposure to cocaine (Bocklisch et al., 2013), and inhibitory transmission from VTA back to the NAc is suppressed (Ishikawa et al., 2013). These studies provide crucial information about upstream inputs onto VTA GABA neurons and may provide further insight into how DA release is controlled under normal and pathological conditions.

DA modulation is important for function of reward, aversion and reward related learning. While much work has been devoted to studying the function of DA neurons and their role in encoding salient information of rewarding or aversive in nature, the role of VTA GABA neurons has been less well characterized. However, as we have discussed, it is now understood that GABA neurons regulate the activity of both VTA DA neurons and target neurons in VTA projection areas. This intimate, bi-directional modulation of DA neurons not only contributes to reward prediction error and DA plasticity, but also encodes the valence state of rewarding and aversive stimuli and also helps orchestrate behavioral responses to salient stimuli. Further characterization of inhibitory circuit function within and beyond the VTA will be necessary to fully understand the function of the mesolimbic system under normal and pathological conditions.

## **REFERENCES**


reinforcement. *Annu. Rev. Neurosci.* 30, 289–316. doi: 10.1146/annurev.neuro. 30.051606.094341


Creed et al. Inhibitory reward circuitry

sites envisioned in Cajal's neuron. *Prog. Brain Res.* 136, 145–155. doi: 10. 1016/s0079-6123(02)36014-x


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 29 November 2013; accepted: 06 January 2014; published online: 22 January 2014.*

*Citation: Creed MC, Ntamati NR and Tan KR (2014) VTA GABA neurons modulate specific learning behaviors through the control of dopamine and cholinergic systems. Front. Behav. Neurosci. 8:8. doi: 10.3389/fnbeh.2014.00008*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2014 Creed, Ntamati and Tan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Local control of striatal dopamine release

# **Roger Cachope1,2 and Joseph F. Cheer 1,3\***

<sup>1</sup> Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, MD, USA

<sup>2</sup> CHDI Foundation, Los Angeles, CA, USA

<sup>3</sup> Department of Psychiatry, University of Maryland School of Medicine, Baltimore, MD, USA

#### **Edited by:**

Anton Ilango, National Institutes of Health (NIH), USA

#### **Reviewed by:**

Kate M. Wassum, University of California Los Angeles, USA Stephan Lammel, Stanford University, USA

#### **\*Correspondence:**

Joseph F. Cheer, Department of Anatomy and Neurobiology, University of Maryland School of Medicine, 20 Penn Street, HSF I, Room 280J, Baltimore, MD 21201, USA

e-mail: jchee001@umaryland.edu

The mesolimbic and nigrostriatal dopamine (DA) systems play a key role in the physiology of reward seeking, motivation and motor control. Importantly, they are also involved in the pathophysiology of Parkinson's and Huntington's disease, schizophrenia and addiction. Control of DA release in the striatum is tightly linked to firing of DA neurons in the ventral tegmental area (VTA) and the substantia nigra (SN). However, local influences in the striatum affect release by exerting their action directly on axon terminals. For example, endogenous glutamatergic and cholinergic activity is sufficient to trigger striatal DA release independently of cell body firing. Recent developments involving genetic manipulation, pharmacological selectivity or selective stimulation have allowed for better characterization of these phenomena. Such termino-terminal forms of control of DA release transform considerably our understanding of the mesolimbic and nigrostriatal systems, and have strong implications as potential mechanisms to modify impaired control of DA release in the diseased brain. Here, we review these and related mechanisms and their implications in the physiology of ascending DA systems.

**Keywords: dopamine, acetylcholine, glutamate, striatum, optogenetics, axonal release, volume transmission**

# **INTRODUCTION: ROLE OF DA IN MOTOR AND LIMBIC FUNCTION**

Dopamine (DA) plays a critical role in the organization of reward-seeking behavior and motor responses (Joshua et al., 2009; Schultz, 2013). Through the mesolimbic and nigrostriatal DA systems, the forebrain receives dopaminergic input that modulates a range of functionally distinct structures, such as the basal ganglia and cerebral cortex (Björklund and Dunnett, 2007; Tritsch and Sabatini, 2012). The mesolimbic system is formed by dopaminergic neurons located in the VTA and their projections to the nucleus accumbens (NAc), cortex, amygdala and hippocampus, which participate in the configuration of reward-seeking behaviors (Björklund and Dunnett, 2007; Stuber et al., 2012; Nieh et al., 2013). The nigrostriatal system has its origin in the substantia nigra pars compacta (SNc) and projects preferentially to the dorsolateral domains of the striatum, having a more defined role in the organization of motor plans (Groenewegen, 2003; DeLong and Wichmann, 2007). Such functional distinction at the level of the striatum seems to have structural and molecular correlates on DA neurons from the SNc (Henny et al., 2012; Schiemann et al., 2012). Additional to these functional implications, dopaminergic transmission is compromised in a variety of neurological conditions such as schizophrenia, Huntington's and Parkinson's disease, drug addiction and obsessive-compulsive disorder, among others (DeLong and Wichmann, 2007; Money and Stanwood, 2013).

The striatum is the main input nucleus of the basal ganglia, and DA modulates how this input is processed (Calabresi et al., 1997; Centonze et al., 2001; Tritsch and Sabatini, 2012). However, in contrast to the traditional view of inter-neuronal chemical excitatory synaptic transmission in which structural and functional specializations are observed at the postsynaptic domains, striatal dopaminergic transmission does not always require such level of postsynaptic structural specialization (Rice and Cragg, 2008; Fuxe et al., 2012). Instead, release occurs in a diffuse manner, DA receptors are extrasynaptic and ultrastructural studies on the extension and density of DA neuron axonal arborization in the striatum point to broad, intricate projections that cover vast areas (Pickel et al., 1981; Smith et al., 1994; Moss and Bolam, 2008; Matsuda et al., 2009). This diffusely spread mode of transmission (in contrast to localized, highly spatially restricted communication), is termed "volume transmission", and is a feature of a number of transmitters such as acetylcholine, norepinephrine, DA and serotonin (Taber and Hurley, 2014). Volume transmission of DA is, however, not exclusive to the striatum and it has its own particularities through different areas (Rice and Cragg, 2008; Fuxe et al., 2012; Martin and Spühler, 2013; Taber and Hurley, 2014). DA as a volume transmitter in the striatum is thought to exert a widespread modulatory influence on excitatory—glutamatergic—transmission arriving from the cortex, basolateral amygdala (BLA), and ventral hippocampus (vHipp; Britt et al., 2012); or on inhibitory—GABAergic transmission incoming from areas such as VTA (Van Bockstaele and Pickel, 1995) and ventral pallidum (Churchill and Kalivas, 1994).

DA modulation of incoming transmission to the striatum plays a key role in the functional expression of reward-seeking behaviors and motor control. Such functions exhibit some stratification within the striatum (Threlfell and Cragg, 2011). For example, dorso-medial and dorso-lateral areas are predominantly involved in motor control, while ventro-medial segments are mostly involved in the expression of reward processing, motivation and salience (Groenewegen, 2003; Voorn et al., 2004; Kreitzer and Berke, 2011; Stuber et al., 2012). Concurrently, cortico-striatal projections also exhibit a stratified distribution in which the motor and cingulate cortices form the primary input to the dorsolateral striatum, while prefrontal and prelimbic cortices project mainly to ventro-medial areas of the striatum (Voorn et al., 2004). Phenomena responsible for regulation of striatal DA release can be VTA/SNc driven, or locally acting, at the striatal level. This latter possibility has long been reported, still attracts considerable attention in terms of mechanistic characterization (Cachope et al., 2012; Threlfell et al., 2012) and is considered as an opportunity for functionally-segregated intervention (Threlfell and Cragg, 2011).

# **MULTIPLICITY OF MECHANISMS IN THE CONTROL OF DOPAMINE RELEASE**

Through what are now seminal papers, Wolfram Schultz et al. demonstrated that firing of DA neurons in the midbrain increases in response to rewarding stimuli in non-human primates (Schultz et al., 1997; Schultz, 1998), while functional imaging studies in humans point to a similar increase in cellular activity (D'Ardenne et al., 2008), suggesting correspondence with Schultz's group reports. Interestingly, it was recently described that VTA GABAergic neurons also encode reward expectation (Cohen et al., 2012). Recordings of DA neurons from the VTA or SNc areas in rodents exhibit slow, tonic firing rates that periodically switch to a high frequency events (Grace and Bunney, 1984a,b). Thus, low levels of DA release have been correlated with low frequency firing rate of DA neurons, while corresponding enhancement in striatal DA release occurs in response to high frequency firing rates (Kawagoe et al., 1992). These findings have sculpted the traditional view of striatal DA release being determined by the rate of neuronal firing of the DA neuron somatas located in either VTA or SNc. However, besides this dominant mechanism of control of DA release, local factors such as reuptake, autoreceptor-dependent modulation, and termino-terminal control exist and are recognized to play a prominent role, independently of VTA/SNc firing rate.

DA neurons projecting to the striatum establish prominent axonal trees at their destination. The volume transmission feature of striatal DA implies that a considerable amount of control is required in terms of uptake and/or negative feedback on future release events. In reaching this goal, two key mechanisms are DA transporter activity (DAT) and D2-like presynaptic autoreceptor activity. DAT activity is thought to limit the radius of DA activity (Rice and Cragg, 2008) and, by doing so, restricts activation of DA receptors (reviewed in Rice et al., 2011). In a similar manner, it is known that blockade of D2-like DA receptors in slices leads to increased DA release in response to repetitive electrical stimulation (Limberger et al., 1991; Patel et al., 1992). This effect, however, is not manifest when single pulse stimulation is used (Limberger et al., 1991; Patel et al., 1992), suggesting that there is not sufficient DA tone elicited by a single pulse to be displaced by the antagonist. Importantly, changes in D2 receptor levels and their subsequent activation are thought to play a prominent role in several neurological conditions in which DA levels are altered (Ford, 2014).

# **LOCAL STRIATAL CONTROL OF DOPAMINE RELEASE GLUTAMATERGIC TRANSMISSION**

Excitatory glutamatergic activity in the striatum originates mainly from frontal cortex, midline and intralaminar thalamus, basal amygdala, and hippocampus (reviewed in Sesack and Grace, 2010; Stuber et al., 2012). Additionally, DA terminals release glutamate (Sulzer et al., 1998; Joyce and Rayport, 2000; Sulzer and Rayport, 2000; Chuhma et al., 2004; Dal Bo et al., 2004; Chuhma et al., 2009; Hnasko et al., 2010), and this has recently been demonstrated by way of selective optogenetic stimulation of DA terminals (Stuber et al., 2010). However, this last report demonstrates that such possibility exists only in DA terminals that reach the NAc, not the dorsal striatum. Still, some debate prevails as to this feature not being present in the adult brain (Bérubé-Carrière et al., 2009; Moss et al., 2011), or being as widespread as initially thought (Stuber et al., 2010; for a review, see Broussard, 2012).

Evidence on the potential role of glutamate as a form of local control of DA release in the striatum has long been reported (Imperato et al., 1990; Cheramy et al., 1991; Krebs et al., 1991; Desce et al., 1992) and both ionotropic and metabotropic glutamate receptors (iGluR; mGluR, respectively) have been implicated. However, most of the initial studies were performed *in vivo* using brain microdialysis as the measuring technique to assess DA levels as well as for local administration of glutamate receptor ligands. Such findings were of course influenced by slow temporal resolution and the effects of the ligand in a complex circuit, among other factors, making a mechanistic interpretation difficult. *In vitro* experimental designs, on the other hand, allowed for more direct mechanistic description while still not directly addressing whether results were equivalent to intact-tissue conditions. These distinct experimental conditions might account for what, at the time, were apparent contradictory results. Initial *in vitro* explorations in slices and synaptosomes accounted not only for glutamate, but for a range of neurotransmitters that could affect striatal DA release locally, including acetylcholine, GABA, glycine and opiates (reviewed in Chesselet, 1984). However, further *in vivo* experiments in freely moving rats were still non conclusive; i.e., activation of AMPA receptors by exogenous ligands led to a decrease in DA release, while an increase was evident only in response to the application of NMDA receptor ligands at high concentrations (Imperato et al., 1990). Blocking uptake of endogenous release, in turn, elevated DA release in a way that was sensitive to the application of either NMDA or AMPA antagonists, suggesting the involvement of both receptor types in that response (Segovia et al., 1997). Similarly, electrical stimulation of the prefrontal cortex, a putative glutamatergic input to striatum, as well as local application of kainate or NMDA increased DA release (Cheramy et al., 1991; Krebs et al., 1991). Development of electrochemical techniques, however, greatly contributed to the clarification of these mechanisms. The use of fast-scan cyclic voltammetry (FSCV) for the detection of DA *in vitro* allowed for better temporal resolution which was less influenced by circuit adaptive responses in the mid-term scale (minutes), which could potentially influence DA readout. Under those conditions, bath application of kainate, AMPA or NMDA elicited inhibition of DA release (Wu et al., 2000; Kulagina et al., 2001; Avshalumov et al., 2003). Moreover, electron microscopy studies were not able to demonstrate labeling of iGluRs in striatal DA terminals (Bernard et al., 1997; Bernard and Bolam, 1998). The lack of expression of iGluRs on DA terminals suggests that iGluRmediated modulation of DA release relates to a more complex process; which may underlie interactions between multiple cellular types and/or chemical mediators. This issue, raised and investigated by Rice's group led to the identification of H2O<sup>2</sup> as a key molecule in the iGluR-mediated decrease of DA release (Avshalumov et al., 2000, 2003, 2008; Avshalumov and Rice, 2003). This model describes how glutamatergic activity on ionotropic receptors in medium spiny neurons (MSNs) triggers production and release of H2O2, which in turn diffuses to adjacent DA terminals and promotes opening of KATP channels leading to reduction of DA release (Avshalumov et al., 2008).

In contrast to iGluRs, labeling of mGluRs has been reported in presynaptic profiles identified as DA axons (Paquet and Smith, 2003). Moreover, blocking glutamate uptake, or high-frequency stimulation of the cortico-striatal pathway modulates DA release, in a mGluR-dependent fashion followed by modulation of Ca++ activated potassium channels (Zhang and Sulzer, 2003). Altogether, the existent evidence points to mGluR-mediated direct action on DA terminals, and a second MSN-mediated mechanism involving iGluR-H2O<sup>2</sup> signaling.

## **CHOLINERGIC TRANSMISSION**

In contrast to striatal glutamatergic activity, which originates mainly from inputs to the striatum, sources of striatal acetylcholine release are mostly from cholinergic interneurons (CINs) that account for about 2–5% of all striatal neurons (Descarries et al., 1997; Descarries and Mechawar, 2000). Additional to CINs, a recent report shows that brainstem-based cholinergic neurons send terminals to the striatum in a topographic fashion with their origin (Dautan et al., 2014). In spite of their low numbers, CINs establish prominent and intricate axonal projections that configure an extensive planar neurotransmission system (Descarries et al., 1997; Descarries and Mechawar, 2000). Electrophysiological characterization shows that CINs are tonically active neurons that fire at a relatively low rate of about 5–10 Hz (Wilson et al., 1990; Aosaki et al., 1995). This rate, however, as in the case of DA neurons, encodes behaviorally relevant reward-related events (Apicella et al., 1991, 2011; Aosaki et al., 1994, 1995; Shimo and Hikosaka, 2001; Morris et al., 2004).

Target receptors of cholinergic activity in the striatum are both of nicotinic and muscarinic types (nAChR and mAChR, respectively). While mAChRs are seven trans-membrane domain G-protein coupled receptors, nAChRs consist of five subunits arranged as homomers or heteromers that, in mammals, are formed by subfamilies II (α7) and III (α2-6, β2-4) (Le Novère et al., 2002). Particularly, striatal DA axons express a high density of α4, α5, α6, β2 and β3 subunits in an arrangement of two αβ pairs that could be α4-β2 and/or α6-β2 and/or α4-β4, plus a fifth subunit that can be α5 or β3 (Champtiaux et al., 2003; reviewed in Threlfell and Cragg, 2011). Additionally, the β2 subunit is expressed on striatal DA axons (Jones et al., 2001) and is included in all nAChRs at these terminals. This characterization is functionally relevant because some segregation exists in which predominance of different α subunits occurs between dorsolateral striatum and the NAc. More specifically, a significant amount of work has shown that α4(non-α6)-nAChRs play a prominent role in dorsal striatum, while α4α6-nAChRs are dominant in NAc (Exley et al., 2008, 2011, 2012). Given the distinct functional role of the dorsolateral and the ventromedial striatum, it has been proposed that such differences could be taken into account as a substrate for region-specific intervention (Threlfell and Cragg, 2011).

mAChRs, in turn, are classified in two groups according to their coupling to either G<sup>s</sup> (M1, M3, M5) or G<sup>i</sup> (M2, M4) subunits of G proteins, with M<sup>2</sup> and M<sup>4</sup> predominantly expressed in CINs (Yan and Surmeier, 1996). In a similar way to what has been described for nAChRs, mAChRs exhibit some dorso-ventral gradient in their ability to regulate DA release. While M2/M<sup>4</sup> receptors are necessary for such regulation in the dorsal striatum, M4 is prevalent in the NAc (Threlfell et al., 2010). Additionally, expression of M<sup>5</sup> receptors has been reported in nigrostriatal DA neurons, although their pattern of expression on striatal DA terminals and subsequent potential role in local control of DA release remains unclear (reviewed in Threlfell and Cragg, 2011; Zhang and Sulzer, 2012).

Involvement of presynaptic cholinergic receptors on DA regulation was inferred early, mainly from experiments describing increase of DA release in response to AChR activation in slices or synaptosomes (Giorguieff et al., 1976, 1977; Wonnacott et al., 1989; Rapier et al., 1990). In a similar way to what occurred with the characterization of glutamatergic-dependent DA modulation, transition to electrochemical methods to quantify DA allowed for a better temporal resolution. Importantly, FSCV has been critical in determining a high dependence of DA release on stimulation frequency under the effect of nicotine. More specifically, in a striatal slice, the maximum peak of DA release does not change significantly through different frequencies (5, 10, 25, 50 Hz) of electrical stimulation. However, in the presence of nicotine or the nAChR antagonist mecamylamine, DA release at low frequencies is decreased, while at high frequencies release is enhanced (Rice and Cragg, 2004).

While electrical stimulation combined with pharmacological and genetic manipulations have produced a wealth of information on cholinergic control of DA release (Giorguieff et al., 1977; Cheramy et al., 1991; Krebs et al., 1991; Desce et al., 1992; Tremblay et al., 1992; Chéramy et al., 1996; Schmitz et al., 2003; Zhang and Sulzer, 2003, 2012; Exley and Cragg, 2008; Exley et al., 2011; reviewed by Cragg, 2006; Rice et al., 2011; Threlfell and Cragg, 2011; Zhang and Sulzer, 2012), the advent of optogenetics offered the previously unseen possibility of selective control of CINs. This would allow inducing AChR activation by means of endogenous release of ACh, obtained by selective stimulation of striatal CINs. Taking advantage of *in vitro* slices, electrochemistry, and optogenetics, both the laboratory of Stephanie Cragg (Threlfell et al., 2012) and ours (Cachope et al., 2012) were interested in characterizing changes in striatal DA levels in response to endogenous AChR activation. Interestingly, both research groups were advancing on characterizing similar phenomena in functionally different striatal areas, showing how endogenous release of ACh directly triggers DA release in the dorsal striatum and the NAc, respectively. Additionally to demonstrating how selective activation of CINs is enough to trigger DA release in striatum and NAc, respectively, both reports confirmed the role of nAChRs and mAChRs in modulating such output. Moreover, Cragg's report very nicely unveiled circuital mechanisms by which thalamic input synchronizes CIN firing, subsequently promoting DA release (Threlfell et al., 2012). Our experiments focused instead on the possibility of CINs-triggered DA release *in vivo* (Cachope et al., 2012). Consistently with the *in vitro* data, optogenetic stimulation of CINs in the anesthetized mouse was sufficient to trigger elevation of DA concentrations in the NAc. Also, following Sabatini's group's report regarding the ability of CINs to evoke glutamatergic responses (Higley et al., 2011), we showed that AChevoked DA release is sensitive to AMPA blockers (Cachope et al., 2012).

In the case of mAChRs, they can also locally modulate DA release in the striatum. *In vitro* experiments with FSCV show how a wide range mAChR antagonist (oxotremorine) decreases DA release evoked by single pulse electrical stimulation, but enhances

reduction of DA release. Activation of iGluRs on MSNs elevates production of

DA levels in response to train stimulation (Threlfell et al., 2010). A similar effect was observed using selective optogenetic stimulation, in which single pulse optical stimulation did not affect DA release, but instead 5 and 10 Hz stimulation enhanced DA release under application of the mAChR antagonist scopolamine (Cachope et al., 2012; Threlfell et al., 2012).

A complex interaction between diverse neurotransmission and neuromodulatory systems takes place in the control of striatal DA release. Although we have focused on the effect of glutamatergic and cholinergic systems, a handful other receptors have been identified as able to alter striatal DA levels; including GABA, cannabinoid, purinergic and opioid. Interestingly (and, up to some point expected), the possibilities for diversity on this local control are dependent on the type of receptor, not just the type of transmitter/modulator being released. Both in the case of glutamate and acetylcholine, different receptors lead to distinct and even opposite effects. As illustrated in **Figure 1**, mGluR activation on DA terminals and iGluR activation on MSNs result both in modulation of K conductances decreasing DA release. In contrast, activation of nAChRs on DA terminals leads to increased DA release, while activation of mAChR autoreceptors expectedly result in decreased DA release. More importantly, all these results demonstrate that firing rate at the VTA and SNc does not entirely determine striatal DA output, leaving enough room for control mechanisms driven by input from other areas (glutamatergic), as well as by interneurons (cholinergic), which might exert considerable impact on it.

further nAChR activation, which would result in decreased DA release.

The role of DA in essential behaviors such as reward-seeking, motivation and motor control has been extensively studied. Regulation of DA release at both the dorso-lateral striatum and the NAc is considered to be mainly the consequence of changes in firing rate at the level of DA somata in the SNc and the VTA, correspondingly. Local control at the level of the striatum has been traditionally linked to DA reuptake and to feedback control on DA release through activation of D2 autoreceptors. However, reports on termino-terminal control of DA release, although scarce decades ago provided key findings in understanding a more complex control system than the one defined just by firing rate at DA neuronal somata. To date, the influence of non-DA striatal terminals on striatal DA release has been explored in a variety of experimental conditions, including synaptosomes, *in vitro* slices and *in vivo* preparations. Not only pharmacological, but genetic, optogenetic, electrophysiological and electrochemical strategies have been used to unveil the localization, role, extent and functional impact of such local influences. Glutamatergic and cholinergic systems have attracted the most attention so far. Still, although highly characterized in terms of types of receptors and neurotransmitters involved, there is not enough evidence on the functional impact of these forms of regulation in the behavioral setting. CINs modify their firing rate in animals subject to behavioral tasks encoding reward delivery as a decrease in firing rate, following a mild increase in frequency of firing (Apicella et al., 1991, 2011; Aosaki et al., 1994; Shimo and Hikosaka, 2001; Morris et al., 2004). Also, a recent report shows a differential role of DA neurons modulating CINs firing in dorsal striatum and NAc (Chuhma et al., 2014). However, there is no clarity as to how prominent all those interactions are in terms of their ability to affect DA release, and even less is known about the role of such variations, if they might impact behavior, or if DA transmission is otherwise still VTA- and SNc-driven.

One of the main strategies to fully develop yet is the potential of targeting these modulation systems to affect striatal DA release in conditions such as Parkinson's disease, schizophrenia, addiction, Huntington's disease, in which DA levels have been reported to be altered. As already outlined by Threlfell and Cragg (2011), modulating the striatal cholinergic system through subunit-specific modulation of nAChR and mAChR promises to be a useful approach. Temporal dynamics are a critical feature of inter-neuronal transmission. Behavioral events have, for example, phasic changes in striatal DA levels as correlates in the limbic and motor areas (O'Neill and Fillenz, 1985; Schultz, 2007a,b; Joshua et al., 2009). A significant proportion of therapeutic strategies are based on ligands that exert a sustained effect on neurotransmitter receptors, cancelling such changes over time. While DA neuron somata drive phasic changes in DA release, termino-terminal control might be seen as a mechanism that allows for fine regulation over that main drive, still preserving most of the temporal dynamics.

## **REFERENCES**

Aosaki, T., Kimura, M., and Graybiel, A. M. (1995). Temporal and spatial characteristics of tonically active neurons of the primate's striatum. *J. Neurophysiol.* 73, 1234–1252.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 02 March 2014; accepted: 07 May 2014; published online: 23 May 2014*. *Citation: Cachope R and Cheer JF (2014) Local control of striatal dopamine release. Front. Behav. Neurosci. 8:188. doi: 10.3389/fnbeh.2014.00188*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience*.

*Copyright © 2014 Cachope and Cheer. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

# Optogenetic dissection of basolateral amygdala projections during cue-induced reinstatement of cocaine seeking

# **Michael T. Stefanik\* and Peter W. Kalivas**

Department of Neuroscience, Medical University of South Carolina, Charleston, SC, USA

#### **Edited by:**

Mary Kay Lobo, University of Maryland School of Medicine, USA

#### **Reviewed by:**

David Dietz, State University of New York at Buffalo, USA Alexxai V. Kravitz, National Institutes of Health, USA

#### **\*Correspondence:**

Michael T. Stefanik, Department of Neuroscience, Medical University of South Carolina, 173 Ashley Avenue, BSB 403, Charleston, SC 29425, USA e-mail: stefanik@musc.edu

Stimuli previously associated with drugs of abuse can become triggers that elicit craving and lead to drug-seeking behavior. The basolateral amygdala (BLA) is a key neural structure involved in cue-induced reinstatement of cocaine seeking. Previous studies have also implicated projections from the BLA directly to the nucleus accumbens (NAc) in these behaviors. However, other structures critically involved in cocaine seeking are targets of BLA innervation, including the prelimbic prefrontal cortex (PL). It has been shown that BLA or PL innervation direct to the NAc can modulate reward-related behaviors but the BLA also projects to the PL, and given the importance of the PL projection to the NAc for reinstated drug seeking, we hypothesized the BLA to PL projection may indirectly influence behavior via PL innervation to the NAc. We delivered a virus expressing the inhibitory optogenetic construct ArchT into the BLA and implanted fiber optics above the injection site or axon terminal fields in either the NAc or PL. Rats then went through 12 days of cocaine self-administration followed by extinction training. Following extinction, animals underwent cue-induced reinstatement sessions in the presence or absence of optical inhibition. Inactivation of the BLA and either the BLA core subcompartment of the NAc (BLA-to-NAcore) BLA-to-PL projections inhibited cue-induced reinstatement. These data demonstrate that the BLA projection either directly into the NAc, or indirectly via the PL, is a necessary regulator of drug-seeking behavior.

**Keywords: optogenetics, prelimbic cortex, nucleus accumbens, cocaine, reinstatement**

## **INTRODUCTION**

Exposure to drug-associated cues elicits craving and increases the probability that drug users will relapse, even after extended periods of abstinence. Understanding the neural circuits that underlie relapse is imperative in order to identify targets for therapeutic intervention. A dynamic interaction between basolateral amygdala (BLA) and prefrontal cortex (PFC) inputs to the nucleus accumbens (NAc) is part of the neural circuitry underpinning cue-induced reinstatement of drug seeking. Excitatory inputs carrying goal- and reward-related information from both cortical and limbic structures converge on the NAc, are integrated, and ultimately influence reward-directed actions. Encoding of rewardpredictive stimuli by the NAc relies on synaptic activity from both the BLA and prelimbic region (PL; McGinty and Grace, 2008). The BLA is critical for generating a response to conditioned cues (Buffalari and See, 2010), and direct innervation of the NAc by the BLA is necessary for cue-induced reward seeking (Setlow et al., 2002; Di Ciano and Everitt, 2004; Ambroggi et al., 2008; Mashhoon et al., 2010; Shiflett and Balleine, 2010; Stuber et al., 2011). However, these functional studies have assessed only one projection or relied on unilateral disconnections between the two regions. This leaves open the possibility that a BLAcore subcompartment of the NAc (NAcore) disconnection could impair behavior by nonspecifically interrupting other pathways, including a projection from the PL of the PFC that receives BLA afferents and in turn projects to the NAcore. Consistent with this hypothesis, previous work has demonstrated BLA control over NAc activity depends on an interaction with the PFC (Jackson and Moghaddam, 2001; McGinty and Grace, 2008), and that this interaction plays a role in cue-induced reinstatement (Fuchs et al., 2007; Mashhoon et al., 2010). Similarly, PFC neurons projecting to the NAc are also excited by conditioned stimuli (Ishikawa et al., 2008; McGinty and Grace, 2008), are essential for reward-seeking behavior (Park et al., 2002; McFarland et al., 2003; Stefanik et al., 2013b), and it has been hypothesized maladaptive changes in this pathway may be a common neural substrate that underlies the unmanageable drive to seek drugs (Goldstein and Volkow, 2002; Kalivas and Volkow, 2005).

Given that the BLA also projects to the PL (Sarter and Markowitsch, 1983; Reep, 1984), and that BLA inactivation can influence NAc projecting neurons from the PL (Jackson and Moghaddam, 2001; McGinty and Grace, 2008), it seems possible that this indirect pathway to the NAc might be an additional route in which BLA activity is influencing cued reinstatement. To test this hypothesis, we employed an inhibitory optogenetic strategy in which we delivered an adeno-associated virus (AAV) coding for the light sensitive proton pump archaerhodopsin (ArchT; Chow et al., 2010) into the BLA and selectively inhibited axon terminals in either the NAc or PL during cue-induced reinstatement of cocaine seeking.

**FIGURE 1 | Inhibition of BLA cell bodies reduces cue-primed reinstatement of cocaine seeking**. **(A)** Active lever pressing during self-administration and extinction training for this group of rats (n = 10). **(B)** DAB staining shows ArchT-expressing neurons in the BLA. Dashed line outlines BLA. Scale bar, 300 µm. **(C)** Inactivation of ArchT-expressing neurons in the BLA significantly reduces cue-primed reinstatement of cocaine seeking. **(D)** Illumination of control GFP virus has no effect on reinstated

## **MATERIALS AND METHODS**

#### **ANIMAL HOUSING AND SURGERY**

All methods used were in compliance with the National Institutes of Health *Guide for the Care and Use of Laboratory Animals* and were approved by the Medical University of South Carolina's Institutional Animal Care and Use Committee. Male Sprague Dawley rats (250–300g, Charles River Laboratories) were individually housed under temperature- and humidity-controlled conditions with a 12 h reverse light/dark cycle (lights on at 6:00 P.M.). Rats were fed *ad libitum* until 7 days post-surgery, after which food was restricted to 25 g of chow pellets per day.

Following one week of handling and acclimation, rats underwent surgery for injection of AAV, implantation of fiber optics, and implantation of indwelling jugular catheters. Animals were anesthetized with ketamine HCl (87.5 mg/kg, i.m.) and xylazine (5 mg/kg, i.m.). Ketorolac (3mg/kg, i.p.) was administered before surgery to provide analgesia. Intra-jugular catheters were implanted as previously described (LaLumiere et al., 2012). Catheters were flushed daily with cefazolin (0.2 mL of 0.1 g/mL) and heparin (0.2 mL of 100 IU) for 1 week, then daily with heparin for the remainder of the experiment to maintain catheter patency.

For virus injections, 0.7 µl of virus (rAAV2-CAG-ArchT-GFP or rAAV2-CMV-GFP for BLA cell body experiment, ∼10<sup>12</sup> viral particles/ml) was delivered bilaterally through 33 gauge needles (0.14 µL/min for 5 min). Needles were left in place for 10 min following injection to allow for virus diffusion away from injection site. For the virus injections, coordinates from Bregma were as follows: BLA: −2.8 mm anteroposterior, ±5.0 mm mediolateral, −8.5 mm dorsoventral. For fiber optic implantation, chronically implantable fiber optics (Precision Fiber) were implanted 0.5 mm dorsal to the site intended to receive light stimulation, coordinates from Bregma: NAcore: +1.5 mm anteroposterior, +3.5 mm mediolateral, −6.5 dorsoventral (10◦ angle); PL: +3.1 mm anteroposterior, +2.0 mm mediolateral, −4.0 dorsoventral (12◦ angle). Fibers we secured to the skull using small screws and dental acrylic and animals recovered for 1 week before behavioral testing.

cocaine seeking. #p < 0.05 compared with extinction levels of lever pressing for ArchT-expressing animals using a paired Student's t-test with probability values adjusted for multiple comparisons according to Bonferroni, Sham t(5) = 5.50, p < 0.01; Laser t(5) = 4.15, p < 0.01. \*p < 0.05 comparing laser and sham treatments, t(5) = 3.99, p = 0.010. For GFP controls, Sham t(3) = 2.91, p = 0.06; Laser t(3) = 4.32, p = 0.02. Comparing laser and sham treatments, t(3) = 0.77, p = 0.50.

#### **SELF-ADMINISTRATION, EXTINCTION AND REINSTATEMENT PROCEDURES**

Self-administration, extinction, and reinstatement procedures occurred in standard operant chambers equipped with two retractable levers, a house light, cue light, and 2900 Hz tone generator (Med Associates). Before cocaine self-administration training, animals were food deprived for 24 h and then underwent a single 15 h food training session in which presses on the active lever resulted in the delivery of a single food pellet (45mg, Noyes) on a fixed-ratio 1 (FR1) schedule of reinforcement. Following food training, animals were restricted to 25 g of food per day, given immediately after the behavioral session, for the remainder of the experiment. One day later, animals began 2-h sessions cocaine self-administration on an FR1 schedule with a 20 s time out. Each active lever press resulted in a 0.05 ml infusion of 0.20 mg cocaine (∼15–20 mg/kg per animal total over 2-h session, dissolved in 0.9% sterile saline, NIDA) and the drug-paired cues (concurrent illumination of the stimulus light above the active lever and tone) for 5 s. Active lever presses made during the time out were counted but did not result in drug delivery and inactive lever presses were of no consequence. Rats underwent self-administration 6 days/week for at least 2 weeks (minimum of 12 days), until they met maintenance criteria of ≥10 infusions of cocaine over 10 days, as well as discrimination between active and inactive levers (>75% lever presses on active lever). A total of three rats not reaching these criteria after 4 weeks were excluded from the study.

Following successful acquisition and maintenance of cocaine self-administration, extinction training (2 h/day) began. During extinction, presses on the previously active lever were recorded but no longer produced drug or presentation of the drug-paired cues. All rats underwent at least 10 days of extinction, until active lever pressing fell to <30% of the average responding during self-administration. Animals were habituated to the fiber optic leashes for ≥3 sessions of both self-administration and extinction. Immediately before testing, fibers were attached and remained in place for the duration of the session. During the reinstatement sessions, active lever presses produced the light/tone drug-paired

cues that had been presented during self-administration, but no drug was delivered. All animals underwent two reinstatement sessions, counterbalanced with respect to whether illumination was given.

## **OPTICAL INHIBITION**

Optical inhibition was delivered as previously described (Stefanik et al., 2013a,b). Briefly, chronically implantable optical fibers were housed inside stainless steel ferrules (for construction details, see Sparta et al., 2011) and implanted bilaterally ∼0.5 mm dorsal to the site intended to receive light. To permit bilateral inhibition, the single end of 2 × 1 fiber splitter (Precision Fiber) was connected via FC/PC connection to a rotating optical commutator, which was then attached via a fiber to a laser (diode-pumped solid-state, 200 mW, 561 nm multimode FC/PC fiber coupler connection, OEM Laser Systems). The two split ends of the fiber splitter were threaded through a metal leash, and epoxied into stainless steel ferrules which were then connected to the animal's head via ceramic sleeves. Light output was measured with an optical power meter and adjusted to ∼10 mW of 561 nm light. Based on *in vivo* measurements, of light output in mammalian brain tissue, these parameters would be expected to provide sufficient light to at least 0.4 mm<sup>3</sup> of tissue (Yizhar et al., 2011). Light was applied continuously for the 2 h reinstatement session, a procedure previously shown to inhibit neuronal firing without significantly desensitizing the opsin (Huff et al., 2013; Stefanik et al., 2013a,b; Tsunematsu et al., 2013). As work in our lab has previously shown no effect of light delivery to the NAcore or PL on behavior, GFP control experiments were conducted at the BLA cell bodies to control laser light on cue-induced reinstatement (Stefanik et al., 2013a,b).

#### **IMMUNOHISTOCHEMISTRY AND IMAGING**

For immunohistochemistry and imaging, animals were anesthetized with pentobarbital (100 mg/ml, i.p.) and then transcardially perfused with phosphate-buffered saline (PBS, pH 7.4) followed by PBS containing 4% (w/v) paraformaldehyde. Brains were post-fixed for 24 h at room temperature in the perfusion solution. Coronal sections (75 µm thick) were incubated for 60 min in 1% hydrogen peroxide, rinsed three times in PBS, and then incubated overnight in PBS containing 0.25% triton-X, 0.01% sodium azide, and anti-GFP (rabbit, 1:50,000, Abcam) antibody. Sections were then rinsed once in PBS and incubated for 30 min in PBS containing the biotinylated secondary antibody (donkey, 1:1000, Jackson Immunoresearch) for 30 min, rinsed four times in PBS, and incubated for 1 h in an ABC Kit (Vector Labs). Sections were then rinsed once in PBS and incubated in PBS with 0.05% diaminobenzidine with 0.05% hydrogen peroxide for 5 min. Slices were then mounted and ArchT-expression was visualized on a light microscope.

## **DATA ANALYSIS**

Statistics were performed using Prism (GraphPad Software). Reinstatement sessions were compared between extinction pressing, cue-induced reinstatement without laser, and cue-induced reinstatement with laser, using a two-tailed paired Student's *t*test with probability values adjusted according to the method of Bonferroni. Data are presented as mean ± SEM.

## **RESULTS**

## **OPTICAL INHIBITION OF THE BASOLATERAL AMYGDALA (BLA) REDUCES CUE-INDUCED REINSTATEMENT**

The BLA is critically involved in the processing of cue-related information (Buffalari and See, 2010). To examine whether optical inhibition of this structure could alter cue-induced reinstated cocaine seeking, an AAV expressing the light-activated inhibitory proton pump ArchT (Chow et al., 2010) was microinjected into the BLA and fiber optics were implanted ∼0.5 mm dorsal to the injection site. Following self-administration and extinction training (**Figure 1A**), animals underwent cue-induced reinstatement in the presence or absence of ∼10 mW of 561 nm light in the BLA using a counterbalanced, within-subjects design. Rats microinjected with a control AAV virus expressing only GFP were treated identically. **Figure 1B** shows DAB staining for ArchT-GFP expression in the BLA. ArchT-expressing animals receiving laser light showed a reduction to nearly extinction levels of active lever-pressing compared to sham (no laser delivery) treatment (**Figure 1C**). To control for the potential effects of light delivery

to this region, we showed that animals receiving the control GFP virus showed no difference in lever pressing regardless of the presence or absence of laser light (**Figure 1D**).

## **INHIBITING THE BASOLATERAL AMYGDALA (BLA) TO CORE SUBCOMPARTMENT OF THE NAc (NAcore) PROJECTION INHIBITS CUE-INDUCED REINSTATEMENT**

Multiple studies point to the projection from the BLA to the NAcore as being critically involved cue-induced reward seeking (Setlow et al., 2002; Di Ciano and Everitt, 2004; Ambroggi et al., 2008; Mashhoon et al., 2010; Shiflett and Balleine, 2010; Stuber et al., 2011). To assess the involvement of this connection in cue-induced reinstated cocaine seeking, ArchT was microinjected into the BLA and optic fibers were implanted above terminal regions in the NAcore. Animals underwent self-administration and extinction training (**Figure 2A**). DAB staining for ArchT-GFP confirmed strong expression of the virus at the terminal fields in the NAcore (**Figure 2B**). **Figure 2C** shows significant reduction in active lever pressing produced by optical inhibition compared to sham treatment.

## **INHIBITING THE BASOLATERAL AMYGDALA (BLA) TO PRELIMBIC (PL) PROJECTION INHIBITS CUE-INDUCED REINSTATEMENT**

The BLA also sends strong projections to the PL that transfer reward-related information (Fuchs et al., 2007; Mashhoon et al., 2010). Given the importance of the PL to NAcore projection in the relapse to drug seeking (Goldstein and Volkow, 2002; McFarland et al., 2003; Kalivas et al., 2005; Stefanik et al., 2013b), BLA neurotransmission in PL could regulate reinstated cocaine seeking. To test the involvement of the projection from the BLA to the PL in cue-induced reinstatement, ArchT was microinjected into the BLA and optic fibers were implanted in the PL. **Figure 3A** shows the self-administration and extinction data for these animals. Strong virus expression was detected from BLA injections at the terminal fields in the PL (**Figure 3B**). Optical inhibition of this pathway attenuated cue-induced cocaine seeking in laser treated animals, but not in sham animals (**Figure 3C**).

# **HISTOLOGY**

**Figure 4** shows the location of the fiber optic termination in each experiment. The area of illumination was estimated from the tip of the histologically identified fiber optic tip to expand in a cone shape for 0.5 mm in length and diameter from the most ventral penetration (Yizhar et al., 2011). While virus spread can be seen in a more widely distributed area of the amygdala, the placement of fiber implantations in the BLA were tightly focused just dorsal to the structure and medial to the external capsule and allow for precise targeting of the BLA alone (**Figure 4A**). Animals with fiber placement outside of the histologically identified BLA were excluded from analysis. Fiber implantations in the NAcore were located primarily dorsal to the anterior commissure (**Figure 4B**). **Figure 4C** shows the location fiber optic implants in the PL located lateral to the midline.

# **DISCUSSION**

The BLA is a key neural structure involved in cue-induced cocaine seeking (Buffalari and See, 2010). Projections from the BLA to the NAc have also been implicated in these behaviors. As previous studies have relied primarily on unilateral pharmacological inactivation to demonstrate the role of the projection, they suffer from a potential interpretational inaccuracy since inactivating projections from the BLA to other regions that influence drug seeking behavior that in turn project to the NAc could be mediating the behavioral inhibition. The PL is one region that might be indirectly influenced since pharmacological and optogentic inhibition of the PL to NAc show that this pathway is critically involved in cocaine reinstatement (McFarland et al., 2003; Stefanik et al., 2013b). Additional evidence also suggests that the BLA-to-PL projection is critical to cue-induced reward seeking (Fuchs et al., 2007; Mashhoon et al., 2010). To address this issue, we used the inhibitory optogenetic construct ArchT to selectively inactivate the BLA or its projections terminating in the NAc or PL. We demonstrate that inhibition of the BLA or either projection is sufficient to markedly reduce cue-induced cocaine seeking. Together, these findings suggest a more nuanced influence of BLA projections on NAc function that previously thought. Thus, while

BLA neurotransmission directly to the NAc provides necessary information to facilitate cue-induced cocaine seeking, the BLA can indirectly influence NAc function via selective activation of PL projections to the NAcore.

A role for activation of the PL to initiate behavior may seem paradoxical in the context of a recent finding by Chen et al. (2013) who were able to suppress compulsive cocaine seeking by optogenetically stimulating hypofunctioning PL neurons during the seeking phase of a cocaine-seeking task. They theorize that chronic drug use induces a hypofunctional state in the PFC causing habitual behavior to supersede cognitive control, and thereby promoting compulsive drug seeking. In line with this view, the hypofunctioning PFC and compulsive drug seeking observed by Chen et al. (2013) were reduced by enhancing activity in the PL. Conversely, the results of this and previous work (McFarland et al., 2003; Stefanik et al., 2013b) support a view that PFC activity is needed in order to recognize and integrate information from a number of structures to initiate drug-seeking behavior. Although seemingly paradoxical in the context of drug seeking paradigms, these studies are consistent with a well-established role for the PFC as a site where computations and decisions are made to execute adaptive responses that can include both behavioral activation and inhibition (Ghazizadeh et al., 2012).

The BLA conveys information to the PL about the cues that have been previously paired with reward delivery (Fuchs et al., 2007; McGinty and Grace, 2008; Mashhoon et al., 2010). Interestingly, while the glutamatergic projection from the BLA has been shown to excite PL neurons (Little and Carter, 2013), a proportion of the BLA projections inhibit pyramidal cell firing (Floresco and Tse, 2007) by synapsing onto interneurons in the PL (Dilgen et al., 2013), providing a potentially paradoxical feed forward inhibition of neurotransmission. BLA activation of both pyramidal neurons and interneurons might cause selective attention for cocaine-associated cues while simultaneously suppressing responses for non-drug stimuli. Consistent with this hypothesis, stimulation of BLA neurons can excite or inhibit PL pyramidal neurons projecting to the NAc (McGinty and Grace, 2008, 2009a,b). Work in other brain regions also supports the idea of selective suppression of information may work to funnel information into specific downstream circuits (Liang et al., 2013). Selective input from the BLA onto interneurons in the PL could be serving to gate other information arriving into the PL that might compete with the drug-paired stimuli. Coupled with direct excitation of NAc medium spiny neurons by BLA afferents, the NAc can then process a multi-layered association between the environmental stimulus and a given behavior. Given this possibility, future studies need to address the relationship between BLA-to-PL neurotransmission on selective populations of PL neurons projecting to NAc during the processing of drugassociated cues.

## **AUTHOR CONTRIBUTIONS**

Michael T. Stefanik and Peter W. Kalivas were responsible for study concept and design. Michael T. Stefanik conducted the behavioral studies and histology. Michael T. Stefanik and Peter W. Kalivas wrote the manuscript.

#### **ACKNOWLEDGMENTS**

This work was supported by grants DA015369, DA 012513, DA003906, and T32 DA7288 from the National Institutes of Health.

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 08 November 2013; accepted: 11 December 2013; published online: 24 December 2013*.

*Citation: Stefanik MT and Kalivas PW (2013) Optogenetic dissection of basolateral amygdala projections during cue-induced reinstatement of cocaine seeking. Front. Behav. Neurosci. 7:213. doi: 10.3389/fnbeh.2013.00213*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience*.

*Copyright © 2013 Stefanik and Kalivas. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

# Optogenetic modulation of descending prefrontocortical inputs to the dorsal raphe bidirectionally bias socioaffective choices after social defeat

# *Collin Challis 1,2, Sheryl G. Beck2,3 and Olivier Berton1,2\**

*<sup>1</sup> Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA*

*<sup>2</sup> Neuroscience Graduate Group, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA*

*<sup>3</sup> Department of Anesthesiology, Children's Hospital of Philadelphia and University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA*

#### *Edited by:*

*Mary K. Lobo, University of Maryland School of Medicine, USA*

#### *Reviewed by:*

*Daniel W. Wesson, Case Western Reserve University, USA Xiao-Dong Wang, Zhejiang University, China*

#### *\*Correspondence:*

*Olivier Berton, Department of Psychiatry, Center for Neurobiology and Behavior, University of Pennsylvania Perelman School of Medicine, 125 S. 31st Street, Philadelphia, PA 19104, USA e-mail: bertonol@ mail.med.upenn.edu*

It has been well established that modulating serotonin (5-HT) levels in humans and animals affects perception and response to social threats, however the circuit mechanisms that control 5-HT output during social interaction are not well understood. A better understanding of these systems could provide groundwork for more precise and efficient therapeutic interventions. Here we examined the organization and plasticity of microcircuits implicated in top-down control of 5-HT neurons in the dorsal raphe nucleus (DRN) by excitatory inputs from the ventromedial prefrontal cortex (vmPFC) and their role in social approach-avoidance decisions. We did this in the context of a social defeat model that induces a long lasting form of social aversion that is reversible by antidepressants. We first used viral tracing and *Cre*-dependent genetic identification of vmPFC glutamatergic synapses in the DRN to determine their topographic distribution in relation to 5-HT and GABAergic subregions and found that excitatory vmPFC projections primarily localized to GABA-rich areas of the DRN. We then used optogenetics in combination with *cFos* mapping and slice electrophysiology to establish the functional effects of repeatedly driving vmPFC inputs in DRN. We provide the first direct evidence that vmPFC axons drive synaptic activity and immediate early gene expression in genetically identified DRN GABA neurons through an AMPA receptor-dependent mechanism. In contrast, we did not detect vmPFC-driven synaptic activity in 5-HT neurons and *cFos* induction in 5-HT neurons was limited. Finally we show that optogenetically increasing or decreasing excitatory vmPFC input to the DRN during sensory exposure to an aggressor's cues enhances or diminishes avoidance bias, respectively. These results clarify the functional organization of vmPFC-DRN pathways and identify GABAergic neurons as a key cellular element filtering top-down vmPFC influences on affect-regulating 5-HT output.

**Keywords: dorsal raphe, ventromedial prefrontal cortex, serotonin, optogenetics, electrophysiology, depression and anxiety disorders, social perception, social defeat**

# **INTRODUCTION**

The capacity to detect and interpret the affective state of others using non-verbal social cues (e.g., facial expression, vocal prosody, posture, body movement, and olfactory cues) is a necessary survival skill shared by many animal species (Chang et al., 2013; Oliveira, 2013). It allows individuals to anticipate harmful intentions of others and adapt through rapid approach or avoidance decisions (O'Connell and Hofmann, 2012). The capacity to conduct social-cognitive appraisal is also a determining aspect of human social competence (Todorov, 2008; Volman et al., 2011) and dysfunction of the neural systems that mediate socioaffective decisions are thought to contribute to excessive reassurance-seeking behaviors and social withdrawal, which are two symptomatic dimensions shared across several affective disorders, including major depression, and social phobia (Heuer et al., 2007; Seidel et al., 2010; Derntl et al., 2011; Stuhrmann et al., 2011; Cusi et al., 2012; Moser et al., 2012).

Serotonin (5-HT) is a neurotransmitter system that plays an evolutionarily conserved role in regulating affiliative and antagonistic behaviors (Canli and Lesch, 2007; Dayan and Huys, 2009; Rogers, 2011). Increases in 5-HT output, such as resulting from treatment with SSRI antidepressants, have consistently been shown to positively bias socioaffective appraisals and facilitate social affiliation and dominance in human and animals (Raleigh et al., 1991; Knutson et al., 1998; Tse and Bond, 2002; Bond, 2005; Harmer and Cowen, 2013). In contrast, 5-HT depletion facilitates socially defensive behaviors and aggression (Young and Leyton, 2002; Munafo et al., 2006). The fact that the output of ascending 5-HT neurons located in the dorsal raphe nucleus (DRN) is under top-down control by multiple forebrain areas (Peyron et al., 1998; Freedman et al., 2000; Chiba et al., 2001; Celada et al., 2002; Lee et al., 2003; Vertes, 2004) suggests a potentially key role for DRN afferent systems in the modulation of socioaffective responses. Studies conducted *in vivo* in anesthetized rodents combining electrical stimulation of the ventromedial prefrontal cortex (vmPFC) and extracellular recordings in the DRN demonstrated the rapid inhibition of putative 5- HT neurons (Varga et al., 2001; Celada et al., 2002). Parallel histological tracing studies demonstrated that DRN GABAergic neurons that are preferential targets of vmPFC projections could mediate the inhibitory responses recorded *in vivo* (Jankowski and Sesack, 2004). However, due to the limited specificity of electrophysiological signatures to predict neurochemical cell-type (Calizo et al., 2011), the identities of neuronal populations that compose the vmPFC-DRN microcircuit have not been fully elucidated. Furthermore, there is a lack of information about the possible topographical distribution of various DRN cellular populations thereby limiting the progress of studies assessing their causal role in socioaffective responses and other behaviors.

In recent studies we used a murine model of chronic social defeat stress (CSDS) that induces long lasting avoidance bias responsive to antidepressants to characterize the role of DRN microcircuits in the development and expression of social aversion (Espallergues et al., 2012; Challis et al., 2013; Crawford et al., 2013; Veerakumar et al., 2013). In mice susceptible to CSDS, but not in ones resilient, we detected a sustained sensitized synaptic inhibition of DRN 5-HT neurons, associated with a state of dramatically reduced intrinsic excitability of 5-HT neurons. Furthermore, we identified a subset of *GAD2*<sup>+</sup> GABA neurons with sensitized excitatory synaptic input and intrinsic excitability, which monosynaptically inhibits nearby 5-HT neurons. Using optogenetic photo silencing we provided evidence of their key role in the associative process that underlie the development of social avoidance in susceptible mice (Challis et al., 2013). Interestingly, we noted that these sensitized GABAergic neurons appear to be located in circumscribed lateral subregions of the DRN heavily innervated by the vmPFC. These observations suggest a potentially unique role of inputs from the vmPFC in driving stress-induced plasticity of GABA neurons within the DRN that underlie the stabilization of avoidance bias after CSDS.

In the present study, we set out to test this hypothesis. We used *in vivo* optogenetics to drive or inhibit the synaptic inputs from vmPFC axons locally within the DRN during the sensory contact phase of CSDS. We also used viral tracing, whole-cell recordings, and optogenetic methods in slice preparations to further characterize the anatomical and functional organization of the vmPFC-DRN pathway. Our results directly show that excitatory projections from the vmPFC preferentially target and synaptically activate GABA neurons that are topographically distributed within the DRN. We also show that activation of these terminals paired temporally with exposure to social cues potentiates negative socioaffective bias and social avoidance, while inhibition of these inputs facilitates the maintenance of social engagement after defeat, a characteristic of resilient individuals. These results provide fundamentally novel insights about neural mechanisms implicated in the top-down control of 5-HT during socioaffective tasks and have important implications for the understanding and treatment of affective disorders.

## **MATERIALS AND METHODS**

#### **ANIMALS**

Eight- to twelve-week old male mice bred onto a C57BL/6 background were used for all experiments. Mice were housed on a 12-h light/dark cycle with food and water available *ad libitum*. All studies were conducted according to protocols approved by the University of Pennsylvania Institutional Animal Care and Use Committee. All procedures were performed in accordance with institutional guidelines. The large cohort of defeated mice used to determine social choice consisted of male C57 Black mice (*C57BL/6J*: JAX stock number 000664). Trained aggressor mice were retired CD-1 male breeder mice (*Crl:CD1*; Charles River Laboratories, Malvern, PA). To generate a mouse line with fluorescently labeled *GAD65*-containing GABAergic or serotonergic neurons, male knocking *GAD2-Cre* mice (*Gad*2tm2*(*cre*)*Zjh*/J*; JAX stock number 010802) (Taniguchi et al., 2011) or BAC transgenic *Pet1-Cre* mice (*B6.Cg-Tg(Fev-cre*)1Esd/J; JAX stock number 012712) (Scott et al., 2005) were respectively crossed to female floxed-stop controlled *tdTomato* (RFP variant) mice (*B6.Cg-Gt(ROSA)26Sor*tm9*(*CAG−tdTomato*)*Hze*/J*; JAX stock number 007908) (Madisen et al., 2010) to achieve fluorescent labeling of *Cre* containing cells. To achieve expression of optogenetic probes or fluorescent tracers in glutamatergic vmPFC neurons we used *CaMKIIa-Cre* mice (*B6.Cg-Tg(CamK2a-Cre)T29-1Stl/J*; JAX stock number 005359) (Tsien et al., 1996). With the exception of the CD-1 strain, all mice were procured from the Jackson Laboratory (Bar Harbor, ME).

## **VIRUS AND SURGERY**

To express optogenetic or fluorescent proteins in glutamatergic neurons, adeno-associated virus (AAV) vectors were produced by and purchased from the University of Pennsylvania vector core (Philadelphia, PA) and injected into *CaMKIIa-Cre* mice. In this work we used AAVs for the *Cre*-inducible expression of the excitatory optogenetic probe *Channelrho dopsin* (AAV2/9.EF1a.DIO.hChR2(H134R)-EYFP.WPRE.hGH; Addgene #20298), inhibitory optogenetic probe *Archaerhodopsin* (AAV2/9.flex.CBA.Arch-GFP.W.SV40; Addgene #22222), fluorescent protein *tdTomato* (AAV2/1.CAG.FLEX.tdTomato.WPRE. bGH; Allen Institute #864) and *GFP* tagged Synaptophysin (AAV2/9.CMV.FLEX.Synaptophysin-Venus.WPRE.hGH; plasmid kindly provided by Anton Maximov, PhD, Department of Molecular and Cellular Neuroscience, The Scripps Research Institute). For stimulation of excitatory vmPFC terminals in the DRN of *GAD2-tdTomato* or *Pet1-tdTomato* mice we used an AAV for the *CaMKIIa-*driven expression of *Channelrhodopsin* fused to *YFP* (AAV2/9.CaMKII.ChR2-YFP.SV40; Stanford) (Mattis et al., 2012).

For viral injections, mice were anesthetized with isofluorane and stereotaxically injected unilaterally in the prelimbic region of the vmPFC (from Bregma, in mm: +1.8 AP, +0.8 ML, −2.7 DV, 15◦ angle) with 0.5μl of virus. Viral yields (in GC) were 3*.*54 × 10<sup>12</sup> for *ChR2-YFP,* 6*.*962 × 10<sup>11</sup> for *Arch-GFP*, 2*.*049 × 10<sup>12</sup> for *tdTomato* and 4*.*347 × 10<sup>12</sup> for *CaMKIIa-ChR2-YFP*. Social defeat began 4 weeks post-surgery for non-cannulated mice to allow time for recovery and viral expression.

For *in vivo* optical stimulation, precut guide cannulae (Plastics One, Roanoke, VA) targeting the DRN (from Lambda, in mm: 0.0 AP, +0.8 ML, −3.3 DV, 15◦ angle) were secured to the skull using stainless steel skull screws and acrylic cement. A fitted dustcap dummy was secured atop the guide cannula and mice were placed back in homecages and allowed 6 weeks to recover. Body weight and behavior was monitored during recovery. Three days before the start of experiment, a homemade fiber optic with ferrule connector (described below) was inserted into the guide cannula and secured with acrylic cement.

## **PREPARATION OF OPTICAL FIBERS**

A Two hundredμm core, 0.37 NA standard multimode fiber (Thorlabs, Newton, NJ) was stripped of cladding, passed through a 230μm multimode ceramic zirconia ferrule (Precision Fiber Products, Milpitas, CA), and secured in place using fiber optic connector epoxy (Fiber Instrument Sales, Oriskany, NY). Ferrules were then polished and cut to length to target the DRN. They were tested for light output and sterilized with 70% ethanol.

## **CHRONIC SOCIAL DEFEAT STRESS**

We use a modified chronic social defeat stress (CSDS) paradigm to induce social avoidance (Golden et al., 2011; Challis et al., 2013). Our model consists of exposing male mice to alternating periods of physical contact with a trained CD1 aggressor male mouse (5 min) and protected sensory contact via separation by a perforated Plexiglass partition (20 min) before returning to home cages overnight. The 20 min of sensory contact is sufficient to induce a significant decrease in social interaction compared to undefeated mice or mice that were not exposed to a sensory period after physical contact. This effect has been previously described Challis et al. (2013). This continued for 10 consecutive days with exposure to a novel aggressor each day. Control animals were also singly housed and were only exposed to daily sensory contact with novel aggressors. On day 11, social approach or avoidance behavior toward an unfamiliar CD1 social target was assessed in a two-trial social interaction task. In the first 2.5-min trial ("no target"), experimental mice explored a dimly lit (55 lux) open-field arena containing an empty wire mesh cage on one edge of the arena (see **Figure 6A**). In the second 2.5-min trial ("target present"), experimental mice were reintroduced to the arena now with an unfamiliar CD1 aggressor positioned in the mesh cage. TopScan video tracking software (CleverSys, Reston, VA) was used to measure the time spent in the interaction zone surrounding the target box.

## **IMMUNOHISTOCHEMISTRY**

Animals were transcardially perfused with 4% paraformaldehyde and brains were processed for standard single or dual immunolabeling methods as previously described (Espallergues et al., 2012). For detection of *cFos*, we used an affinity purified rabbit polyclonal antibody raised against the N-terminus of human *cFos* (1:1000 dilution; SC-52, Santa Cruz Biotechnology, Santa Cruz, CA). To enhance *GFP* expression we used a chicken anti-*GFP* antibody (1:1000 dilution; GFP-1020, Aves Labs, Inc., Tigard, OR). Primary antibodies were detected using fluorescent secondary antibodies obtained from Jackson ImmunoResearch Laboratories (1:500 dilution; West Grove, PA).

## **CELL COUNTING**

To map neuronal populations in the DRN, 30μm serial sections of the DRN were collected every 120μm between −4.36 mm and −4.96 mm from Bregma. Native *tdTomato* fluorescence and immuno-enhanced *GFP* fluorescence of *SynP* labeled vmPFC terminals were visualized using confocal microscopy. Slices from corresponding rostro-caudal levels between mice were aligned on a map based on location of the aqueduct. Neurons and terminals were manually drawn for each level of the DRN.

To quantify *cFos* colocalization with *tdTomato*<sup>+</sup> neurons, slices were stained for *cFos* and labeled neurons were manually counted in the DRN of each section. Colocalization with *tdTomato* was defined as nuclear localization of the *cFos* signal and was manually counted by an experimenter blind to the experimental condition of the mice from which the slices originated. There was not a significant variation of total number of *tdTomato*<sup>+</sup> cells within each strain.

To determine whether spatial distribution of synaptic vmPFC inputs traced using *SynP-GFP* correlated with the distribution of *GAD2-tdTomato* or *Pet1-tdTomato* neurons, we divided corresponding coronal views of the DRN in *GAD2-tdTomato*, *Pet1 tdTomato* and *SynP-GFP* injected *CaMKIIa-Cre* mice into 10 × 10 grids and tested for correlations between *SynP-GFP* and *tdTomato* fluorescence across the grid. This was done at each of the 6 rostrocaudal levels across the DRN. Fluorescent intensity within each grid box was calculated using the ImageJ "Measure" function which converts red, green, and blue (RGB) pixel values to brightness using the formula V = (R + G + B)/3. These intensity values were then normalized to the grid box with the highest intensity. Correlations were tested using the Pearson coefficient and plotted using linear regression.

## **ELECTROPHYSIOLOGY**

Brain slices were prepared as previously described (Crawford et al., 2010, 2013; Calizo et al., 2011; Espallergues et al., 2012; Challis et al., 2013; Howerton et al., 2013). The 200μm coronal slices containing DRN were placed in aCSF (in mM, NaCl 124, KCl 2.5, NaH2PO4 1.25, MgSO4 2.0, CaCl2 2.5, dextrose 10, NaHCO3 26) at 37◦C, aerated with 95% O2/5% CO2. After 1 h, slices were kept at room temperature. Tryptophan (2.5 mM) was included in the holding chamber to maintain 5-HT synthesis, but was not in the aCSF perfusing the slice in the recording chamber. Individual slices were placed in a recording chamber (Warner Instruments, Hamden, CT) and perfused with aCSF at 2 ml/min maintained at 32◦C by an in-line solution heater (TC-324, Warner Instruments). Neurons were visualized using a Nikon E600 upright microscope fitted with a 60X water immersion objective and targeted under DIC or fluorescent filters. Resistance of electrodes was about 8–10 MOhms when filled with a recording solution composed of (in mM) K-gluconate (130), NaCl (5), Na phosphocreatine (10), MgCl2 (1), EGTA (0.02), HEPES (10), MgATP (2) and Na2GTP (0.5) with 0.1% biocytin and a pH of 7.3. Whole-cell recordings were obtained using a Multiclamp 700 B amplifier (Molecular Devices, Sunnyvale, CA). Cell characteristics were recorded using current clamp techniques as previously described (Crawford et al., 2010; Espallergues et al., 2012). Signals were collected and stored using Digidata 1320 analog-to-digital converter and pClamp 9.0 software (Molecular Devices). Collection of EPSC data was as previously described (Crawford et al., 2011) and performed with bath application of 20μM bicuculline to block GABA synaptic activity. To characterize light-evoked ESPC activity, 20μM DNQX was applied to the bath to block AMPA receptor activity. All drugs were made in stock solutions, diluted on the day of the experiment, and added directly to the ACSF.

## **ELECTROPHYSIOLOGY DATA ANALYSIS**

Synaptic properties were analyzed using MiniAnalysis (Synaptosoft, Decatur, GA) as previously described (Crawford et al., 2011, 2013). Synaptic events were analyzed using parameters optimized for each cell with the detection threshold set beyond the maximum values of the all-points noise histogram for a portion of the trace containing no detectable synaptic events. This threshold generally ranged from 5 to 8 pA. MiniAnalysis generates a summary table containing the mean and median values for the frequency, amplitude, rise time (10–90%), decay time, and event half width (50%). For each cell, at least 200 events were chosen at random and manually filtered to exclude multiple peaks then combined to obtain an averaged EPSC or IPSC for each cell to obtain values for decay time, event area, and event time half-width. Additional statistical analysis is described below. Data reported are means ± s.e.m.

#### **OPTICAL STIMULATION**

For *in vivo* stimulation, mice with previously implanted fiber optic ferrules were connected to a 200μm, 0.37 NA patch cord via zirconia sleeve that was then connected to a diode-pumped solidstate (DPSS) laser through an FC/PC adaptor and rotary joint. We used blue (473 nm, BL-473-00100-CWM-SD-05-LED-0) and yellow (561 nm, GR-561-00100-CWM-SD-05-LED-F) DPSS lasers obtained from OEM Laser Systems (Bluffdale, UT). Power output was measured using an optical sensor (Thorlabs, Newton, NJ) to be about 10 mW. Intensity was calculated using a model predicting irradiance in mammalian tissues (http://www*.*stanford*.* edu/group/dlab/cgi-bin/graph/chart*.*php). From a 200μm fiber optic tip, estimated intensity was 7.33 mW mm−<sup>2</sup> for blue laser stimulation and 7.05 mW mm−<sup>2</sup> for yellow laser stimulation. For stimulation of vmPFC terminals expressing *ChR2* to determine DRN neuronal activation, the day before the stimulation mice were connected to the laser and housed in home cages overnight. The following day we performed sustained blue light stimulation at 25 Hz with 10 ms pulse width for 20 min without disturbing the mouse. For stimulation of *ChR2* during CSDS mice were connected to the laser after physical defeat and we performed sustained blue light stimulation at 25 Hz and 10 ms pulse width during 20 min of sensory contact. For stimulation of *Arch* during CSDS we performed constant yellow light stimulation for 20 min.

For stimulation of brain slices expressing *ChR2* in vmPFC terminals, a prepared 200μm core, 0.37 NA standard multimode fiber was lowered into the recording chamber and submerged below ACSF. The tip of the fiber was positioned approximately 1 mm from the vmPFC or DRN, illuminating the entire region. Stimulation of the DRN was either performed at 0.5 Hz with a 10 ms pulse width for an 8 s epoch with 22 s between sweeps or at 25 Hz with a 5 ms pulse width for a 20 s epoch with 10 s between sweeps. Stimulation of the vmPFC was performed at either 5, 25, or 100 Hz with a 5 ms pulse width for a 2 s epoch with 18 s between sweeps. Laser intensity was estimated to be 18.07 mW mm-2.

## **DATA ANALYSIS AND STATISTICS**

For multiple group comparisons, all variables were distributed normally based on Bartlett's test and analyzed using parametric statistics (i.e., One-, Two-Way ANOVAs, between group or with repeated measures, followed by Fisher's PLSD test where appropriate). Comparisons between two groups were performed using Student's *t*-test. Statistical analysis was performed using Statistica (StatSoft, Tulsa, OK). To calculate spatial correlation between *SynP* and *tdTomato* fluorescence, the Pearson correlation coefficient (Pearson's r) was calculated. To determine rate of cumulative time spent per second, the slope of the linear regression and goodness of fit (*r*2) was calculated. Statistical significance was defined as a *p* value *<* 0.05. All data are presented as the mean ± s.e.m. Outlying values (3 standard deviations from the mean) were excluded from group means.

# **RESULTS**

## **EXCITATORY vmPFC TERMINALS AND GABAergic NEURONS IN THE DRN HAVE OVERLAPPING TOPOGRAPHIC DISTRIBUTIONS**

To assess the distribution of vmPFC axon terminals in the DRN we performed viral mediated tracing using a *Cre*-dependent AAV vector coding for a *GFP*-tagged variant of the synaptic protein *Synaptophysin* (*SynP*) (Veerakumar et al., 2013). To selectively target excitatory neurons in the vmPFC (Lee et al., 2003; Commons et al., 2005) the vector was injected in male mice of the *CaMKIIa-Cre* line (Calhoun et al., 1996) (**Figure 1A**). We then assessed the distribution of excitatory vmPFC terminals by visualizing *SynP-GFP* fluorescence in the DRN (**Figure 1B**). The distribution pattern of vmPFC terminals shows a striking similarity to images from the Allen Brain Connectivity Atlas after injetion of an AAV expressing *EGFP* in the vmPFC (**Figure 1C**). To determine whether synapses formed by these terminals occur preferentially in areas enriched in 5-HT or GABA neuron subtypes, we compared the topographic distribution of *SynP-GFP* punctas with that of genetically labeled GABA (*GAD2-tdTomato*) or 5-HT (*Pet1-tdTomato*) neurons at similar rostro-caudal levels (Challis et al., 2013) (**Figure 2**). We found that GABA neurons tended to be primarily distributed in the lateral aspects of the DRN, while 5-HT neurons were concentrated in the midline in the anterior and posterior DRN and were in the midline as well as branched to the dorsolateral DRN, or lateral wings (Crawford et al., 2010), in the mid DRN. Glutamatergic vmPFC terminals on the other hand clustered in the dorsolateral and ventrolateral DRN in the anterior to mid DRN before gathering in the dorsomedial and ventromedial DRN of the most posterior slices. We compared the relative fluorescent intensity of *SynP-GFP* with the intensities of *GAD2-tdTomato* or *Pet1-tdTomato* signals to determine if there was a topographic correlation in the DRN (**Figure 3**). Scatter plots summarize the correlation found *SynP-GFP* intensity and either *GAD2-tdTomato* (**Figure 3B**) or *Pet1-tdTomato* (**Figure 3C**) intensity. We found that throughout the DRN, distribution of vmPFC terminals correlated more strongly with GABA neurons than with 5-HT neurons except in the most caudal extent of the DRN as determined by calculation of Pearson correlation coefficients [Number of mice (slices per mouse) = 3(6); **Figure 3D**].

**FIGURE 1 | Visualization of vmPFC axon terminals in the DRN (A)** An AAV vector coding for the *Cre*-dependent expression of the fluorescently tagged synaptic protein *Synaptophysin* (*SynP-GFP*) was injected in the vmPFC of *CaMKIIa-Cre* mice. Topographic distribution of vmPFC axon terminals in the DRN was visualized using confocal microscopy. **(B)** Distribution of vmPFC terminals as determined by *SynP-GFP* expression as reported in this study. Scale bar 50μm. **(C)** Anterograde tracing using an AAV-GFP vector injected in the vmPFC as reported by the Allen Brain Connectivity Atlas. Picture is courtesy of the Allen Mouse Brain Atlas [Internet]. ©2012 Allen Institute for Brain Science (Seattle, WA). Available from: http://mouse.brain-map.org. Despite use of different tracing methods, the pattern of innervation revealed is strikingly similar, with sparse innervation of the midline area of the DRN, classically containing 5-HT neurons, and denser innervation of lateral portions of the DRN. Scale bar 50μm.

## **DESCENDING EXCITATORY PROJECTIONS FROM THE vmPFC PREFERENTIALLY DRIVE DRN** *cFOS* **INDUCTION IN GABAergic NEURONS**

Using immediate early gene mapping, we previously established that exposure to CSDS activates DRN GABA neurons preferentially over 5-HT neurons and that the topographic distribution of these neurons overlaps with that of vmPFC terminals (Challis et al., 2013). Here, we tested whether direct activation of the terminals would increase *cFos* primarily in GABA neurons. We did this by stereotaxic infusion of an AAV vector leading to *CaMKIIa*-driven expression of *YFP*-tagged *Channelrhodopsin-2* (*ChR2-YFP*) in the vmPFC (Ji and Neugebauer, 2012) (**Figures 4A,B**). Previous studies have shown that this approach restricts expression chiefly to pyramidal neurons (Tsien et al., 1996). Twenty-eight days after surgery we observed robust expression of *ChR2-YFP* in the vmPFC that spread through infralimbic (IL) and prelimbic (PL) regions. We confirmed the expression and function of *ChR2* in the vmPFC by performing currentclamp recordings of *YFP*<sup>+</sup> neurons during exposure to trains of pulsed light (**Figure 4C**). Photostimulation frequencies from 5 Hz up to 25 Hz resulted in pulse-locked action potentials, however at 100 Hz, a stimulation frequency similar to that of deep brain stimulation (DBS), this fidelity was lost. To then stimulate terminals directly in the DRN, we implanted cannulae targeting the DRN 3 weeks after injection (**Figure 4D**). Three days before stimulation fiber optic ferrules were inserted in the cannulae and secured to the skull. The day prior to testing, mice were connected to the laser via fiber optic patch cable and remained isolated in home cages overnight. On the day of testing we performed laser stimulation without disturbing the mice to prevent activation by handling. We used a selective photoexcitation protocol of vmPFC axon terminals in the DRN similar to an approach that has previously been shown to produce robust time-locked behavioral effects dependent on the resulting local release of glutamate in the DRN (Warden et al., 2012). Here, photostimulation of the vmPFC terminals in the DRN for 20 min (473 nm, 10 mW, 25 Hz, 10 ms pulse width) resulted in a significant overall increase in *cFos* expression compared to unstimulated controls [Student's *t*-test, *t(*10*)* = 14*.*89, *p <* 0*.*001; *n* = 6–8 per group; **Figures 4E,F**]. In *GAD2-tdTomato* and *Pet1-tdTomato* mice, this stimulation protocol led to significantly higher activation of *GAD2*- over *Pet1* labeled neurons [Two-Way ANOVA, genotype × stim, *F(*3*,* <sup>13</sup>*)* = 102*.*07, *p <* 0*.*001, **Figures 4G,H**]. Control mice that were connected to the laser, but not stimulated did not display an increase in *cFos* immunoreactivity. Mice that were injected with sham virus also did not display an increase in *cFos* expression (data not shown). This outcome, in line with previous neuroanatomical and ultrastructural data, implicates GABAergic neurons as the primary postsynaptic targets of vmPFC afferents in the DRN.

## **PHOTOSTIMULATION OF vmPFC TERMINALS IN DRN DRIVES TIMED-LOCKED AMPA-MEDIATED POSTSYNAPTIC RESPONSES IN GABAergic BUT NOT 5-HT NEURONS**

To determine if the vmPFC drives synaptic activity of GABA neurons in the DRN we again injected *CaMKIIa*-driven *ChR2* into the vmPFC of *GAD2-tdTomato* mice. After 6 weeks, we then prepared slices of the DRN for whole-cell patch clamp

column 1) and 5-HT (*Pet1-tdTomato*, column 2) neurons as well as antibody

was overlaid on a map using the aqueduct as a reference. Scale bar 50 μm.

strong correlation, *r* = 0 signifying no correlation and *r* = −1 signifying strong negative correlation. **(D)** Overall there is a stronger correlation of topographic distribution of GABA neurons with vmPFC terminals than 5-HT neurons except in the most caudal DRN. (Bars with asterisks indicate Pearson coefficients that are significantly non-zero; <sup>∗</sup>*p <* 0*.*05, ∗∗*p <* 0*.*005, ∗∗∗*p <* 0*.*001).

electrophysiology and recorded from genetically labeled *GAD2*<sup>+</sup> GABA neurons (**Figure 5A**). Brief pulses of 473 nm laser stimulation (0.5 Hz, 10 mW, 10 ms pulse width) resulted in pulse-locked EPSC events (**Figure 5C**) that remained in high fidelity up to 25 Hz (**Figure 5D**). In the presence of DNQX these events disappeared, indicating that the recorded excitatory events were mediated by AMPA receptors (**Figure 5B**). Comparing laser-evoked EPSCs to spontaneous events revealed significant differences in event rise time [Student's *t*-test, *t(*12*)* = 4*.*56, *p <* 0*.*001; number of mice (number of neurons) = 2(12)] and decay time [Student's *t*-test, *t(*12*)* = 2*.*16, *p <* 0*.*05] and trends toward significance in event amplitude and charge transfer (**Figure 5B** and **Table 1**). These differences indicated that the photostimulation of vmPFC fibers resulted in a unique postsynaptic response that was distinguishable from spontaneous quantal release. Using these stimulation parameters we were able to record postsynaptic responses in 25% of the recorded GABA neurons (12 total neurons). In contrast, recording from identified 5-HT neurons in *Pet1-tdTomato* mice did not yield any stimulated postsynaptic responses (12 total neurons; **Figure 5E**). These results reinforce the premise that the vmPFC sends glutamatergic projections directly to GABAergic neurons in the DRN.

## **PHOTOACTIVATION AND PHOTOINHIBITION OF vmPFC TERMINALS IN THE DRN DURING POST-DEFEAT SENSORY CONTACT PERIOD HAS OPPOSITE EFFECTS ON AVOIDANCE BEHAVIOR**

We have previously demonstrated that inhibition of DRN GABAergic neurons prevents the acquisition of social avoidance after defeat, but did not change expression of an already acquired avoidance phenotype (Challis et al., 2013). To determine whether vmPFC terminals that drive GABA neurons' activity in the DRN also contribute to the encoding of social aversion, we expressed optogenetic probes in *CaMKIIa-Cre* neurons in the vmPFC and photostimulated or photoinhibited terminals directly in the DRN. To activate glutamatergic vmPFC projections, we used *ChR2* (473 nm) and to inhibit we expressed *Archaerhodopsin* (*Arch*, 543 nm). Because we have previously shown that a period of 20 min of post-defeat sensory exposure is necessary and sufficient to trigger a significant avoidance response (Challis et al., 2013), mice were connected to the laser via fiber optic connector and stimulated daily during this period before returning to home cages overnight (**Figure 6A**). This was repeated for 10 days with exposure to a novel CD1 aggressor mouse every day. On day 11, approach-avoidance choices were evaluated by performing the social interaction test using a novel social target (**Figure 6B**).

Mice from the control group injected with a sham vector and receiving laser stimulation in the DRN displayed interaction times similar to that previously reported in defeated naïve mice indicating that the cannulation and potential thermal artifacts caused by laser manipulation, do not *per se* significantly alter the development of social avoidance (Challis et al., 2013) (**Figure 6C**). In contrast, mice whose vmPFC terminals were photoinhibited in the DRN did not display typical social avoidance and maintained high levels of approach during social interaction testing [Two-Way ANOVA, virus × stim, *F(*11*,* <sup>50</sup>*)* = 6*.*58, *p <* 0*.*001; *n* = 6–10 mice per group]. On the other hand, defeated mice whose vmPFC terminals were photoactivated tended to show reductions in social interaction compared to mice injected with sham virus, although this difference did not reach statistical significance due to a floor effect on the expression of social avoidance. Interestingly, control mice that did not undergo defeat, but received photoactivation

of vmPFC terminals in the DRN in the presence of a CD1 social target also subsequently displayed a significant decrease in time spent [Two-Way ANOVA, virus × stim, *F(*<sup>11</sup> *,*50*)* = 4*.*50, *p* = 0*.*002; **Figure 6D**] and in total entries in the social interaction zone [Two-Way ANOVA, virus × stim, *F(*11*,* <sup>50</sup>*)* = 6*.*85, *p <* 0*.*001; **Figure 6E**].

## **INCREASED vmPFC DRIVE OF DRN DELAYS DECISION TO APPROACH NOVEL SOCIAL TARGET**

To gain further insight into how manipulation of vmPFC-DRN during CSDS training alters subsequent avoidance behaviors, we examined the effect of this manipulation on the time-course of social approach-avoidance behaviors during the interaction test. We first characterized the temporal distribution of the bouts of interaction during the course of the tests in a large cohort (*n* = 117) of unimplanted control and defeated mice, stratified as "resilient" or "susceptible" as previously reported (Krishnan et al., 2007; Golden et al., 2011; Challis et al., 2013). Examining the cumulative time spent in the social interaction and corner zones we found that the behavior of susceptible mice significantly diverged from control and resilient as early as 4 s into the test (Repeated measures ANOVA, defeat × time, *F(*298*,* <sup>15049</sup>*)* = 49*.*894, *p <* 0*.*001; **Figure 7A**). Many mice in the latter two groups entered the social interaction zone immediately, with almost all entering under 40 s (**Figure 7B**), and continued to investigate the social target throughout the entire duration of the trial such that average interaction time accrued quasi-linearly in these groups (Linear regression, slope in cumulative time in seconds/second elapsed = 0.464 ± 0.005 for control, 0.4558 ± 0.004 for resilient, *r*<sup>2</sup> = 0*.*741 for control, 0.616 for resilient; **Figure 7A**). In contrast, susceptible mice considerably delayed their decision to first enter the social interaction zone compared to resilient mice (under 50% had entered by 40 s; Kolmogorov-Smirnov test, *p <* 0*.*001; **Figure 7B**) and rarely returned to interaction zone after their first entry (Linear regression, slope = 0.1482 ± 0.002, *r*<sup>2</sup> = 0*.*436 for susceptible; **Figure 7A**). Together, we interpret these data as an indication that the interindividual variability during the social interaction test reflects the execution of a binary choice between two behavioral strategies made a few seconds after the initiation of the task.

We applied the same time-course analysis to the dataset obtained from undefeated control mice receiving chronic photostimulation of vmPFC terminals during sensory exposure to novel aggressor mice. We found that the behavioral profile of undefeated mice that were implanted, but not stimulated, followed the same behavioral approach pattern as unimplanted control To characterize light-evoked ESPC activity, 20μM DNQX was applied to the bath to block AMPA receptor activity. All drugs were made in stock solutions, diluted on the day of the experiment, and added directly to the ACSF.

## **ELECTROPHYSIOLOGY DATA ANALYSIS**

Synaptic properties were analyzed using MiniAnalysis (Synaptosoft, Decatur, GA) as previously described (Crawford et al., 2011, 2013). Synaptic events were analyzed using parameters optimized for each cell with the detection threshold set beyond the maximum values of the all-points noise histogram for a portion of the trace containing no detectable synaptic events. This threshold generally ranged from 5 to 8 pA. MiniAnalysis generates a summary table containing the mean and median values for the frequency, amplitude, rise time (10–90%), decay time, and event half width (50%). For each cell, at least 200 events were chosen at random and manually filtered to exclude multiple peaks then combined to obtain an averaged EPSC or IPSC for each cell to obtain values for decay time, event area, and event time half-width. Additional statistical analysis is described below. Data reported are means ± s.e.m.

#### **OPTICAL STIMULATION**

For *in vivo* stimulation, mice with previously implanted fiber optic ferrules were connected to a 200μm, 0.37 NA patch cord via zirconia sleeve that was then connected to a diode-pumped solidstate (DPSS) laser through an FC/PC adaptor and rotary joint. We used blue (473 nm, BL-473-00100-CWM-SD-05-LED-0) and yellow (561 nm, GR-561-00100-CWM-SD-05-LED-F) DPSS lasers obtained from OEM Laser Systems (Bluffdale, UT). Power output was measured using an optical sensor (Thorlabs, Newton, NJ) to be about 10 mW. Intensity was calculated using a model predicting irradiance in mammalian tissues (http://www*.*stanford*.* edu/group/dlab/cgi-bin/graph/chart*.*php). From a 200μm fiber optic tip, estimated intensity was 7.33 mW mm−<sup>2</sup> for blue laser stimulation and 7.05 mW mm−<sup>2</sup> for yellow laser stimulation. For stimulation of vmPFC terminals expressing *ChR2* to determine DRN neuronal activation, the day before the stimulation mice were connected to the laser and housed in home cages overnight. The following day we performed sustained blue light stimulation at 25 Hz with 10 ms pulse width for 20 min without disturbing the mouse. For stimulation of *ChR2* during CSDS mice were connected to the laser after physical defeat and we performed sustained blue light stimulation at 25 Hz and 10 ms pulse width during 20 min of sensory contact. For stimulation of *Arch* during CSDS we performed constant yellow light stimulation for 20 min.

For stimulation of brain slices expressing *ChR2* in vmPFC terminals, a prepared 200μm core, 0.37 NA standard multimode fiber was lowered into the recording chamber and submerged below ACSF. The tip of the fiber was positioned approximately 1 mm from the vmPFC or DRN, illuminating the entire region. Stimulation of the DRN was either performed at 0.5 Hz with a 10 ms pulse width for an 8 s epoch with 22 s between sweeps or at 25 Hz with a 5 ms pulse width for a 20 s epoch with 10 s between sweeps. Stimulation of the vmPFC was performed at either 5, 25, or 100 Hz with a 5 ms pulse width for a 2 s epoch with 18 s between sweeps. Laser intensity was estimated to be 18.07 mW mm-2.

## **DATA ANALYSIS AND STATISTICS**

For multiple group comparisons, all variables were distributed normally based on Bartlett's test and analyzed using parametric statistics (i.e., One-, Two-Way ANOVAs, between group or with repeated measures, followed by Fisher's PLSD test where appropriate). Comparisons between two groups were performed using Student's *t*-test. Statistical analysis was performed using Statistica (StatSoft, Tulsa, OK). To calculate spatial correlation between *SynP* and *tdTomato* fluorescence, the Pearson correlation coefficient (Pearson's r) was calculated. To determine rate of cumulative time spent per second, the slope of the linear regression and goodness of fit (*r*2) was calculated. Statistical significance was defined as a *p* value *<* 0.05. All data are presented as the mean ± s.e.m. Outlying values (3 standard deviations from the mean) were excluded from group means.

# **RESULTS**

## **EXCITATORY vmPFC TERMINALS AND GABAergic NEURONS IN THE DRN HAVE OVERLAPPING TOPOGRAPHIC DISTRIBUTIONS**

To assess the distribution of vmPFC axon terminals in the DRN we performed viral mediated tracing using a *Cre*-dependent AAV vector coding for a *GFP*-tagged variant of the synaptic protein *Synaptophysin* (*SynP*) (Veerakumar et al., 2013). To selectively target excitatory neurons in the vmPFC (Lee et al., 2003; Commons et al., 2005) the vector was injected in male mice of the *CaMKIIa-Cre* line (Calhoun et al., 1996) (**Figure 1A**). We then assessed the distribution of excitatory vmPFC terminals by visualizing *SynP-GFP* fluorescence in the DRN (**Figure 1B**). The distribution pattern of vmPFC terminals shows a striking similarity to images from the Allen Brain Connectivity Atlas after injetion of an AAV expressing *EGFP* in the vmPFC (**Figure 1C**). To determine whether synapses formed by these terminals occur preferentially in areas enriched in 5-HT or GABA neuron subtypes, we compared the topographic distribution of *SynP-GFP* punctas with that of genetically labeled GABA (*GAD2-tdTomato*) or 5-HT (*Pet1-tdTomato*) neurons at similar rostro-caudal levels (Challis et al., 2013) (**Figure 2**). We found that GABA neurons tended to be primarily distributed in the lateral aspects of the DRN, while 5-HT neurons were concentrated in the midline in the anterior and posterior DRN and were in the midline as well as branched to the dorsolateral DRN, or lateral wings (Crawford et al., 2010), in the mid DRN. Glutamatergic vmPFC terminals on the other hand clustered in the dorsolateral and ventrolateral DRN in the anterior to mid DRN before gathering in the dorsomedial and ventromedial DRN of the most posterior slices. We compared the relative fluorescent intensity of *SynP-GFP* with the intensities of *GAD2-tdTomato* or *Pet1-tdTomato* signals to determine if there was a topographic correlation in the DRN (**Figure 3**). Scatter plots summarize the correlation found *SynP-GFP* intensity and either *GAD2-tdTomato* (**Figure 3B**) or *Pet1-tdTomato* (**Figure 3C**) intensity. We found that throughout the DRN, distribution of vmPFC terminals correlated more strongly with GABA neurons than with 5-HT neurons except in the most caudal extent of the DRN as determined by calculation of Pearson correlation coefficients [Number of mice (slices per mouse) = 3(6); **Figure 3D**].

back in homecages and allowed 6 weeks to recover. Body weight and behavior was monitored during recovery. Three days before the start of experiment, a homemade fiber optic with ferrule connector (described below) was inserted into the guide cannula and secured with acrylic cement.

## **PREPARATION OF OPTICAL FIBERS**

A Two hundredμm core, 0.37 NA standard multimode fiber (Thorlabs, Newton, NJ) was stripped of cladding, passed through a 230μm multimode ceramic zirconia ferrule (Precision Fiber Products, Milpitas, CA), and secured in place using fiber optic connector epoxy (Fiber Instrument Sales, Oriskany, NY). Ferrules were then polished and cut to length to target the DRN. They were tested for light output and sterilized with 70% ethanol.

## **CHRONIC SOCIAL DEFEAT STRESS**

We use a modified chronic social defeat stress (CSDS) paradigm to induce social avoidance (Golden et al., 2011; Challis et al., 2013). Our model consists of exposing male mice to alternating periods of physical contact with a trained CD1 aggressor male mouse (5 min) and protected sensory contact via separation by a perforated Plexiglass partition (20 min) before returning to home cages overnight. The 20 min of sensory contact is sufficient to induce a significant decrease in social interaction compared to undefeated mice or mice that were not exposed to a sensory period after physical contact. This effect has been previously described Challis et al. (2013). This continued for 10 consecutive days with exposure to a novel aggressor each day. Control animals were also singly housed and were only exposed to daily sensory contact with novel aggressors. On day 11, social approach or avoidance behavior toward an unfamiliar CD1 social target was assessed in a two-trial social interaction task. In the first 2.5-min trial ("no target"), experimental mice explored a dimly lit (55 lux) open-field arena containing an empty wire mesh cage on one edge of the arena (see **Figure 6A**). In the second 2.5-min trial ("target present"), experimental mice were reintroduced to the arena now with an unfamiliar CD1 aggressor positioned in the mesh cage. TopScan video tracking software (CleverSys, Reston, VA) was used to measure the time spent in the interaction zone surrounding the target box.

## **IMMUNOHISTOCHEMISTRY**

Animals were transcardially perfused with 4% paraformaldehyde and brains were processed for standard single or dual immunolabeling methods as previously described (Espallergues et al., 2012). For detection of *cFos*, we used an affinity purified rabbit polyclonal antibody raised against the N-terminus of human *cFos* (1:1000 dilution; SC-52, Santa Cruz Biotechnology, Santa Cruz, CA). To enhance *GFP* expression we used a chicken anti-*GFP* antibody (1:1000 dilution; GFP-1020, Aves Labs, Inc., Tigard, OR). Primary antibodies were detected using fluorescent secondary antibodies obtained from Jackson ImmunoResearch Laboratories (1:500 dilution; West Grove, PA).

#### **CELL COUNTING**

To map neuronal populations in the DRN, 30μm serial sections of the DRN were collected every 120μm between −4.36 mm and −4.96 mm from Bregma. Native *tdTomato* fluorescence and immuno-enhanced *GFP* fluorescence of *SynP* labeled vmPFC terminals were visualized using confocal microscopy. Slices from corresponding rostro-caudal levels between mice were aligned on a map based on location of the aqueduct. Neurons and terminals were manually drawn for each level of the DRN.

To quantify *cFos* colocalization with *tdTomato*<sup>+</sup> neurons, slices were stained for *cFos* and labeled neurons were manually counted in the DRN of each section. Colocalization with *tdTomato* was defined as nuclear localization of the *cFos* signal and was manually counted by an experimenter blind to the experimental condition of the mice from which the slices originated. There was not a significant variation of total number of *tdTomato*<sup>+</sup> cells within each strain.

To determine whether spatial distribution of synaptic vmPFC inputs traced using *SynP-GFP* correlated with the distribution of *GAD2-tdTomato* or *Pet1-tdTomato* neurons, we divided corresponding coronal views of the DRN in *GAD2-tdTomato*, *Pet1 tdTomato* and *SynP-GFP* injected *CaMKIIa-Cre* mice into 10 × 10 grids and tested for correlations between *SynP-GFP* and *tdTomato* fluorescence across the grid. This was done at each of the 6 rostrocaudal levels across the DRN. Fluorescent intensity within each grid box was calculated using the ImageJ "Measure" function which converts red, green, and blue (RGB) pixel values to brightness using the formula V = (R + G + B)/3. These intensity values were then normalized to the grid box with the highest intensity. Correlations were tested using the Pearson coefficient and plotted using linear regression.

## **ELECTROPHYSIOLOGY**

Brain slices were prepared as previously described (Crawford et al., 2010, 2013; Calizo et al., 2011; Espallergues et al., 2012; Challis et al., 2013; Howerton et al., 2013). The 200μm coronal slices containing DRN were placed in aCSF (in mM, NaCl 124, KCl 2.5, NaH2PO4 1.25, MgSO4 2.0, CaCl2 2.5, dextrose 10, NaHCO3 26) at 37◦C, aerated with 95% O2/5% CO2. After 1 h, slices were kept at room temperature. Tryptophan (2.5 mM) was included in the holding chamber to maintain 5-HT synthesis, but was not in the aCSF perfusing the slice in the recording chamber. Individual slices were placed in a recording chamber (Warner Instruments, Hamden, CT) and perfused with aCSF at 2 ml/min maintained at 32◦C by an in-line solution heater (TC-324, Warner Instruments). Neurons were visualized using a Nikon E600 upright microscope fitted with a 60X water immersion objective and targeted under DIC or fluorescent filters. Resistance of electrodes was about 8–10 MOhms when filled with a recording solution composed of (in mM) K-gluconate (130), NaCl (5), Na phosphocreatine (10), MgCl2 (1), EGTA (0.02), HEPES (10), MgATP (2) and Na2GTP (0.5) with 0.1% biocytin and a pH of 7.3. Whole-cell recordings were obtained using a Multiclamp 700 B amplifier (Molecular Devices, Sunnyvale, CA). Cell characteristics were recorded using current clamp techniques as previously described (Crawford et al., 2010; Espallergues et al., 2012). Signals were collected and stored using Digidata 1320 analog-to-digital converter and pClamp 9.0 software (Molecular Devices). Collection of EPSC data was as previously described (Crawford et al., 2011) and performed with bath application of 20μM bicuculline to block GABA synaptic activity. histological tracing studies demonstrated that DRN GABAergic neurons that are preferential targets of vmPFC projections could mediate the inhibitory responses recorded *in vivo* (Jankowski and Sesack, 2004). However, due to the limited specificity of electrophysiological signatures to predict neurochemical cell-type (Calizo et al., 2011), the identities of neuronal populations that compose the vmPFC-DRN microcircuit have not been fully elucidated. Furthermore, there is a lack of information about the possible topographical distribution of various DRN cellular populations thereby limiting the progress of studies assessing their causal role in socioaffective responses and other behaviors.

In recent studies we used a murine model of chronic social defeat stress (CSDS) that induces long lasting avoidance bias responsive to antidepressants to characterize the role of DRN microcircuits in the development and expression of social aversion (Espallergues et al., 2012; Challis et al., 2013; Crawford et al., 2013; Veerakumar et al., 2013). In mice susceptible to CSDS, but not in ones resilient, we detected a sustained sensitized synaptic inhibition of DRN 5-HT neurons, associated with a state of dramatically reduced intrinsic excitability of 5-HT neurons. Furthermore, we identified a subset of *GAD2*<sup>+</sup> GABA neurons with sensitized excitatory synaptic input and intrinsic excitability, which monosynaptically inhibits nearby 5-HT neurons. Using optogenetic photo silencing we provided evidence of their key role in the associative process that underlie the development of social avoidance in susceptible mice (Challis et al., 2013). Interestingly, we noted that these sensitized GABAergic neurons appear to be located in circumscribed lateral subregions of the DRN heavily innervated by the vmPFC. These observations suggest a potentially unique role of inputs from the vmPFC in driving stress-induced plasticity of GABA neurons within the DRN that underlie the stabilization of avoidance bias after CSDS.

In the present study, we set out to test this hypothesis. We used *in vivo* optogenetics to drive or inhibit the synaptic inputs from vmPFC axons locally within the DRN during the sensory contact phase of CSDS. We also used viral tracing, whole-cell recordings, and optogenetic methods in slice preparations to further characterize the anatomical and functional organization of the vmPFC-DRN pathway. Our results directly show that excitatory projections from the vmPFC preferentially target and synaptically activate GABA neurons that are topographically distributed within the DRN. We also show that activation of these terminals paired temporally with exposure to social cues potentiates negative socioaffective bias and social avoidance, while inhibition of these inputs facilitates the maintenance of social engagement after defeat, a characteristic of resilient individuals. These results provide fundamentally novel insights about neural mechanisms implicated in the top-down control of 5-HT during socioaffective tasks and have important implications for the understanding and treatment of affective disorders.

## **MATERIALS AND METHODS**

#### **ANIMALS**

Eight- to twelve-week old male mice bred onto a C57BL/6 background were used for all experiments. Mice were housed on a 12-h light/dark cycle with food and water available *ad libitum*. All studies were conducted according to protocols approved by the University of Pennsylvania Institutional Animal Care and Use Committee. All procedures were performed in accordance with institutional guidelines. The large cohort of defeated mice used to determine social choice consisted of male C57 Black mice (*C57BL/6J*: JAX stock number 000664). Trained aggressor mice were retired CD-1 male breeder mice (*Crl:CD1*; Charles River Laboratories, Malvern, PA). To generate a mouse line with fluorescently labeled *GAD65*-containing GABAergic or serotonergic neurons, male knocking *GAD2-Cre* mice (*Gad*2tm2*(*cre*)*Zjh*/J*; JAX stock number 010802) (Taniguchi et al., 2011) or BAC transgenic *Pet1-Cre* mice (*B6.Cg-Tg(Fev-cre*)1Esd/J; JAX stock number 012712) (Scott et al., 2005) were respectively crossed to female floxed-stop controlled *tdTomato* (RFP variant) mice (*B6.Cg-Gt(ROSA)26Sor*tm9*(*CAG−tdTomato*)*Hze*/J*; JAX stock number 007908) (Madisen et al., 2010) to achieve fluorescent labeling of *Cre* containing cells. To achieve expression of optogenetic probes or fluorescent tracers in glutamatergic vmPFC neurons we used *CaMKIIa-Cre* mice (*B6.Cg-Tg(CamK2a-Cre)T29-1Stl/J*; JAX stock number 005359) (Tsien et al., 1996). With the exception of the CD-1 strain, all mice were procured from the Jackson Laboratory (Bar Harbor, ME).

## **VIRUS AND SURGERY**

To express optogenetic or fluorescent proteins in glutamatergic neurons, adeno-associated virus (AAV) vectors were produced by and purchased from the University of Pennsylvania vector core (Philadelphia, PA) and injected into *CaMKIIa-Cre* mice. In this work we used AAVs for the *Cre*-inducible expression of the excitatory optogenetic probe *Channelrho dopsin* (AAV2/9.EF1a.DIO.hChR2(H134R)-EYFP.WPRE.hGH; Addgene #20298), inhibitory optogenetic probe *Archaerhodopsin* (AAV2/9.flex.CBA.Arch-GFP.W.SV40; Addgene #22222), fluorescent protein *tdTomato* (AAV2/1.CAG.FLEX.tdTomato.WPRE. bGH; Allen Institute #864) and *GFP* tagged Synaptophysin (AAV2/9.CMV.FLEX.Synaptophysin-Venus.WPRE.hGH; plasmid kindly provided by Anton Maximov, PhD, Department of Molecular and Cellular Neuroscience, The Scripps Research Institute). For stimulation of excitatory vmPFC terminals in the DRN of *GAD2-tdTomato* or *Pet1-tdTomato* mice we used an AAV for the *CaMKIIa-*driven expression of *Channelrhodopsin* fused to *YFP* (AAV2/9.CaMKII.ChR2-YFP.SV40; Stanford) (Mattis et al., 2012).

For viral injections, mice were anesthetized with isofluorane and stereotaxically injected unilaterally in the prelimbic region of the vmPFC (from Bregma, in mm: +1.8 AP, +0.8 ML, −2.7 DV, 15◦ angle) with 0.5μl of virus. Viral yields (in GC) were 3*.*54 × 10<sup>12</sup> for *ChR2-YFP,* 6*.*962 × 10<sup>11</sup> for *Arch-GFP*, 2*.*049 × 10<sup>12</sup> for *tdTomato* and 4*.*347 × 10<sup>12</sup> for *CaMKIIa-ChR2-YFP*. Social defeat began 4 weeks post-surgery for non-cannulated mice to allow time for recovery and viral expression.

For *in vivo* optical stimulation, precut guide cannulae (Plastics One, Roanoke, VA) targeting the DRN (from Lambda, in mm: 0.0 AP, +0.8 ML, −3.3 DV, 15◦ angle) were secured to the skull using stainless steel skull screws and acrylic cement. A fitted dustcap dummy was secured atop the guide cannula and mice were placed

# Optogenetic modulation of descending prefrontocortical inputs to the dorsal raphe bidirectionally bias socioaffective choices after social defeat

# *Collin Challis 1,2, Sheryl G. Beck2,3 and Olivier Berton1,2\**

*<sup>1</sup> Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA*

*<sup>2</sup> Neuroscience Graduate Group, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA*

*<sup>3</sup> Department of Anesthesiology, Children's Hospital of Philadelphia and University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA*

#### *Edited by:*

*Mary K. Lobo, University of Maryland School of Medicine, USA*

#### *Reviewed by:*

*Daniel W. Wesson, Case Western Reserve University, USA Xiao-Dong Wang, Zhejiang University, China*

#### *\*Correspondence:*

*Olivier Berton, Department of Psychiatry, Center for Neurobiology and Behavior, University of Pennsylvania Perelman School of Medicine, 125 S. 31st Street, Philadelphia, PA 19104, USA e-mail: bertonol@ mail.med.upenn.edu*

It has been well established that modulating serotonin (5-HT) levels in humans and animals affects perception and response to social threats, however the circuit mechanisms that control 5-HT output during social interaction are not well understood. A better understanding of these systems could provide groundwork for more precise and efficient therapeutic interventions. Here we examined the organization and plasticity of microcircuits implicated in top-down control of 5-HT neurons in the dorsal raphe nucleus (DRN) by excitatory inputs from the ventromedial prefrontal cortex (vmPFC) and their role in social approach-avoidance decisions. We did this in the context of a social defeat model that induces a long lasting form of social aversion that is reversible by antidepressants. We first used viral tracing and *Cre*-dependent genetic identification of vmPFC glutamatergic synapses in the DRN to determine their topographic distribution in relation to 5-HT and GABAergic subregions and found that excitatory vmPFC projections primarily localized to GABA-rich areas of the DRN. We then used optogenetics in combination with *cFos* mapping and slice electrophysiology to establish the functional effects of repeatedly driving vmPFC inputs in DRN. We provide the first direct evidence that vmPFC axons drive synaptic activity and immediate early gene expression in genetically identified DRN GABA neurons through an AMPA receptor-dependent mechanism. In contrast, we did not detect vmPFC-driven synaptic activity in 5-HT neurons and *cFos* induction in 5-HT neurons was limited. Finally we show that optogenetically increasing or decreasing excitatory vmPFC input to the DRN during sensory exposure to an aggressor's cues enhances or diminishes avoidance bias, respectively. These results clarify the functional organization of vmPFC-DRN pathways and identify GABAergic neurons as a key cellular element filtering top-down vmPFC influences on affect-regulating 5-HT output.

**Keywords: dorsal raphe, ventromedial prefrontal cortex, serotonin, optogenetics, electrophysiology, depression and anxiety disorders, social perception, social defeat**

# **INTRODUCTION**

The capacity to detect and interpret the affective state of others using non-verbal social cues (e.g., facial expression, vocal prosody, posture, body movement, and olfactory cues) is a necessary survival skill shared by many animal species (Chang et al., 2013; Oliveira, 2013). It allows individuals to anticipate harmful intentions of others and adapt through rapid approach or avoidance decisions (O'Connell and Hofmann, 2012). The capacity to conduct social-cognitive appraisal is also a determining aspect of human social competence (Todorov, 2008; Volman et al., 2011) and dysfunction of the neural systems that mediate socioaffective decisions are thought to contribute to excessive reassurance-seeking behaviors and social withdrawal, which are two symptomatic dimensions shared across several affective disorders, including major depression, and social phobia (Heuer et al., 2007; Seidel et al., 2010; Derntl et al., 2011; Stuhrmann et al., 2011; Cusi et al., 2012; Moser et al., 2012).

Serotonin (5-HT) is a neurotransmitter system that plays an evolutionarily conserved role in regulating affiliative and antagonistic behaviors (Canli and Lesch, 2007; Dayan and Huys, 2009; Rogers, 2011). Increases in 5-HT output, such as resulting from treatment with SSRI antidepressants, have consistently been shown to positively bias socioaffective appraisals and facilitate social affiliation and dominance in human and animals (Raleigh et al., 1991; Knutson et al., 1998; Tse and Bond, 2002; Bond, 2005; Harmer and Cowen, 2013). In contrast, 5-HT depletion facilitates socially defensive behaviors and aggression (Young and Leyton, 2002; Munafo et al., 2006). The fact that the output of ascending 5-HT neurons located in the dorsal raphe nucleus (DRN) is under top-down control by multiple forebrain areas (Peyron et al., 1998; Freedman et al., 2000; Chiba et al., 2001; Celada et al., 2002; Lee et al., 2003; Vertes, 2004) suggests a potentially key role for DRN afferent systems in the modulation of socioaffective responses. Studies conducted *in vivo* in anesthetized rodents combining electrical stimulation of the ventromedial prefrontal cortex (vmPFC) and extracellular recordings in the DRN demonstrated the rapid inhibition of putative 5- HT neurons (Varga et al., 2001; Celada et al., 2002). Parallel


dorsal raphe nucleus using juxtacellular labelling methods. *Eur. J. Neurosci.* 25, 119–126. doi: 10.1111/j.1460-9568.2006.05276.x


prefrontal-amygdala connectivity while viewing facial signals of aggression. *Biol. Psychiatry* 71, 36–43. doi: 10.1016/j.biopsych.2011.07.033


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 09 December 2013; paper pending published: 03 January 2014; accepted: 29 January 2014; published online: 17 February 2014.*

*Citation: Challis C, Beck SG and Berton O (2014) Optogenetic modulation of descending prefrontocortical inputs to the dorsal raphe bidirectionally bias socioaffective choices after social defeat. Front. Behav. Neurosci. 8:43. doi: 10.3389/fnbeh.2014.00043 This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2014 Challis, Beck and Berton. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Optogenetics reveals a role for accumbal medium spiny neurons expressing dopamine D2 receptors in cocaine-induced behavioral sensitization

**Shelly Sooyun Song<sup>1</sup>† , Byeong Jun Kang<sup>1</sup>† , Lei Wen2,3,4† , Hyo Jin Lee<sup>1</sup> , Hye-ri Sim<sup>1</sup> , Tae Hyong Kim<sup>1</sup> , Sehyoun Yoon<sup>1</sup> , Bong-June Yoon<sup>1</sup> , George J. Augustine2,3,4 and Ja-Hyun Baik<sup>1</sup>\***

<sup>1</sup> Department of Life Sciences, Molecular Neurobiology Laboratory, School of Life Sciences and Biotechnology, Korea University, Seoul, South Korea

<sup>2</sup> Center for Functional Connectomics (CFC), KIST, Seoul, South Korea

<sup>3</sup> Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore

4 Institute of Molecular and Cell Biology, Singapore, Singapore

#### **Edited by:**

Anton Ilango, Leibniz Institute for Neurobiology, Germany

#### **Reviewed by:**

Simona Cabib, Università "Sapienza", Roma, Italy Vincent Pascoli, Geneva University, Switzerland

#### **\*Correspondence:**

Ja-Hyun Baik, Department of Life Sciences, Molecular Neurobiology Laboratory, School of Life Sciences and Biotechnology, Korea University, Nokji Bldg, Rm 402, Anam-dong, Sungbuk-gu, Seoul, 136-713, South Korea e-mail: jahyunb@korea.ac.kr †These authors have contributed

equally to this work.

Long-lasting, drug-induced adaptations within the nucleus accumbens (NAc) have been proposed to contribute to drug-mediated addictive behaviors. Here we have used an optogenetic approach to examine the role of NAc medium spiny neurons (MSNs) expressing dopamine D2 receptors (D2Rs) in cocaine-induced behavioral sensitization. Adeno-associated viral vectors encoding channelrhodopsin-2 (ChR2) were delivered into the NAc of D2R-Cre transgenic mice. This allowed us to selectively photostimulate D2R-MSNs in NAc. D2R-MSNs form local inhibitory circuits, because photostimulation of D2R-MSN evoked inhibitory postsynaptic currents (IPSCs) in neighboring MSNs. Photostimulation of NAc D2R-MSN in vivo affected neither the initiation nor the expression of cocaine-induced behavioral sensitization. However, photostimulation during the drug withdrawal period attenuated expression of cocaine-induced behavioral sensitization. These results show that D2R-MSNs of NAc play a key role in withdrawal-induced plasticity and may contribute to relapse after cessation of drug abuse.

**Keywords: optogenetics, medium spiny neurons, dopamine D2 receptors, cocaine, drug addiction**

#### **INTRODUCTION**

Dopamine (DA) signaling is associated with reward expectation and goal-directed behavior (Wise, 2004; Goto and Grace, 2005; Berridge, 2007). One of the well-known pathologies of dopaminergic disorders is drug addiction (Robinson and Berridge, 1993, 2003). Following repeated exposure to addictive substances, adaptive changes occur at the molecular and cellular level in the DA mesolimbic pathway; these can lead to drug dependence, which is a chronic, relapsing disorder in which compulsive drug-seeking and drug-taking behaviors persist despite their serious negative consequences (Thomas et al., 2008; Baik, 2013). Characterization of the modifications that take place in the mesolimbic dopaminergic system is thus key to understanding drug addiction.

Dopamine D1 receptors (D1R) and D2 receptors (D2R) are highly expressed in the medium spiny neurons (MSNs) of the striatum. It has been suggested that long-lasting drug-induced adaptations in the ventral striatum, better known as the nucleus accumbens (NAc), contribute to the development of addiction as well as drug-seeking and relapse behaviors (Lobo and Nestler, 2011; Smith et al., 2013). Dopaminergic cell bodies from the ventral tegmental area mostly innervate the NAc. Over 95% of the cells within the NAc are MSNs, which receive excitatory inputs from four major brain regions: the prefrontal cortex, the ventral subiculum of the hippocampus, the basolateral amygdala, and the thalamus (Sesack and Grace, 2010; Lüscher and Malenka, 2011). MSNs within the NAc can be divided into two major subpopulations: direct pathway MSNs that express D1Rs and project directly to midbrain DA areas, and indirect pathway MSNs that express D2Rs and project to the ventral pallidum (Kreitzer and Malenka, 2008; Sesack and Grace, 2010; Lüscher and Malenka, 2011; Smith et al., 2013). Because MSN are GABAergic, activation of MSNs neurons will inhibit their downstream targets which are also GABAergic (Chevalier and Deniau, 1990). Therefore, activation of D1R-MSNs will excite midbrain DA neurons, which then contributes to the regulation of reward-related behaviors (Lüscher and Malenka, 2011; Bocklisch et al., 2013).

Recent studies using genetically engineered mice that express Cre recombinase in a cell-type specific manner have revealed different roles for D1R-MSNs and D2R-MSNs in cocaine addiction behaviors. Such mice enable genetic targeting of specific toxins, optogenetic probes or DREADD (designer receptors exclusively activated by a designer drug) to selectively manipulate D1R-MSN or D2R-MSN. This approach has led to some consensus about the role of MSNs in addictive behaviors: D1R-MSNs apparently promote addictive behaviors, while no specific role (or an inhibitory role) in the development of drug-induced addictive behaviors has been suggested for D2R-MSNs (Hikida et al., 2010; Lobo et al., 2010; Ferguson et al., 2011; Bock et al., 2013). Cocaine exposure apparently induces synaptic modification and alterations in gene expression in both MSN populations (Lobo et al., 2010; Lobo and Nestler, 2011; Grueter et al., 2013). Although it appears that D1R-MSNs and D2R-MSNs play opposing roles in cocaine-mediated addictive behaviors, the precise role of D2R-MSNs is not clear.

Previously it has been shown that D2R knockout (KO) mice display normal cocaine-mediated behavioral sensitization and cocaine-seeking behaviors, with only a slight decrease in sensitivity caused by the absence of D2R (Baik et al., 1995; Chausmer et al., 2002; Sim et al., 2013). However, exposure to stress during drug withdrawal suppresses expression of cocaine-induced behavioral sensitization as well as cocaine-seeking and relapse behaviors in D2R KO mice (Sim et al., 2013). Specific knockdown of D2R in the NAc does not affect basal locomotor activity, nor cocaine-induced behavioral sensitization, but does confer the ability of stress to inhibit expression of cocaine-induced behavioral sensitization (Sim et al., 2013). These findings strongly suggest that blockade of D2R in the NAc does not prevent cocaine-mediated behavioral sensitization. Rather, it appears that D2R in the NAc play a distinct role in regulation of the stresstriggered synaptic modifications during withdrawal that lead to an increase in cocaine-seeking and relapse behaviors (Sim et al., 2013).

Here we have used optogenetics to further evaluate the role of NAc D2R-MSNs in cocaine-induced behavioral sensitization. Using brain slices, we find that photostimulation of D2R-MSNs activates local inhibitory circuits within NAc involving neighboring MSNs. Photostimulation of NAc D2R-MSNs *in vivo* affects neither the initiation nor the expression of cocaineinduced behavioral sensitization. However, repetitive activation of NAc D2R-MSNs during the drug withdrawal period attenuates cocaine-induced addictive behavior. Our results show that D2R-MSNs of NAc play a key role in withdrawal-induced plasticity and may contribute to relapse after cessation of drug abuse.

# **MATERIALS AND METHODS**

## **MICE**

D2-Cre BAC transgenic mice on a C57Bl/6 background were obtained from MMRRC (Mutant Mouse Regional Resource Centers, B6.FVB(Cg)-Tg(Drd2-cre)ER44Gsat/Mmucd). In behavioral experiments, littermates lacking the D2-Cre transgene were used as controls for the D2-Cre mice. Mice were maintained in a specific pathogen-free barrier facility under constant conditions of temperature and humidity, and on a 12-h light, 12-h dark schedule. Animal care and handling were performed in accordance with standards approved by the Institutional Animal Care and Use Committees of Korea University and KIST.

## **VIRUS VECTOR PREPARATION**

pAAV-EF1a-DIO-hChR2(H134R)-EYFP-WPRE was generously provided by Karl Deisseroth (Stanford Univ.). For preparation of AAV, HEK293T cells were grown in DMEM media with antibiotics and FBS. The day before transfection, four plates beyond 90% confluence from 10-cm dishes were plated onto five 15-cm dishes and incubated for 18–22 h or until 60 to 70% confluence. HEK293T cells were transfected with pAAV-DIO-ChR2-EYFP, pAAV-DJ and pHelper using jetPEI transfection reagent (QBiogene). The DNA/DMEM/PEI cocktail was vortexed and incubated at room temperature for 20 min. After incubation, the transfection mixture was added to each 15 cm dish. Transfected cells were harvested 48 h after transfection and incubated with 0.5% sodium deoxycholate (Sigma; D6750) and 50 units/ml of benzonase nuclease (Sigma; E1014) at 37◦C for 1 h. After removing cellular debris by centrifuging at 3000 × g for 15 min, the supernatant was filtered through a 0.45 mm PVDF filter (Millipore). Purification of AAV- DJ particles was performed using HiTrap heparin affinity columns (GE Healthcare). For concentration of AAV, Amicon ultra-15 centrifugal filter units with a 100,000 molecular weight cutoff were used. Concentrated virus aliquoted and frozen for storage at −80◦C. The final viral concentrations was 3∼6 × 10<sup>12</sup> virus particles per ml for each AAV.

## **STEREOTAXIC INJECTION AND OPTICAL FIBER PLACEMENT**

Animals were anesthetized by i.p. injections of 1.6 µl of Zoletil and 0.05 µl of xylazine (Rompun, Bayer) per gram of body weight and placed in a stereotaxic apparatus (David Kopf Instruments, Tujunga, CA). For injection of viruses, a 31-gauge syringe needle was used to bilaterally infuse 2 µl of virus into NAc at an angle of 0◦ (AP +1.7; ML ±1.3; DV −4.5) at a rate of 0.1 ul/min. The needle was left in place for 10 min after injection before being slowly withdrawn. The fiber-optic cannula for implantation consisted of a zirconia ferrule (1.25 mm in diameter and 4.5 mm long) and flat tip of an optical fiber (200 µm in diameter). The implantation of the fiber-optic cannula into NAc for illumination of D2-MSNs was performed immediately after injection of viruses. The coordinates for implantation of the fiber-optic cannula were an angle of 0◦ (AP +1.7; ML ±1.35; DV −4.2) for targeting NAc. To help anchor the optical fiber, two screws were anchored into the skull to the rear of the implantation site of the fiber-optic cannula. To fix the fiberoptic cannula on the skull, C&B Superbond (Sun Medical) was applied to the surface of the skull around the base of the cannula. Once the C&B Superbond hardened, the cannula was released from the holder and dental cement (Poly-F, Dentsply) was applied around the cannula and screws. To close the incision around the cannulation site, Vetbond tissue adhesive (3 M, 7003449) was used. After implantation, mice were given subcutaneous injection of antibiotics (Enrofloxacin, 5 mg/kg, q 12 h) and analgesia (Carprofen, 5 mg/kg, q 24 h) for 3 consecutive days.

## **IN VIVO PHOTOSTIMULATION**

A 200 µm patch cord was connected to the external portion of the chronically implantable optical fiber using a sleeve. Optical fibers were attached through an FC/PC adaptor to a blue laser diode (473 nm, MBL-III 473-150 mW), and light pulses were generated through a stimulator (BNC 575). For photostimulation of ChR2-expressing neurons, the stimulation paradigm was 20 Hz frequency, 5 ms pulse duration and 2–5 mW of light power. Light power emitted from the patch cord was measured using a power meter (PM100D) with a S121C light sensor.

## **BEHAVIORAL ANALYSIS**

Behavioral experiments were performed with male D2-Cre mice at 11–13 weeks of age, with the exception of mice subjected to electrophysiological analysis which were 5–6 weeks of age. Agematched D2-Cre and Cre negative control mice were injected with virus and housed individually and allowed to acclimate to the cage until the behavioral test. For each manipulation, mice were transferred to the experimental room 60 min before the onset of the experiment to allow for habituation and to reduce stress (brightness of the experimental room was 70 lux). Each experimental apparatus was cleaned with 70% ethanol between experiments to remove any potential odor cues.

## **COCAINE SENSITIZATION**

For initiation of cocaine sensitization, mice were habituated to saline injections (i.p.) for 3 consecutive days and then injected with saline or cocaine (15 mg kg−<sup>1</sup> , i.p) for 5 consecutive days. Mice were injected intraperitoneally (i.p.) with either cocaine hydrochloride (Johnson Mattney, Edinburgh, UK) dissolved in saline (0.9% NaCl) or saline with a 30 G needle. Immediately after each injection, mice were tested for horizontal locomotor activity in an open-field chamber for 30 min. For measurement of the effect of photostimulation on the initiation and expression of sensitization (**Figure 5**), mice were given blue light illumination bilaterally through dual fiber-optic patch cords onto the NAc for four 3-min periods during 30 min sessions in home cages. Patch cords from the fiber-optic cannula located on the mouse skull was removed and mice were given at least 10 min rest. Mice were then injected with either cocaine or saline (coc 1dcoc 5d). After initiation of sensitization, cocaine was withdrawn for 14 days without any injection of saline. During this withdrawal period, no photostimulation was applied. The expression of behavioral sensitization to cocaine was then determined by injection of a challenge dose of the drug (10 mg kg−<sup>1</sup> , i.p.) after photostimulation of the NAc as illustrated in **Figure 5A**. To measure the effect of photostimulation during the cocaine withdrawal period (**Figure 6**), mice were subjected to the same protocol for sensitization as described above (for **Figure 5**) except photostimulation was given. After initiation of cocaine sensitization, photostimulation was applied to the NAc daily for 1 h during the total withdrawal period of 14 days. After 14 days of withdrawal, all groups of mice were injected with the challenge dosage of cocaine, (10 mg kg−<sup>1</sup> ).

#### **IMMUNOFLUORESCENCE AND CONFOCAL LASER MICROSCOPY**

For immunofluorescence, mice were anesthetized with Zoletil (Virbac, 1.6 µl/g, intraperitoneally) and 0.05 µl/g Rompun (Bayer) and perfused with filter-sterilized 0.1 M PBS followed by fixation using 4% paraformaldehyde/PBS solution (Sigma). The brain was then removed and post-fixed for 4 h with icecold fixative as above. The brains were then dehydrated in 30% sucrose/0.1 M PBS for 2 days. Brains were then frozen and 40-µmthick consecutive coronal sections were prepared on a cryostat (Leica CM 1900, Germany). Sections (40 µm) were blocked for 1 h in 0.1 M PBS containing 5% normal goat serum and 0.2% Triton X-100 and incubated with rabbit polyclonal anti-D2R (1:500, Millipore, AB5084P) at 4◦C overnight. After washes with PBS containing 0.2% Triton X-100, samples were incubated at RT for 1 h with Alexa Fluor 568 goat anti-rabbit IgG (1:500; Molecular Probes, Eugene, OR, USA) and 0.2 µl/ml 4, 6-diamidino-2 phenyl-indole HCl (DAPI; Sigma, St. Louis, MO, USA) in PBS containing 1% normal goat serum and 0.2% Triton X-100. As a negative control, samples were incubated with DAPI and the secondary antibody only. Sections were examined on a C1 Plan Apo × 40/1.4 water confocal laser scanning system (LSM 700, Zeiss, Berlin, Germany).

## **ELECTROPHYSIOLOGY AND PHOTOSTIMULATION IN NUCLEUS ACCUMBENS SLICES**

Mice were used for experiments 4 weeks after virus injection, to achieve optimal expression of ChR2-EYFP. Mice were then anesthetized and decapitated for preparation of acute brain slices. The brain was quickly removed and immediately placed in ice-cold cutting solution containing (in mM) 250 Sucrose, 26 NaHCO3, 10 D-Glucose, 3 Myo-inositol, 2.5 KCl, 2 Na-pyruvate, 1.25 NaH2PO4, 0.5 Ascorbic acid, 1 Kynurenic acid and 7 MgCl<sup>2</sup> which was bubbled with 95% O2/5% CO<sup>2</sup> (pH = 7.4). Coronal brain slices (250 µm thick) containing the NAc were prepared using a vibratome (Leica VT 1200 S) and were then incubated in gassed artificial cerebrospinal fluid (ACSF) containing (in mM): 11 D-glucose, 125 NaCl, 25 NaHCO3, 1.25 NaH2PO4, 2.5 KCl, 1.25 MgCl<sup>2</sup> and 2.5 CaCl<sup>2</sup> at 34◦C for 1 h before recording. Slices were then transferred to a submersion recording chamber in which O2 saturated ACSF solution was continuously superfused. Cells in NAc and VTA were visualized using a 2-photon microscope (Olympus FV1000 MPE, Tokyo, Japan) equipped with a 25X water immersion lens and infrared DIC optics. Whole-cell patch clamp recordings were obtained from NAc cells with a Multiclamp 700B amplifier and Digidata 1440A digitizer (Molecular Devices, LLC). Data were sampled using pCLAMP 10.2 software and further analyzed using Clampfit 10.2 software (Molecular Devices, LLC). Patch electrodes with resistances between 3–5 M were filled with an internal solution containing (in mM): 130 K-gluconate, 2 NaCl, 2 MgCl2, 20 HEPES, 4 Na2ATP, 0.4 Na3GTP, 0.5 EGTA and 10 Na2- phosphocreatine, with pH adjusted to 7.3 using 1 N KOH. Bicuculline (10 µM) was bathapplied to brain slice to block GABA receptors in a subset of experiments.

NAc cells expressing ChR2-EYFP were photostimulated by a LED light source (460 ± 27 nm, UHP-Mic-LED-460, Prizmatix). Blue light from the LED was further filtered and attenuated by a filter cube equipped with an excitation filter (470–495 nm); flashes of light (10 ms duration, 0.0366–0.354 mW/mm<sup>2</sup> ) were delivered to the brain slice via the 25X objective lens at frequencies of 5–40 Hz. In a subset of experiments, photocurrents were measured in ChR2-expressing cells in response to 2 s duration light flashes.

## **STATISTICAL ANALYSIS**

Data are presented as means ± s.e.m. and were analyzed with the two-tailed Student's *t*-test, or with two-way analysis of variance followed by Bonferroni's *post hoc* test. A *P*-value of <0.05 was considered statistically significant.

background figure, 1 mm: insert, 200 µm. **(B)** Confocal images of D2R-Cre mouse brain slices infected with AAV-DIO-ChR2-EYFP virus showing the co-expression of ChR2 (green) in NAc neurons with dopamine D2 receptors (red) (D2R+ChR2). **(C)** Confocal image of neurons in a living NAc slice.

structure (C2). aca: anterior part of anterior commissure; NAc: nucleus accumbens core. **(D)** ChR2-mediated photocurrent (black traces) evoked by light flashes (470 nm, blue traces at bottom) of various intensities.

(Continued)

#### **FIGURE 1 | Continued**

**(E)** Relationship between peak amplitude of photocurrents and light intensity. Points indicate mean ± S.E.M (n = 4), while the curve is fitted with the Hill equation. **(F)** Representative membrane potential responses to trains of light flashes (10 ms, 0.35 mW/mm<sup>2</sup> ) at the indicated frequencies. **(G)** Relationship between light intensity and probability of light-evoked action potentials (APs) during trains of light flashes, as in F (n = 5). Curves are fitted with the Hill equation.

### **RESULTS**

## **SELECTIVE PHOTOSTIMULATION OF MEDIUM SPINY NEURONS IN NUCLEUS ACCUMBENS**

To determine the role of NAc D2R-MSNs in cocaine-mediated addictive behaviors, we used an optogenetic approach to stimulate NAc D2R neurons. To selectively control the activity of D2R-MSNs in NAc by light, viral vectors coding AAV-DIO-ChR2-EYFP were stereotaxically injected into the NAc of D2R-Cre BAC transgenic mice. 4 weeks after viral injection, robust expression of ChR2-EYFP was observed in the NAc (**Figure 1A**). The specificity of ChR2 expression in D2R-MSNs was confirmed by immunofluorescence confocal analysis: expression of YFPtagged ChR2 was co-localized with D2R in NAc (**Figure 1B**), showing that ChR2 was expressed in D2R-expressing neurons in NAc.

Although such an approach has been used in other studies (e.g., Lobo et al., 2010), the details of virus injection procedures will vary from one lab to another, making it important to document optogenetic control under our specific experimental conditions. We assessed the functional expression of ChR2 by making whole-cell patch clamp recordings from MSNs in NAc slices. MSNs were identified by: (1) a relatively hyperpolarized resting membrane potential (RMP), typically more negative than −80 mV; (2) a regular pattern of AP firing in response to applied current pulses; (3) long latency to firing of the first AP during a current pulse; (4) absence of a voltage "sag" during hyperpolarization caused by a hyperpolarization-activated cation current (Ih); and (5) relatively small size of their cell bodies (Chang and Kitai, 1985; O'Donnell and Grace, 1993; Le Moine and Bloch, 1996; Taverna et al., 2008). Blue light (470 nm) was applied over the entire field of view (0.78 mm<sup>2</sup> ) while voltage-clamping the MSNs at a holding potential of −69 mV. Some MSNs expressed ChR2, evident as YFP fluorescence in their somata (arrows in **Figures 1C1,C3**). Such neurons exhibited substantial photocurrents, with brighter light stimuli eliciting larger photocurrents (**Figure 1D**). The relationship between peak photocurrent amplitude and light intensity (**Figure 1E**) had a half-maximal light sensitivity of 0.054 ± 0.0023 mW/mm<sup>2</sup> and a maximal peak amplitude of 1.16 ± 0.16 nA (mean ± s.e.m., *n* = 4).

Under current-clamp conditions, MSNs expressing ChR2 fired APs reliably in response to trains of light pulses (10 ms duration; **Figure 1F**). Under these conditions, light intensities greater than 0.1 mW/mm<sup>2</sup> were sufficient to evoke APs (**Figure 1G**, *n* = 5). APs were reliably evoked at photostimulation frequencies up to 20 Hz, while at 40 Hz the light-induced responses summed to cause a sustained depolarization that was less effective at evoking APs (**Figures 1F,G**).

**circuits. (A)** Confocal image of a live NAc slice, showing a dye-filled neuron that does not express ChR2 and a neighboring cell (arrowhead) that expressed ChR2 and could be photostimulated. **(B)** IPSC (black trace) induced by a light pulse (10 ms duration, 0.35 mW/mm<sup>2</sup> ) at a holding potential of −60 mV. The IPSC was blocked by bath application of bicuculline (red trace). **(C)** IPSC evoked by light pulses (10 ms, 0.35 mW/mm<sup>2</sup> ) when the membrane potential was clamped at 10 mV increments from −99 mV to −39 mV. **(D)** Voltage dependence of light-induced IPSCs. The relationship between IPSC amplitude and membrane potentials (n = 6) could be fit by a straight line and had an estimated reversal potential of −81 mV. **(E)** Four different classes of responses of NAc neurons to photostimulation.

## **PHOTOSTIMULATION OF D2R-MSNs DRIVES LOCAL INHIBITORY CIRCUITS**

To investigate the consequences of D2R-MSNs activity on local circuits in NAc, we photostimulated presynaptic MSN expressing ChR2 while measuring postsynaptic responses in ChR2-negative MSNs (**Figure 2A**). The neuron shown in **Figure 2A** does not express ChR2, as indicated by the absence of EYFP fluorescence as well as the absence of short-latency photocurrents like those shown in **Figure 1D**. However, when the postsynaptic MSNs were held as a potential of −69 mV, 10 ms duration light flashes evoked outward currents after a latency of 9.0 ± 0.42 ms (**Figure 2B**, *n* = 15). To determine the nature of these responses, the postsynaptic membrane potential was varied between −99 mV to −39 mV, while a light flash was applied (**Figure 2C**). Lightinduced responses varied with membrane potential (**Figure 2D**, *n* = 6) and reversed their polarity at −81 ± 3.4 mV. Given that the equilibrium potential for chloride ions is −80 mV under our ionic conditions, the light-induced outward currents could be due to chloride flux mediated by postsynaptic GABA<sup>A</sup> receptors. To

# Optogenetics reveals a role for accumbal medium spiny neurons expressing dopamine D2 receptors in cocaine-induced behavioral sensitization

**Shelly Sooyun Song<sup>1</sup>† , Byeong Jun Kang<sup>1</sup>† , Lei Wen2,3,4† , Hyo Jin Lee<sup>1</sup> , Hye-ri Sim<sup>1</sup> , Tae Hyong Kim<sup>1</sup> , Sehyoun Yoon<sup>1</sup> , Bong-June Yoon<sup>1</sup> , George J. Augustine2,3,4 and Ja-Hyun Baik<sup>1</sup>\***

<sup>1</sup> Department of Life Sciences, Molecular Neurobiology Laboratory, School of Life Sciences and Biotechnology, Korea University, Seoul, South Korea

<sup>2</sup> Center for Functional Connectomics (CFC), KIST, Seoul, South Korea

<sup>3</sup> Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore

4 Institute of Molecular and Cell Biology, Singapore, Singapore

#### **Edited by:**

Anton Ilango, Leibniz Institute for Neurobiology, Germany

#### **Reviewed by:**

Simona Cabib, Università "Sapienza", Roma, Italy Vincent Pascoli, Geneva University, Switzerland

#### **\*Correspondence:**

Ja-Hyun Baik, Department of Life Sciences, Molecular Neurobiology Laboratory, School of Life Sciences and Biotechnology, Korea University, Nokji Bldg, Rm 402, Anam-dong, Sungbuk-gu, Seoul, 136-713, South Korea e-mail: jahyunb@korea.ac.kr †These authors have contributed

equally to this work.

Long-lasting, drug-induced adaptations within the nucleus accumbens (NAc) have been proposed to contribute to drug-mediated addictive behaviors. Here we have used an optogenetic approach to examine the role of NAc medium spiny neurons (MSNs) expressing dopamine D2 receptors (D2Rs) in cocaine-induced behavioral sensitization. Adeno-associated viral vectors encoding channelrhodopsin-2 (ChR2) were delivered into the NAc of D2R-Cre transgenic mice. This allowed us to selectively photostimulate D2R-MSNs in NAc. D2R-MSNs form local inhibitory circuits, because photostimulation of D2R-MSN evoked inhibitory postsynaptic currents (IPSCs) in neighboring MSNs. Photostimulation of NAc D2R-MSN in vivo affected neither the initiation nor the expression of cocaine-induced behavioral sensitization. However, photostimulation during the drug withdrawal period attenuated expression of cocaine-induced behavioral sensitization. These results show that D2R-MSNs of NAc play a key role in withdrawal-induced plasticity and may contribute to relapse after cessation of drug abuse.

**Keywords: optogenetics, medium spiny neurons, dopamine D2 receptors, cocaine, drug addiction**

#### **INTRODUCTION**

Dopamine (DA) signaling is associated with reward expectation and goal-directed behavior (Wise, 2004; Goto and Grace, 2005; Berridge, 2007). One of the well-known pathologies of dopaminergic disorders is drug addiction (Robinson and Berridge, 1993, 2003). Following repeated exposure to addictive substances, adaptive changes occur at the molecular and cellular level in the DA mesolimbic pathway; these can lead to drug dependence, which is a chronic, relapsing disorder in which compulsive drug-seeking and drug-taking behaviors persist despite their serious negative consequences (Thomas et al., 2008; Baik, 2013). Characterization of the modifications that take place in the mesolimbic dopaminergic system is thus key to understanding drug addiction.

Dopamine D1 receptors (D1R) and D2 receptors (D2R) are highly expressed in the medium spiny neurons (MSNs) of the striatum. It has been suggested that long-lasting drug-induced adaptations in the ventral striatum, better known as the nucleus accumbens (NAc), contribute to the development of addiction as well as drug-seeking and relapse behaviors (Lobo and Nestler, 2011; Smith et al., 2013). Dopaminergic cell bodies from the ventral tegmental area mostly innervate the NAc. Over 95% of the cells within the NAc are MSNs, which receive excitatory inputs from four major brain regions: the prefrontal cortex, the ventral subiculum of the hippocampus, the basolateral amygdala, and the thalamus (Sesack and Grace, 2010; Lüscher and Malenka, 2011). MSNs within the NAc can be divided into two major subpopulations: direct pathway MSNs that express D1Rs and project directly to midbrain DA areas, and indirect pathway MSNs that express D2Rs and project to the ventral pallidum (Kreitzer and Malenka, 2008; Sesack and Grace, 2010; Lüscher and Malenka, 2011; Smith et al., 2013). Because MSN are GABAergic, activation of MSNs neurons will inhibit their downstream targets which are also GABAergic (Chevalier and Deniau, 1990). Therefore, activation of D1R-MSNs will excite midbrain DA neurons, which then contributes to the regulation of reward-related behaviors (Lüscher and Malenka, 2011; Bocklisch et al., 2013).

Recent studies using genetically engineered mice that express Cre recombinase in a cell-type specific manner have revealed different roles for D1R-MSNs and D2R-MSNs in cocaine addiction behaviors. Such mice enable genetic targeting of specific toxins, optogenetic probes or DREADD (designer receptors exclusively activated by a designer drug) to selectively manipulate D1R-MSN or D2R-MSN. This approach has led to some consensus about the role of MSNs in addictive behaviors: D1R-MSNs apparently promote addictive behaviors, while no specific role (or an inhibitory role) in the development of drug-induced addictive behaviors has

(**Figure 4B**). No such effects were observed in the control D2R-Cre(−) NAc-ChR2 mice (**Figures 4B,C**), indicating that the effects of photostimulation were caused by activation of ChR2, rather than possible non-specific effects such as heating of brain tissue. Therefore, our data indicated that photostimulation of D2R-MSNs in NAc elicited a decrease in locomotor activity.

These results established our ability to control the activity of D2R-MSNs within NAc *in vivo*. We next used this capability to examine the influence of D2R-MSN activity on behavioral sensitization to repeated administration of cocaine. Behavioral sensitization refers to the process that allows an initial exposure to psychostimulants, such as cocaine, to enhance the ability of subsequent drug exposures to stimulate locomotor activity. This process can be separated into initiation and expression phases: initiation describes the immediate neural events that induce behavioral sensitization (Vanderschuren and Kalivas, 2000; Sim et al., 2013), while expression is known to be a long-lasting form of behavioral plasticity that persists after drug withdrawal (Vanderschuren and Kalivas, 2000; Sim et al., 2013).

**FIGURE 5 | Effects of activation of D2-MSN during sensitization to cocaine. (A)** Experimental scheme for photo-stimulation of D2-MSNs during initiation and expression of sensitization to cocaine. Blue-light illumination (2∼5 mW, 5 ms, 20 Hz) was delivered for four 3-min periods during 40-min sessions before measuring locomotion as indicated by blue bars. Sal, saline; Coc, cocaine; IS, initation of sensitization; WD, withdrawal; ES, expression of sensitization. **(B)** The effects of photo-stimulation during the initiation of behavioral sensitization (IS) with five daily injections of either saline or cocaine (15 mgkg−<sup>1</sup> , i.p.) and during the expression of behavioral sensitization (ES) induced by challenge dose of cocaine (10 mgkg−<sup>1</sup> , i.p.) after 14 days of drug withdrawal (For coc 1d-5d: cocaine effect F(1,11) > 24.55, p < 0.05. For ES: cocaine effect in F(1,11) = 40.14, p < 0.0001; cocaine × D2-Cre interaction F(1,11) = 0.7147, p = 0.407, n = 6∼7). Two-way analysis of variance (ANOVA) post hoc test: \* P < 0.05, \*\* P < 0.01, \*\*\* P < 0.001 vs. saline-treated group). Data are means ± s.e.m. **(C)** Detailed time course of locomotor activity for D2-Cre(–) or D2-Cre(+) mice during ES to cocaine (10 mgkg–1, i.p.) following blue-light illumination. Arrows denote time of injection with either saline or cocaine (For 20–50 m: cocaine effect F(1,11) > 1.812, P < 0.1919, n = 6∼7). Two-way analysis of variance (ANOVA) post hoc test: \* P < 0.05, \*\* P < 0.01, \*\*\* P < 0.001 vs. saline-treated group). Data are means ± s.e.m.

## **BEHAVIORAL ANALYSIS**

Behavioral experiments were performed with male D2-Cre mice at 11–13 weeks of age, with the exception of mice subjected to electrophysiological analysis which were 5–6 weeks of age. Agematched D2-Cre and Cre negative control mice were injected with virus and housed individually and allowed to acclimate to the cage until the behavioral test. For each manipulation, mice were transferred to the experimental room 60 min before the onset of the experiment to allow for habituation and to reduce stress (brightness of the experimental room was 70 lux). Each experimental apparatus was cleaned with 70% ethanol between experiments to remove any potential odor cues.

## **COCAINE SENSITIZATION**

For initiation of cocaine sensitization, mice were habituated to saline injections (i.p.) for 3 consecutive days and then injected with saline or cocaine (15 mg kg−<sup>1</sup> , i.p) for 5 consecutive days. Mice were injected intraperitoneally (i.p.) with either cocaine hydrochloride (Johnson Mattney, Edinburgh, UK) dissolved in saline (0.9% NaCl) or saline with a 30 G needle. Immediately after each injection, mice were tested for horizontal locomotor activity in an open-field chamber for 30 min. For measurement of the effect of photostimulation on the initiation and expression of sensitization (**Figure 5**), mice were given blue light illumination bilaterally through dual fiber-optic patch cords onto the NAc for four 3-min periods during 30 min sessions in home cages. Patch cords from the fiber-optic cannula located on the mouse skull was removed and mice were given at least 10 min rest. Mice were then injected with either cocaine or saline (coc 1dcoc 5d). After initiation of sensitization, cocaine was withdrawn for 14 days without any injection of saline. During this withdrawal period, no photostimulation was applied. The expression of behavioral sensitization to cocaine was then determined by injection of a challenge dose of the drug (10 mg kg−<sup>1</sup> , i.p.) after photostimulation of the NAc as illustrated in **Figure 5A**. To measure the effect of photostimulation during the cocaine withdrawal period (**Figure 6**), mice were subjected to the same protocol for sensitization as described above (for **Figure 5**) except photostimulation was given. After initiation of cocaine sensitization, photostimulation was applied to the NAc daily for 1 h during the total withdrawal period of 14 days. After 14 days of withdrawal, all groups of mice were injected with the challenge dosage of cocaine, (10 mg kg−<sup>1</sup> ).

## **IMMUNOFLUORESCENCE AND CONFOCAL LASER MICROSCOPY**

For immunofluorescence, mice were anesthetized with Zoletil (Virbac, 1.6 µl/g, intraperitoneally) and 0.05 µl/g Rompun (Bayer) and perfused with filter-sterilized 0.1 M PBS followed by fixation using 4% paraformaldehyde/PBS solution (Sigma). The brain was then removed and post-fixed for 4 h with icecold fixative as above. The brains were then dehydrated in 30% sucrose/0.1 M PBS for 2 days. Brains were then frozen and 40-µmthick consecutive coronal sections were prepared on a cryostat (Leica CM 1900, Germany). Sections (40 µm) were blocked for 1 h in 0.1 M PBS containing 5% normal goat serum and 0.2% Triton X-100 and incubated with rabbit polyclonal anti-D2R (1:500, Millipore, AB5084P) at 4◦C overnight. After washes with PBS containing 0.2% Triton X-100, samples were incubated at RT for 1 h with Alexa Fluor 568 goat anti-rabbit IgG (1:500; Molecular Probes, Eugene, OR, USA) and 0.2 µl/ml 4, 6-diamidino-2 phenyl-indole HCl (DAPI; Sigma, St. Louis, MO, USA) in PBS containing 1% normal goat serum and 0.2% Triton X-100. As a negative control, samples were incubated with DAPI and the secondary antibody only. Sections were examined on a C1 Plan Apo × 40/1.4 water confocal laser scanning system (LSM 700, Zeiss, Berlin, Germany).

## **ELECTROPHYSIOLOGY AND PHOTOSTIMULATION IN NUCLEUS ACCUMBENS SLICES**

Mice were used for experiments 4 weeks after virus injection, to achieve optimal expression of ChR2-EYFP. Mice were then anesthetized and decapitated for preparation of acute brain slices. The brain was quickly removed and immediately placed in ice-cold cutting solution containing (in mM) 250 Sucrose, 26 NaHCO3, 10 D-Glucose, 3 Myo-inositol, 2.5 KCl, 2 Na-pyruvate, 1.25 NaH2PO4, 0.5 Ascorbic acid, 1 Kynurenic acid and 7 MgCl<sup>2</sup> which was bubbled with 95% O2/5% CO<sup>2</sup> (pH = 7.4). Coronal brain slices (250 µm thick) containing the NAc were prepared using a vibratome (Leica VT 1200 S) and were then incubated in gassed artificial cerebrospinal fluid (ACSF) containing (in mM): 11 D-glucose, 125 NaCl, 25 NaHCO3, 1.25 NaH2PO4, 2.5 KCl, 1.25 MgCl<sup>2</sup> and 2.5 CaCl<sup>2</sup> at 34◦C for 1 h before recording. Slices were then transferred to a submersion recording chamber in which O2 saturated ACSF solution was continuously superfused. Cells in NAc and VTA were visualized using a 2-photon microscope (Olympus FV1000 MPE, Tokyo, Japan) equipped with a 25X water immersion lens and infrared DIC optics. Whole-cell patch clamp recordings were obtained from NAc cells with a Multiclamp 700B amplifier and Digidata 1440A digitizer (Molecular Devices, LLC). Data were sampled using pCLAMP 10.2 software and further analyzed using Clampfit 10.2 software (Molecular Devices, LLC). Patch electrodes with resistances between 3–5 M were filled with an internal solution containing (in mM): 130 K-gluconate, 2 NaCl, 2 MgCl2, 20 HEPES, 4 Na2ATP, 0.4 Na3GTP, 0.5 EGTA and 10 Na2- phosphocreatine, with pH adjusted to 7.3 using 1 N KOH. Bicuculline (10 µM) was bathapplied to brain slice to block GABA receptors in a subset of experiments.

NAc cells expressing ChR2-EYFP were photostimulated by a LED light source (460 ± 27 nm, UHP-Mic-LED-460, Prizmatix). Blue light from the LED was further filtered and attenuated by a filter cube equipped with an excitation filter (470–495 nm); flashes of light (10 ms duration, 0.0366–0.354 mW/mm<sup>2</sup> ) were delivered to the brain slice via the 25X objective lens at frequencies of 5–40 Hz. In a subset of experiments, photocurrents were measured in ChR2-expressing cells in response to 2 s duration light flashes.

## **STATISTICAL ANALYSIS**

Data are presented as means ± s.e.m. and were analyzed with the two-tailed Student's *t*-test, or with two-way analysis of variance followed by Bonferroni's *post hoc* test. A *P*-value of <0.05 was considered statistically significant.

been suggested for D2R-MSNs (Hikida et al., 2010; Lobo et al., 2010; Ferguson et al., 2011; Bock et al., 2013). Cocaine exposure apparently induces synaptic modification and alterations in gene expression in both MSN populations (Lobo et al., 2010; Lobo and Nestler, 2011; Grueter et al., 2013). Although it appears that D1R-MSNs and D2R-MSNs play opposing roles in cocaine-mediated addictive behaviors, the precise role of D2R-MSNs is not clear.

Previously it has been shown that D2R knockout (KO) mice display normal cocaine-mediated behavioral sensitization and cocaine-seeking behaviors, with only a slight decrease in sensitivity caused by the absence of D2R (Baik et al., 1995; Chausmer et al., 2002; Sim et al., 2013). However, exposure to stress during drug withdrawal suppresses expression of cocaine-induced behavioral sensitization as well as cocaine-seeking and relapse behaviors in D2R KO mice (Sim et al., 2013). Specific knockdown of D2R in the NAc does not affect basal locomotor activity, nor cocaine-induced behavioral sensitization, but does confer the ability of stress to inhibit expression of cocaine-induced behavioral sensitization (Sim et al., 2013). These findings strongly suggest that blockade of D2R in the NAc does not prevent cocaine-mediated behavioral sensitization. Rather, it appears that D2R in the NAc play a distinct role in regulation of the stresstriggered synaptic modifications during withdrawal that lead to an increase in cocaine-seeking and relapse behaviors (Sim et al., 2013).

Here we have used optogenetics to further evaluate the role of NAc D2R-MSNs in cocaine-induced behavioral sensitization. Using brain slices, we find that photostimulation of D2R-MSNs activates local inhibitory circuits within NAc involving neighboring MSNs. Photostimulation of NAc D2R-MSNs *in vivo* affects neither the initiation nor the expression of cocaineinduced behavioral sensitization. However, repetitive activation of NAc D2R-MSNs during the drug withdrawal period attenuates cocaine-induced addictive behavior. Our results show that D2R-MSNs of NAc play a key role in withdrawal-induced plasticity and may contribute to relapse after cessation of drug abuse.

# **MATERIALS AND METHODS**

## **MICE**

D2-Cre BAC transgenic mice on a C57Bl/6 background were obtained from MMRRC (Mutant Mouse Regional Resource Centers, B6.FVB(Cg)-Tg(Drd2-cre)ER44Gsat/Mmucd). In behavioral experiments, littermates lacking the D2-Cre transgene were used as controls for the D2-Cre mice. Mice were maintained in a specific pathogen-free barrier facility under constant conditions of temperature and humidity, and on a 12-h light, 12-h dark schedule. Animal care and handling were performed in accordance with standards approved by the Institutional Animal Care and Use Committees of Korea University and KIST.

## **VIRUS VECTOR PREPARATION**

pAAV-EF1a-DIO-hChR2(H134R)-EYFP-WPRE was generously provided by Karl Deisseroth (Stanford Univ.). For preparation of AAV, HEK293T cells were grown in DMEM media with antibiotics and FBS. The day before transfection, four plates beyond 90% confluence from 10-cm dishes were plated onto five 15-cm dishes and incubated for 18–22 h or until 60 to 70% confluence. HEK293T cells were transfected with pAAV-DIO-ChR2-EYFP, pAAV-DJ and pHelper using jetPEI transfection reagent (QBiogene). The DNA/DMEM/PEI cocktail was vortexed and incubated at room temperature for 20 min. After incubation, the transfection mixture was added to each 15 cm dish. Transfected cells were harvested 48 h after transfection and incubated with 0.5% sodium deoxycholate (Sigma; D6750) and 50 units/ml of benzonase nuclease (Sigma; E1014) at 37◦C for 1 h. After removing cellular debris by centrifuging at 3000 × g for 15 min, the supernatant was filtered through a 0.45 mm PVDF filter (Millipore). Purification of AAV- DJ particles was performed using HiTrap heparin affinity columns (GE Healthcare). For concentration of AAV, Amicon ultra-15 centrifugal filter units with a 100,000 molecular weight cutoff were used. Concentrated virus aliquoted and frozen for storage at −80◦C. The final viral concentrations was 3∼6 × 10<sup>12</sup> virus particles per ml for each AAV.

## **STEREOTAXIC INJECTION AND OPTICAL FIBER PLACEMENT**

Animals were anesthetized by i.p. injections of 1.6 µl of Zoletil and 0.05 µl of xylazine (Rompun, Bayer) per gram of body weight and placed in a stereotaxic apparatus (David Kopf Instruments, Tujunga, CA). For injection of viruses, a 31-gauge syringe needle was used to bilaterally infuse 2 µl of virus into NAc at an angle of 0◦ (AP +1.7; ML ±1.3; DV −4.5) at a rate of 0.1 ul/min. The needle was left in place for 10 min after injection before being slowly withdrawn. The fiber-optic cannula for implantation consisted of a zirconia ferrule (1.25 mm in diameter and 4.5 mm long) and flat tip of an optical fiber (200 µm in diameter). The implantation of the fiber-optic cannula into NAc for illumination of D2-MSNs was performed immediately after injection of viruses. The coordinates for implantation of the fiber-optic cannula were an angle of 0◦ (AP +1.7; ML ±1.35; DV −4.2) for targeting NAc. To help anchor the optical fiber, two screws were anchored into the skull to the rear of the implantation site of the fiber-optic cannula. To fix the fiberoptic cannula on the skull, C&B Superbond (Sun Medical) was applied to the surface of the skull around the base of the cannula. Once the C&B Superbond hardened, the cannula was released from the holder and dental cement (Poly-F, Dentsply) was applied around the cannula and screws. To close the incision around the cannulation site, Vetbond tissue adhesive (3 M, 7003449) was used. After implantation, mice were given subcutaneous injection of antibiotics (Enrofloxacin, 5 mg/kg, q 12 h) and analgesia (Carprofen, 5 mg/kg, q 24 h) for 3 consecutive days.

## **IN VIVO PHOTOSTIMULATION**

A 200 µm patch cord was connected to the external portion of the chronically implantable optical fiber using a sleeve. Optical fibers were attached through an FC/PC adaptor to a blue laser diode (473 nm, MBL-III 473-150 mW), and light pulses were generated through a stimulator (BNC 575). For photostimulation of ChR2-expressing neurons, the stimulation paradigm was 20 Hz frequency, 5 ms pulse duration and 2–5 mW of light power. Light power emitted from the patch cord was measured using a power meter (PM100D) with a S121C light sensor.

#### **FIGURE 1 | Continued**

**(E)** Relationship between peak amplitude of photocurrents and light intensity. Points indicate mean ± S.E.M (n = 4), while the curve is fitted with the Hill equation. **(F)** Representative membrane potential responses to trains of light flashes (10 ms, 0.35 mW/mm<sup>2</sup> ) at the indicated frequencies. **(G)** Relationship between light intensity and probability of light-evoked action potentials (APs) during trains of light flashes, as in F (n = 5). Curves are fitted with the Hill equation.

#### **RESULTS**

## **SELECTIVE PHOTOSTIMULATION OF MEDIUM SPINY NEURONS IN NUCLEUS ACCUMBENS**

To determine the role of NAc D2R-MSNs in cocaine-mediated addictive behaviors, we used an optogenetic approach to stimulate NAc D2R neurons. To selectively control the activity of D2R-MSNs in NAc by light, viral vectors coding AAV-DIO-ChR2-EYFP were stereotaxically injected into the NAc of D2R-Cre BAC transgenic mice. 4 weeks after viral injection, robust expression of ChR2-EYFP was observed in the NAc (**Figure 1A**). The specificity of ChR2 expression in D2R-MSNs was confirmed by immunofluorescence confocal analysis: expression of YFPtagged ChR2 was co-localized with D2R in NAc (**Figure 1B**), showing that ChR2 was expressed in D2R-expressing neurons in NAc.

Although such an approach has been used in other studies (e.g., Lobo et al., 2010), the details of virus injection procedures will vary from one lab to another, making it important to document optogenetic control under our specific experimental conditions. We assessed the functional expression of ChR2 by making whole-cell patch clamp recordings from MSNs in NAc slices. MSNs were identified by: (1) a relatively hyperpolarized resting membrane potential (RMP), typically more negative than −80 mV; (2) a regular pattern of AP firing in response to applied current pulses; (3) long latency to firing of the first AP during a current pulse; (4) absence of a voltage "sag" during hyperpolarization caused by a hyperpolarization-activated cation current (Ih); and (5) relatively small size of their cell bodies (Chang and Kitai, 1985; O'Donnell and Grace, 1993; Le Moine and Bloch, 1996; Taverna et al., 2008). Blue light (470 nm) was applied over the entire field of view (0.78 mm<sup>2</sup> ) while voltage-clamping the MSNs at a holding potential of −69 mV. Some MSNs expressed ChR2, evident as YFP fluorescence in their somata (arrows in **Figures 1C1,C3**). Such neurons exhibited substantial photocurrents, with brighter light stimuli eliciting larger photocurrents (**Figure 1D**). The relationship between peak photocurrent amplitude and light intensity (**Figure 1E**) had a half-maximal light sensitivity of 0.054 ± 0.0023 mW/mm<sup>2</sup> and a maximal peak amplitude of 1.16 ± 0.16 nA (mean ± s.e.m., *n* = 4).

Under current-clamp conditions, MSNs expressing ChR2 fired APs reliably in response to trains of light pulses (10 ms duration; **Figure 1F**). Under these conditions, light intensities greater than 0.1 mW/mm<sup>2</sup> were sufficient to evoke APs (**Figure 1G**, *n* = 5). APs were reliably evoked at photostimulation frequencies up to 20 Hz, while at 40 Hz the light-induced responses summed to cause a sustained depolarization that was less effective at evoking APs (**Figures 1F,G**).

**circuits. (A)** Confocal image of a live NAc slice, showing a dye-filled neuron that does not express ChR2 and a neighboring cell (arrowhead) that expressed ChR2 and could be photostimulated. **(B)** IPSC (black trace) induced by a light pulse (10 ms duration, 0.35 mW/mm<sup>2</sup> ) at a holding potential of −60 mV. The IPSC was blocked by bath application of bicuculline (red trace). **(C)** IPSC evoked by light pulses (10 ms, 0.35 mW/mm<sup>2</sup> ) when the membrane potential was clamped at 10 mV increments from −99 mV to −39 mV. **(D)** Voltage dependence of light-induced IPSCs. The relationship between IPSC amplitude and membrane potentials (n = 6) could be fit by a straight line and had an estimated reversal potential of −81 mV. **(E)** Four different classes of responses of NAc neurons to photostimulation.

## **PHOTOSTIMULATION OF D2R-MSNs DRIVES LOCAL INHIBITORY CIRCUITS**

To investigate the consequences of D2R-MSNs activity on local circuits in NAc, we photostimulated presynaptic MSN expressing ChR2 while measuring postsynaptic responses in ChR2-negative MSNs (**Figure 2A**). The neuron shown in **Figure 2A** does not express ChR2, as indicated by the absence of EYFP fluorescence as well as the absence of short-latency photocurrents like those shown in **Figure 1D**. However, when the postsynaptic MSNs were held as a potential of −69 mV, 10 ms duration light flashes evoked outward currents after a latency of 9.0 ± 0.42 ms (**Figure 2B**, *n* = 15). To determine the nature of these responses, the postsynaptic membrane potential was varied between −99 mV to −39 mV, while a light flash was applied (**Figure 2C**). Lightinduced responses varied with membrane potential (**Figure 2D**, *n* = 6) and reversed their polarity at −81 ± 3.4 mV. Given that the equilibrium potential for chloride ions is −80 mV under our ionic conditions, the light-induced outward currents could be due to chloride flux mediated by postsynaptic GABA<sup>A</sup> receptors. To

background figure, 1 mm: insert, 200 µm. **(B)** Confocal images of D2R-Cre mouse brain slices infected with AAV-DIO-ChR2-EYFP virus showing the co-expression of ChR2 (green) in NAc neurons with dopamine D2 receptors (red) (D2R+ChR2). **(C)** Confocal image of neurons in a living NAc slice.

structure (C2). aca: anterior part of anterior commissure; NAc: nucleus accumbens core. **(D)** ChR2-mediated photocurrent (black traces) evoked by light flashes (470 nm, blue traces at bottom) of various intensities.

(Continued)

# Nociception and pain: lessons from optogenetics

## **Fiona B. Carr and Venetia Zachariou\***

Fishberg Department of Neuroscience, Department of Pharmacology and Systems Therapeutics and Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY, USA

#### **Edited by:**

Mary Kay Lobo, University of Maryland School of Medicine, USA

#### **Reviewed by:**

Philippe Seguela, Montreal Neurological Institute, Canada Feng Tao, Johns Hopkins University School of Medicine, USA

#### **\*Correspondence:**

Venetia Zachariou, Fishberg Department of Neuroscience, Department of Pharmacology and Systems Therapeutics and Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, 1425 Madison Avenue, New York, NY 10029, USA e-mail: venetia.zachariou@mssm.edu The process of pain perception begins in the periphery by activation of nociceptors. From here nociceptive signals are conveyed via the dorsal horn of the spinal cord to multiple brain regions, where pain is perceived. Despite great progress in pain research in recent years, many questions remain regarding nociceptive circuitry and behavior, in both acute nociception and chronic pain states. Techniques that allow for selective activation of neuronal subpopulations in vivo can provide a better understanding of these complex pathways. Here we review the studies to date that have employed novel optogenetic tools to improve our understanding of the pain pathway at the peripheral, spinal and supraspinal levels.

**Keywords: pain, nociception, nociceptor, spinal cord, amygdala, prefrontal cortex**

## **CIRCUITS AND CELL TYPES INVOLVED IN PAIN**

Chronic pain represents a significant clinical problem affecting up to 20% of the general population (Breivik et al., 2006; Reid et al., 2011) and is a symptom of a variety of different underlying pathologies including arthritis, nerve injury, depression and cancer. An understanding of nociceptive mechanisms and the neurobiology of pain perception are necessary to better treat chronic pain conditions. Here we provide a brief overview of the circuitry and cell types involved, and review studies to date that have used optogenetic tools to manipulate these pathways.

Unlike some neurobiological disorders which can be attributed to specific brain regions, pain incorporates multiple components of the nervous system, at the peripheral, spinal, and supraspinal levels. The first components of the pain pathway are the peripheral nociceptors. These are a heterogeneous population, and can be classified in a variety of ways including myelination properties, expression of transducer molecules, peptidergic content and voltage gated sodium channel expression (Gold and Gebhart, 2010). Transgenic mice have been invaluable in demonstrating that nociceptor heterogeneity is of functional significance, with different nociceptors contributing to distinct nociceptive modalities such as heat and mechanical pain (Abrahamsen et al., 2008; Cavanaugh et al., 2009). Nociceptor subtypes also contribute in distinct ways to different forms of chronic pain such as inflammatory and neuropathic states (Cavanaugh et al., 2009; Seal et al., 2009; Minett et al., 2012). Activation of specific nociceptor subtypes in the intact system will be helpful in confirming modality specificity in pain, and determining appropriate peripheral targets for novel therapeutic strategies.

The dorsal horn of the spinal cord, where the majority of nociceptors terminate, provides the next level of complexity in pain processing. An exception to this are the trigeminal nociceptors, which innervate the head and facial areas and terminate in the medullary dorsal horn (Ren and Dubner, 1999). Interneurons are the most numerous population in the dorsal horn (Todd, 2010) and the balance between excitation and inhibition within the dorsal horn may be disrupted in chronic pain states. For instance, it is known that decreased GABAergic and glycinergic inhibition contribute to neuropathic pain hypersensitivity (Moore et al., 2002; Lu et al., 2013). In addition subtypes of glutamatergic interneurons may play distinct roles in acute, inflammatory and neuropathic pain states (Wang et al., 2013). To date our understanding of interneuron circuitry in the dorsal horn is based largely on electrophysiological and pharmacological approaches. Given the pivotal role of interneurons in the gate control theory of pain (Melzack, 1999), regional and cell type specific activation using optogenetics will be a powerful way to investigate this important circuitry.

The principal direct targets of dorsal horn projection neurons are the parabrachial area and thalamus (Gauriau and Bernard, 2002, 2004). From here a range of areas are activated including the somatosensory cortex, anterior cingulate, prefrontal cortex (PFC), and amygdala (Apkarian et al., 2005; Tracey and Mantyh, 2007; Baliki et al., 2013). It is important to note that chronic pain states are frequently accompanied by psychological symptoms such as anxiety and depression (Bushnell et al., 2013) and understanding the role of brain areas in both the sensory and affective components of pain is important for the development of novel treatment approaches. Another important aspect of pain circuitry to be considered is the top-down pain modulatory system, which allows higher brain structures to signal to the dorsal horn to regulate nociceptive processing. This may be either inhibitory or

facilitatory in nature, arises from a variety of brain structures and is relayed to the dorsal horn via the rostral ventromedial medulla (RVM; Ossipov et al., 2010). Optogenetic approaches will allow further investigation of these ascending and descending circuits in animal pain behavior (**Figure 1**).

#### **OPTOGENETIC STIMULATION OF NOCICEPTORS IN C. ELEGANS**

C. elegans was the first multicellular organism to be investigated using optogenetic methods (Nagel et al., 2005), and is one of the most useful simple organisms in the study of genetics and molecular biology. Many advances in neuroscience and behavior have also been made using this system. C. elegans is particularly useful in the study of nociception as it exhibits a clear and reproducible withdrawal behavior, involving a reversal and change in direction away from the noxious stimulus (Wittenburg and Baumeister, 1999; Tobin and Bargmann, 2004). Recently, optogenetic activation of C. elegans nociceptors has been used to characterize molecules associated with nociception (Husson et al., 2012). Channelrhodopsin 2 was expressed under the control of the F49H12.4 promoter, predominantly found within PVD nociceptive neurons. Stimulation of the worm with blue light therefore allowed activation of the nociceptive network, without activating other non-noxious mechanical sensing neurons. Interestingly, activation of these nociceptors is less susceptible to habituation than other sensory responses in C. elegans. The authors used this technique to screen for genes involved in mediating the nociceptive response in the PVD neuron, investigating the contribution of RNA interference for various target genes to the optogenetically induced nociceptive behavior. This led to the identification of a number of genes involved in nociceptive transmission in the organism, including voltage gated calcium channel subunits and melastatin-related transient receptor (TRPM) channels. This study highlights the potential of using optogenetically triggered behavior as a method to screen for novel nociceptive mediators, or even analgesics.

## **OPTOGENETIC STIMULATION OF SPECIFIC NOCICEPTOR POPULATIONS IN MICE**

Optogenetic stimulation of a subset of peripheral nociceptor has also been used as a tool to characterize the spinal cord circuitry associated with that nociceptor subtype (Wang and Zylka, 2009). Mas-related G protein coupled receptor D (Mrgprd) is a marker for a subset of non-peptidergic nociceptors in the rat and mouse. These neurons appear to be involved in mediating mechanical nociception, but not noxious thermal or cold stimuli (Cavanaugh et al., 2009). Although anatomically it is known that Mrgprd-positive fibers terminate largely in lamina II of the dorsal horn, the precise connectivity of this nociceptor subpopulation was not known. In contrast to the variety of molecular markers available to characterize nociceptor populations, relatively few markers are available to characterize neuronal populations in the dorsal horn. To characterize the dorsal horn neurons innervated by Mrgprd expressing primary afferents, Wang and colleagues used a combined optogenetic and electrophysiological approach (Wang and Zylka, 2009). Knock-in mice were generated to express channlerhodopsin-2 at the Mrgprd locus, allowing this nociceptor population to be selectively activated by blue light. Patch clamp recordings from lamina II dorsal horn neurons were made using a slice preparation and stimulation of Mrgprd expressing terminals by brief high frequency blue light application. This was sufficient to induce excitatory post synaptic potentials in 50% of the lamina II neurons recorded. The non-responsive lamina II neurons are therefore likely to be innervated by other afferents, such as peptidergic nociceptors, supporting the view that pain processing is modality specific at both the peripheral and spinal levels. Interestingly this preparation illustrates that stimulation of channelrhodopsin-2 on axon terminals is sufficient to evoke post synaptic activity, as the dorsal root ganglions (DRGs) were not attached. This study highlights the potential of optogenetics in understanding the complex pain related circuitry within the dorsal horn. Unlike traditional electrophysiological studies which use electrical stimulation of afferents or application of a noxious stimulus, the optogenetic approach allows precise stimulation a defined nociceptor population. This is of great interest for the understanding connectivity between peripheral nociceptors and neurons in the spinal cord.

Recently, an alternative approach has been used to explore the use of optogenetics in pain studies. *Nav1.8* encodes a voltage gated sodium channel expressed in nociceptors, and Nav1.8 expressing primary afferent neurons have been previously shown to contribute to inflammatory and neuropathic pain (Abrahamsen et al., 2008). Many studies have used the Nav1.8-Cre mice as a background to study the role of particular genes in nociception and pain-like behaviors (Stirling et al., 2005). This mouse model has now been used to express channelrhodopsin-2 specifically in nociceptors. Stimulation of the hindpaw with blue light was sufficient to induce a number of characteristic nocifensive behaviors, including paw withdrawal, licking, jumping and vocalizations, in freely moving mice. Therefore this study provides the first evidence that optical stimulation of peripheral nociceptors can trigger nocifensive behaviors in the awake and freely moving animal (Daou et al., 2013). Furthermore, c-Fos labeling of the dorsal horn following this pattern of optical activation indicated activation of both lamina I and lamina II dorsal horn neurons, suggesting that both peptidergic nociceptors and non-peptidergic nociceptors are activated. Interestingly, the authors found that optogenetic stimulation of Nav1.8 nociceptors is also capable of driving long term hypersensitivity to mechanical and thermal stimuli, mimicking aspects of chronic pain models in mice. As pain has an important affective component, the authors also demonstrated that blue light could drive conditioned place aversion in these animals.

In contrast to the approach taken by Zlyka et al. in which channelrhodopsin was inserted into the locus of a low abundance marker of a subset of nociceptors, the Nav1.8-Cre approach may be more useful for behavioral studies, and investigations of spinal and supraspinal activation, given that a greater number of nociceptors will be activated in this way.

#### **OPTOGENETIC STUDIES OF BRAIN REGIONS INVOLVED IN PAIN**

To date few studies have used optogenetic techniques to address the role of specific brain regions in pain behavior in rodents. One example has incorporated optogenetic stimulation to investigate the role of the central nucleus of the amygdala (CeA) in a mouse model of visceral pain (Crock et al., 2012). Visceral pain is a common complaint, and can be modeled in rodents by bladder or colorectal distension. In the study by Crock and colleagues, bladder distention was carried out, and the visceromotor response (VMR, elecromyographic response of the abdominal muscle) used as an index of visceral sensitivity. The amygdala is important in the affective dimensions of the pain response, and previous work had shown that the c-fos expression is increased in the CeA following noxious visceral stimulation (Han and Neugebauer, 2004). To explore this further, the right CeA was injected with herpes simplex virus (HSV) viral vector expressing channelrhodopsin-2. Optogenetic stimulation was carried out for 30 min before, during or after bladder distension. Optical stimulation before distension increased VMR sensitivity. This hypersensitivity was maintained for up to 15 min following the termination of the optical stimulation, providing direct evidence that neuronal activity in the CeA can lead to increased visceral nociception. A limitation of this study is the nature of the bladder distension protocol which requires the animals to be lightly anesthetized in order to perform measures of the VMR. Future studies to incorporate stimulation in awake, moving animals will provide further insights into the role of specific brain regions in the modulation of pain states.

Another important region involved in the modulation of pain is the PFC. As with the amygdala, this region is particularly associated with the affective component of the pain experience (Tracey and Mantyh, 2007). In a recent investigation, the PFC was injected with an adenoassociated virus (AAV), expressing channelrhodopsin-2 under the control of the calcium/calmodulin dependent kinase type II alpha (CaMKIIα) promoter (Ji and Neugebauer, 2012). This allows selective expression within pyramidal neurons only, which are the major excitatory population in the region, and avoids expression within inhibitory interneurons. Two distinct regions of the PFC were explored, the infralimbic (IL) and prelimbic (PL) areas. Following optogenetic stimulation of the IL region, extracellular recordings were made from pyramidal neurons in the IL and the nearby PL region with or without peripheral stimulation, and also following noxious mechanical stimulation of the knee joint. This approach helps provide a better understanding of circuitry in the PFC. Excitation of neurons in the IL region not only increases spontaneous activity of other neurons in this area, but also the evoked responses to noxious and non-noxious stimulation. Optical stimulation of the IL conversely led to a decrease in pyramidal neuron excitability in the PL, suggesting an inverse relationship between IL output and between PL activity (Ji and Neugebauer, 2012). This study is a good example of the type of analysis that can be achieved by combining optogenetic stimulation with electrophysiological recordings. Future studies may be useful to investigate the effects of chronic pain states, as opposed to acute noxious stimulation, on specific neuronal populations in the brain.

Recently we have applied an optogenetic approach to explore novel brain regions involved in opiate analgesia. Opioids are important clinically, however their use is limited through the development of tolerance and addiction (Ling et al., 2011). Understanding the brain regions and cell types involved in these processes are is important to help tackle the problem. We had previously demonstrated that regulator of G protein signaling 9-2 (Rgs9-2) is a negative modulator of morphine tolerance (Zachariou et al., 2003). In our recent study we demonstrated a potent role of Rgs9-2 in the nucleus accumbens (NAc, part of the brain reward center) in modulation of morphine tolerance. Taking advantage of an optogenetic strategy, we have showed that activation of Rgs9 expressing neurons in the NAc leads to the rapid development of analgesic tolerance in the hot plate test (Gaspari et al., 2014). Using bacterial artificial chromosome (BAC) lines expressing Cre recombinase, we were also able to selectively stimulate the two main NAc neuronal subpopulations: those expressing primarily D1 dopamine receptors (D1-type, direct pathway) and those enriched in D2 dopamine receptors (D2-type, indirect pathway). Activation of channelrhodopsin-2 in each of these neuronal subpopulation demonstrated that morphine tolerance is modulated by D1 type neurons, which is the population that mostly expresses mu opioid receptors (MORs; Cui et al., 2014). In accord with our earlier findings on a role of Rgs9-2 in the NAc in morphine tolerance, activation of D1-type but not D2 type neurons increases Rgs9-2 levels in the NAc (Gaspari et al., 2014).

## **OPTOPHARMACOLOGY AND NOVEL OPTORECEPTORS IN PAIN RESEARCH**

Optogenetic approaches require the expression of an exogenous light sensitive molecule in the target system. Although this has proved a very useful tool in basic neurobiological research, by definition this approach is of limited use clinically. An alternative approach is the use of "optopharmacology", that is administration of compounds that confer light sensitivity onto a specific cell type (Kramer et al., 2013). One recent example of this approach is the discovery of a novel small molecule, optovin, which confers light sensitivity on transient receptor potential cation channel, member A1 (TRPA1; Kokel et al., 2013). This ion channel is part of the transient receptor potential (TRP) family that act as transducers of noxious stimuli at the periphery. TRPA1 is activated by noxious cold and can also respond to chemical irritants such as formalin and mustard oil, and may also play a role in mediating mechanical nociception (Vay et al., 2012). The ability to optically stimulate peripheral fibers expressing TRPA1 is therefore of experimental interest to investigate the role of these neurons in animal and human pain sensation. Optovin was identified by screening a library of compounds in zebrafish embryos, which are normally insensitive to light. This molecule was found to confer sensitivity to violet light, and the effect was mediated by sensory neurons, as a spinalized preparation retained the response. A subset of mouse DRG cells also responded to optovin treatment, in a light dependent manner. These were identified as TRPA1 expressing nociceptors. Using zebrafish homozygous for TRPA1 it was demonstrated that this ion channel is required for the effects of optovin. Expression of human TRPA1 in cell line derived from human embryonic kidney (HEK) cells also conferred sensitivity to optovin. The significance of this study lies in the ability to modulate the pain sensing TRPA1 neurons, without the need for genetic manipulations to confer this sensitivity. This approach may be useful for research and treatment of pain in the future. In particular, such an approach would allow selective optical control of subpopulations of peripheral nociceptors without the need for genetic manipulations.

Another approach has been the recent development of a photoactive MOR, the main receptor mediating the analgesic actions of morphine (Barish et al., 2013). This study designed a recombinant receptor, containing the optically active component of the rhodopsin molecule and the parts of the mu opioid receptor responsible for MOR receptor protein and G protein signaling. In a HEK cell system, this receptor is activated by light and has the ability to activate endogenous MOR receptor intracellular signaling pathways. Further development would be required, but expression of this mutant MOR receptor could be a useful way to overcome the side effects of MOR mediated analgesia, including analgesic tolerance, which is associated with alterations in MOR signal transduction pathways. Future work will determine if selective activation of the MOR in the periphery by light application prevents the CNS problems in patients requiring long term morphine treatment.

## **FUTURE DIRECTIONS IN PAIN OPTOGENETICS**

The power of optogenetics lies in the ability to achieve regional and cell type specific neuronal activation. These methods provide an unprecedented opportunity to probe the complexities of the pain pathway at the peripheral, spinal and supraspinal levels. Among the most exciting developments in the field to date is the ability to produce pain-like behaviors in transgenic mice by optical stimulation of nociceptors in the skin (Daou et al., 2013). This allows direct investigation of nociceptor activation in awake, freely moving animals, avoiding the stimulation of nonnociceptive neurons and other cell types in the periphery. The stimulus is also under precise temporal control, and avoids the need for using other artificial stimuli such as chemical irritants. Future studies to target specific subtypes of nociceptor will lead to greater advances in our understanding of modality specificity, and contribution of nociceptor subtypes to different forms of chronic pain.

Somewhat surprisingly, to date no studies have taken advantage of these tools within the dorsal horn however this may reflect technical challenges in inserting optogenetic fibers into the spinal cord. Considering the importance of this component of the pain pathway, however, it is likely that optogenetics will also provide valuable new insights into this complex circuitry. Interneuron based optogenetic experiments have been performed within the ventral horn of the spinal cord, to investigate the contribution of particular subsets to locomotor activity (Dougherty et al., 2013). It is feasible that a similar approach could be employed in the dorsal horn, given our increasing anatomical and molecular knowledge of interneuron populations (Todd, 2010).

As described, a small number of studies have explored brain areas in pain using optogenetics, however many other areas of the pain matrix remain to be studied in this way. Both brain studies to date have relied either on electrophysiological recordings (Ji and Neugebauer, 2012) or behavioral outcomes in the anesthetized animal (Crock et al., 2012). One of the most important directions to be explored will be the combination of optogenetic brain stimulation with behavior in the awake animal.

The studies described here are only the beginning of what we expect to be fruitful and informative exploration of pain circuits using optogenetic tools. It is likely that subsequent studies will move beyond these initial proof of concept studies, and use optogenetic tools to tackle unanswered questions regarding pain circuitry.

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 05 December 2013; accepted: 18 February 2014; published online: 25 March 2014.*

*Citation: Carr FB and Zachariou V (2014) Nociception and pain: lessons from optogenetics. Front. Behav. Neurosci. 8:69. doi: 10.3389/fnbeh.2014.00069*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2014 Carr and Zachariou. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Optogenetic dissection of amygdala functioning

# **Ryan T. LaLumiere\***

Department of Psychology, University of Iowa, Iowa City, IA, USA

## **Edited by:**

Mary Kay Lobo, University of Maryland School of Medicine, USA

#### **Reviewed by:**

Christa McIntyre, University of Texas, USA Joshua Johansen, RIKEN Brain

#### Science Institute, Japan **\*Correspondence:**

Ryan T. LaLumiere, Department of Psychology, University of Iowa, 11 Seashore Hall E, Iowa City, IA 52242, USA e-mail: ryan-lalumiere@uiowa.edu

Studies of amygdala functioning have occupied a significant place in the history of understanding how the brain controls behavior and cognition. Early work on the amygdala placed this small structure as a key component in the regulation of emotion and affective behavior. Over time, our understanding of its role in brain processes has expanded, as we have uncovered amygdala influences on memory, reward behavior, and overall functioning in many other brain regions. Studies have indicated that the amygdala has widespread connections with a variety of brain structures, from the prefrontal cortex to regions of the brainstem, that explain its powerful influence on other parts of the brain and behaviors mediated by those regions. Thus, many optogenetic studies have focused on harnessing the powers of this technique to elucidate the functioning of the amygdala in relation to motivation, fear, and memory as well as to determine how the amygdala regulates activity in other structures. For example, studies using optogenetics have examined how specific circuits within amygdala nuclei regulate anxiety. Other work has provided insight into how the basolateral and central amygdala nuclei regulate memory processing underlying aversive learning. Many experiments have taken advantage of optogenetics' ability to target either genetically distinct subpopulations of neurons or the specific projections from the amygdala to other brain regions. Findings from such studies have provided evidence that particular patterns of activity in basolateral amygdala (BLA) glutamatergic neurons are related to memory consolidation processes, while other work has indicated the critical nature of amygdala inputs to the prefrontal cortex and nucleus accumbens (NA) in regulating behavior dependent on those downstream structures. This review will examine the recent discoveries on amygdala functioning made through experiments using optogenetics, placing these findings in the context of the major questions in the field.

**Keywords: basolateral amygdala, central amygdala, memory, consolidation, fear, anxiety, channelrhodopsin**

Despite its relatively small size in the human brain, the amygdala influences a wide variety of neural functions, especially those related to emotion and memory. As a result, considerable research has targeted this structure and investigated how it regulates different processes and interacts with other brain regions to do so. The development of optogenetics over the past decade has opened new doors for investigations into neural circuits, both on a systems-circuitry level and on a microcircuitry level, and into how those circuits govern behavior. The amygdala, with its rich history of research in neuroscience, has been a ripe target for optogenetic investigations and, thus, many studies have applied this technique to address deeper issues regarding amygdala functioning, providing new avenues of research into "old" questions about the amygdala. Because optogenetics permits stimulation and inhibition of specific projection pathways through illumination of opsin-expressing axons and axon terminals (LaLumiere, 2011; Yizhar et al., 2011), many studies have used this approach to explore specific circuits. Moreover, because opsin expression can be genetically targeted to distinct cell types, many studies have also taken advantage of this capability to control activity in specific classes of neurons to determine their functional roles. This review will explore the recent studies that have used optogenetics to

better understand the function of the amygdala in the brain and behavior, with a particular focus on how these studies have elucidated better understandings of the systems regulating emotion, fear, and memory as well as of the functional neural circuits that involve the amygdala.

## **AMYGDALA AND EMOTION**

Perhaps the oldest line of research on the amygdala's role in the brain originates with how the amygdala regulates emotion and emotional output. Indeed, early work showed that amygdala lesions in monkeys produced the now-classic Kluver-Bucy syndrome (Kluver and Bucy, 1937; Weiskrantz, 1956). The symptoms of the syndrome involved profound alterations in the monkeys' emotional behavior, especially those involving fear-based behavior. These early findings have led to a considerable number of studies that have expounded upon this function for the amygdala. Studies in humans have confirmed the findings on amygdala lesions, as selective amygdala lesions also appear to produce deficits in emotion-related behaviors, especially those regarding fear (Adolphs et al., 1994, 2005).

Thus, it is not surprising that studies have harnessed the power of optogenetics to develop an improved understanding for how the amygdala influences emotional behavior. In a series of studies, Kay Tye et al. have made several inroads into understanding amygdala functioning in anxiety and anxiety-related behavior. Tye et al. (2011) used an optogenetic approach to distinguish the roles of the basolateral amygdala (BLA) inputs to the medial vs. lateral portions of the central amygdala (CEA) in such behavior in mice. Indeed, previous work had suggested that the medial CEA and its inputs from the BLA drive anxiety and/or fear-related behaviors, whereas BLA inputs to the lateral CEA provide feedforward inhibition of the medial CEA (Paré et al., 2004; Ciocchi et al., 2010). To distinguish these pathways, Tye et al. (2011) transduced BLA neurons with either the depolarizing cation channel channelrhodopsin-2 (ChR2) or the hyperpolarizing chloride pump halorhodopsin and illumination was provided to the BLA terminals in the lateral CEA. The expression of the opsins was limited to the pyramidal neurons of the BLA through the use of a CaMKIIα promoter. The results indicated that stimulation and inhibition of the BLA terminals in the lateral CEA reduced and increased, respectively, anxiety in the mice, as measured in different tasks. Moreover, the findings strongly suggested that this effect was due to feedforward inhibition of the medial CEA. Illumination of the BLA cells themselves did not produce the same behavioral effect and sometimes produced the opposite effect. With low light levels, illumination of the terminals did not produce reliable antidromic propagation of action potentials back to the BLA, largely eliminating the concern that such propagation could be responsible for the behavioral effect, though it is clear that optical stimulation of axon terminals can produce reliable antidromic propagation to cell bodies (Jennings et al., 2013). These results not only provided a clearer understanding of the amygdala circuits driving behavior but illustrated the importance of targeting specific downstream projection targets, as stimulating or inhibiting specific projections may produce rather different effects than might be found with stimulating or inhibiting the entire structure.

Since this initial work, Tye et al. have extended their focus on anxiety by examining the projections from the BLA to the ventral hippocampus in the regulation of anxiety-related behaviors (Felix-Ortiz et al., 2013). Previous studies had indicated that the BLA projects to the ventral hippocampus and that the ventral hippocampus is also involved in anxiety (Pitkänen et al., 1995, 2000; Bannerman et al., 2003). Using an open-arm plus maze and an open-field chamber, Felix-Ortiz et al. found that inhibition of BLA afferents to the ventral hippocampus, via activation of halorhodopsin, decreased anxiety-related behaviors. Similarly, stimulation of such afferents, using 20 Hz light pulses to activate ChR2, increased anxiety-related behaviors. Importantly, control experiments demonstrated that the effects of the BLA terminal stimulation were not due to antidromic stimulation of the BLA itself or orthodromic stimulation of fibers of passage. Moreover, electrophysiology analysis suggested that the effects of stimulating BLA inputs to the ventral hippocampus were mediated through local circuit mechanisms involving both direct activation of principal cells in the hippocampus and indirect recruitment of inhibitory neurons. In other work, Felix-Ortiz and Tye (2014) examined optical stimulation and inhibition of the BLA axon terminals in the ventral hippocampus during tests of social behavior. In a resident-juvenile-intruder test, stimulation of these terminals, using 20 Hz light pulses, reduced social interactions of the resident, whereas inhibition of these terminals increased such interactions, suggesting that activation of this pathway increased anxiety in the animals whereas silencing of this pathway produced the opposite effect. The effects with stimulation were blocked with microinjections of glutamate receptor antagonists into the ventral hippocampus, indicating that the results were due to stimulation of BLA axon terminals and the concomitant release of glutamate from those terminals. Similar behavioral effects with stimulation were observed with a three-chamber sociability test. Together, the studies by Tye et al. have produced a wealth of knowledge regarding how the amygdala, and especially the BLA, influences anxiety-related behavior through different outputs to other regions. Moreover, the findings have also demonstrated the potential for using optogenetics to develop an improved understanding of the circuits underlying behavior.

Other work has focused on how inputs from the hypothalamus regulate amygdala activity and fear-related behaviors. Previous work had indicated that oxytocin exerts its effects on a variety of behaviors, at least in part, through activation of oxytocin receptors in the CEA (Viviani et al., 2011). However, while oxytocin neurons are located in the hypothalamus, it has not been clear whether the oxytocin is released via dendritic mechanisms and then spreads passively to the amygdala or is released via classic axonal mechanisms (Landgraf and Neumann, 2004). To address this issue, Knobloch et al. (2012) transduced hypothalamic oxytocin neurons using an oxytocin promoter. Optical stimulation of these neurons' axonal terminals in the CEA produced oxytocindependent effects on CEA neuronal activity, indicating that the hypothalamic neurons are able to directly release oxytocin from their axon terminals. In particular, optical stimulation appeared to increase neuronal activity in the lateral CEA and inhibit activity in medial CEA neurons. Anatomical analysis found that the hypothalamic inputs terminated in the lateral, but not medial, CEA. A separate experiment providing stimulation during a test for contextual fear conditioning found an attenuation of freezing behavior with the optical stimulation. Indeed, the overall evidence from this work suggests that the hypothalamic oxytocin neuronal input to the CEA makes synaptic contacts on the lateral, rather than medial, aspect of the CEA, consistent with the function of the lateral CEA in inhibiting medial CEA output and reducing the expression of fear and fear-related behavior. On a larger level, these findings contribute to our understanding of how specific inputs, and even genetically distinct inputs, to the amygdala regulate both behavior and local circuit activity.

## **AMYGDALA AND MEMORY**

Other work has focused on the use of optogenetics to understand the relationship between the amygdala and memory, especially aversive learning such as fear conditioning. One of the earliest uses of optogenetics in studies of the amygdala examined whether optogenetic stimulation of the lateral amygdala, combined with tones, produces fear conditioning (Johansen et al., 2010). In these experiments, the authors targeted the pyramidal cells of the lateral amygdala. These early findings confirmed that different frequencies of light produced robust firing in lateral amygdala neurons, though with the early version of ChR2 used, the fidelity of the neurons' responding to 50 Hz light pulses was not as good as that found in response to 20 Hz light pulses. Nonetheless, with 20 Hz stimulation, there was a strong c-fos response in the neurons. Optical stimulation, paired with a tone, produced an increase in freezing both during training and in a later retention test. However, the authors noted that the levels of freezing were considerably lower than had been found in previous studies, an effect that the authors suggest indicates that other mechanisms must also be involved in order to produce full fear conditioning. These findings contributed to previous work showing that activity in the lateral amygdala is critical for the normal development and retention of tone fear conditioning. Moreover, through the use of optogenetics, these experiments were able to selectively target the pyramidal cells while providing temporally precise stimulation, setting the stage for future experiments to build on these early results and develop a clearer understanding of the amygdala's role in fear conditioning and memory.

Our own work has used an optogenetic approach to examining the role of the BLA in modulating memory consolidation for inhibitory avoidance, a similar aversive learning task (Huff et al., 2013). Prior work suggested that BLA activity in the gamma frequency range (35–45 Hz) is important for synchronizing activity in downstream structures and promoting the consolidation of learning (Bauer et al., 2007; Popescu et al., 2009). However, such work has depended on physiological recordings, which cannot determine whether driving activity in that range alters memory consolidation. Therefore, in our experiments, the BLA pyramidal neurons were transduced with ChR2(E123A), a "ChETA" version of channelrhodopsin that permits high-fidelity responding to light pulses up to 200 Hz (Gunaydin et al., 2010; Yizhar et al., 2011). We found that stimulating the BLA pyramidal cells with bursts of gamma-frequency light pulses (40 Hz) for 15 min immediately after inhibitory avoidance training enhanced retention 2 days later (Huff et al., 2013). Stimulation with bursts of 20 Hz pulses did not produce a significant effect on retention. Previous work has shown that other types of posttraining stimulation (e.g., electrical) of the amygdala produce an inverted-U curve with regard to retention (Gold et al., 1975), but it is not known whether optical stimulation of the BLA also produces such an effect. In another groups of rats, we inhibited BLA neuronal activity immediately after training via activation of the outward proton pump ArchT and found that 15 min of neuronal inhibition, but not 1 min, impaired retention of the learning (Huff et al., 2013). These findings indicated that BLA stimulation in the gamma frequency range enhances memory consolidation and, critically, have provided a set of effective parameters for using optogenetic approaches to modulate memory consolidation that we are continuing to use in new experiments addressing BLA interactions with efferent brain regions.

Other work has investigated a subpopulation of BLA neurons to examine its role in fear conditioning (Jasnow et al., 2013). Specifically, Jasnow et al. targeted the glutamatergic pyramidal cells found in the BLA by driving ChR2 expression with the Thy1 promoter, which limits expression to a specific subpopulation of glutamatergic cells in the BLA and other forebrain regions (Sugino et al., 2006). Optical stimulation of this specific class of BLA neurons during tone fear conditioning impaired retention of the learning while having no effect on the expression of freezing itself during the conditioning (Jasnow et al., 2013). Moreover, optical stimulation of the neurons paired with the tone alone during extinction training enhanced the retention of the extinction learning, again without having any effect on the freezing itself during the extinction training. Generally, it has been thought that activity in the lateral amygdala and medial CEA drive the expression of fear, but electrophysiological characterization of this subpopulation suggests that it shunts activity in lateral amygdala neurons and inhibits activity of medial CEA neurons. Indeed, optical activation of this neuronal subpopulation had no effect on the acute expression of fear but, rather, appeared to influence consolidation specifically for memories that *oppose* fear conditioning.

Recent work has also investigated the interactions of distinct subpopulations within the BLA with efferent targets in relation to fear conditioning. Prior work has noted the existence of neurons within the basal nucleus of the BLA that are responsive to cues associated with footshocks and other neurons that are responsive to cues previously associated with footshocks that have been extinguished. Senn et al. (2014) investigated whether such neurons also show distinct projection patterns to the medial prefrontal cortex (mPFC). To perform a functional investigation of these subpopulations of neurons, retrogradely transported viruses containing Cre-recombinase were injected into either the prelimbic (PL) or infralimbic (IL) cortex while a conditional viral vector expressing the opsins in a Cre-dependent manner was injected into the BLA. As a result, the opsins were selectively expressed in the IL-projecting neurons or the PLprojecting neurons, enabling illumination of the entire BLA to only stimulate or inhibit the specific subpopulation of neurons. Consistent with previous work suggesting a dichotomy between the dorsal regions of the mPFC (PL) and the ventral regions (IL) with regard to fear conditioning (Peters et al., 2009), the authors demonstrated that BLA neurons projecting to the PL are activated by unextinguished cues whereas those projecting to the IL are activated by extinguished cues. Moreover, inhibition of IL-projecting neurons during cue extinction training produced a significant impairment in the retention of the extinction learning, compared to stimulating such neurons. Conversely, inhibition of the PL-projecting neurons during such training enhanced the retention of the extinction learning, compared to stimulating such neurons. While much previous research has focused on the mPFC inputs to the BLA in regulating fear extinction and expression, these findings provide evidence that BLA inputs to distinct mPFC regions also differentially influence such behaviors.

## **AMYGDALA AND REWARD AND ADDICTION**

Considerable evidence indicates that the amygdala is involved in reward and addiction-related behaviors (Everitt et al., 1999; See et al., 2003; Di Ciano and Everitt, 2004). As part of understanding the precise role played by the BLA in such behaviors, Stuber et al. (2011) examined the pathway from the BLA to the nucleus accumbens (NA) core in modulating and driving reinforcement of operant behavior. Although previous evidence had suggested that the BLA could directly influence dopamine release in the NA (Howland et al., 2002), a key mechanism for reinforcement, it was not clear whether activity in the BLA inputs to the NAcore could actually reinforce operant behavior or was necessary for reward seeking. Stuber et al. found that mice engaged in an operant behavior (nose pokes) in order to receive optical stimulation (burst of 20 Hz light pulses) of the BLA axon terminals in the NAcore. Similarly, optical inhibition, via activation of halorhodopsin, of the BLA terminals decreased responding for a sucrose reward. Strikingly, optical stimulation of mPFC inputs to the NAcore did not produce the same effects. Other work has shown that optical stimulation of BLA inputs to the NAshell are also reinforcing (Britt et al., 2012), though, in contrast to the mPFC inputs to the NAcore, stimulation of mPFC inputs or ventral hippocampal inputs to the NAshell was also found to be reinforcing. Of particular interest in this study was that the authors used the ability to selectively control different glutamatergic inputs to the NAshell, via illumination of opsin-transduced axons from different regions, to determine the degree of plasticity in the connections following cocaine exposure. Together, these findings illustrate not only the increasing evidence for a role for BLA glutamatergic inputs to the NA in regulating reinforcement and reward but also that, depending on the part of the NA targeted, other glutamatergic inputs do not necessarily produce identical results.

Other studies have indicated that, consistent with its role in different forms of associative learning, the BLA is a key part of the neural circuit that drives drug-seeking behavior triggered by drug-associated cues (See et al., 2003). However, as the BLA projects to both the PL and the NAcore, both of which are considered critical components of a general drug-seeking circuit (McFarland and Kalivas, 2001; McFarland et al., 2004; LaLumiere and Kalivas, 2008), it has not been clear which pathway is responsible for driving cue-induced drug-seeking. Recent work by Stefanik and Kalivas (2013) has used optogenetics to investigate this issue. The BLA was transduced with ArchT. Inhibition of BLA terminals in either the PL or NAcore reduced cue-induced reinstatement of cocaine-seeking, indicating that activity in both pathways is obligatory for associative cue-driven cocaine-seeking.

# **CIRCUITRY OF THE AMYGDALA AND INTERACTIONS WITH OTHER BRAIN REGIONS**

A number of studies have used optogenetic approaches to develop a better understanding of the functional connections between the amygdala and other brain regions. For example, Li et al. (2012) have investigated the role of kappa opioid receptor signaling in the bed nucleus of the stria terminalis (BNST). Patch-clamp recordings in the BNST provided evidence that activation of kappa opioid receptors inhibits GABAergic transmission via presynaptic mechanisms. As the CEA provides an important GABAergic input to the BNST, one that has been implicated as a critical pathway in the central stress system (Jasnow et al., 2004; Walker and Davis, 2008), the authors used optogenetics to target and control activity in this pathway. The results indicated that kappa opioid receptor activation inhibited GABAergic transmission in this pathway specifically, a result that would have been difficult to demonstrate using other techniques.

Other research has used optogenetics to delineate precisely how the amygdala influences activity in other regions. Luna and Morozov (2012) contrasted the microcircuitry of BLA inputs vs. anterior piriform inputs to the posterior piriform cortex. Although both structures were found to innervate deep pyramidal cells of the posterior piriform, the BLA and anterior piriform connected with different kinds of interneurons. Specifically, the BLA produced strong connections with fast-spiking interneurons, whereas the anterior piriform had its strongest synapses on irregular-spiking interneurons. As these different classes of interneurons synapse on different regions of the pyramidal cells (somatic vs. distal dendritic), the feedforward inhibition from BLA vs. anterior piriform inputs would be expected to have profoundly different effects on the likelihood of spiking in the principal cells of the posterior piriform.

Several studies have used optogenetics to understand amygdala function and interactions in combination with a variety of other techniques, an approach that will likely dominate much future work as optogenetics affords significant advantages in our ability to gain deeper knowledge regarding neural circuitry. For example, experiments have focused on the well-known connections between the BLA and the mPFC. The reciprocal connections between these regions appear to be involved in a wide variety of behavioral and higher cognitive functions. Yet, the mPFC receives inputs from many other structures and the distinctions among the connections formed by these inputs have not been clear. Therefore, Little and Carter (2012) investigated how BLA, ventral hippocampal, midline thalamus, and contralateral mPFC inputs to the layer two pyramidal neurons of the mPFC make synaptic connections. The authors used optogenetics to target specific pathways by transducing the efferent structures with ChR2 and providing illumination to their axonal terminals in the mPFC. Moreover, they combined their optogenetic manipulations with two-photon microscopy in order to determine the functional connections on a subcellular level. The results from this study indicate that the different regions do, in fact, make different subcellular connections. The BLA appears to make synaptic connections much closer to the soma, relative to other regions, especially the thalamic inputs. Additionally, the BLA inputs target spines of an "intermediate" size, along with ventral hippocampal inputs, in contrast to the thalamic inputs to the large spines and the contralateral mPFC inputs to the small spines. As both the size of the spine and the distance from the soma govern the relative strengths of the inputs, these findings shed light on how different regions influence local circuit activity in other regions. In a follow-up study, Little and Carter (2013) extended their findings, again using optogenetics combined with two-photon microscopy. In this case, their findings indicated that BLA inputs to the mPFC were considerably stronger on mPFC neurons that innervated the BLA, compared to mPFC neurons that provide inputs to the contralateral mPFC. Together, these findings have contributed to a deeper understanding for how the BLA and mPFC interact and, critically, provide a foundation for understanding how BLA inputs to the mPFC may regulate PFC activity and PFCdependent functions.

Work has examined other brain regions' inputs to the amygdala. In a recent study, Carter et al. (2013) found evidence of a circuit involving projections from the parabrachial nucleus in the brainstem to the CEA that suppresses appetite. After genetically identifying and targeting neurons in the parabrachial nucleus to determine their ability to suppress appetite in mice, the authors then transduced these cells with ChR2 and provided illumination to downstream targets. Although stimulating parabrachial axon terminals in the BNST had no effect on food intake, stimulation between 20–40 Hz of the axon terminals in the lateral CEA reduced food consumption. In contrast to many studies that have used 20 Hz stimulation but consistent with our own work (e.g., Huff et al., 2013), Carter et al. found that 20 Hz stimulation did not produce as strong a behavioral effect as found with the 30 and 40 Hz stimulation. By utilizing both the genetic targeting ability of combining optogenetics with transgenic mice and by targeting the axon terminals, these findings provide a significant step forward in understanding how genetically distinct neuronal populations connect with different regions in the brain and, in turn, regulate appetite-related behavior.

Optogenetic studies have also targeted specific interneuron populations in the BLA to understand local circuits. Chu et al. (2012) examined how dopamine influences parvalbumin-positive interneurons in the BLA and, thereby, influence principal cell activity. This issue is of importance because previous work has shown that dopamine influences BLA activity and modulates memory consolidation (Bissiere et al., 2003; LaLumiere et al., 2004, 2005). Prior work has found that D2 receptor activation in the BLA suppresses feedforward inhibition, thereby providing a gating mechanism for synaptic plasticity in the amygdala (Bissiere et al., 2003), while other work has also found that dopamine disinhibits the BLA via inhibition of intercalated cells in a D1 receptor-dependent manner (Marowsky et al., 2005). Using a Cre line of transgenic mice, Chu et al. were able to target ChR2 expression to the parvalbumin-positive cells of the BLA, which are believed to be the major class of interneurons in the structure. The authors then demonstrated that dopamine selectively reduced GABAergic transmission to principal cells, but not to other interneurons, and that this occurred in a presynaptic D2 receptordependent mechanism. These findings provide additional confirmation of the critical role of dopamine, and especially D2 receptors, in modulating BLA activity. Moreover, these findings were demonstrated in a specific subclass of interneurons, an important issue as other work has suggested that different stimuli influence different subtypes of interneurons in the BLA (Bienvenu et al., 2012).

## **CONCLUSIONS**

As a heterogeneous collection of nuclei with a variety of influences on neural functioning and behavior, the amygdala has been the subject of countless neurobiological studies. Our ability to investigate this structure and its role in the brain and behavior has been significantly altered by the development of optogenetics. The studies reviewed here and listed in **Table 1** have provided an understanding for the kinds of questions that this approach can address with regard to the amygdala and, critically, have provided roadmaps for continuing research into amygdala functioning. The findings from these studies indicate that the local circuitry in the amygdala regulates a number of emotion-related behaviors and have produced a clearer picture for how this circuitry works. Other work has begun providing glimpses into how the BLA influences activity in efferent structures, even delineating the local



microcircuits to determine the precise differences between BLA inputs to the region compared to other inputs. Moreover, other studies presented here have demonstrated that specific frequencies of BLA activity differentially influence behavior and memory consolidation. These findings will prove invaluable as neuroscience builds a deeper understanding of the full connectivity of the nervous system on a functional level. All of this work and the findings from these experiments, however, would not have been possible without the temporal, spatial, and/or genetic precision afforded by optogenetics. As future studies utilizing optogenetic approaches build upon these early findings, we should expect considerable advancement in our understanding of amygdala functioning in behavior and in interactions with other brain systems.

#### **ACKNOWLEDGMENTS**

The author would like to acknowledge NIH support through grants MH097111 (Ryan T. LaLumiere) and DA034684 (Ryan T. LaLumiere).

## **REFERENCES**


amygdala. *J. Neurosci.* 33, 15333–15342. doi: 10.1523/JNEUROSCI.2385-13. 2013


**Conflict of Interest Statement**: The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 03 February 2014; accepted: 13 March 2014; published online: 26 March 2014.*

*Citation: LaLumiere RT (2014) Optogenetic dissection of amygdala functioning. Front. Behav. Neurosci. 8:107. doi: 10.3389/fnbeh.2014.00107*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience*.

*Copyright © 2014 LaLumiere. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

# *Ex vivo* dissection of optogenetically activated mPFC and hippocampal inputs to neurons in the basolateral amygdala: implications for fear and emotional memory

#### *Cora Hübner 1,2†, Daniel Bosch1†, Andrea Gall 1, Andreas Lüthi <sup>3</sup> and Ingrid Ehrlich1 \**

*<sup>1</sup> Hertie Institute for Clinical Brain Research and Centre for Integrative Neuroscience, University of Tuebingen, Tuebingen, Germany*

*<sup>2</sup> Graduate School of Neural and Behavioral Sciences, IMPRS, Tuebingen, Germany*

*<sup>3</sup> Friedrich Miescher Institute for Biomedical Research, Basel, Switzerland*

#### *Edited by:*

*Anton Ilango, National Institutes of Health, USA*

#### *Reviewed by:*

*Pankaj Sah, Queensland Brain Institute, Australia Francisco Sotres-Bayon, Universidad Nacional Autónoma de México, Mexico*

#### *\*Correspondence:*

*Ingrid Ehrlich, Hertie Institute for Clinical Brain Research and Centre for Integrative Neuroscience, Otfried-Mueller-Str. 25, 72076 Tuebingen, Germany e-mail: ingrid.ehrlich@ uni-tuebingen.de*

*†These authors have contributed equally to this work.*

Many lines of evidence suggest that a reciprocally interconnected network comprising the amygdala, ventral hippocampus (vHC), and medial prefrontal cortex (mPFC) participates in different aspects of the acquisition and extinction of conditioned fear responses and fear behavior. This could at least in part be mediated by direct connections from mPFC or vHC to amygdala to control amygdala activity and output. However, currently the interactions between mPFC and vHC afferents and their specific targets in the amygdala are still poorly understood. Here, we use an *ex-vivo* optogenetic approach to dissect synaptic properties of inputs from mPFC and vHC to defined neuronal populations in the basal amygdala (BA), the area that we identify as a major target of these projections. We find that BA principal neurons (PNs) and local BA interneurons (INs) receive monosynaptic excitatory inputs from mPFC and vHC. In addition, both these inputs also recruit GABAergic feedforward inhibition in a substantial fraction of PNs, in some neurons this also comprises a slow GABAB-component. Amongst the innervated PNs we identify neurons that project back to subregions of the mPFC, indicating a loop between neurons in mPFC and BA, and a pathway from vHC to mPFC via BA. Interestingly, mPFC inputs also recruit feedforward inhibition in a fraction of INs, suggesting that these inputs can activate dis-inhibitory circuits in the BA. A general feature of both mPFC and vHC inputs to local INs is that excitatory inputs display faster rise and decay kinetics than in PNs, which would enable temporally precise signaling. However, mPFC and vHC inputs to both PNs and INs differ in their presynaptic release properties, in that vHC inputs are more depressing. In summary, our data describe novel wiring, and features of synaptic connections from mPFC and vHC to amygdala that could help to interpret functions of these interconnected brain areas at the network level.

**Keywords: medial prefrontal cortex, hippocampus, amygdala, conditioned fear, optogenetics**

## **INTRODUCTION**

Emotional information is processed in distinct neural circuits. Salient emotions such as fear and anxiety are among those most intensely investigated, because resulting behaviors can be easily evoked and studied in the laboratory and the underlying brain areas are highly conserved among mammalian species from mice to man (Ledoux, 2000; Phelps and Ledoux, 2005). The most powerful models used to date to elucidate the neural circuits and mechanisms that control fear are classical Pavlovian fear conditioning and extinction of acquired fear (Maren, 2001; Ehrlich et al., 2009; Herry et al., 2010; Pape and Pare, 2010). Fear conditioning involves pairing of a previous neutral stimulus (CS) with an aversive stimulus (US), such that a CS-US association is formed. During extinction training, the CS is repeatedly presented without the US, which leads to a decrease in the learned fear response. Many lines of evidence suggest that fear and extinction learning create two distinct memory traces, and which memory is retrieved depends on the retrieval context (Bouton et al., 2006; Myers and Davis, 2007; Quirk and Mueller, 2008).

Although the amygdala is one of the most important brain areas for mediating fear and its extinction, the medial prefrontal cortex (mPFC) and hippocampus (HC), structures that are reciprocally connected to the amygdala, are implicated in aspects of acquisition, consolidation and retrieval of fear and extinction memories (Myers and Davis, 2007; Quirk and Mueller, 2008; Maren, 2011). Projections from mPFC to amygdala originate from layers 2 and 5 of different subregions, including the prelimbic (PL) and infralimbic (IL) areas and form asymmetric synapses (Pinto and Sesack, 2000, 2008). Tracing studies are not completely consistent regarding target nuclei in the amygdala. Projections from PL appear to target mainly the basal nucleus of the basolateral amygdala (BA) and portions of the capsular subdivision of the central amygdala (CeC), whereas IL projections are generally less dense and target large parts of the amygdaloid complex including the intercalated cells, and more densely a specialized lateral part of the CeC, and the ventromedial part of the LA as well as the magnocellular division of the BA (McDonald et al., 1996; Vertes, 2004; Pinard et al., 2012). Moreover, PL and IL receive amygdala projections originating mainly from the BA (Conde et al., 1995; Hoover and Vertes, 2007). Recently it has been shown that BA neurons projecting to IL and PL have opposing roles in expression of fear following extinction learning (Senn et al., 2014). Together, this raises the possibility that IL and PL may interact with the amygdala by virtue of their reciprocal connections to influence the outcome of fear and extinction learning. Projections from HC to amygdala originate in the temporal subiculum and the adjacent part of CA1. Subicular projections are dense in the accessory basal (AB) and medial part of BA, but moderate in LA and light in central amygdala, while CA1 projections to the amygdala mainly terminate in the BA, with lighter projections to LA and AB (Canteras and Swanson, 1992; Pitkanen et al., 2000). The ventral HC (vHC) is thought to contribute contextual information following extinction learning either via direct amygdala projections or indirectly by strong projections to the mPFC, which subsequently projects to the amygdala (Hoover and Vertes, 2007; Pape and Pare, 2010; Orsini et al., 2011).

Systems-level studies started to elucidate specific functions of mPFC and hippocampal regions and their interactions with the amygdala in fear learning, fear expression and extinction of fear (Maren and Quirk, 2004; Pape and Pare, 2010; Maren, 2011). For example, synchronization of activity in amygdala-hippocampalprefrontal cortical circuits plays a critical role in anxiety, acquired fear, extinction learning, and fear discrimination (Seidenbecher et al., 2003; Lesting et al., 2011; Likhtik et al., 2014), but the underlying connectivities of neurons and microcircuits are still incompletely understood. The mPFC appears to play a double role in high and low fear states. Activation of the IL suppresses fear by suppressing amygdala output possibly via intercalated cells and central amygdala inhibition (Quirk et al., 2003; Paré et al., 2004; Maren, 2011), and/or via local BLA interneurons (Rosenkranz and Grace, 2001, 2002). The PL is thought to excite the amygdala to increase fear output during fear expression and renewal (Vidal-Gonzalez et al., 2006; Orsini et al., 2011; Sierra-Mercado et al., 2011). However, *in vivo* recordings of neuronal responses in the BLA during mPFC stimulation have yielded conflicting results about amygdala activation (Rosenkranz and Grace, 2001, 2002; Likhtik et al., 2005). Also, few data are available on how hippocampal activity influences BLA activity (Maren and Fanselow, 1995; Hobin et al., 2003; Maren and Hobin, 2007). It has been proposed that both hippocampal and PL projections to the BA mediate context-dependent fear renewal (Orsini et al., 2011), but if and how these inputs converge in the BA has not been studied.

Thus, a key open question that will guide our understanding and interpretation of systems-level functions and mechanisms, is to decipher the functional connectivities in amygdalahippocampal-prefrontal circuits including innervation of distinct cell types, delineation of similarities or differences in synaptic input properties, and the recruitment of specific microcircuits. Here, we use an *ex vivo* optogenetic approach to study the properties of mPFC and vHC inputs to specific subtypes of BA neurons and describe distinct wiring principles and synaptic properties between these three structures.

# **MATERIALS AND METHODS**

# **ANIMALS**

For all experiments, we used adult male mice (8–12 week old at time of slice recordings) of the following lines: C57BL/6J (Harlan, Netherlands), glutamate decarboxylase 67 (GAD67)–green fluorescent protein (GFP) transgenic mice (Tamamaki et al., 2003) backcrossed to C57BL/6J, and Parvalbumin-Cre (PV-Cre, Jackson stock 008069) mice crossed to Ai14 reporter mice (Jackson stock 007914) that were backcrossed to C57BL/6J. All experimental procedures were in accordance with the EU directive on use of animals in research and approved by the Regierungspraesidium Tuebingen, state of Baden-Wuerttemberg, Germany.

## **STEREOTACTIC INJECTIONS**

Four to six week old mice were maintained under isoflurane anesthesia, fixed in a stereotactic frame (Stoelting, USA) and injected bilaterally in either the mPFC or ventral hippocampus or a combination of both at the following coordinates from bregma (in mm). mPFC: posterior 1.9, lateral ±0.3, ventral −2.1; ventral hippocampus: posterior −3.1, lateral ±3.4, ventral −3.8. Pressure injections were performed using glass capillaries (1B150F-4, WPI, Germany) attached to a Toohey Spritzer (Toohey Company, USA). For mPFC inputs, the mPFC was injected either with 0.5μl of rAAV-CAG-hChR2(H134R) mCherry (serotype 2/1 or 2/9, Penn Vector Core, USA) alone or with a 0.5μl mix of rAAV-hSyn.hChR2(H134R)-eYFP (serotype 2/9, Penn Vector Core, USA) and red retrobeads (Lumafluor, USA). For hippocampal inputs, the ventral hippocampus was injected with 0.5μl rAAV-hSyn.hChR2(H134R)-eYFP and the mPFC was injected with 0.4μl red retrobeads. In all cases, viral preps were diluted such that they had comparable titers (1 × 1012 GC/ml). Retrobeads were dialyzed against 0.32 M sucrose prior to use to avoid osmotic damage of the tissue. Four to six weeks postinjection, amygdala slices were prepared for slice recordings.

## **SLICE RECORDINGS**

Coronal or horizontal (tilted 35◦ from horizontal plane) (Morozov et al., 2011) acute brain slices were prepared in icecold artificial cerebrospinal fluid (ACSF) supplemented with 8.7 mM MgSO4 at 320μm thickness using a vibrating microtome (HM650V, Microm, Germany) equipped with a sapphire blade (Delaware Diamond Knives, USA). Slices were recovered at 37◦C for 45 min and stored at room temperature in ACSF composed of (in mM): 124 NaCl, 1.25 NaH2PO4, 1.3 MgSO4, 2.7 KCl, 26 NaHCO3, 2 CaCl2, 18 D-glucose, 4 L-ascorbic acid and oxygenated with 95% O2, 5% CO2 until recording. Slices containing the amygdala were transferred to a submersion recording chamber, superfused with oxygenated ACSF at a speed of 1–2 ml/min, and maintained at 30–31◦C. Oblique infrared and fluorescence illumination were used to target unlabeled or retrobead-labeled principal neurons, and GFP- or dtTomato-expressing interneurons for recording. Whole-cell patch-clamp recordings were performed using pipettes pulled from borosilicate glass capillaries (GB150F-8P, Science Products, Germany) with resistances of 5–8 M*-*. For most whole cell recordings, the intracellular solution contained (in mM): 130 K-Gluconate, 5 KCl, 4 Mg-ATP, 0.4 Na-GTP, 10 Na2-phosphocreatine, 10 HEPES, 0.6 EGTA and had an osmolarity of 290–295 mOsm and pH of 7.2–7.3. In some recordings 0.5% w/v biocytin was included in the intracellular solution. Some recordings were performed in Cs-based internal solution containing (in mM): 135 Cs-Methylsulphonate, 6 CsCl, 4 Mg-ATP, 0.4 Na-GTP, 10 Na2-phosphocreatine, 10 HEPES, 0.6 EGTA and had an osmolarity of 290–295 mOsm and pH of 7.2–7.3. Data were acquired using a Multiclamp 700 B amplifier, Digidata 1440 AD-board, and Clampex software (all from MDS, USA). Signals were filtered at 2 kHz and digitized at 5 kHz for synaptic current recordings and filtered at 10 kHz and digitized at 20 kHz for current-clamp recordings. Series resistance was monitored throughout each experiment and data were excluded if it changed *>*20%. ChR2-expressing fibers were activated with brief light pulses (0.6–2 ms, 5–10 mW/mm2) from a light emitting diode (470 nm, KSL70, Rapp Opto-Electronics, Germany) delivered to the whole field through the 40 × 0*.*8 NA objective of the upright microscope (BX51WI, Olympus, Japan). All chemicals were reagent grade (from Roth, Merck, or Sigma, Germany). CNQX was obtained from Biotrend (Germany), Picrotoxin was obtained from Sigma (Germany).

## **IMMUNOSTAINING AND IMAGING**

After recording, amygdala slices were fixed in 4% PFA in phosphate buffered saline (PBS) for 16–24 h at 4◦C. Slices were embedded in a block of 2% Agar-Agar and resectioned at 70μm. For visualization of projections within the amygdala, some sections were stained with Neurotrace (1:200, Invitrogen). Other sections that contained filled cells were permebealized in 0.3% Triton-X100 in PBS, and biocytin-filled cells were revealed using fluorescently-conjugated Steptavidin-Cy5 (1:200, Dianova, Germany). Immunostainings for parvalbumin were performed using standard procedures using mouse anti-Parvalbumin (Sigma, 1:2000) and Alexa-405-conjugated goatanti-mouse (Invitrogen, 1:1000) antibody. Sections were imaged using a laser scanning confocal microscope (LSM 710, Carl Zeiss, Germany) equipped with a 25 × 0*.*8◦ NA for overview of projections or filled cells, or a 63 × 1*.*4◦ NA objective and the pinhole set to 1 airy unit for colocalization of markers in filled cells.

## **LOCALIZATION OF INJECTION SITES AND FIBERS**

Coronal hippocampal sections were cut right after amygdala sections, immediately imaged on a fluorescent stereoscope (SCX16, Olympus, Japan) to confirm viral injection sites, and slices were fixed in 4% PFA in PBS for further analyses. The frontal cortex was removed, fixed overnight in 4% PFA in PBS, resectioned at 70μm, and stained with Neurotrace (1:200, Invitrogen). Bead only injection sites in the mPFC were imaged using a fluorescent stereoscope. Viral and bead injection sites in the mPFC were imaged on a laser scanning confocal microscope either with a 10 × 0*.*3◦ NA or a 25 × 0*.*8◦ NA objective with the pinhole open or set to one airy unit as indicated. All images were overlaid with the mouse brain atlas (Paxinos and Franklin, 2001).

## **DATA ANALYSIS AND STATISTICS**

All electrophysiological data were analyzed using the NeuroMatic suite of macros (http://www*.*neuromatic*.*thinkrandom*.*com/) and additional custom-written macros in IgorPro (Wavemetrics, USA). Input resistance (Rinput), series resistance (Rseries), membrane time constant and capacitance were calculated from 100 ms long, −5 mV voltage steps applied from a holding potential (Vhold) of −70 mV and were monitored throughout the experiment. Resting membrane potential was measured right after breaking into the cell by switching to current-clamp mode. Spiking patterns were elicited by applying depolarizing currents from 0 to +200◦ pA in 50◦ pA steps. Spike parameters were determined from the smallest current step that evoked one or a few action potentials. Spike threshold was determined as the voltage at which a *>*8-fold change in the rate of rise (in mV/ms) occurred. Spike amplitude was measured as the voltage difference between spike threshold and the peak of the spike. The spike half-width was measured as time difference between up- and downstroke of the spike at half-maximal amplitude. The fast afterhyperpolarization (fAHP) was measured in a 15 ms window after the peak of the spike, as the most negative membrane potential relative to the spike threshold. Synaptic current parameters were measured using Neuromatic functions on an average response generated from at least 10 individual sweeps. Amplitudes were measured as a negative or positive peak, or for late inhibitory currents as the average in a 1 ms time window 300 ms after stimulation. EPSC rise time was the time between 10–90% of maximal amplitude, and EPSC decay time was determined as time it took for the EPSC peak to decay to 37% of maximal amplitude. Latencies were measured as time between onset of stimulation and onset of the synaptic response.

Since recordings for specific group comparisons were all conducted with the same solutions and under identical conditions, we did not correct our data for the liquid junction potential. All data are reported as mean ± standard error of the mean. Statistical comparisons were performed using SPSS software (IBM, USA). For nominal data, Fisher's Exact Test or χ2-Test were used as indicated. Scaled data comparisons were performed using unpaired or paired Students *t*-test as indicated.

# **RESULTS**

To dissect which neurons in the amygdala receive inputs from mPFC and vHC, to characterize properties of these inputs, and to assess activated microcircuits, we used an *ex-vivo* optogenetic approach. We injected mice with recombinant Adeno-associated virus (rAAV) expressing the light activatable protein channelrhodopsin fused to either mCherry or eYFP. Viral injections into the mPFC either infected neurons mainly located in PL or IL, or a larger area of the mPFC, encompassing PL and IL, and sometimes parts of adjacent regions (**Figures 1A,E–G**). For all mPFC injection conditions, dense fluorescently labeled fibers were observed in the medial BA (**Figure 1B**). Injections into the hippocampus were targeted toward the caudal and ventral part (vHC, **Figures 2A,B**) resulting in labeled fibers in the medial BA and BMA (**Figures 2G,H**). In some of the animals with viral injections to mPFC or vHC, we also injected a retrograde tracer (retrobeads) to label mPFC-projecting principal neurons in the BA for recording (**Figures 1C,D**, **2C,H**). To identify interneurons in live brain slices, we used GAD-67-GFP reporter mice (Tamamaki et al.,

**FIGURE 1 | Viral and bead injection sites for studying mPFC inputs to BLA. (A)** Confocal image of a representative brain slice of an animal injected in the mPFC with rAAV-ChR2(H134R)-eYFP (green). Scale bar: 500μm. **(B)** Confocal image of a 35◦ tilted horizontal brain slice of the BLA with mPFC projections (green) corresponding to the injection site of **(A)**. Scale bar: 250μm. **(C)** Confocal image of a representative brain slice with retrobead injection site restricted to the PL region of the mPFC

(red). Scale bar: 250μm. **(D)** Image of an *ex vivo* recorded retrogradely labeled PN in the BA. Scale bar: 10 μm. **(E–G)** Overlay of mPFC viral injection sites (green) with the mouse brain atlas for animals categorized as having main injection sites in **(E)** mPFC (*n* = 13), **(F)** PL (*n* = 17) and **(G)** IL (*n* = 2). **(H–J)** Overlay with the mouse brain atlas for animals categorized as having the main retrobead injection site in **(H)** mPFC (*n* = 2), **(I)** PL (*n* = 13) and **(J)** IL (*n* = 2).

2003), and in a few experiments PV-Cre mice crossed with a red reporter mouse (Madisen et al., 2010).

## **PREFRONTAL AND HIPPOCAMPAL INPUTS DIFFERENTIALLY RECRUIT EXCITATORY AND INHIBITORY RESPONSES IN BA PRINCIPAL NEURONS AND INTERNEURONS**

We focused our recordings on neurons located in the medial part of the BA, the region where both, labeled mPFC and vHC axons were reliably observed in acute brain slices. Importantly, passive and active properties of principal neurons that received inputs from mPFC or the vHC were nearly indistinguishable (**Table 1**), and consistent with those recently described for recordings from the magnocellular region of the BA in mice (Senn et al., 2014). To study responses elicited by activation of channelrhodopsinpositive axons from the mPFC or vHC, we first recorded from principal neurons (PNs) and interneurons (INs) in currentclamp mode. As expected from a glutamatergic projection, all light-responsive PNs and INs displayed an initial depolarizing response that resembled an excitatory postsynaptic potential (EPSP, **Figures 3A,B**). With increasing stimulation intensity, a fraction of both PNs and INs responded with a spike arising from the EPSP (**Figure 3A**, bottom). In PNs, spikes were more readily elicited by mPFC- than vHC-fiber stimulation, while INs were equally likely to show spike responses for the two input pathways (**Figure 3B**). Under conditions where we did not elicit spikes in PNs, we observed also two types of hyperpolarizing responses, resembling fast and slow inhibitory postsynaptic potentials, which we called early and late IPSPs (**Figure 3C**). Stimulation of mPFC inputs elicited IPSPs in a substantial and similar fraction of PNs and INs (46–66%). In contrast, vHC afferent stimulation recruited IPSPs in a significantly larger fraction of PNs than INs (60 vs. 13%, **Figure 3D**). When comparing inputs, IPSPs were equally prevalent in PNs following stimulation of either mPFC or vHC inputs, whereas in INs, mPFC inputs were more likely to evoke IPSPs than vHC inputs (**Figure 3D**). This suggests that activation of the vHC recruits less inhibition onto INs when compared to PNs, or to neurons innervated by the mPFC.

In a second step, we addressed if activation of IL- vs. PLafferents would show different response profiles, by analyzing

rAAV-ChR2(H134R)-eYFP (green). **(B)** Overlay of vHC viral injection sites with the mouse brain atlas for all animals analyzed (*n* = 23). **(C)** Stereoscopic picture of a representative brain slice with retrobead injection site in the mPFC (red) of the same animal. **(D–F)** Overlay of main retrobead injection

(green) and retrogradely labeled principle neurons projecting to the mPFC (red) of the animal shown in **(A)** and **(C)**. Scale bar: 250μm. **(H)** Close-up of insert from **(G)** with vHC projections and retrobead-labeled neurons in the medial BA. Scale bar: 20μm.

subsets of neurons activated by fibers from localized IL and PL injections. Interestingly, we found no significant difference in excitatory or inhibitory response types onto PNs, or in the prevalence of excitatory and inhibitory input types from the PL to PNs vs. INs (**Figures 3E,F**). Although the dataset for IL inputs onto INs is very limited, overall our data suggest that IL and PL fiber activation leads to similar response type profiles for excitation and inhibition onto medial BA PNs and INs in naïve animals.

## **SYNAPTIC RESPONSES ARE COMPRISED OF EARLY EPSCs AND GABA<sup>A</sup> AND GABA<sup>B</sup> MEDIATED FEEDFORWARD IPSCs**

To confirm that light responses were generated by axonal activation and to dissect synaptic components, we performed voltage-clamp recordings. In keeping with our initial observation, we found that all PNs and INs showed light-evoked inward currents at −70 mV, which resembled excitatory postsynaptic currents (EPSCs, **Figures 4A–F**). These putative EPSCs were completely abolished by application of the sodium channel blocker tetrodotoxin (TTX, 1μM), indicating that they were driven by action potentials in ChR2-expressing axons (*n* = 3, data not shown). Neurons with only EPSP-like responses in current-clamp mode displayed a single component current response with a reversal potential close to 0 mV following mPFC or vHC fiber stimulation (**Figures 4A,D,G**). These responses were completely blocked by the AMPA/Kainate receptor antagonist CNQX (10μM, *n* = 3, **Figure 4H** left), indicating that they represent glutamatergic EPSCs. Recordings from neurons with a depolarizing and early hyperpolarizing profile revealed two current components in voltage-clamp recordings: an early component with a reversal potential close to 0 mV and a second component with a reversal potential close to −70 mV, the expected equilibrium potential for chloride, and thus GABAA-mediated inhibitory postsynaptic currents (IPSCs, **Figures 4B,E,G**). Consistent with the notion of an EPSC/earlyIPSC sequence, the second component was blocked by picrotoxin (PTX, 100μM, *n* = 9), and the first component was blocked by subsequent application of CNQX (*n* = 6, **Figure 4I**, left). Furthermore, the biphasic EPSC/earlyIPSC was also completely abolished by CNQX alone (*n* = 2, **Figure 4H**, right), a finding that is in agreement with feedforward inhibition. Lastly, we examined neurons with a late hyperpolarization in current-clamp mode. Here, we always found three current components, the first reversing around 0 mV (consistent with an EPSC), a second reversing around −70 mV (consistent with the early IPSC described above) and a third, small component with a reversal potential close to −90 mV, the expected equilibrium potential for potassium und thus, the effector channels of GABAB receptors (**Figures 4C,F,G**). Here, the GABAA antagonist PTX only blocked the early IPSC, and subsequent application of CNQX abolished the EPSC and late IPSC (*n* = 1, **Figure 4I**, right). Thus, neurons with a late


Example traces of excitatory response types showing subthreshold EPSP and EPSP-spike (E-spike) responses. Scale bar: 10 mV/25 ms. **(B)** Relative distribution (%) of excitatory response types in principal neurons (PN) and interneurons (IN) receiving synaptic inputs from mPFC (black) and vHC (blue). Legend for **(B,E)** is shown in panel **(E)**. Spikes were more readily elicited in PNs by mPFC inputs than vHC inputs (Relative occurrence of spikes was for mPFC→PN: E-spike 57%, *n* = 65 vs. vHC→PN: E-spike 18%, *n* = 45; Fisher's Exact Test, <sup>∗</sup>*p <* 0*.*001). No difference was observed in INs with mPFC and vHC inputs (mPFC→IN: E-spike 57%, *n* = 35 vs. vHCI→N: E-spike 38%, *n* = 16; Fisher's Exact Test, *p* = 0*.*237) or INs vs. PNs within each input type (Fisher's Exact Test, *p* = 1 for mPFC and *p* = 0*.*164 for vHC). **(C)** Example traces showing early and late inhibitory response types. Scale bar: 2 mV/50 ms. **(D)** Relative distribution (%) of purely excitatory (open bars) and additional inhibitory response types (early IPSP: closed bars, late IPSP: striped bars) in PNs and INs receiving input from mPFC and vHC. Legend for **(D,F)** is shown in panel **(F)**. Occurrence of inhibitory response types for mPFC→PN: early 55%, late 11%, *n* = 65; mPFC→IN: early 34%, late 11%, *n* = 35; vHC→PN: early 49%, late 11%, *n* = 45; and vHC→IN: early 12%, *n* = 16. Inputs from vHC recruited *(Continued)*

#### **FIGURE 3 | Continued**

IPSPs more readily in PNs compared to INs (*n* = 27*/*45 vs. *n* = 2*/*16; Fisher's Exact Test, <sup>∗</sup>*p* = 0*.*001). In INs, mPFC inputs recruited IPSPs more readily than vHC inputs (*n* = 16*/*35 vs. *n* = 2*/*16; Fisher's Exact Test, <sup>∗</sup>*p* = 0*.*028). **(E)** Relative distribution (%) of excitatory response types in neurons receiving synaptic input from prelimbic (PL) and infralimbic (IL) regions of the mPFC. Relative occurrence of spikes was for PL→PN: 57%, *n* = 35; IL→PN: 57%, *n* = 14 neurons; PL→IN: 53%, *n* = 15; IL→IN: 50%, *n* = 2. No difference in excitatory response types was observed (Fisher's Exact Test, all *p* = 1) **(F)** Relative distribution (%) of purely excitatory (open bars) and early and late inhibitory (closed and striped bars, respectively) response types in neurons receiving synaptic input from PL and IL regions of the mPFC. Occurrence of inhibitory response types for PL→PN: early 60%, late 11%, *n* = 35; and PL→IN: early 40%, late 7%, *n* = 15; IL→PN: early 36%, late 14%; *n* = 14; IL→IN: early 0%, late 50%, *n* = 2. There was no difference in inhibitory response types (Fisher's Exact Test, all *p >* 0*.*25).

inhibition also display an early inhibitory response in voltageclamp mode. We also isolated EPSCs and early IPSCs in Cs-based internal solution to precisely determine their onset latencies. Latencies of EPSC were consistently shorter than latencies of early IPSC latencies in a within-cell comparison (**Figures 5A,B**). Furthermore, the values are well in line with recent studies on optogenetic monosynaptic and disynaptic activation of EPSCs and IPSCs, respectively (Cho et al., 2013; Felix-Ortiz et al., 2013). Late IPSCs were not observed in Cs-based recordings, lending further support to their potassium-channel/GABAB receptor based mechanism.

In conclusion, our data show that neurons in the BA receive either exclusively excitatory glutamatergic inputs or a combination of excitatory and inhibitory inputs from mPFC and vHC. Although we cannot completely exclude a contribution of feedback inhibition, our findings strongly suggest that at least early inhibition is due to feedforward processes. Feedforward inhibition was frequently observed in BA PNs for both inputs (*>*60%), and prominent at mPFC inputs to INs, but rarely observed for vHC inputs onto INs (**Figure 5C**). Postsynaptic inhibition can either be mediated by GABAA receptors, or a combination of GABAA and GABAB receptors.

## **PROPERTIES OF mPFC- AND vHC-EVOKED EXCITATORY INPUTS IN BA DEPEND ON INPUT AND TARGET CELL TYPE**

To address if excitatory inputs onto different types of BA neurons have distinct properties, we compared EPSCs between PNs and INs in each input pathway and between input pathways. In all cases, synaptic latencies of EPSCs were consistent with monosynaptic activation (**Tables 2**, **5**). When comparing EPSC kinetics between neuron types, INs showed more rapid rise and decay times than PNs in both input pathways (**Figures 6A,B**), a feature previously described for local interneurons in hippocampus and amygdala (Mahanty and Sah, 1998; Jonas et al., 2004). Consistent with that, EPSCs in INs had similarly fast kinetics when comparing mPFC and vHC inputs (**Table 5**, **Figures 6A,B**). Interestingly, when comparing PNs, we found that EPSCs evoked by vHC input had a decreased latency and slightly but significantly faster rise and decay times than those originating from mPFC inputs (**Table 2**, **Figures 6A,B**).

To assess presynaptic properties, we performed paired-pulse stimulation and analysis. Because our TTX experiments suggested action potential-dependent neurotransmitter release, and ChR2(H134R) can follow stimulation frequencies up to 40 Hz reliably (Berndt et al., 2011), we used intervals between 50 and 300 ms for stimulation. At all intervals tested, we found a significant difference in the paired-pulse ratio (PPR) between mPFC and vHC inputs onto PNs and INs with consistently lower values for vHC inputs (**Figures 6C,D**). At the 50 ms interval, the PPR of vHC inputs was strongly depressing, suggesting a high release probability of these synapses, whereas mPFC inputs to PNs and INs showed higher values (around 1), suggesting a lower release probability (**Figures 6C,D**).

In our dataset, mPFC inputs to PNs evoked larger EPSCs than vHC inputs (**Table 2**). To rule out that the observed differences in PPR and EPSC kinetics between inputs may be due to amplitude differences, we used two approaches. Firstly, we performed amplitude-restricted analysis of EPSC properties (criterion: amplitudes *<*500 pA; mPFC→PN: −2*.*1 ± 22 pA, *n* = 40; vHC→PN: −2*.*0 ± 23 pA, *n* = 37; *p* = 0*.*081) and still detected significant differences in latency, rise, decay and PPR (*p* ≤ 0*.*01 for all). Secondly, we performed correlation analysis on both datasets. We found no correlation between amplitude vs. latency, kinetics, or PPR for vHC inputs (*p* ≥ 0*.*05 for all), and an opposite than expected correlation between amplitude and latency and amplitude and rise time for mPFC inputs (i.e., larger EPSC had faster latencies and shorter rise times, *p <* 0*.*05). Thus, in conclusion, differences in mPFC vs. vHC input properties did not result from amplitude differences. If faster latencies and kinetics of EPSCs in vHC inputs would result from dendritic filtering and/or synapse location, these parameters should be positively correlated. Indeed, we found a highly significant correlation between latency and rise time (*n* = 42, *R*<sup>2</sup> = 0*.*43; *p <* 0*.*0001) and rise and decay time (*n* = 42, *R*<sup>2</sup> = 0*.*21; *p* = 0*.*002).

In summary, EPSCs in INs displayed faster rise and decay times and lower PPRs when compared to their PN counterparts, features that would allow them to rapidly and reliably function in feedforward inhibitory circuits. Furthermore, vHC inputs are generally faster and more depressing than mPFC inputs. This is unlikely due to differences in the passive and active properties of target principal neurons (**Table 1**), or variability of EPSC amplitudes, but likely a feature resulting from differences in pre- and postsynaptic properties and location of specific inputs.

## **OVERALL INHIBITION/EXCITATION RATIO IS SIMILAR FOR DIFFERENT INPUT TYPES**

Although feedforward inhibition was observed with equal likelihood for mPFC and vHC inputs onto PNs, and also for mPFC inputs onto INs, one possibility is that the amount of inhibition could be different. We estimated the inhibitory drive by calculating the inhibition/excitation ratio (I/E ratio) from the peak amplitudes of the inward and outward components of the biphasic EPSC/IPSC recorded at −50 mV (Shin et al., 2006) (**Figure 7A**). The I/E ratio was highly variable, but on average not significantly different for mPFC vs. vHC inputs onto PNs (**Figure 7B**). There was also no difference in the I/E ratio between mPFC inputs onto PNs vs. INs (**Figures 7A,B**). Furthermore, activation of fibers

from subregions of the mPFC (large mPFC injections vs. specific PL and IL injections) did not show any significant differences in I/E ratios (**Figure 7C**). Taken together, in naive animals, our data indicate no clear differences in the strength of recruited feedforward inhibition when normalized to excitation between PNs and INs, and between different input types.

IPSC: −63*.*65 ± 0*.*94 mV, *n* = 42; late IPSC: −84*.*06 ± 2*.*56 mV, *n* = 7), INs

for PNs with mPFC input (EPSC: 1.88 ± 2.67 mV, *n* = 42; early

## **DISSECTION OF PROJECTION NEURON POPULATIONS TARGETED BY mPFC AND vHC AFFERENTS**

Projection neurons within the BA have diverse targets within and outside of the amygdala. In fear and extinction learning, changes in the activity of mPFC-projecting BA neurons play a critical role (Herry et al., 2008; Senn et al., 2014), while other types of projection neurons regulate anxiety-like behavior (Tye et al., 2011; Felix-Ortiz et al., 2013). Thus, we investigated properties of mPFC and vHC inputs onto mPFC-projecting BA neurons in a subset of animals that were co-injected with retrobeads in the mPFC (**Figures 1**, **2**). We first compared mPFC-backprojecting principal neurons (bPN) with their unlabeled neighboring cells (uPN). Although we cannot rule out false-negatives among uPNs, we assume that the vast majority of these cells do not project to mPFC. Overall, both mPFC and vHC inputs to bPN and uPN displayed identical response type distributions that included excitation and feedforward inhibition (**Figures 8A,B**). Additionally, these distributions resembled those observed for the entire PN population for both mPFC and vHC inputs (c.f. **Figure 3D**, all Fisher's Exact Tests: *p >* 0*.*15).

remaining EPSC (left and right panel), and late IPSC (right panel). Scale bars:

50 pA/5 ms (left) and 25 pA/25 ms (right).

When assessing properties of mPFC- and vHC-evoked EPSCs, we detected no difference between uPNs and bPNs within each input type (**Table 2**, all *t*-tests: *p >* 0*.*06). However, when comparing mPFC and vHC inputs to either bPNs or uPNs, we found the same significant changes in EPSC kinetics (except decay time), and in paired-pulse properties (**Table 2**). This confirms and supports our previous findings, and suggests that EPSC properties in

within-cell comparison of latencies of EPSCs (L exc) and IPSCs (L inh) in BA PNs for mPFC (*n* = 6) and vHC (*n* = 2) inputs. IPSC latencies were significantly slower that EPSC latencies (paired Students *t*-test: <sup>∗</sup>*p <* 0*.*01). **(C)** Proposed wiring scheme of excitatory and feedforward inhibitory connections from mPFC and vHC with neurons in the BA based on results shown in **Figures 3**–**5**. Numbers (%) represent the prevalence for recruitment of feed-forward inhibition from **Figure 3D**.

BA PNs are determined by afferent specificity rather than projection target specificity. Interestingly, when analyzing the I/E ratio, we revealed that cells that do not project to the mPFC (uPNs) received significantly less inhibition upon mPFC compared to vHC input stimulation (**Figure 8C**).

We next dissected inputs and outputs of BA PNs for subregions of the mPFC. In our dataset, we found the following combinations of inputs and outputs: mPFC→PL-projecting PN (plPN) (*n* = 3), mPFC→IL-projecting PN (ilPN) (*n* = 1), IL→plPN (*n* = 2) and PL→plPN (*n* = 17), vHC→plPN (*n* = 11), and vHC→ilPN (*n* = 10). When comparing input properties in the three most frequently observed groups (PL→plPN, vHC→plPN, and vHC→ilPN, **Table 3**), we again revealed feedforward inhibition as a salient feature, which was particularly prevalent at vHC inputs to PL-projecting cells, but not significantly different from the overall population of PNs (c.f **Figures 3D**, **8D,E**; Fisher's Exact Test, *p* = 0*.*14). Similar to the results above, properties of EPSCs appeared to be determined by input rather than output type for PNs (**Table 3**). Interestingly, the I/E ratio in neurons with feedforward inhibition was similar for vHC inputs onto PL- and IL- projecting cells, and similar for vHC and PL inputs onto PL-projecting cells (**Figure 8F**).


**Table 2 | Properties**

 **of EPSCs onto principal neurons.**

**FIGURE 6 | Properties of mPFC- and vHC-evoked EPSCs in BA neurons. (A)** Example traces of EPSCs recorded at −70 mV in BA principal neurons (PN) and interneurons (IN) receiving input from mPFC (top) or vHC (middle) and amplitude-scaled overlays showing faster rise and decay of EPSCs in IN. Amplitude-scaled overlays of mPFC- or vHC-evoked EPSCs in PNs and INs show faster EPSCs of vHC inputs in PNs (bottom). Scale bars: 100 pA/10 ms (top) and 5 ms (bottom). **(B)** Summary graphs for rise and decay times of EPSCs in PNs and INs for mPFC (black) and vHC inputs (blue), all values are in **Table 2** (all PNs) and **Table 5** (all INs). Rise and decay times were significantly faster for mPFC inputs onto INs vs. PNs (rise time: *p <* 0*.*001; decay time: *p <* 0*.*001) and vHC inputs onto INs vs. PNs (rise time: *p* = 0*.*002; decay time: *p <* 0*.*001). vHC inputs evoked faster EPSCs in PNs than mPFC inputs

In summary, our data suggest reciprocal connections from mPFC to mPFC-projecting cells in the BA, and a pathway from vHC to mPFC via BA projection neurons that target both, PL and IL. Synaptic properties in these pathways and connections are regulated by input rather than output specificity.

## **mPFC AND vHC INPUTS TARGET DIVERSE CLASSES OF INTERNEURONS IN THE BA**

The basolateral amygdala harbors different types of local interneurons with partially distinct physiological and molecular signatures (Ehrlich et al., 2009; Spampanato et al., 2011). To address the diversity of interneurons receiving input from mPFC and vHC, we analyzed some of their passive and active properties, but found no overall differences (**Table 4**). However, in spike input-output curves, we observed a tendency of INs with vHC input to fire action potentials with higher frequencies (not shown). One group of INs that can be unequivocally identified electrophysiologically, are fast-spiking interneurons. Therefore, (rise time: *p <* 0*.*001, decay time: *p* = 0*.*01). **(C)** Example traces of EPSCs evoked by paired pulse stimulation (interval: 50 ms) at −70 mV in PNs and INs after stimulation of mPFC (top left) or vHC (top right) inputs. Overlays scaled to the amplitude of the first EPSC illustrate differences in paired pulse ratio (bottom). Scale bars: 100 pA/25 ms (top) and 10 ms (bottom). **(D)** Summary graph for paired pulse ratios (PPR) of EPSCs at different stimulation intervals in PNs and INs for mPFC (black) and vHC inputs (blue), all values are in **Table 2** (all PNs) and **Table 5** (all INs). PPRs of mPFC inputs onto INs vs. PNs were not significantly different (*p >* 0*.*05 for all intervals). PPRs of vHC inputs onto PNs vs. INs were significantly larger (∗*p <* 0*.*05 for all intervals). PNs and INs receiving mPFC input showed significantly larger PPRs than PNs and INs receiving vHC inputs (PN: <sup>∗</sup>*p <* 0*.*01; IN: <sup>∗</sup>*p <* 0*.*05 for all intervals).

we classified individual INs as fast-spiking (fsIN) or non-fast spiking (nfsINs) based on previously published criteria including firing rate and patterns, and spike waveform (Rainnie et al., 2006; Woodruff and Sah, 2007b; Spampanato et al., 2011). Indeed, in both datasets (mPFC or vHC input) cells classified as fsINs exhibited significantly higher spike frequencies in input-output curves than nfsINs, and little spike frequency adaptation (**Figures 9A,B**). Furthermore, fsINs displayed a significantly shorter spike halfwidth and a more pronounced fast afterhyperpolarization (fAHP) (**Figures 9C,D**). Some of the fsINs were *post-hoc* identified as positive for the calcium binding protein parvalbumin (**Figure 9F**). Within the population of INs with input from mPFC, only 17% were fsINs, whereas among those with vHC input, 44% were fsINs (**Figure 9E**). When comparing this with the expected prevalence of fsINs of ∼20% amongst all INs (McDonald and Mascagni, 2001, 2002; Woodruff and Sah, 2007b), fsIN were overrepresented in the population with vHC, but not with mPFC input (χ2-tests, *p* = 0*.*02 and *p* = 0*.*65, respectively). Furthermore, fsINs were

more frequently targeted by vHC than mPFC afferents (Fisher's Exact Test, *p <* 0*.*05). When assessing inputs from specific mPFC injection regions, we observed the following prevalences in connectivity: mPFC→fsIN (*n* = 2*/*15, 13 %), PL→fsIN (*n* = 4*/*16, 25 %), IL→fsIN (*n* = 0*/*4, 0 %), suggesting differences between PL and IL in innervation of INs.

We also tested directly if mPFC and vHC inputs activate parvalbumin (PV)-expressing INs (pvINs) by recording from cells in PV-reporter mice. In a small sample, we found that mPFC afferent stimulation evoked light responses in pvINs that resembled fsINs (*n* = 5) and vHC afferent stimulation evoked light response in pvINs with more diverse firing patterns and lower average spike frequency (*n* = 4, **Figure 9G**). However, all targeted pvINs displayed a short spike half-width and large fAHP (**Figure 9H**). Light responses in pvINs had latencies of 2.5 ± 0.26 ms and 2.15 ± 0.22 ms (for mPFC and vHC input, respectively) consistent with monosynaptic activation.

We next compared response type profiles for different IN types. In agreement with findings for the overall population of INs (**Figure 3D**), a fraction of nfsINs, fsINs, and pvINs showed feedforward inhibition upon mPFC input activation (**Figures 9H,I**). In contrast, vHC afferent stimulation elicited feedforward inhibition only in ≤25% of nfsINs and pvINs (1/4 cells, not firing at high frequency), but in none of the fsINs (**Figures 9I,J**). In summary, mPFC and vHC inputs activate nfsINs and fsINs including PV-positive cells. Inputs from mPFC target diverse populations of INs and consistently evoke feedforward inhibition in all IN types. In contrast, vHC inputs are more likely to target fsINs, which in turn do not receive feedforward inhibitory inputs.

# **DISCUSSION**

We investigated cellular and synaptic interactions between mPFC and vHC with target neurons in the medial BA, a region innervated by both areas. As expected, PNs and local INs received monosynaptic, excitatory inputs from mPFC and vHC. In addition, both inputs recruited GABAergic feedforward inhibition in a substantial fraction of PNs, but mPFC inputs more frequently recruited feedforward inhibition onto INs, suggesting activation of dis-inhibitory circuits in the BA. Amongst the innervated PNs we identify neurons that project back to subregions of the mPFC, indicating a loop between neurons in mPFC and BA, and a pathway from vHC to mPFC via BA. A general feature of both mPFCand vHC-evoked EPSCs onto local INs is that they show faster rise and decay kinetics compared to PNs. However, mPFC and vHC inputs to both PNs and INs differ in their presynaptic properties. Our data describe wiring principles and features of synaptic connections from mPFC and vHC to amygdala that may help to interpret functional interactions of these brain areas at the network level.

## **FEEDFORWARD INHIBITION ONTO BA PROJECTION NEURONS IS A SALIENT FEATURE**

We identified feedforward inhibition onto PNs in the BA as a prominent feature of mPFC and vHC inputs. Previous *in vivo* studies have yielded conflicting results about recruitment of local inhibition in the BLA by mPFC stimulation, possibly due to methodological constraints (Rosenkranz and Grace, 2001, 2002; Likhtik et al., 2005). Our approach is not compromised by activation of en-passant fibers or backfiring of BA projection neurons, and also allowed for detection of small inhibitory currents. We were able to recruit inhibition even at stimulation intensities that did not fire BA PNs. Together with latency and pharmacological analyses, this provides strong evidence for feedforward, rather than exclusive feedback inhibition. Thus, vHC and mPFC inputs to BA are similarly controlled by local feedforward inhibition. As at sensory inputs to the LA, this may serve to limit excitation, and to gate activity and plasticity (Li et al., 1996; Lang and Paré, 1997; Szinyei et al., 2000; Bissière et al., 2003; Shin et al., 2006).

In naïve animals, we found a highly variable inhibition/excitation ratio, even when comparing subgroups of PNs with specific inputs or outputs. This could reflect variability in inhibitory synapse location (perisomatic or proximal vs. distal dendritic), synaptic strength, number of inhibitory synapses on the target cell, or local interaction of inputs. However, we cannot rule out that this may also be partially influenced by technical variations (e.g., number of infected axons, differences in viability of INs, or IN-connectivity in slices). It is conceivable that mPFCand vHC- driven feedforward inhibition is a substrate for plastic changes in BA inhibitory synaptic markers (Chhatwal et al., 2005; Heldt and Ressler, 2007), or in inhibitory innervation of

**FIGURE 8 | mPFC and vHC connectivity with and properties of inputs to BA projection neurons. (A)** Scheme of projections observed onto retrobead-labeled mPFC-projecting (bPN) and unlabeled BA principal neurons (uPN). **(B)** Relative distribution (%) of purely excitatory (open bars) and additional inhibitory response types (early IPSP: closed bars, late IPSP: striped bars) in uPNs and bPNs. Occurrence of inhibitory response types for mPFC→uPN: early 62%, late 10%, *n* = 21; mPFC→bPN: early 62%, late 8%, *n* = 26; vHC→uPN: early 43%, late 21%, *n* = 14; and vHC→bPN: early 67%, late 8%, *n* = 24. No significant differences were found between inputs or cell types (Fisher's Exact Test, *p >* 0*.*05). **(C)** Summary graph of inhibition-to-excitation ratio (I/E ratio) in uPNs and bPNs for mPFC (black) and vHC inputs (blue). All values are in **Table 2**. Individual data points illustrate the high variability of I/E ratios. Synaptic inputs onto uPNs showed significantly lower I/E ratios for mPFC inputs

functionally identified PNs (Trouche et al., 2013) upon fear and extinction learning.

#### **IDENTITY OF INTERNEURONS RECRUITED BY DIFFERENT INPUTS**

Additional evidence for feedforward inhibition stems from reliable activation of local INs in the BA. In keeping with observations in other systems, EPSCs in INs displayed faster kinetics than EPSCs in PNs. This likely results from expression of glutamate receptors with fast kinetics, and enables rapid and temporally precise signaling in feedforward circuits (Jonas et al., 2004; Polepalli vs. vHC inputs (uPN mPFC vs. vHC: *p* = 0*.*027; all other comparisons: <sup>∗</sup>*p >* 0*.*05). **(D)** Scheme of specific projections observed onto retrobead labeled PNs projecting to PL (plPN) and IL (ilPN) subdivisions of the mPFC. **(E)** Relative distribution (%) of purely excitatory (open bars) and additional inhibitory response types (early IPSP: closed bars, late IPSP: striped bars) in plPNs and ilPNs. Occurrence of inhibitory response types for PL→plPN: early 56%, late 11%, *n* = 12; vHC→plPN: early 80%, late 10%, *n* = 10; vHC→ilPN: early 45%, late 9%, *n* = 11. No significant differences were found between inputs or cell types (Fisher's Exact Test, *p >* 0*.*05). **(F)** Summary graph of I/E ratio in plPNs and ilPNs for PL (black) and vHC inputs (blue). All values are in **Table 3**. Individual data points illustrate the high variability of I/E ratios. No significant difference in I/E ratios was observed for mPFC or vHC inputs onto plPNs (*p* = 0*.*413), or vHC input onto plPNs vs. ilPNs (*p* = 0*.*740).

et al., 2010). Our data suggest that several types of interneurons are part of feedforward inhibitory circuits, including fast-spiking PV-positive (PV+) cells, and non-fast spiking PV+ and PVnegative cells. It has been proposed that fast-spiking PV+ INs are part of feedforward circuits in the BA based on their lower than expected innervation by local glutamatergic afferents (Woodruff and Sah, 2007b). On the other hand, cortical (including mPFC) innervation of PV+ cells in the BA was underrepresented compared to local innervation, suggesting that PV+ cells also participate in feedback inhibition (Smith et al., 2000). Our data may


*PPR, paired-pulse ratio; I/E ratio, inhibition-to-excitation ratio; PN, principal neuron; PL, prelimbic cortex; IL, infralimbic cortex; plPN, principal neuron projecting to prelimbic cortex; ilPN, principal neuron projectingto infralimbic cortex; values represent mean* ± *s.e.m. (n, number of recorded cells); comparisons: superscript numbers represent pairs of significantly different values.*


*ExcitabilityspikefrequencypAinjection;afterhypeipolarization;spikingspikingmean* ± *s.e.m. (n, number of recorded cells); statistical comparisons: superscript numbers represent pairs of significantly different values.*

**FIGURE 1 | Viral and bead injection sites for studying mPFC inputs to BLA. (A)** Confocal image of a representative brain slice of an animal injected in the mPFC with rAAV-ChR2(H134R)-eYFP (green). Scale bar: 500μm. **(B)** Confocal image of a 35◦ tilted horizontal brain slice of the BLA with mPFC projections (green) corresponding to the injection site of **(A)**. Scale bar: 250μm. **(C)** Confocal image of a representative brain slice with retrobead injection site restricted to the PL region of the mPFC

(red). Scale bar: 250μm. **(D)** Image of an *ex vivo* recorded retrogradely labeled PN in the BA. Scale bar: 10 μm. **(E–G)** Overlay of mPFC viral injection sites (green) with the mouse brain atlas for animals categorized as having main injection sites in **(E)** mPFC (*n* = 13), **(F)** PL (*n* = 17) and **(G)** IL (*n* = 2). **(H–J)** Overlay with the mouse brain atlas for animals categorized as having the main retrobead injection site in **(H)** mPFC (*n* = 2), **(I)** PL (*n* = 13) and **(J)** IL (*n* = 2).

2003), and in a few experiments PV-Cre mice crossed with a red reporter mouse (Madisen et al., 2010).

## **PREFRONTAL AND HIPPOCAMPAL INPUTS DIFFERENTIALLY RECRUIT EXCITATORY AND INHIBITORY RESPONSES IN BA PRINCIPAL NEURONS AND INTERNEURONS**

We focused our recordings on neurons located in the medial part of the BA, the region where both, labeled mPFC and vHC axons were reliably observed in acute brain slices. Importantly, passive and active properties of principal neurons that received inputs from mPFC or the vHC were nearly indistinguishable (**Table 1**), and consistent with those recently described for recordings from the magnocellular region of the BA in mice (Senn et al., 2014). To study responses elicited by activation of channelrhodopsinpositive axons from the mPFC or vHC, we first recorded from principal neurons (PNs) and interneurons (INs) in currentclamp mode. As expected from a glutamatergic projection, all light-responsive PNs and INs displayed an initial depolarizing response that resembled an excitatory postsynaptic potential (EPSP, **Figures 3A,B**). With increasing stimulation intensity, a fraction of both PNs and INs responded with a spike arising from the EPSP (**Figure 3A**, bottom). In PNs, spikes were more readily elicited by mPFC- than vHC-fiber stimulation, while INs were equally likely to show spike responses for the two input pathways (**Figure 3B**). Under conditions where we did not elicit spikes in PNs, we observed also two types of hyperpolarizing responses, resembling fast and slow inhibitory postsynaptic potentials, which we called early and late IPSPs (**Figure 3C**). Stimulation of mPFC inputs elicited IPSPs in a substantial and similar fraction of PNs and INs (46–66%). In contrast, vHC afferent stimulation recruited IPSPs in a significantly larger fraction of PNs than INs (60 vs. 13%, **Figure 3D**). When comparing inputs, IPSPs were equally prevalent in PNs following stimulation of either mPFC or vHC inputs, whereas in INs, mPFC inputs were more likely to evoke IPSPs than vHC inputs (**Figure 3D**). This suggests that activation of the vHC recruits less inhibition onto INs when compared to PNs, or to neurons innervated by the mPFC.

In a second step, we addressed if activation of IL- vs. PLafferents would show different response profiles, by analyzing

#### **FIGURE 9 | Continued**

(black) or vHC input (blue). Scale bar: 20 mV/2 ms. Right: Graphs of spike half-width and fast after hyperpolarisation (fAHP) for nfsINs and fsINs with mPFC (black) or vHC inputs (blue), all values are in **Table 4**. Spike half width was broader in nfsINs vs. fsINs (mPFC: *p <* 0*.*001; vHC: *p* = 0*.*005) and the fAHP was smaller nfsINs vs. fsINs for mPFC inputs (mPFC: *p* = 0*.*003; vHC:*p* = 0*.*127). **(E)** Graph showing the distribution of BA IN-types with light responses (%). fsIN were more likely to be recruited by vHC vs. mPFC inputs (Fisher's Exact Test, *p <* 0*.*05). **(F)** Confocal image of fsIN recorded in a GAD67-GFP mouse (green), filled with Biocytin (pink), and identified as PV-positive (blue). Scale bar: 5μm. **(G)** Left: Example traces of spike responses to a 200 pA/500 ms current injection in PV-positive INs from PV-Cre reporter mice (pvIN) receiving mPFC (black) or vHC inputs (blue). Scale bar: 20 mV/100 ms. Right:

and BA-projecting neurons in the vHC and PL are activated upon renewal (Orsini et al., 2011). This has lead to a model in which convergent inputs from PL and vHC drive BA activity to increase fear output. Although we have no direct evidence, the fact that PL and vHC target PL-projecting BA neurons, supports the idea of input convergence at least onto PL-projecting PNs. Our findings also delineate reciprocal connections between PL and BA PNs. In fact, a modeling study indicates that this bidirectional PL-BA loop can contribute to sustained CS-responses in PL and BA during fear expression and attributes a major role to PL inputs to BLA in this process (Pendyam et al., 2013). This fits well with our observation of non-depressing excitatory PL inputs and activation of dis-inhibitory circuits in the BA, features that could support sustained activation in PL→BA circuits. Additionally or in parallel, fear responses can also be regulated at the level of the mPFC. Here, vHC-mediated inhibition is thought to gate BLA-driven PL activation, resulting in turn in a net decrease in BLA activation and reduced fear output (Sotres-Bayon et al., 2012).

At the level of the BLA, it emerges that output specificity of BLA projection neurons also defines their behavioral role. Neurons targeting mPFC subregions or vHC have distinct roles in acquired and extinguished fear states and anxiety. For example, activity of PL- vs. IL-projecting BA cells supports the expression of high fear or low fear after extinction, respectively (Senn et al., 2014). The projection from BA to vHC controls anxiety by mediating excitation and polysynaptic inhibition in the vHC (Felix-Ortiz et al., 2013). Furthermore, vHC-projecting BA neurons also become activated during expression of acquired fear (Senn et al., 2014). Because vHC-projecting neurons are located in the magnocellular BA, the site of convergence of vHC and mPFC inputs, it is likely that their activity is also controlled by mPFC and vHC inputs to generate behavioral outputs.

We hoped to identify differences in vHC and mPFC inputs onto mPFC-projecting BA neurons (which include fear and extinction neurons) that could help to explain their responses and roles during distinct high and low fear behavioral states (Herry et al., 2008; Senn et al., 2014). However, in naïve animals, wiring of mPFC and vHC inputs and response types in PL- and ILprojecting cells and unlabeled counterparts (largely comprised of neurons projecting outside the mPFC) were similar. One important consideration is that not all PL- and IL-projecting cells are fear or extinction neurons, respectively, and input specificity Input-output curves show that pvINs receiving mPFC vs. vHC inputs fire with significantly higher frequency (∗*p <* 0*.*05). **(H)** Left: Spike waveforms in pvINs with mPFC and vHC input. Scale bar: 20 mV/2 ms. Right: Spike half-width and fAHP were similar for pvINs with mPFC or vHC input. Spike half width: 0.6 ± 0.1 ms vs. 0.5 ± 0.0 ms; fAHP −17*.*9 ± 1*.*1 vs. −19*.*7 ± 2*.*1 mV; *p >* 0*.*05. **(I)** Relative distribution (%) of purely excitatory (open bars) and additional inhibitory response types (early IPSP: closed bars, late IPSP: striped bars) in nfsINs, fsINs, and pvINs. Occurrence of inhibitory response types for mPFC→nfsIN: early 34%, late 14%, *n* = 29; mPFC→fsIN: early 33%, late 0%, *n* = 6; vHC→nfsIN: early 22%, late 0%, *n* = 9; and vHC→fsIN: early 0%, late 0%, *n* = 7. No significant differences were found between inputs or cell types (Fisher's Exact Test, *p >* 0*.*05). **(J)** Wiring scheme of different interneuron types in upon activation of mPFC and vHC afferents.

could be limited to a subset of these anatomically defined neurons (Senn et al., 2014). Secondly, specific response types may only emerge through synaptic plasticity upon learning (Vouimba and Maroun, 2011; Cho et al., 2013).

Based on the opposing roles of PL and IL in fear expression and extinction one would expect differences in amygdalar activation by these inputs (Milad and Quirk, 2002; Burgos-Robles et al., 2009; Sierra-Mercado et al., 2011). Our data show that PNs in magnocellular BA are similarly activated and receive similar feedforward inhibition. This is consistent with a recent study suggesting no difference in IL and PL inputs onto BA PNs in naïve animals (Cho et al., 2013). While these authors also suggest that PL and IL inputs onto BA PNs might show equal changes upon fear extinction learning, our data imply that the impact could differ because of differential recruitment of INs that participate in feedforward circuits. Moreover, the effect of PL/IL inputs might also depend on the identity of the postsynaptic PN, and thus might have been overlooked by Cho et al. Indeed, in our hands, IL inputs target mainly non-fast spiking INs, which may be dendrite-targeting INs that either control local plasticity, or are subject to differential modulatory control (Klausberger, 2009; Spampanato et al., 2011; Chiu et al., 2013). Thus, different interneuron subtypes may participate in routing information from defined inputs to distinct and functionally diverse postsynaptic cell populations in the basolateral amygdala.

Taken together, while on the one hand our findings identify some clear differences and specializations, our data reveal a number of general and preserved features of inputs from the IL, PL, and the vHC to neurons and microcircuits in the BA. Changes in the activity of specific BA neurons upon fear and extinction learning (Herry et al., 2008; Amano et al., 2011) could then emerge either in a state-dependent manner, controlled by neuromodulators, and/or due to cellular or synaptic plasticity upon learning. This plasticity may alter recruitment of excitation, feedforward inhibition, or dis-inhibition in a cell-type specific manner. The wiring principles and synaptic features of connections from mPFC and vHC to BA described here serve as a foundation for further investigations into the roles of these inputs during fear and anxiety-related behavior, and into elucidating specific sites and mechanisms of plasticity within these circuits *in vivo* and *ex vivo*.

# **ACKNOWLEDGMENTS**

This work was supported by funding from the Hertie Foundation, the Centre for Integrative Neuroscience (DFG, Exc 307), and a NARSAD Young Investigator Award (to Ingrid Ehrlich). We would like to thank members of the Ehrlich lab for comments throughout the work and Verena Senn for comments on an earlier version of the manuscript. We acknowledge support by the Deutsche Forschungsgemeinschaft and Open Access Publishing Fund of Tuebingen University.

## **REFERENCES**


lateral amygdala in the expression and extinction of conditioned fear. *Neuropsychopharmacology* 36, 529–538. doi: 10.1038/npp.2010.184


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 30 November 2013; paper pending published: 21 January 2014; accepted: 13 February 2014; published online: March 2014. 05*

*Citation: Hübner C, Bosch D, Gall A, Lüthi A and Ehrlich I (2014) Ex vivo dissection of optogenetically activated mPFC and hippocampal inputs to neurons in the basolateral amygdala: implications for fear and emotional memory. Front. Behav. Neurosci. 8:64. doi: 10.3389/fnbeh.2014.00064*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2014 Hübner, Bosch, Gall, Lüthi and Ehrlich. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Identification and optogenetic manipulation of memory engrams in the hippocampus

#### **Steve Ramirez<sup>1</sup> , Susumu Tonegawa1,2 and Xu Liu1,2\***

<sup>1</sup> Department of Biology and Department of Brain and Cognitive Sciences, RIKEN-MIT Center for Neural Circuit Genetics at the Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, MA, USA

<sup>2</sup> Howard Hughes Medical Institute, Massachusetts Institute of Technology, Cambridge, MA, USA

#### **Edited by:**

Anton Ilango, National Institutes of Health, USA

#### **Reviewed by:**

Mazahir T. Hasan,

Charité-Universitätsmedizin, Germany Yu Zhou, Medical College of Qingdao University, China

#### **\*Correspondence:**

Xu Liu, Department of Biology and Department of Brain and Cognitive Sciences, RIKEN-MIT Center for Neural Circuit Genetics at the Picower Institute for Learning and Memory and Howard Hughes Medical Institute, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Building 46-5261, Cambridge, MA 02139, USA e-mail: xuliu@mit.edu

With the accumulation of our knowledge about how memories are formed, consolidated, retrieved, and updated, neuroscience is now reaching a point where discrete memories can be identified and manipulated at rapid timescales. Here, we start with historical studies that lead to the modern memory engram theory. Then, we will review recent advances in memory engram research that combine transgenic and optogenetic approaches to reveal the underlying neuronal substrates sufficient for activating mnemonic processes. We will focus on three concepts: (1) isolating memory engrams at the level of single cells to tag them for subsequent manipulation; (2) testing the sufficiency of these engrams for memory recall by artificially activating them; and (3) presenting new stimuli during the artificial activation of these engrams to induce an association between the two to form a false memory. We propose that hippocampal cells that show activitydependent changes during learning construct a cellular basis for contextual memory engrams.

**Keywords: optogenetics, memory engram, IEG, ChR2, false memory**

#### **A BIOLOGICAL LOCUS FOR MEMORY**

Memories thread and unify our overall personal narrative. Disruption of the putative neural correlates for memories in humans leads to devastating maladies and dramatically impairs cognition. Even when they are not subject to experimenter manipulations or natural insults, memories are not fully veridical representations of past experiences. Recalling a memory makes it labile, which can distort the mental representation of an event, incorporate misinformation, and sometimes even fabricate illusory episodes entirely (Schacter and Loftus, 2013). Despite the importance of memories in our daily lives and the comprehensive studies on this subject, the process by which memories emerge through the interactions of neurons distributed across various brain regions is a poorly understood phenomenon (Eichenbaum, 2004; Squire et al., 2007).

The biological conceptualization of a memory was given a name—an "engram", by the German Zoologist Richard Semon in 1921 (Semon, 1921). A decade later, the American Psychologist Lashley et al. (1932) pioneered a systematic hunt for engrams in the rodent brain by lesioning various parts of cerebral cortex and relating the size and the location of the lesion to behavioral performance on a maze task. His experiments lead him to formulate the *mass action principle*, which posits that memories are spread throughout the cortex and not localized to discrete brain regions (Lashley et al., 1932). For Lashley, the biological locus for a single memory remained elusive. Years later, the Canadian Neurosurgeon Penfield and Rasmussen (1950) observed the first tantalizing hint that certain memories could be localized in defined brain regions. During his surgeries for patients with epilepsy, Penfield applied small jolts of electricity to the brain to reveal which regions were centers for causing seizures. Remarkably, while stimulating parts of the medial temporal lobe (MTL), he observed that 8% of his patients reported vivid recall of random episodic memories (Penfield and Rasmussen, 1950). This finding suggests that the MTL region harbors the biological locus for episodic memory.

Then in 1953, the American Neurosurgeon William Scoville and British Neuropsychologist Brenda Milner tested the conjecture that the MTL had distinct contributions to episodic memories (Scoville and Milner, 1957). To treat the epileptic convulsions that incapacitated his patient Henry Molaison (H.M.), Scoville and Milner (1957), like Penfield and Rasmussen (1950), resected the problematic neural tissue, which involved removal of large sections of the hippocampus and adjacent areas in this case. For the decades to come, H.M. lost his ability to form new memories—to bridge personal events across large spans of time (anterograde amnesia)—while simultaneously failing to recall events for years leading up to his surgery (retrograde amnesia). Scoville and Milner (1957) work on H.M. also pointed to the MTL in general and to the hippocampus in particular as an essential locus for episodic memory.

Since then, a large number of subsequent studies in humans (Rempel-Clower et al., 1996; Schmolck et al., 2002), as well as in non-human primates and rodents (Jarrard, 1993; Zola and Squire, 2001), have established that the hippocampus is crucial for the formation of memories that include "what-wherewhen" components, or context-temporal-informational domains, which are known as episodic memories (Eichenbaum, 2004). Additionally, the hippocampus' structure and function has been extraordinarily conserved across mammalian clades, permitting a thorough experimental interrogation and deconstruction of its functions in animal models (Eichenbaum, 2003).

As a candidate mechanism supporting mnemonic processes, the strengths of synapses throughout the hippocampus are thought to be altered in an experience-dependent manner. The idea of neural plasticity dates back to Plato, who originally conjectured that memories leave a stamp or trace in the mind analogous to the impression on a wax tablet left by a signet ring (Campbell, 1883). In the 21st Century, Hebb (1949) put a modern spin on Plato's dialogue and hypothesized that neurons that "fire together" also "wire together" (Hebb, 1949)—a conceptual antecedent of long-term potentiation (LTP). Bliss and Lomo (1973) experimental demonstration of LTP (Bliss and Lomo, 1973), followed by the essential role of NMDA receptors in LTP induction (Collingridge et al., 1983) opened a way to investigate LTP as synaptic mechanism underlying certain forms of learning and memory. The result of the initial pharmacological blockade experiments conducted with an NMDA receptor (NMDAR) antagonist, AP5, were consistent with the notion that LTP is essential for spatial learning (Morris et al., 1986), and the validity of this notion was demonstrated with the more definite targeted genetic ablation of the NMDAR in the CA1 region of the hippocampus (Tsien et al., 1996), although the cortical NMDAR may also have contributed to the phenotype (Fukaya et al., 2003). A subsequent report concluded that CA1 NMDAR were dispensable for spatial learning *per se*, but the same report showed an easily detectable level of NMDAR RNA in CA1 and hence the possibility that the remaining CA1 NMDAR supported spatial learning cannot be excluded (Bannerman et al., 2012). The authors reactivated an old hypothesis (Vinogradova, 1975) that ascribes CA1/DG NMDAR a different role in learning and memory processes. However, this provocative and controversial hypothesis would require careful and critical examination and further investigation in the future. Mice with NMDAR ablation in DG and CA3 showed impairments of pattern separation and pattern completion, respectively (McHugh et al., 1996, 2007; Nakazawa et al., 2003). Another study did not detect the effect of NMDAR deletion in the DG cells on pattern separation (Niewoehner et al., 2007), but this may be due to the fact that different behavioral paradigms were used. Overall, these observations supported the role of NMDARdependent synaptic plasticity, including LTP, in hippocampaldependent memory.

#### **MOLECULAR SIGNATURES OF MEMORIES**

In more recent years, the role of the different circuits within the hippocampal-entorhinal cortex network, as well as young vs. old DG granule cells, in specific aspects of hippocampaldependent learning and memory has been identified by using targeted genetic manipulations (Nakashiba et al., 2008, 2009, 2012; Clelland et al., 2009; Drew et al., 2010; Suh et al., 2011). While these past studies have been pivotal in our understanding of the role of the different subfields and circuits in learning and memory, they were conducted without distinguishing between cells that were activated by specific sensory or cognitive stimuli and cells that remained inactive.

To identify which cells are active during the formation of a memory, one can rely on the activity-dependent nature of immediate early genes (IEGs). It is believed that the formation of long-term memory (LTM) requires gene transcription and protein translation at the time of training to alter neural morphology, receptor densities, and overall excitability of the cells (Jones et al., 2001). Multiple rounds of transcription have been identified after learning and most studies focus on IEGs, which are transcribed within minutes in an experience-dependent manner by transcription factor proteins already present in the cytoplasm of a neuron (Guzowski, 2002).

The most well characterized IEGs are *zif268*, *c-fos*, and *Arc/Arg3.1*, and all of them have been implicated in supporting memory formation. Mice with a deletion of *zif268* show deficits in contextual fear conditioning (CFC) and the hidden platform variant of the Morris Water Maze (MWM) test (Jones et al., 2001). A similar result was obtained in mice lacking the *c-fos* gene in the central nervous system (Fleischmann et al., 2003). These mice also had impaired LTP, but the developmental effects of *c-fos* gene deletion could not be excluded from contributing to the observed phenotypes. Post-developmental antisense oligodeoxynucleotide (As-ODN)-mediated blockade of *c-fos* translation in the hippocampus caused impaired consolidation of inhibitory avoidance, MWM, and socially transmitted food preference behaviors (Guzowski and McGaugh, 1997; Guzowski, 2002; Countryman et al., 2005), all of which are tasks thought to depend on the integrity of the hippocampus. Translational inhibition of *Arc* by As-ODN, and mice with global genetic deletion of *Arc*, have demonstrated an obligatory role for this IEG in memory consolidation for MWM, fear conditioning, conditioned taste aversion, and novel object recognition tasks (Guzowski et al., 2000; Plath et al., 2006).

In addition, studies of *Arc* and *c-fos* expression after behavioral training have shown that the proportion of cells expressing these IEGs in DG (2–6%), CA3 (20–40%), and CA1 (40–70%) after exposure to a novel environment resembles the proportion of hippocampal excitatory cells physiologically active in a given environment, which further validates the use of IEGs as an indicator of recent neural activity (Vazdarjanova and Guzowski, 2004). The cellular expression pattern of *c-fos* and *Arc* is different for different contexts, but remains stable upon re-exposure to the same context. These observations indicate that memory engram is highly conjunctive in nature but with remarkably labile synaptic properties that allow flexible memory updating (Guzowski et al., 2006; Richards and Frankland, 2013). It is in light of these studies that our first hypothesis for memory engrams emerges—namely, cells expressing *c-fos* after a training episode are participating in the encoding of the memory for that specific experience. Therefore, these cells may represent a component of the stored memory engram.

## **EXPERIMENTAL EVIDENCE FOR MEMORY ENGRAMS**

To pinpoint a biological process as the underlying mechanism for a specific phenomenon, three types of evidence are normally required. These are: correlation, blockade, and mimicry. Correlation is to record the parallel occurrence between the phenomenon and the process, which will show an indirect relationship between these two; blockade means interrupting the candidate process, and if this also interferes with the phenomenon, then this shows the *necessity* of the process for the expression of the phenomenon; mimicry is to artificially generate the process, and if by doing so one can recreate the phenomenon, then this demonstrates *sufficiency*.

These principles also apply if one wants to demonstrate that engram-bearing cells are the basis for memories. For correlation experiments, molecular and physiological changes were found in specific neuronal ensembles accompanying memory formation from insects to humans. In the *Drosophila* olfactory learning circuit, defined neuronal populations or even single neurons change their response properties selectively towards odors used in training after olfactory conditioning (Yu et al., 2006; Liu and Davis, 2009). In mice, overlapping populations of cells in the amygdala are activated during the acquisition and recall of a fear memory (Reijmers et al., 2007). IEGs are expressed in largely overlapping populations of neurons in the rat hippocampus and neocortex during repeated exposure to the same environment (Guzowski et al., 1999; Vazdarjanova et al., 2002). In addition, single neurons recorded from the human hippocampus and MTL are shown to respond reliably to the same images or episodes (Quiroga et al., 2005; Gelbard-Sagiv et al., 2008). These observations suggest that, if there is a cellular basis for memory across different species, then it is sparsely encoded in a stable population of neurons.

Researchers have also conducted blockade experiments on selected cells to show the necessity of engram-bearing cells for different types of memories. In two studies, researchers pioneered a novel loss-of-function approach to perturb a component of a memory engram. By selectively ablating or inhibiting sparse population of cells in the amygdala that are preferentially recruited into the representation of a fear memory, the researchers interfered with the recall of that memory in mice (Han et al., 2009; Zhou et al., 2009). Moreover, in rats, selective inactivation of a small population of neurons in the nucleus accumbens that were previously activated by cocaine has also been shown to attenuate the memory for the drug-associated environment (Koya et al., 2009).

Compared to the observation and blockade experiments, the mimicry experiments for memory engram studies remained a considerable challenge. Although it had been widely recognized and agreed that such experiments are essential to test the engram hypothesis (Martin and Morris, 2002; Gerber et al., 2004), the lack of tools that could precisely label and control selected neurons involved in a particular memory posed a formidable obstacle to carry out these experiments. Demonstrating the existence of memory engrams at the cellular level requires a system that can selectively label and activate the memory engram-bearing cells to induce the predicted behavioral changes caused by learning.

## **IDENTIFYING MEMORY ENGRAMS**

To selectively activate a cell population bearing an engram for a particular memory, one needs to be able to isolate and label these cells for future manipulation. Our first goal was to develop and characterize an activity-dependent and inducible system to label only the cells involved in the formation of a specific memory with channelrhodopsin-2 (ChR2). ChR2 is a light-sensitive channel that allows the influx of cations when illuminated by ∼470 nm blue light (Nagel et al., 2003), resulting in the activation of the neurons expressing this channel. We began by expressing ChR2- EYFP fusion protein in an activity dependent, doxycycline (Dox) regulated manner. This approach ensures that only neurons active during a defined episode become labeled for subsequent control by light stimulation.

We used the TetTag mouse (Reijmers et al., 2007), which harbors a pivotal transgene of interest, *c-fos*- tetracycline transactivator (tTA). This transgene contains the *c-fos* promoter, which drives the expression of the tTA. In this transgenic line, tTA can mimic the expression pattern of endogenous *c-fos* and only transiently appear in activated cells. The tTA protein will bind to the tetracycline-responsive element (TRE) to trigger the expression of a downstream target gene. The binding of tTA to TRE is blocked by Dox, which can be administered through an animal's diet. If Dox is removed from the food, a window for activity-dependent labeling is opened and tTA can bind to TRE to turn on the expression of a gene of interest—ChR2-EYFP in this case—only in activated cells (**Figure 1A**).

Consistent with its role in processing various aspects of spatial and temporal information, the hippocampus is known to be critical for the formation of the contextual component of fear memories (Kim and Fanselow, 1992; Phillips and Ledoux,

**FIGURE 1 | Activity-dependent labeling of neurons. (A)** Training-induced neuronal activity drives the expression of tTA, which in turn activates downstream gene expression through TRE promoter and labels active cells with ChR2-EYFP (yellow). This process can be blocked by the presence of Dox. **(B)** The animals are kept on Dox (green background) before the training, so any irrelevant stimuli (represented by various symbols) during this period will not cause cells to express ChR2-EYFP. The animals are taken off Dox during training, so cells active during the memory formation will be labeled. Animals are put back on Dox diet post-training and during testing, so no further labeling happens.

1992). Of the various hippocampal subregions, computational models predict that DG in particular orthogonalizes inputs from entorhinal cortex into separate spatial representations (Treves and Rolls, 1994). Behavioral data support DG's essential role in discriminating between similar contexts (McHugh et al., 2007), and *in vivo* recordings in freely moving animals show that DG granule cells are exquisitely sensitive to subtle changes in contextual information (Leutgeb et al., 2007). Cellular studies of IEG expression show that sparse populations of DG granule cells (2–6%) are activated in a given context (Chawla et al., 2005; Schmidt et al., 2012). Moreover, whereas a largely overlapping population of DG granule cells is activated repeatedly in the same environment, different environments or even different tasks in the same environment activate different populations of DG (Kubik et al., 2007; Satvat et al., 2011). These lines of evidence point to the DG as an ideal target for the formation of contextual memory engrams that represent discrete environments, guiding us to select the DG as our target for potential engram cell labeling.

We targeted the DG of *c-fos*-tTA transgenic mice with an AAV virus vector carrying TRE-ChR2-EYFP and implanted an optical fiber directly above the site of injection for light delivery. The mice were raised on a Dox diet to prevent any tTA-dependent transcription. This ensures that tTA produced by unintended activity throughout development or other experiences prior to our behavioral training will not induce the expression of ChR2- EYFP. When the mice are fear conditioned while on a Dox-free diet, the tTA expresses in a similar pattern as *c-fos* and enables the transcription of ChR2-EYFP, thus tagging putative fear engrambearing cells. Then the mice are put back on diet containing Dox right after training to prevent the labeling of new cells in response to experiences after training (**Figure 1B**).

### **ACTIVATING MEMORY ENGRAMS**

Prior to fear conditioning, we performed habituation sessions to measure the animals' basal freezing levels to a novel context (context A), during which they were on Dox diet and ChR2-EYFP was not expressed. As expected, the animals showed minimal amounts of freezing behavior during both light-on and lightoff epochs. They were then taken off Dox to open a window for activity-dependent labeling and fear conditioned in a different context (context B) to label fear memory engram-bearing cells, and placed back on Dox diet immediately after training. A day later, the animals were placed back into context A and stimulated with light to activate the neurons labeled during fear conditioning in context B and test the behavioral consequences.

This experiment directly tests the hypothesis that the *c-fos*expressing cells in the hippocampus activated during training are sufficient for memory recall. Indeed, optogenetic reactivation of these cells resulted in freezing behavior indicative of fear memory recall (Liu et al., 2012), thus demonstrating their causal contributions to activating the behavioral expression of a memory.

Two control groups were also tested, and none of them showed any light induced freezing before or after training. The first control group did not receive foot shocks during the training. Histological data suggested that simply exposing mice to a novel context elicited as much DG activity as exposure to a novel context plus shock, as similar numbers (∼6%) of DG cells became *c-fos* positive after either condition. This group demonstrated that the freezing in the experimental groups was not due to the optical activation of a population of hippocampal neurons unrelated to a fear memory. The second control group underwent the exactly same training as the experimental group but expressed EYFP alone instead of ChR2-EYFP. This control demonstrated that the light-induced freezing in the experimental groups was not due to changes in the salience of light after fear conditioning or other non-specific effects induced by light.

We also tested whether the light-activated memory recall was context-specific. We began by exposing animals that were off Dox to context A and label DG cells with ChR2-EYFP, then the animals were placed back on Dox to prevent any further labeling. They were then fear conditioned the next day in context B. Thus, these animals have both a ChR2-labeled non-fear memory engram for neutral context A and an unlabeled fear memory engram for context B. Crucially, our neuronal data showed that two statistically independent populations of DG cells were recruited to encode two discrete environments. These observations argue that DG orthogonalizes input at the neuronal ensemble level and recruits distinct sets of cells for distinct experiences. As predicted, this group of animals did not show increased freezing upon light stimulation, despite the presence of a fear memory from context B. This result argues that DG memory engram cell populations are context-specific.

Together, these data suggest that re-activating DG cells that were active during fear conditioning training is sufficient to induce the recall and behavioral expression of that fear memory. Accordingly, we propose that these cells form a cellular basis of a memory engram, and that two different contexts are parsed out as independent experiences represented by independent neuronal ensembles in DG.

Our study provides a methodological framework to study how an animal's environment is represented in neuronal ensembles in the hippocampus. The strength of our system lies in its precision of tagging only relevant cells for future manipulation. This system can target specific brain regions, specific cell types, and also specific cell populations involved in a particular memory, which are otherwise indistinguishable from their neighboring cells. It focuses the tremendous power of optogenetics onto behaviorally relevant cell ensembles and enables the circuit and functional mapping of multiple memory engrams throughout the brain.

#### **FALSE MEMORIES IN THE BRAIN**

In the early 1930s, the British psychologist Frederic Bartlett constructed and recited short but slightly inconsistent fables, most famously *The War of the Ghosts,* to test subjects from various cultural backgrounds (Bartlett, 1932). Strikingly, when asked to recall the fable, many subjects unknowingly modified the fable into a logical story that also contained new elements that fit within their cultural milieu. Bartlett (1932) discovered that memory distortion can occur in such a way that contextual information currently in mind (i.e., information being recalled) can act as a backdrop for the addition of new information.

The integration of new information into an already constructed memory has been shown to occur in both humans and animals. Mnemonic processes are reconstructive in nature, as the act of recalling a memory renders it labile and highly susceptible to modifications (Nader et al., 2000; Debiec et al., 2002; Tse et al., 2007). Memory's imperfections are not limited to pathological cases, as they are also present in healthy humans, in whom distortions and illusions of memories occur frequently. Such modifications can occur through the incorporation of misinformation into memory from external sources, such as leading questions, deception, and other causes—a phenomenon termed *suggestibility.* They can also occur through the phenomenon of *misattribution*, when retrieved information is assigned to the wrong source. Striking examples abound demonstrating the dramatic instances in which suggestibility and misattribution errors distort memories of crime scenes, childhood events, and traumatic experiences, which were often recalled under interrogation in the court of law or during psychotherapy sessions (Loftus et al., 1978; Schacter and Loftus, 2013).

Interestingly, the activity of the anterior MTL in general and the hippocampus in particular have been positively correlated with the strength of both veridical and false memory recall (Cabeza et al., 2001), thus making the hippocampus an ideal candidate region for interrogating the neuronal conditions that support false memory formation. Amnesic patients with MTL atrophy, likewise, are sometimes less susceptible to false recognition than normal controls (Schacter et al., 1996). Taken together, these results suggest that MTL networks participating in generating episodic imagery are also involved in misremembering imagined events as previously experienced episodes.

Human studies utilizing behavioral and fMRI techniques, however, have not been able to delineate which hippocampal subregions are sufficient for false memory formation, thus necessitating animal models for a more spatially and temporally precise analysis of these neural circuits. While cognitive psychology has greatly enhanced our understanding of false memories through functional neuroimaging and behavioral studies, the underlying neuronal and circuit level processes that enable these cognitive quirks remain vastly unexplored. Few studies to date have utilized rodent models specifically to study the neural substrates underlying false memories. Two studies (McTighe et al., 2010; Romberg et al., 2012) investigated object recognition memory in rats with surgical or pathological perirhinal cortex lesions and found that experimental rats tended to treat novel experiences as familiar, thus leading to the false recognition of objects. Three caveats abound, however: such lesions are difficult to restrict spatially and to reproduce reliably across subjects in terms of volume of brain damaged; these lesions are temporally imprecise; and finally, they do not directly target the cells that participate in forming the engram under investigation.

#### **OPTOGENETIC INCEPTION OF A FALSE MEMORY**

Building on our previous finding that DG hippocampal cells recruited during learning define an active neural population that is sufficient for memory recall upon subsequent activation (Liu et al., 2012), we asked the following question: can an artificially activated contextual memory engram serve as a conditioned stimulus (CS) and become associated with an unconditioned stimulus (US) to form an artificial CS-US association?

To test this, we began by taking the animals off Dox and labeling cells active during the exploration of a neutral context (context A) with ChR2-mCherry. We then put the animals back on Dox and fear conditioned them in a different context (context B) while optically activating the labeled cells involved in encoding the context A. We hypothesized that light-activated context A cells could produce an artificial CS while the mice were simultaneously administered an US to form an artificial associative fear memory. Indeed, when placed back in context A, the experimental group of animals displayed increased freezing levels and hence were freezing to a context in which they were never actually shocked before (Ramirez et al., 2013)—a result that perhaps parallels some types of false recognition memories in humans. Importantly, when placed in a novel context (context C), the animals showed low freezing levels, which indicated that the freezing is contextspecific and is not simply a result of generalization (**Figure 2**).

It is possible that the light-induced activity from context A cells interfered with natural fear memory acquisition of context B. To test this possibility, we re-exposed the trained animals to context B and measured freezing levels across groups. Three findings emerged: (1) the experimental group of animals showed decreased freezing compared to control groups of animals trained in context B alone, suggesting that the activity of context A cells interfered with the animal's ability to form a representation of context B normally; (2) when given light-on epochs, the experimental group displayed an increase in freezing, which indicated that a natural fear memory (for context B) and an artificiallyinducted false fear memory (for context A) can have an additive effect; and (3) in a group of animals that had context A cells labeled with ChR2-mCherry but in which light was omitted during fear conditioning in context B (so that context A cells still represent a neutral context), light-on epochs decreased the freezing responses while re-exposed to context B, suggesting that the activity of cells representing a neutral context A may have a competitive effect on natural fear memory recall for context B.

**FIGURE 2 | Inception of a false fear memory.** Top: The behavior paradigm for the experimental animals. Animals were kept on Dox post-surgery (green background), then taken off Dox and allowed to explore context A to label active cells with ChR2. Then they were put back on Dox and fear conditioned (lightening symbol) in context B while receiving light stimulation (blue shower symbol) to activate cells representing context A. When they were put back to context A, they showed a false fear memory for A (freezing indicated by wavy lines) where they were never actually shocked. They showed no fear memory for a control context C and a genuine fear memory for context B where they were shocked. Bottom: Cellular activity. Red, gray, and white circles indicate neurons representing contexts A, B, and C respectively. Asterisks indicate neurons activated either naturally by contextual exposure or artificially by light stimulation.

We could now also probe the behavioral relevance of the DG cells that were artificially associated with an aversive event of high valence (e.g., US). If an artificial CS-US association was generated using our experimental parameters, then activation of these labeled cells should now be sufficient to elicit the associated behavioral output (i.e., freezing). Indeed, when placed in a novel context D and stimulated with light, the experimental group displayed light-induced freezing, suggesting that these activated DG cells have become part of a fear engram as a result of being associated with an US. In a sense, the false memory became a real memory.

To map the downstream brain areas involved in light-induced false memory formation, we measured *c-fos* expression after three different treatments: the false fear memory recall in context A, the natural fear memory recall in context B, and the neutral memory recall in context C. We measured the number of *c-fos* positive cells in the amygdala, which is an essential site for forming fear memories (Rogan et al., 1997). Previous electrophysiological and immunohistochemical studies have shown positive correlations between amygdala IEG activity and freezing levels (Holahan and White, 2004; Knapska and Maren, 2009). Accordingly, we observed that natural fear memory recall in context B and false memory recall in context A elicited similarly robust levels of *c-fos* expression in the basolateral amygdala (BLA) as well as in the central amygdala (CeA), whereas animals exploring context C showed low basal *c-fos* levels. Quantifications revealed that the recall of a natural and false memory both activated 25% of BLA and CeA (Ramirez et al., 2013). These results show that natural fear memory recall and false fear memory recall activate similar proportions of amygdala cells, arguing that both memories recruit similar circuits involved in producing fear memories.

Interestingly, a previous study attempted a similar experimental schedule to generate an artificial memory using pharmacosynthetic methods, and failed to see increased freezing upon re-exposure to either context A or context B alone. Instead, a synthetic memory was observed, which can be recalled only with the presence of both context A (artificially activated by a pharmacosynthetic ligand) and context B (naturally occurring during re-exposure; Garner et al., 2012). Compared to our viralbased optogenetic manipulations, a key difference in their system is that all *c-fos*-expressing cells in the forebrain and midbrain were labeled and activated over the span of minutes to hours with a synthetic ligand. The differences observed here have two implications: the region-specific optogenetic manipulations with millisecond precision, when compared to forebrain-wide pharmacogenetic perturbations that last several minutes, perhaps more reliably recapitulates the endogenous neural activity required for behavioral expression; and, perhaps not all *c-fos*expressing brain regions are sufficient to elicit the recall of a CS. To further explore this conjecture, we performed the same experiments described above but targeted CA1 instead. We found that 40–70% of CA1 neurons are labeled with ChR2-mCherry in response to context exposure, and we hypothesized that optical activation of such a large population of CA1 neurons, which are also known to utilize highly precise temporal codes to encode information, may not activate a context-specific representation. Indeed, optical stimulation of CA1 engram-bearing cells failed to act as a discrete CS during the presentation of an US to construct an association between the two, as was observed in the DG (Ramirez et al., 2013).

Together, these experiments showed that optical activation of a hippocampal contextual memory engram could act as an artificial CS during fear conditioning to form an artificial CS-US association, or a putative false memory, because the artificial memory never had its contiguous experiences naturally linked. Along similar lines, a recent optogenetic study elegantly showed that pairing lateral amygdala (LA) stimulation—such that activated LA cells substituted as an US—with an auditory CS was sufficient to induce freezing responses later when the CS was presented alone (Johansen et al., 2010), while a second study demonstrated that activating a random population of piriform cortex neurons paired with rewards or shocks could elicit the associated appetitive or aversive behavioral output upon stimulation of the same neurons (Choi et al., 2011). These two studies provide strong evidence that CS and US information can be artificially driven in subpopulations of neurons, and our data expand these findings by providing activity-dependent and context-specific leverage over a defined memory.

While the relationship between our animal model and human false memories remains unclear presently, it does enable future study of memory-updating processes at the level of discrete engrams. Notably, the formation of false memories in humans often occurs as a result of recombining mnemonic elements of discrete experiences into a new, reconstructed memory that is not a veridical representation of the past. These memories are often not *de novo* and require pre-existing memories as a scaffold onto which distinct experiences can be incorporated to update the memory itself (Gershman et al., 2013). Similarly, in our mouse model, our artificial memory is not a *de novo* construction; rather, it is a result of artificially linking a pre-existing memory and an event of high valence. Whether or not the reactivated memory is purely Pavlovian in nature or contains episodic components is a topic currently under investigation.

### **CONCLUSIONS**

In summary, we have shown that hippocampal engram-bearing cell populations underlying previously acquired memories can activate the recall of the associated memories upon subsequent stimulation, and that this activity can form a functional CS, which in turn can be integrated into the simultaneous formation of a discrete fear memory. Our work also suggests that IEGexpressing neurons can form a cellular basis for memory engrams and that these cells have direct, causal relevance in producing memory recall. Identifying the neural underpinnings behind engram formation may yield important clues into the treatment of patients with pathological MTL atrophy, in whom episodic memory is profoundly impaired (Carlesimo and Oscar-Berman, 1992; Fleischman and Gabrieli, 1999).

Since episodic memory is essential to a normal life, an understanding of the normal circuitry underlying hippocampal function is crucial if we are to understand the diseased state in disorders that give rise to episodic memory impairments (Hodges et al., 1990; Storandt et al., 2002; Tamminga, 2013). The findings presented here enable the cellular and functional mapping of memories in different brain regions and the causal dissection of their role in producing associated behaviors.

Indeed, the DG cells we activated to produce recall could also serve as both a technical and conceptual gateway to access memory engrams distributed throughout the brain to produce a variety of behaviors. Such activity-dependent neuronal ensembles are not limited to the DG of the hippocampus, and their roles are not limited to contextual memory either. For example, a recent study found that activating specific ensemble of neurons in the LA was sufficient to induce the recall of an established fear memory (Kim et al., 2014). Similar neuronal ensembles in areas like the prefrontal cortex (PFC), thalamus, BLA, and nucleus accumbens (NAc) have also been shown to play important roles in conditioned addiction (Cruz et al., 2013).

Looking into the future of memory engram research, many challenging questions still remain to be addressed. For example: what is happening at synapse level in these engram-bearing cells (Takeuchi et al., 2013); what is the minimum cell population required to activate a memory (Deng et al., 2013); what occurs to these cells during memory consolidation (Tayler and Wiltgen, 2013) and extinction (Trouche et al., 2013); whether or not defined engram-bearing cells are necessary for memory recall (Drew et al., 2013); and, whether or not similar principles also apply to appetitive memories. From a broader point of view, it is foreseeable that similar engram technologies can be applied to other neuronal circuits to understand complex processes such as anxiety (Felix-Ortiz et al., 2013), depression (Li et al., 2013), and social interaction (Adolphs, 2010; Stowers et al., 2013). On the technology development front, new tools are concurrently being generated to meet these increasing demands. These tools include innovative molecular biology constructs (Mayford, 2013), engineered artificial promoters (Kawashima et al., 2013), novel transgenic animal models (Guenthner et al., 2013; Huang and Zeng, 2013; Sando III et al., 2013), advanced optics (Packer et al., 2013), and cutting-edge nanotools (Alivisatos et al., 2013). Equipped with these powerful tools, memory engram studies will continue to advance our understanding of the brain by causally probing the neuronal basis of learning and memory.

## **AUTHOR CONTRIBUTIONS**

Steve Ramirez, Susumu Tonegawa and Xu Liu wrote the paper.

## **ACKNOWLEDGMENTS**

We thank R. L. Redondo and T. J. Ryan for their comments on the manuscript. This work was supported by RIKEN Brain Science Institute and Howard Hughes Medical Institute.

#### **REFERENCES**


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 22 November 2013; paper pending published: 06 December 2013; accepted: 27 December 2013; published online: 17 January 2014*.

*Citation: Ramirez S, Tonegawa S and Liu X (2014) Identification and optogenetic manipulation of memory engrams in the hippocampus. Front. Behav. Neurosci. 7:226. doi: 10.3389/fnbeh.2013.00226*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience*.

*Copyright © 2014 Ramirez, Tonegawa and Liu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

# Optogenetic and chemogenetic insights into the food addiction hypothesis

## **Michael J. Krashes and Alexxai V. Kravitz\***

Diabetes, Endocrinology, and Obesity Branch, National Institute of Diabetes and Digestive and Kidney Diseases, National Institutes of Health, Bethesda, MD, USA

#### **Edited by:**

Mary K. Lobo, University of Maryland School of Medicine, USA

#### **Reviewed by:**

Scott Sternson, Howard Hughes Medical Institute, USA Ingo Willuhn, Netherlands Institute for Neuroscience, Netherlands

#### **\*Correspondence:**

Alexxai V. Kravitz, Diabetes, Endocrinology, and Obesity Branch, National Institute of Diabetes and Digestive and Kidney Diseases, National Institutes of Health, Building 10-CRC, Room 5-5932, 10 Center Drive, Bethesda, MD 20814, USA e-mail: lex.kravitz@nih.gov

Obesity is clinically diagnosed by a simple formula based on the weight and height of a person (body mass index), but is associated with a host of other behavioral symptoms that are likely neurological in origin. In recent years, many scientists have asked whether similar behavioral and cognitive changes occur in drug addiction and obesity, lending many to discuss the potential for "food addiction". Advances in understanding the circuitry underlying both feeding behaviors and drug addiction may allow us to consider this question from the viewpoint of neural circuits, to complement behavioral perspectives. Here, we review advances in understanding of these circuits and use them to consider whether drawing comparisons to drug addiction is helpful for understanding certain forms of obesity.

**Keywords: obesity, addiction, optogenetics, food, feeding, arcuate, striatum**

Drug addiction is a chronic, relapsing disorder that is characterized by physical signs such as tolerance and withdrawal, as well as emotional and behavioral symptoms such as sensations of craving and compulsive reward-seeking. Tolerance describes a phenomenon in which higher doses of a drug are required to achieve an effect, while withdrawal signs describe a range of physiological and emotional consequences that occur when an addict stops taking a drug. The behavioral changes associated with drug addiction can be broadly grouped into three main categories (Koob and Volkow, 2010). First, drugs and associated cues exert strong effects on reinforcement processes, driving drugdirected behavior to become compulsive. Second, drug addiction is accompanied by impaired inhibitory control processes, which normally act as the brakes on behavior. Finally, drug addiction is complemented by negative emotional states such as anxiety and depression, which can serve as triggers to drive further drug use. Indeed, drug-abstinent humans and animals are most vulnerable to relapse during periods of emotional stress or hardship (Epstein et al., 2006; Koob, 2008; Erb, 2010; Sinha et al., 2011). These three classes of symptoms may reflect alterations in distinct circuitry, which work together to facilitate drug use in addicted individuals. We will describe recent optogenetic and chemogenetic studies that have provided hypothetical maps of what this circuitry might be.

The term "food addiction" was introduced into the literature in the 1950s (Randolph, 1956), but there were few published studies on this topic in the subsequent 60 years. Instead, a large number of researchers addressed drug addiction during this time (**Figure 1**). This has changed in very recent years, during which a small but growing number of researchers have begun investigating food addiction. Modern researchers are in an ideal position to investigate this link, as the United States and many other countries have become entrenched in an obesity epidemic that must be addressed (Centers for Disease Control, 2013), and societal acceptance of "food addiction" is commonplace, as evidenced by the large number of support groups for over-eating, many of them based on the 12-step framework developed to address drug and alcohol dependence (Weiner, 1998; Russell-Mayhew et al., 2010). Indeed, several measures of substance use (particularly cigarette smoking) in the US have been on the decline in recent decades, while the prevalence of obesity has risen steadily (Centers for Disease Control, 2013).

Like drug addiction, obesity is a complex disorder with multiple causes and symptoms. For example, a small number of obese individuals have monogenic receptor mutations (such as in the leptin and melanocortin receptors) that cause extreme weight gain (Farooqi and O'Rahilly, 2008). However, the majority of obesity that has developed in the past 30 years is not believed to be the result of monogenic mutations, but rather changes in our food supply and lifestyles during this time (Farooqi and O'Rahilly, 2008). The behavioral signs and symptoms that are associated with this obesity can be loosely mapped to the same categories as drug addiction: compulsive overconsumption, difficulty controlling the intake of food, and the emergence of negative emotional states such as anxiety and depression (Kenny, 2011a; Sharma and Fulton, 2013; Sinha and Jastreboff, 2013; Volkow et al., 2013). Therefore, it is possible that the circuit changes underlying these processes in obesity are similar to those that occur during drug addiction. It is worth noting, however, that like drug addiction, specific obese individuals often exhibit subsets of these dysfunctions, such that an individual is likely to exhibit different specific symptoms, and alterations in circuitry. In addition, feeding depends on homeostatic feeding circuitry which is critical for survival, a distinct difference from drug addiction.

Neuroscience Information Framework (NIF). RIID:nif-0000-25673

Conceptually, feeding has often been regarded as the product of two independent networks that integrate and control food intake, hunger and hedonic pleasure (Kenny, 2011b). In addition to reward circuitry that likely contributes to both drug addiction and obesity, a homeostatic system also regulates food intake based on caloric need by circulating blood borne factors such as glucose, free fatty acids, leptin, ghrelin and insulin (Myers and Olson, 2012; Adan, 2013; Hellström, 2013). These engage hypothalamic and brainstem circuits to promote or blunt feeding responses, thus contributing to normal energy balance. This is one way in which obesity differs from drug addiction, as obesity may reflect alterations in homeostatic feeding circuitry, in addition to changes in reward circuitry. Importantly, novel tools have been developed that allow neuroscientists to manipulate circuits with unprecedented precision and control (Fenno et al., 2011; Rogan and Roth, 2011; Tye and Deisseroth, 2012). In this review, we outline recent research on the circuitry underlying both feeding and drug addiction, and discuss the degree to which analysis of this circuitry can shed new light on the similarities and differences between obesity and drug addiction.

#### **CIRCUITRY MEDIATING HOMEOSTATIC FEEDING**

Studying the mechanisms of homeostatic food intake is challenging due to slow temporal kinetics of the parameters mediating the switch between hunger and satiety. Hormones need to be released from peripheral tissues, travel to the brain and signal nutrient-sensing neurons to direct food-seeking and consumption behavior. These prolonged changes in energy deficit considerably hamper the examination of the contributing relationships between deprivation-sensitive sensory systems and the downstream brain circuits they engage. To side-step this difficulty, manipulations of molecularly circumscribed nutrient-sensing neurons can be used to prove the central control of feeding. Once identified, the afferent and efferent pathways modulating both hunger and satiety can be further analyzed in detail (Sternson, 2013).

The arcuate nucleus (ARC) of the hypothalamus constitutes a variety of diverse cell-types that are ideally situated to integrate blood-borne signals released from peripheral tissues, as the ARC rests at the base of the brain adjacent to the third ventricle and median eminence. Specifically, two distinct ARC subpopulations, orexigenic agouti-related protein (AGRP) and anorexigenic proopiomelanocortin (POMC) neurons have been substantially linked to alterations in food intake. Both heterogeneous subtypes are conversely stimulated and inhibited by the fat-derived hormone leptin (Myers and Olson, 2012) and the energy signals glucose (Claret et al., 2007; Fioramonti et al., 2007) and insulin (Konner et al., 2007; Hill et al., 2010). Moreover, AGRP neurons are directly activated by the hunger-promoting gut-derived hormone ghrelin (Cowley et al., 2003; van den Top et al., 2004). Further bolstering their respective contributions to eating, pharmacological injections into the brain of the neuromodulators released by AGRP neurons, the peptides AGRP and neuropeptide Y (NPY) escalate feeding (Semjonous et al., 2009), while α-melanocyte stimulating hormone (α-MSH) and adrenocorticotrophic hormone (ACTH), released from POMC neurons, attenuate food intake (Poggioli et al., 1986).

Optogenetic or chemogenetic (Aponte et al., 2011; Krashes et al., 2011, 2013; Atasoy et al., 2012) activation of AGRP neurons is sufficient to rapidly elicit voracious food intake, even in calorically replete animals, linking activation of these neurons to the perception of hunger and subsequent feeding. Importantly, the degree of consumption is dependent on both the number of excitable neurons and stimulation frequency (Aponte et al., 2011). Chronic activation of these neurons and the resulting hyperphagia and reduced energy expenditure leads to marked weight gain, accompanied by increased fat stores (Krashes et al., 2011). Furthermore, the neuromediators released by AGRP neurons drive biphasic feeding episodes with GABA and/or NPY promoting acute food intake while the peptide AGRP orchestrates food consumption over a delayed, chronic scale (Atasoy et al., 2012; Krashes et al., 2013). Interestingly, animals with acutely stimulated AGRP neurons during a normal resting period, in the absence of food, display intense, unabated locomotor activity that is completely reversed in the presence of food, strongly suggesting a foraging role for these neurons (Krashes et al., 2011). Furthermore, remote AGRP-induction significantly increases an animal's willingness to work for food in a classic nosepoke assay (Krashes et al., 2011).

To investigate the downstream functional contributions of AGRP neurons on feeding, long-range axon projections were photostimulated and resulting food intake was assessed. Selective terminal-field activation in the paraventricular (PVN) hypothalamus evoked feeding in a similar magnitude to direct somatic AGRP activation, implicating a crucial role for neurons in this brain site in directing appetite signaling (Atasoy et al., 2012). To definitively demonstrate this, two forms of chemogenetic inhibition were used to silence the majority of PVN neurons, resulting in escalated *ad lib* food intake and motivation to work for food. Furthermore, elegant occlusion studies whereby

**118**

AGRP afferents to the PVN and downstream PVN neurons marked by a mouse oxytocin (OXT) promoter fragment were cotransduced with channelrhodopsin-2 (ChR2) and simultaneously photostimulated, completely reversing the AgRP→PVN-evoked increase in food intake. Finally, by applying combinatorial optoand chemogenetic manipulations with pharmacology, alternative downstream circuits of AGRP neurons were implicated in eliciting feeding behavior. Recently, it was revealed that AGRP axonal projections to the bed nucleus of the stria terminalis (BNST), lateral hypothalamus (LH) or paraventricular thalamus (PVT), in addition to the PVN, are sufficient to drive feeding (Betley et al., 2013; need to add this ref PMID: 24315102). Importantly, distinct AGRP axonal projections that target different anatomical brain regions originate from specific subpopulations, whereby a "oneto-one" axon collateral configuration for AGRP neurons governs downstream connectivity (Betley et al., 2013).

Conversely to experiments testing AGRP sufficiency, tools used to acutely suppress AGRP neurons revealed their necessity in feeding (Krashes et al., 2011), which parallels the hypophagic response in animals following conditional ablation of these cells (Gropp et al., 2005; Luquet et al., 2005). This neural ablation approach led to the identification of an anorexia circuit in the parabrachial nucleus (PBN; Wu et al., 2009), which receives inhibitory input from AGRP neurons (Atasoy et al., 2012) and critical excitatory input from the nucleus of the solitary tract (NTS), which in turn is activated via serotonergic projections from the raphe magnus and obscurus (Wu et al., 2012). Notably, acutely abrogating glutamatergic signaling from the PBN increases food intake, implicating the importance of excitatory tone from this anatomical region in guiding feeding behavior (Wu et al., 2012). To further demonstrate the PBN has key regulator of appetite, a novel circuit, marked by calcitonin gene-related peptide-expressing neurons, projecting to the central nucleus of the amygdala has been shown to mediate feeding responses (Carter et al., 2013).

Direct POMC manipulations have the opposite effect on appetite as chronic optogenetic and chemogenetic (Aponte et al., 2011; Zhan et al., 2013) activation of this ARC population decreases food intake. This effect requires intact melanocortin signaling, as mice with constitutively-suppressed melanocortin-4 receptors failed to exhibit this hypophagic response (Aponte et al., 2011). Furthermore, acute stimulation of POMC neurons in the NTS attenuates food intake with fast-acting kinetics (hours) vs. the slower-acting ARC-expressing POMC neurons (days) (Zhan et al., 2013). However, only the latter are necessary for mediating satiety as acute ablation of ARC-expressing POMC neurons causes hyperphagia and obesity (Zhan et al., 2013). Further studies investigating both downstream targets and upstream circuits regulating these AGRP and POMC neurons are required to unravel a functional, wiring diagram modulating appetite control.

While this elegant work has elucidated much of the important circuitry that controls homeostatic feeding under natural conditions, it is not clear whether plasticity in this circuitry contributes to behavioral changes associated with obesity, nor whether targeting this circuitry would be effective for long term weight loss (Halford and Harrold, 2012; Alvarez-Castro et al., 2013; Hellström, 2013). Although obese people eat more, it is not clear whether obese people experience stronger perceptions of hunger or reduced perceptions of satiety, beyond the physiological need to eat more to sustain a larger body size (French et al., 2014). Future studies may investigate the intrinsic firing of these neural populations, as well as plasticity mechanisms among these neurons to address this. Intriguingly, a recent study demonstrated genetic perturbation of AgRP neural activity from development or postnatal ablation of these neurons enhanced exploratory behavior and intensified responses to cocaine, indicating that alterations in these neurons can contribute to behavioral plasticity associated with other brain regions (Dietrich et al., 2012). Chronic manipulations of these circuits may address the extent to which these circuits are altered in obesity, as well as their therapeutic potential for long term weight loss.

## **BEYOND HOMEOSTATIC FEEDING**

Evidence for the potential of animals to engage in nonhomeostatic feeding was demonstrated in classic electrical stimulation and lesion experiments of the lateral hypothalamus (Delgado and Anand, 1953; Margules and Olds, 1962; Wise, 1974; Markou and Frank, 1987), which can cause rodents to eat far beyond homeostatic need. Recent work has elucidated that this likely depended on inhibitory projections from the BNST, marked by Vesicluar GABA transporter (VGAT) to the LH (Jennings et al., 2013). Optogenetic stimulation of these GABAergic projections evoked robust feeding in sated mice and time spent in a designated food zone, while inhibition of these projections diminished feeding in hungry mice. Interestingly, these bidirectional optogenetic perturbations revealed that this GABA*BNST*→Glutamate*LH* circuit had significant influence on motivational valence. Manipulating this pathway in an orexigenic direction evoked appetitive, rewarding responses as assessed using real-time place preference and self-stimulation assays, while manipulation in an anorexigenic direction elicited aversive responses (Jennings et al., 2013). Remarkably, the same study demonstrated both necessity and sufficiency for a glutamatergic sub-population of neurons in the LH marked by the expression of *Vglut2* (glutamate transporter 2; Jennings et al., 2013). While manipulations of the LH can produce a range of effects on motivated behavior (including complete cessation of feeding) (Hoebel, 1971; Wise, 1974), optogenetic stimulation of these VGAT*BNST*→VGLUT*LH* projections or direct optogenetic inhibition of VGLUT*LH* neurons specifically produced voracious feeding behavior, suggesting that explicit hypothalamic afferent projections or populations of LH neurons likely support different aspects of feeding behavior. This point has been noted for decades (Wise, 1974), however the emergence of novel tools and techniques have allowed investigators to understand more specifically which neural populations and projections support different aspects of feeding behavior.

# **CRAVING AND COMPULSIVE CONSUMPTION OF FOOD REWARDS**

Craving is a core feature of drug addiction, which is believed to underlie the compulsive consumption of drugs of abuse (Koob and Volkow, 2010). Obese people often experience craving for food as well, and the circuitry that correlates with craving in obesity appears to be similar to that in drug addiction (Avena et al., 2008; Jastreboff et al., 2013). This includes dopaminergic circuitry, and adaptations in these structures are likely to be responsible for heightened craving in both drug addiction and obesity (Volkow et al., 2002; Wang et al., 2002). The largest populations of dopaminergic neurons reside in the midbrain, in the substantia nigra pars compacta (SNc) and the ventral tegmental area (VTA). Optogenetic activation of midbrain dopaminergic neurons in mice facilitated positive reinforcement during foodseeking behavior in an operant task (Adamantidis et al., 2011) in addition to a more generalized place preference test (Tsai et al., 2009). Similar positive reinforcing properties, as assessed by intracranial self-stimulation, of these neurons were observed in rats (Witten et al., 2011). GABAergic neurons of the VTA directly inhibit dopaminergic VTA cells and optogenetic activation of the former is sufficient to drive conditioned place aversion as well as consummatory behavior (Tan et al., 2012; van Zessen et al., 2012). Intriguingly, in the conditions used in the Adamantidis study, stimulation of dopaminergic terminals alone was not reinforcing, although it facilitated positive reinforcement of food-maintained behavior (Adamantidis et al., 2011). This suggests that a special relationship may exist between reinforcement in feeding contexts, such that animals have a lower threshold for learning about foodrelated information than other information.

The reinforcing actions of dopamine likely depend on dopamine dependent plasticity onto or within striatal neurons that receive input from midbrain dopaminergic structures. These are mainly medium spiny neurons that express either the dopamine D1 or D2 receptor, known as direct pathway (dMSNs) or indirect pathway medium spiny neurons (iMSNs), respectively (Gerfen et al., 1990). A model for how these striatal populations control behavior was introduced in the late 1980s, and is sometimes referred to as the "classic model" of basal ganglia circuitry (Albin et al., 1989). Based largely on anatomical studies, these authors hypothesized that activation of dMSNs facilitated motor output, whereas activation of iMSNs inhibited motor output. Explicit tests of this model have supported it, demonstrating that direct pathway promotes movement, whereas the indirect pathway inhibits movement (Sano et al., 2003; Durieux et al., 2009; Kravitz et al., 2010).

However, just as dopamine can promote both reinforcement and movement, dMSNs and iMSNs also exhibit an opposing influence over reinforcement, which may suggest physiological links between movement and reinforcement (Kravitz and Kreitzer, 2012). The dopamine D1 receptor is an excitatory Gs coupled receptor, and thus dopamine can excite dMSNs through this receptor (Planert et al., 2013), which may be integral to the reinforcing properties of dopamine. Indeed, optogenetic stimulation of dMSNs is sufficient to drive operant reinforcement in mice (Kravitz et al., 2012), and modulation of dMSNs activity can modulate the reinforcing properties of cocaine and amphetamine (Lobo et al., 2010; Ferguson et al., 2011) and natural rewards (Hikida et al., 2010) in a manner consistent with the effects of direct dMSN stimulation. The dopamine D2 receptor is an inhibitory Gi coupled receptor, and thus dopamine inhibits iMSNs through this receptor (Planert et al., 2013). Optogenetic activation of D2 receptor expressing iMSNs promotes aversion (Kravitz et al., 2012), and also reduces preference (Lobo et al., 2010), and self-administration of cocaine (Bock et al., 2013). Consistent with this, chemogenetic inhibition of these neurons enhances the rewarding properties of amphetamine and cocaine (Ferguson et al., 2011; Bock et al., 2013). Similarly, when food deprived rats were given a choice between palatable food (chocolate biscuits) and their normal chow, the D1 agonist SKF 38393 increased their preference for the palatable food, while the D2 agonist quinpirole reduced it (Cooper and Al-Naser, 2006). In this way, dopamine release can promote reinforcement through two independent basal ganglia circuits. Dopamine may promote reinforcement through activating dMSNs and activity through the direct pathway, as well as through inhibiting iMSNs and activity through the indirect pathway (Kravitz and Kreitzer, 2012).

While dopamine release is normally reduced as animals learn reinforcement relationships, sucrose binging can repeatedly evoke high levels of dopamine release, repeatedly providing a reinforcement signal following behaviors directed at these foods (Rada et al., 2005; Hoebel et al., 2009). Whether repeated dopamine release occurs with high fat or other palatable diets is not known. The repeated dopamine release during sucrose binging may be similar to what happens with addictive drugs, which also continue to stimulate dopaminergic function through pharmacological actions, irrespective of how well the animal has learned the association between a behavior and drug delivery (Di Chiara and Imperato, 1988). Therefore, as animals consume such diets, dopamine mediated reinforcement processes may occur at repeated and super-physiological levels. Indeed, obesity has been associated with enhanced activity in areas of the brain that process salience and reward in response to visual food stimuli (Rothemund et al., 2007; Stoeckel et al., 2008; Jastreboff et al., 2013), although other studies have reported opposing findings on this point (Stice et al., 2010). Importantly, especially when considering similarities and differences between drug addiction and sucrose addiction, different subset of striatal neurons are activated when animals self-administer cocaine vs. food or water, indicating that different "functional units" throughout the basal ganglia may subserve behaviors directed at drug vs. food reinforcers (Carelli et al., 2000). Despite this functional organization, it is possible that similar pathological changes in dopamine mediated reinforcement processes may contribute to compulsive consumption in subset of striatal units that subserve both food and drug addiction. The above studies elucidated pathways that can modulate the reinforcing properties of drugs of abuse, and suggest that these pathways may be altered in drug addiction. However, this is only one component of addiction, which is a complex disease involving many brain circuits. In addition to drug-mediated reinforcement through basal ganglia circuits described above, other circuits mediate impairments in inhibitory control, and the emergence of negative emotional states. While the above have better elucidated the role of the dopaminergic system in mediating reinforcement, it is important to note that not all reinforcement is addiction. For example, the vast majority of individuals that experience drugs of abuse do not become addicted, despite finding the drugs reinforcing. Therefore, other circuitry changes are likely involved in drug addiction, such as those underlying deficits in inhibitory control over behavior, and the emergence of negative emotional states.

## **IMPAIRMENTS IN INHIBITORY CONTROL**

Drug addiction is accompanied by impairments in medial prefrontal and orbitofrontal cortical function, and resulting deficits in executive control over behavior (Koob and Volkow, 2010; Volkow et al., 2013). In animals, a recent study demonstrated that prolonged cocaine self-administration decreases cellular excitability of pre-frontal cortical neurons, potentially pointing to a mechanism for how repeated cocaine use impairs frontal circuitry (Chen et al., 2013). To directly test the role of PFC neurons in compulsive cocaine seeking, these authors optogenetically stimulated and inhibited these neurons, which attenuated or increased compulsive cocaine seeking, respectively (Chen et al., 2013). Although in a different behavioral paradigm, different results were reported with cue-induced reinstatement of cocaine seeking, where inhibition of this structure impaired cue-induced reinstatement of cocaine seeking (Stefanik et al., 2013). This difference indicates that prefrontal impairments in the human studies may not be reflective of simple decreases in prefrontal activity, but rather more specific changes in distinct prefrontal circuits in ways that enhance relapse potential. Indeed, optogenetic stimulation studies demonstrate that specific PFC neurons projecting to the largely serotonergic dorsal raphe promote active swimming in a forced swim test, while activation of all PFC neurons do not (Warden et al., 2012). It is possible that different pre-frontal cortical circuits facilitate defined aspects of drug-related behavior, and as such, may be revealed by different behavioral paradigms.

Similar cortical deficits may also be associated with obesity. The diet industry is sustained by the inability of humans to control their eating without external interventions. There is increasing evidence that obesity is associated with impairments in cognitive function, including deficits in executive function, working memory, and attention (Gunstad et al., 2007; Bruehl et al., 2009; Mirowsky, 2011). These functions are served by cortical circuitry, which exerts a "top-down" control over subcortical brain circuits discussed above. Brain imaging studies have revealed a number of structural abnormalities associated with obesity, such as decreases in gray matter volume and metabolic activity in frontal regions of obese people, likely contributing to impairments in the ability to inhibit eating (Le et al., 2006; Pannacciulli et al., 2006; Volkow et al., 2009; Smucny et al., 2012; Van den Eynde et al., 2012).

One situation in which humans often find themselves attempting to exert inhibitory control is during dieting. A dieting human is attempting to maintain a calorically-deficient state, while resisting both reinforcement mechanisms (outlined above) and emotional stressors (outlined below). An animal model of this is stress-induced reinstatement of food seeking. In this paradigm, animals are trained to lever-press for food, after which this is extinguished but can be reinstated with stressors, including the pharmacological stress mimic yohimbine (and α2-adrenergic antagonist). Optogenetic inhibition of the medial PFC during yohimbine treatment impaired this reinstatement, similar to reports with cue induced reinstatement of cocaine, suggesting that similar processes may underlie both results (Calu et al., 2013; Stefanik et al., 2013). Again, this indicates that cortical dysfunctions associated with obesity are likely not simple changes in overall activity, but rather the specific activity of specific prefrontal projections. Indeed, a Fos activation study in both food and stress reinstatement paradigms revealed that activated prefrontal neurons exhibit unique synaptic alterations, relative to non-activated neurons (Cifani et al., 2012). A focal point for future research will investigate the terminal projections of these pre-frontal cortical neurons, which have been shown to send axons to reward centers such as the VTA and accumbens core. Such studies will allow us to address the extent to which prefrontal dysfunctions are similar or different between obesity and drug addiction.

## **NEGATIVE EMOTIONAL STATES**

Negative emotional states such as anxiety and depression can be strong triggers that drive drug use in addicts. Addicts are most vulnerable to relapse during periods of stress or emotional distress, and drug use can promote stressful and emotionally distressing situations (Koob, 2008). Similar patterns can occur with over-eating associated with obesity, causing researchers to question whether similar circuitry underlies stress evoked drug and food addiction (Parylak et al., 2011; Sinha and Jastreboff, 2013). For example, periods of stress are often associated with the consumption of highly palatable foods, giving rise to the terms "comfort foods" and "emotional eating". In addition, obese animals exhibit higher levels of anxiety and depression, suggesting that these foods themselves contribute to a cycle in which these negative emotional states contribute to further eating (Yamada et al., 2011; Sharma and Fulton, 2013).

Multiple brain systems regulate negative emotional states, including the dopamine system. Altered dopamine signaling has been heavily implicated in obesity as both obese humans and rodents have lower levels of striatal dopamine D2 receptor (D2R) availability compared with lean people and animals (Wang et al., 2001; Johnson and Kenny, 2010). In addition, polymorphisms in the D2 receptor gene (*Drd2*) have been linked to obesity and multiple forms of drug addiction (Blum et al., 1990; Noble et al., 1993; Stice et al., 2008; Chen et al., 2012). Interestingly, although deficits in D2R availability have also been linked to addiction to cocaine, alcohol, opiates, and nicotine, these addictions are not associated with weight gain. This suggests that the effects of D2 receptor impairments are not linked to weight gain *per se*, but to the overlapping behavioral changes that accompany both obesity and drug addiction. One hypothesis for how reduced D2R function may contribute to behavioral changes associated with both obesity and drug addiction is that animals consume more to compensate for blunted dopaminergic responses as a result of decreased receptor levels (Wang et al., 2002; Stice et al., 2008). In other words, animals require higher levels of dopaminergic stimulation to get the same effect as an animal with a full complement of dopamine receptors. This can be accomplished through pharmacological means, as all drugs of abuse result in dopamine release in the striatum (Di Chiara and Imperato, 1988). Alternatively, it may be accomplished through the consumption of palatable foods, such as food that are high in sugar and fat.

Reduced D2R function may be predicted to elevate activity in iMSNs, as D2R is a Gi coupled receptor. Therefore, it is possible that obese individuals consume foods that over-stimulate dopamine release to inhibit these overactive iMSNs and escape from pervasive negative emotional states. Consistent with this hypothesis, animals that express ChR2 in iMSNs exhibit aversion to stimulation of these cells (Kravitz et al., 2012). When examined in the context of cocaine reward, optogenetic stimulation also impairs (Lobo et al., 2010; Bock et al., 2013), while chemogenetic inhibition of these neurons enhanced cocaine directed behaviors (Ferguson et al., 2011; Bock et al., 2013). Consistent with these findings, increases in the rewarding properties of amphetamine were detected when these neurons were ablated (Durieux et al., 2009). Together, these findings suggest that reductions in D2 expression may produce a pervasive negative emotional state, and that animals will seek super-physiological dopamine release to escape from this state.

In addition to dopamine receptors, alterations in dopamine producing neurons in the VTA may contribute to the emergence of negative emotional states. Through their inputs to the VTA, efferents emanating from the laterodorsal tegmentum and the lateral habenula elicit positive and negative states in mice, respectively (Lammel et al., 2012; Stamatakis and Stuber, 2012). Selective inhibition of VTA DA neurons induced depressionlike phenotypes, as assessed via tail-suspension and forced-swim tests, in addition to anhedonia, quantified through a sucrose preference assay (Tye et al., 2013). To demonstrate bidirectional control of these neurons and their sufficiency in mediating these behaviors, the authors showed that temporally sparse phasic photoactivation of VTA DA neurons rescues stress-induced depression-like phenotypes (Tye et al., 2013). To investigate susceptibility vs. resilience to social-stress-induced behavioral irregularities, it was reported that optogenetic induction of phasic, but not tonic, firing in VTA DA neurons of mice undergoing a subthreshold social-defeat paradigm promoted social avoidance and decreased sucrose preference, two independent readouts of depression (Chaudhury et al., 2013). Dopamine neurons in the VTA have long been known to encode consummatory reward and reward-predictive cues (Bayer and Glimcher, 2005; Pan et al., 2005; Roesch et al., 2007; Schultz, 2007). Electrophysiological studies have also linked VTA DA neurons to stress and negative states (Anstrom et al., 2009; Wang and Tsien, 2011; Cohen et al., 2012) highlighting the complexity of dopaminergic signaling.

Finally, in humans, the amygdala has been linked to both anxiety-disorders (Etkin et al., 2009) and craving (Childress et al., 1999; Wrase et al., 2008), in addition to a host of other emotional processes. Several optogenetic studies have dissected amygdala circuits in connection with a wide array of behaviors from those related to anxiety (Tye et al., 2011; Felix-Ortiz et al., 2013; Kim et al., 2013) or fear (Ciocchi et al., 2010; Haubensak et al., 2010; Johansen et al., 2010) as well as those related to reward-seeking (Stuber et al., 2010; Britt et al., 2012). While electrophysiological studies demonstrate that amygdala neurons encode both positive and negative motivational valence (Paton et al., 2006; Shabel and Janak, 2009), there have not yet been studies genetically identifying the neural encoding dynamics of the partially nonoverlapping populations of neurons that do so. While the neural correlates of negative emotional states associated with obesity are not fully understood, examination of synaptic and cellular alterations in these circuits may be a promising place to look.

# **CONCLUSION**

In recent years, the drug addiction paradigm has been applied to the neural circuits mediating behaviors associated with obesity. This perspective has sparked important insights, while still recognizing that obesity has important differences from drug addiction. Primarily, food is necessary for survival, which makes parsing the adaptive and maladaptive components of feedings a challenge when thinking of potential therapies, as obese people cannot develop strategies to avoid food altogether, as a drug addict might towards drugs of abuse. Given the ability of feeding behaviors to be both necessary for survival and harmful in excess, understanding the neural circuits related to food addiction calls for tools of utmost precision, such as manipulations facilitated by optogenetic and chemogenetic approaches.

# **REFERENCES**


corelease glutamate. *J. Neurosci.* 30, 8229–8233. doi: 10.1523/jneurosci.1754-10. 2010


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 13 December 2013; accepted: 09 February 2014; published online: 28 February 2014.*

*Citation: Krashes MJ and Kravitz AV (2014) Optogenetic and chemogenetic insights into the food addiction hypothesis. Front. Behav. Neurosci. 8:57. doi: 10.3389/fnbeh. 2014.00057*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2014 Krashes and Kravitz. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Investigating habits: strategies, technologies and models

# **Kyle S. Smith<sup>1</sup>\* and Ann M. Graybiel <sup>2</sup>\***

<sup>1</sup> Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA

<sup>2</sup> Department of Brain and Cognitive Sciences, McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA

#### **Edited by:**

Mary K. Lobo, University of Maryland School of Medicine, USA

#### **Reviewed by:**

Susan Ferguson, University of Washington, USA Xiangdong W. Yang, University of California at Los Angeles, USA

#### **\*Correspondence:**

Kyle S. Smith, Department of Psychological and Brain Sciences, Dartmouth College, 6207 Moore Hall, Hanover, NH 03755, USA e-mail: kyle.s.smith@dartmouth.edu Ann M. Graybiel, Department of Brain and Cognitive Sciences, McGovern Institute for Brain Research, Massachusetts Institute of Technology, 46-6133, Cambridge, MA 02139, USA e-mail: graybiel@mit.edu

Understanding habits at a biological level requires a combination of behavioral observations and measures of ongoing neural activity. Theoretical frameworks as well as definitions of habitual behaviors emerging from classic behavioral research have been enriched by new approaches taking account of the identification of brain regions and circuits related to habitual behavior. Together, this combination of experimental and theoretical work has provided key insights into how brain circuits underlying actionlearning and action-selection are organized, and how a balance between behavioral flexibility and fixity is achieved. New methods to monitor and manipulate neural activity in real time are allowing us to have a first look "under the hood" of a habit as it is formed and expressed. Here we discuss ideas emerging from such approaches. We pay special attention to the unexpected findings that have arisen from our own experiments suggesting that habitual behaviors likely require the simultaneous activity of multiple distinct components, or operators, seen as responsible for the contrasting dynamics of neural activity in both cortico-limbic and sensorimotor circuits recorded concurrently during different stages of habit learning. The neural dynamics identified thus far do not fully meet expectations derived from traditional models of the structure of habits, and the behavioral measures of habits that we have made also are not fully aligned with these models. We explore these new clues as opportunities to refine an understanding of habits.

**Keywords: striatum, cortex, action, chunking, learning, flexibility, reward, reinforcement**

There is a distinguished history of scientific attention to habitual behaviors. In early thinking in psychology, much of behavior was framed in terms of lack of mindfulness, and this mode of behavior was considered as habitual. This was, and remains, a straightforward and intuitive way of thinking. This line of thinking was convolved with the classic reflex arcs of Sherrington and his students, the ubiquity of habits emphasized by William James, the "law of effect" of Thorndike, the formalized drive theory of Hull, and with the general behaviorist research tradition. As a consequence, many aspects of animal behavior were viewed in terms of sensory inputs and movement outputs linked by what we would now call neural networks (James, 1890; Thorndike, 1898; Sherrington, 1906; Hull, 1943). As ethologic approaches developed, views emerged emphasizing that speciesspecific instinctual behaviors are characterized by a consistency in performance similar to, or exceeding, that of learned behaviors (Tinbergen, 1950; Lorenz and Leyhausen, 1973). It now is clear that even so-called simple reflex arcs lie within modifiable microcircuits (Marder, 2011), as already forecast in the work of Sherrington and embedded in the ideas of William James.

In other fields, also, there has been pushback against the idea that animals other than humans behave without some form of mindfulness. Affective and incentive motivational processes have been seen as guiding forces for behavior as potent as stimulusresponse (S-R) associations (Bolles, 1972; Bindra, 1978; Toates, 1986; Berridge, 2004; Salamone and Correa, 2012). Many features of behavior have been identified as being driven actively as intentional processes, rather than by habit (Tolman, 1932; Holland, 2008). Habits were, accordingly, sometimes separated from mainstream fields investigating behavior, especially cognitive decision-making behavior. Now, in part due to new methods emerging to approach this issue more directly, an active field is exploring what habits might be and what they might not be (Dickinson, 1985; Graybiel, 1997, 2008; Daw et al., 2005a; Yin and Knowlton, 2006; Wood and Neal, 2007; Holland, 2008; Redish et al., 2008; Belin et al., 2009; Packard, 2009; Balleine and O'Doherty, 2010; Berridge and O'Doherty, 2013; Dolan and Dayan, 2013).

Here we summarize the results of experiments in which we asked whether we could, by employing optogenetic methods, manipulate habitual behaviors—either changing them after they had formed, or preventing them from being formed. This attempt at gaining causal evidence about the control of habits suggests that they are not always stand-alone S-R behaviors, but rather, can be behaviors that are carefully monitored and controlled by neural circuits on-line in real time despite their apparent automaticity. We suggest that habits can be characterized as constituting sequences of actions that have been chunked together through simultaneous cortical and basal ganglia activity dynamics and as arising from multiple core operators in the brain that control habits in real time. Our intent here is to focus on the results and implications of newer optogenetic and recording studies, but we encourage readers to compare and contrast recent articles for comprehensive views on brain mechanisms related to learned actions and habits (Graybiel, 2008; Balleine et al., 2009; Packard, 2009; Balleine and O'Doherty, 2010; Dolan and Dayan, 2013).

## **OPERATIONAL DEFINITIONS AND MEASURES OF HABITUAL BEHAVIOR**

Studies of habitual behaviors use a range of contemporary empirical approaches to probe their structure in species ranging from flies to rodents to humans (Dickinson, 1985; Yin and Knowlton, 2006; Brembs, 2009; Tricomi et al., 2009; Balleine and O'Doherty, 2010). We categorize a few of these approaches here, and the criteria they set up for definitions of habitual behavior, but the list that we give is not exhaustive; nor is each measure exclusive of others. Even within single tasks, key task demands can push behavior either into more mindful, goal-directed, flexible modes, or into more habitual, repetitive or fixed modes (Adams, 1982; Dickinson, 1985; Daw et al., 2005a; Yin and Knowlton, 2006; Balleine et al., 2009; Packard and Goodman, 2013). For example, in order to encourage habits, protocols often include extended and distributed training, increased reward exposure, diluted contingency between actions and rewards or increased informational uncertainty, and reduction in reward contiguity with sensory inputs. The fact that habitual and cognitively driven behaviors can vary inversely when task conditions change (or when experimental brain lesions are imposed) suggests that there is likely to be a competitive or at least parallel architecture in neural circuits driving habitual and goal-directed expression of behavior, a point much emphasized in current theoretical work (Dickinson, 1985; Balleine and Dickinson, 1998; Killcross and Coutureau, 2003; Daw et al., 2005b; Packard and Goodman, 2013) (but also see Dolan and Dayan, 2013).

#### **OUTCOME REPRESENTATION**

In a clever extension of the early logic that S-R and non-S-R modes of behavior might both exist, revaluation of the reinforcement outcome offered was introduced into an operant task in order to pit the two behavioral strategies against one another (Adams, 1982; Dickinson, 1985; Dickinson and Balleine, 2009). Logic held that if behaviors were driven more by "cognitive" associations between an action and a particular expected outcome (goal-driven behavior), then changing the value of the expected outcome should directly influence the action. Conversely, if a behavior were like an inflexible S-R reflex lacking outcome representation, then outcome changes ought to not affect it.

In an early experiment testing this logic (Adams, 1982), animals were trained on a simple operant task (e.g., press a lever for reward). They then received a devaluation of the reward, which involved induction of conditioning an aversion to the reward through pairings of the reward with administration of a nauseogenic agent, lithium chloride. In this paradigm, once the reinforcement is experienced in its newly devalued state, and knowledge about the value reduction is acquired, animals are returned to the task and tested in a probe trial in which no reinforcement is present—so-called extinction conditions. Animals that have been trained in the task, but have not been trained far past their initial acquisition performance levels, reduce their performance of the action that was paired with obtaining the newly devalued reward. This finding suggested that the animals recognize the identity of the outcome that they are working for, and can flexibly adjust whether they will or will not work to receive it when its subjective value changes, providing support for earlier notions of animal purposefulness in behavior (Tolman, 1932; Tolman and Gleitman, 1949; Holland, 2008; Dickinson and Balleine, 2009). By contrast, animals that have been over-trained on the task, that is, trained over and over long after they reached an initial learning criterion, will continue working for it. This persistence of seeking the reinforcement despite its devaluation is considered to be a defining mark of habitual behavior.

Over-training is, however, not necessary for such habitual behavior. Habits can also form in animals without extended training if they are trained on variable interval task schedules (rather than on ratio schedules) (Dickinson et al., 1983). In these variable interval schedules, a reward is given after a varying length of time, once the initial task behavior is established, in order to dilute directly the contingency between each action and each reward experience (Dickinson, 1985; Balleine and Dickinson, 1998). Thus, an additional criterion for defining a behavior as habitual emerged from this line of work, by which animals should be insensitive both to outcome value as well as to changes in the consistency of the action-outcome (A-O) contingency (Dickinson, 1985; Balleine and Dickinson, 1998).

Among associative learning psychologists, these two instrumental learning constructs are contrasted to Pavlovian stimulusstimulus (S-S) learning, because this form of learning (as in bell indicates food) tends to be directly sensitive to reward revaluation (Holland and Rescorla, 1975; Balleine and Dickinson, 1992; Dickinson and Balleine, 2009) and generally insensitive to A-O contingencies (Williams and Williams, 1969; Stiers and Silberberg, 1974; Hershberger, 1986) (see also below for exceptions).

A large number of loss-of-function neuroscience experiments supports this framework for interpreting habits as outcomeinsensitive S-R near-reflexes (**Figure 1**). For example, normally outcome-insensitive behaviors can be rendered outcome-guided following disruption (through lesions, chemical inactivation, or gene knockout) of the sensorimotor striatum (called the dorsolateral striatum or DLS). Similar effects are found after disruption of the dopamine-containing input to the DLS from the *pars compacta* of the substantia nigra, after disconnection of the DLS and the central nucleus of the amygdala (indirectly connected), or after disruption of pallidum-projecting neurons in the striatum in general (Yin et al., 2004; Faure et al., 2005; Yu et al., 2009; Wang et al., 2011; Lingawi and Balleine, 2012). Disruption of the infralimbic subdivision of medial prefrontal cortex (here called the infralimbic (IL) cortex) produces similar effects (Coutureau and Killcross, 2003; Killcross and Coutureau, 2003; Hitchcott et al., 2007; Smith et al., 2012; Barker et al., 2013). The fact that the IL cortex is not directly connected with the DLS, along with the many other regions implicated, suggests a widespread distribution of habit-promoting regions in the brain.

#### **MULTIPLE MEMORY SYSTEMS**

Related to this line of thinking is the notion that behaviors in environments requiring navigation (e.g., mazes) can be guided by spatial cues, or instead can be driven by learned movement plans. In one famous test, a so-called plus maze was used to tease apart such "place" (allocentric) and "response" (egocentric) strategies (Tolman et al., 1947; Packard and McGaugh, 1996). Animals might start at the south arm and be required to turn east (right) to receive reward. After a training period, animals then would be started in the north arm. If they followed the spatial cues, as trained rats often do, they would turn left to enter the east arm. If, instead, they had learned to make a particular response, as overtrained animals might do, they would turn right to enter the west arm. Habitual behavior, according to this work and its precedents, was identified by the adoption of such a response-based maze running strategy. In lesion work, the DLS and dopamine have once again been implicated as required for such a habitual strategy (McDonald and White, 1993; Lee et al., 2008; Packard, 2009; Wang et al., 2011).

#### **LEARNED RESPONSE VS. TREE-SEARCH**

Tasks also are often designed to assess the acquisition and persistence of stimulus-evoked responses, such as in having animals learn to respond in one way to a specific cue in order to receive a reward and to use various decision-making strategies that are more or less cognitive or habitual. Recent computational work has conceived of habits as resulting from model-free learning systems in the brain (Daw et al., 2005a; McDannald et al., 2012; Dolan and Dayan, 2013). In this view, behavior is ultimately brought under the control of two systems—one forward-looking (model-based), and one that stores value based on experience (model-free). Model-free control, analogous to habitual behavioral control, is thought to capture a multitude of behavioral phenomena and corresponding neural findings, the idea being that behaviors that have been stored through prediction-error learning might share a common computational structure (Doya et al., 2002; Daw et al., 2005a; Dolan and Dayan, 2013). This framework of learning covers traditional instrumental and Pavlovian conditioning realms, in which instances of both modelfree and model-based control is thought to occur (Berridge and O'Doherty, 2013; Dolan and Dayan, 2013), and has been used to positive effect in work on brain function in both rodents and humans (Daw et al., 2005a; Bornstein and Daw, 2011; McDannald et al., 2012; Dolan and Dayan, 2013). The emerging views are at a stage of vigorous debate. Lines of evidence have arisen that support the existence of dissociable biological substrates for each system, for competition between them, for toggling between them in single behaviors, for merging them, and even for model-based systems teaching model-free systems as collaborators (Dolan and Dayan, 2013). Nonetheless, once again, such work has implicated the striatum, dopamine, and particular circuits in the neocortex in controlling the acquired model-free behavioral plan (Daw et al., 2005a; McDannald et al., 2012; Dolan and Dayan, 2013).

## **PERFORMANCE OPTIMIZATION**

Reminiscent of early thinking is the idea that habits are behaviors that have become well-practiced, routine, and predictable. Useful hallmarks for the formation of skills and habit-like behaviors include increased speed to start and complete tasks, more stereotypic and routed movements through a task environment, fewer deliberations at decision points, reduced distractibility, indifference to negative feedback, and increased performance accuracy (Poldrack et al., 2005; Graybiel, 2008; Belin et al., 2009; Desrochers et al., 2010; Hikosaka and Isoda, 2010; Smith and Graybiel, 2013a). Such measures do not distinguish the possible covert strategies or task representations that might be controlling behavior; but they do provide an important set of reference points for understanding a behavior as more or less exploratory and might serve as reference points for understanding the progression of habit formation and the involvement of potential brain mechanisms.

## **NEUROPHYSIOLOGICAL CORRELATES OF HABITUAL ACTION LEARNING**

We turn now to insights gained from exploiting relatively recent technological advances, including chronic recording from behaving animals and gene-based targeting strategies for manipulating brain activity in real time by optogenetics. Loss-of-function studies, such as those highlighted above, provided strong evidence for regions of the neocortex and basal ganglia as being critical for the expression of habits. It is possible at some level to map these roles onto changes in neural activity recorded from these regions. However, they have not yet converged on a unified view of the underlying structure of a habitual behavior. This difference is the main focus of our discussion.

Numerous changes in neuronal activity in the neocortex and basal ganglia have been documented in rodents and primates as learned behaviors shift with practice from exploratory to skillful in their execution. Ensembles of neurons develop time-locked responses to well learned or innate sequences of actions such as grooming patterns or sequences of motifs in birdsong, often with distinct representations of distinct steps within a sequence (Berns and Sejnowski, 1998; Brainard and Doupe, 2002; Fujii and Graybiel, 2003, 2005; Aldridge et al., 2004; Jin et al., 2009; Hikosaka and Isoda, 2010; Fee and Goldberg, 2011). Related work in rats has focused on the plasticity of somatotopic representations in the striatum during motor learning. For example, by recording the activity of forelimb-sensitive neurons in rats performing a forelimb reaching task, Carelli and West found that as the movements were repeated over time for reward, there was a decline in the response of movement-related neurons (Carelli et al., 1997; Tang et al., 2007). This result points to a contrast with the findings in cortical recording experiments on related tasks (Karni et al., 1995; Nudo et al., 1996; Plautz et al., 2000); instead of increasing the representation of movements as they are put to use in a task, the striatum appears to become more selective or efficient in such representations. This work raised the important possibility that regions needed for habit expression are themselves key sites of neuroplasticity. The way in which these plasticity changes reshape neural ensemble activity has been surprising.

## **AS THE DORSOLATERAL STRIATUM (DLS) SEES HABITS: ACTION CHUNKING REPRESENTATIONS**

We have explored this plasticity in striatal activity in the DLS region targeted in the loss of function studies, by recording for the entire time during which rats and mice develop habitual behaviors with extensive training on T-maze tasks (in which reward is obtained following correct navigation in response to instruction cues) (**Figure 1**). The findings in these studies have provided evidence for viewing habits fundamentally as sequence of actions that are grouped together, or "chunked", for ready deployment. By recording simultaneously from many putative medium-spiny neurons (MSNs) in the DLS day by day, we asked what neural activity might occur during task training and over-training.

An initial study (Jog et al., 1999) documented a remarkable change in patterning of spike activity in the DLS viewed in the framework of entire run-times. In the relatively naïve animals, DLS ensembles were active during the runs, but as training continued and the runs became well-practiced and faster, many individual units in the DLS ensembles developed activity that was particularly pronounced at the beginning and/or at the end of the runs, and fewer were active mid-run. This beginning-and-end task-bracketing activity provided a compelling candidate neural correlate for the chunking of actions together into a habitual unit (Graybiel, 1998). It has long been noted that individual elements of memories can be chunked together to aid recall (Miller, 1956)—as we recall PIN numbers, passcodes, and phone numbers. It is difficult to remember the elements other than within the whole, as witnessed by the trouble we have in picking up in the middle after interruptions in recall. In the context of actions, plausibly, a well learned action-sequence could similarly be chunked into a performance unit. The recorded DLS activity certainly appeared to reflect this process and suggested indeed that a habit might in part be a chunked-together sequence of behavioral steps.

It soon became clear that not only the numbers of neurons, but also the spike rates of the neurons, contributed to the taskbracketing patterns (Barnes et al., 2005). Moreover, the chunking pattern grew as the animals learned the habit, but changed if conditions changed during extinction and reinstatement tests (Barnes et al., 2005). When the rewards presented at the goal sites during the training phase were then removed or greatly reduced, the DLS bracketing pattern returned to roughly the same pan-run pattern present in the untrained animals, and when the rewards were suddenly returned, the bracketing pattern returned in nearly its fully developed form. Was the neural activity pattern suppressed? Passed to another brain region? Too small to detect with extracellular ensemble recording methods? These are very live issues that need answers. What these dynamics do suggest is that just as habits, once developed, are difficult to forget, so the patterned neural activity that accompanies them can be suppressed but still is maintained so that it can be expressed rapidly when conditions call for it (Pavlov, 1927; Rescorla, 1996; Barnes et al., 2005). The DLS dynamics appear to play a key role in this rapid loss and recovery of learned habits.

This bracketing pattern has since been found in other regions as well. It emerges as part of the neural representation of song repetition in HVC of Bengalese finches (Fujimoto et al., 2011), in primate prefrontal cortex as a series of saccades are performed by highly trained monkeys (Fujii and Graybiel, 2003), and in the substantia nigra of mice as they learn to press a lever repeatedly for reward (Jin and Costa, 2010). The DLS "end-related activity" can phasically precede or follow the end of a performance sequence, or appear as phasic activity around finalizing actions, such as turning in the T-maze task, depending on task-type or potentially the trial-to-trial time-locking of repeated actions for analysis (Barnes et al., 2005; Kubota et al., 2009; Jin and Costa, 2010; Thorn et al., 2010; Smith and Graybiel, 2013a).

Within the DLS, the chunking pattern is not restricted to the MSNs, but can be expressed by interneurons as well, but with interesting differences. Kubota et al. (2009) trained and then overtrained mice in the standard maze T-maze task in which turn directions were instructed by auditory cues, and then without warning switched the modality of the instruction cues from auditory to tactile. Remarkably, in MSNs, the chunking pattern was unaffected by this task change and accompanying drop in performance accuracy; it continued to emphasize the beginning and end of maze runs despite the cue shift and the resulting robust shift in performance. Fast-firing neurons, putative striatal fastspiking interneurons (FSIs), recorded simultaneously with MSNs, also developed the task-bracketing activity pattern. But the activity of these FSIs did change when the sudden task modification was introduced; they developed a phasic response at the onset of the new cue, a response that then faded with several days of further training.

These distinct MSN and FSI dynamics could be important for the striatum to maintain an overall structure of the task a general plan or action set—while still processing changes in task details to adjust performance (Kubota et al., 2009). In other studies, the activity of striatal FSI populations has also been shown to relate to the suppression of unwanted or unselected movements (Wickens et al., 2007; Berke, 2011), suggesting that one important role of their sudden engagement at the T-maze cue shift could be to suppress pre-potent responses to the initial cue. They could effectively halt or segment the chunked behavior to permit flexibility in incorporating the new cue-response segment into a newly chunked behavioral pattern. Another important class of striatal interneurons, the cholinergic cells or tonically active neurons (TANs), show even further distinction in their dynamics related to task performance (Aosaki et al., 1994; Kimura et al., 2003; Apicella, 2007; Graybiel, 2008; Goldberg and Reynolds, 2011; Thorn and Graybiel, in press). The question of how interactions among these subtypes of striatal neurons relates to DLS function can now be addressed with cell-type specific markers.

#### **DORSOMEDIAL STRIATUM (DMS) PLASTICITY TRACKS HABIT FORMATION ALONGSIDE DORSOLATERAL STRIATUM (DLS) STABILITY**

From the point of view of the field focusing on habit learning, a key issue for these T-maze experiments was the relation of the DLS task-bracketing pattern to activity in other striatal regions, particularly in the dorsomedial striatum (DMS). Inhibition of the DMS leads to a loss of behavioral flexibility and outcomesensitivity, and an increase in habitual mode of responding, in multiple task conditions (Yin et al., 2005b; Ragozzino, 2007; Packard, 2009). These results were opposite to the results of inhibiting the DLS, suggesting that the two regions have opposing functions in relation to balancing flexible and inflexible behavior. If the DLS chunking pattern found in the T-maze experiments were related to habits, as suspected, then DMS activity recorded in parallel ought to be quite different, related to cognitive-associative components of the task rather than to habitual performance.

This prediction turned out to have support from the experiments of Thorn et al. (2010), who recorded simultaneously in both the DLS and the DMS while rats were trained on auditory and tactile versions of the T-maze task. The DLS formed its task-bracketing ensemble activity pattern early and maintained it as the animals proceeded through the extensive training protocol. Simultaneously, DMS ensemble activity during the decision time in the task increased as animals were learning the time of lessened activity in the DLS. Then, as the animals became familiar with the task during over-training, this DMS decision-period activity waned. Thus a different set of dynamics marked habit learning in the two striatal regions. The DMS region in which these recordings were made was at about the same anteroposterior level as that of the DLS recordings, and so was anterior to the region studied in the loss of function studies (Yin et al., 2005a,b; Bradfield et al., 2013). A shift in the balance of DMS and DLS activity is emerging as a common marker of acquired habits in neural recording studies. For example, a recent experiment (Gremel and Costa, 2013) adapted a context-dependent habit task (Killcross and Coutureau, 2003) in which rats would behave habitually in one environment (in this case, where reward was delivered on a random interval schedule) but non-habitually in another environment (where reward was delivered on a random ratio schedule). Switching between habitual and non-habitual performance states in these environments was accompanied by a shift, in presumably the same neurons, between higher overall firing activity in DLS vs. DMS, respectively. Similarly, strengthening of DLS activity with decline of DMS activity has also been found in mice undergoing training on a rotarod task (Yin et al., 2009), suggesting this shifted medial-lateral balance in striatal activity also extends to skilled behaviors that were acquired through negative reinforcement.

The contrast in DLS and DMS neural dynamics suggests that, in a reinforcement context, changes in DMS activity might direct the expression of behaviors as they are being formed into habits, but once the DMS activity falls, then DLS-related circuits might take over the control of the behaviors, allowing them to be expressed as habit (Thorn et al., 2010). This notion meshes well with models derived from the results of lesion work, in which the rapid establishment of a habit following DMS (or prelimbic cortex) lesions is considered to reflect the uncovering of a DLSassociated habit that was dormant or occluded when the goaldirected system was intact (Killcross and Coutureau, 2003; Daw et al., 2005a; Yin and Knowlton, 2006; Balleine et al., 2009). In new work (Thorn and Graybiel, in press), evidence is emerging that theta-band oscillatory activity might allow the DMS and DLS to have different information flow through channels favoring subbands of theta. Moreover, in the T-maze task, which involves elements of both place and response learning, oscillatory local field potential activities in the striatum and hippocampus become progressively linked by the development of inverse phase relationships in the theta-band oscillations that they exhibit, especially during the decision-making parts of the runs (DeCoteau et al., 2007a,b; Tort et al., 2008).

If, as these studies suggest, there is parallel operation and possibly competition between circuits engaged by these striatal sub-regions in controlling behavior, then the brain might have a selection or arbitration mechanism for guiding the switch between them (Killcross and Coutureau, 2003; Daw et al., 2005a; Yin and Knowlton, 2006; Dolan and Dayan, 2013). The T-maze work suggested that the dynamics of the DMS activity might accomplish this function (Thorn et al., 2010), but left unresolved was which of the many DMS inputs might be driving the change, as well as the potential existence of similar habitrelated neural dynamics in other brain regions. One potential route toward increasing or decreasing habit strength could be to enhance experimentally the decision-related DMS activity. However, the increase in strength of the bracketing pattern seen in some of these studies (Barnes et al., 2005) together with the increase in overall DLS activity that appears to occur as skills emerge in other studies (Tricomi et al., 2009; Yin et al., 2009; Gremel and Costa, 2013) raises the possibility that some task environments or types of learning might recruit late-stage plasticity in both DMS and DLS. As we discuss below, our evidence suggests that DLS is participating actively in action sequence expression and run-to-run automaticity of behavior at all stages, and is not only keeping the task-bracketing pattern ready for selection according to changes in the activity of other brain regions.

## **DORSOLATERAL STRIATUM (DLS) ENCODES ACTION SEQUENCES AND AUTOMATICITY DURING AN OUTCOME-INSENSITIVE HABIT**

Loss-of-function studies rooted in associative learning theory have largely been done separately from studies of the neural plasticity related to the formation of rewarded action sequences, but important attempts are being made to bridge the two research strategies (Tang et al., 2007; Kimchi et al., 2009; Stalnaker et al., 2010; Fanelli et al., 2013; Gremel and Costa, 2013; Smith and Graybiel, 2013a). In the T-maze, for instance, major questions remained about whether changes in neural activity, particularly in the DLS, related to habit formation as formally defined in outcome representation terms. Did maze performance reflect an A-O or place-based strategy, or an S-R or response-based strategy? Were dynamics of the DLS task-bracketing pattern related in any way to shifts between these strategies? How would S-R learning be reflected in the DLS dynamics?

To address some of these questions, we developed a modified version of the T-maze task in which a distinct reward was to be found at each of the two end-arms of the maze, and the rewardspecific devaluation-sensitivity test was used to assay habitual behavior. Throughout this time, single-unit activity was recorded in both the DLS and its counterpart in the IL cortex, two canonical habit-promoting regions.

In the DLS, we observed the familiar task-bracketing pattern, with the firing of MSNs accentuating the run start, turn, and goal arrival. This pattern formed rapidly, around the time that animals reached a criterion of accuracy on the task. Importantly, however, parallel experiments in a set of control animals established that behavior was markedly goal directed at this early training point. In the probe test given on the day after devaluation, animals trained just to the learning criterion would immediately avoid the devalued goal (Smith et al., 2012). Thus, the DLS chunking pattern was emerging prior to habit expression proper. Once it formed, the DLS pattern remained steady during the over-training period, and, judging from the probe test given after the over-training period, the animals formed a devaluation-insensitive habit. After the probe test, when rewards for correct runs were returned, the animals' behavior changed sharply; runs slowed, deliberative head movements increased, and accuracy dropped. However, as in the cue modality switch study discussed above (Kubota et al., 2009), the DLS pattern remained nearly unchanged. The DLS activity at the start, turn, and end of the runs remained as strong as it had been before. Behaviorally, animals were running correctly to the devalued goal (fewer trials) or to the non-devalued goal, or running the "wrong way" to the non-devalued goal when instructed to the devalued goal. At the session-wide level, it appeared that as long as animals were running a familiar route smoothly, whether the run was rewarded or not, the DLS ensemble pattern persisted. This possibility was supported further when we devalued all goals, which led to extinction of maze runs and a corresponding rapid loss of the chunking pattern (Smith and Graybiel, 2013a).

In trial-by-trial analyses of the DLS ensemble activity, and also in analyses of the activity of other subpopulations of DLS neurons, we did not find any clear relationship between the action-encoding in the DLS in any given trial and the timing of habit formation and suppression. DLS activity similarly lacked any correlation with sensitivity or insensitivity to outcome value on single runs. What we did find, however, was a remarkably close DLS relationship to the automaticity of a given run. During some runs, particularly early in training, animals would deliberate at the junction of the goal-arms, essentially conducting a "vicarious trial and error" by using their head to check each one before making a choice (Muenzinger, 1938; Tolman, 1948; Johnson and Redish, 2007). Both the strength of the DLS activity at the start of the run and the strength of the overall DLS chunking pattern were correlated inversely with these deliberative head movements: the DLS pattern was much stronger on trials that lacked this deliberation. This finding suggested that the DLS keeps track of or controls the level of automaticity or decisiveness of behavior at points of choice options. Much like the increased efficiency of movement-related signals that occurs in the DLS as actions are learned (Carelli et al., 1997; Jog et al., 1999; Graybiel, 2008), an increase in performance optimization appeared to characterize the buildup of DLS activity around major task actions. However, a particularly intriguing aspect of the results is that the major correlate of deliberative head movements occurred in DLS activity at the start of the run, and not in DLS activity at the turn itself when the deliberations were occurring or not (Smith and Graybiel, 2013a). Thus the activity was not related to the head movements themselves, but was anticipatory or predictive. This correlation of DLS activity with non-deliberative running was evident in early training as well as in late over-training, indicating a close involvement of the DLS run-to-run, more than just at the phase of behavioral learning when the behavior was expressed as a habit. In short, although the DLS pattern could mark the early formation of a habitual response plan (and "memory" of this plan after devaluation), it appears to relate quite closely to behavior on a trialto-trial basis, even in demonstrably goal-directed performance phases.

# **IMPLICATIONS FOR UNDERSTANDING HABITS: VIEWING DORSOLATERAL STRIATUM (DLS) ACTIVITY AS A STARTING POINT RATHER THAN A FINISHING LINE**

#### **A PUZZLING DISCONNECT BETWEEN DORSOLATERAL STRIATUM (DLS) PHYSIOLOGY AND STIMULUS-RESPONSE (S-R) BEHAVIORAL THEORY**

Some of these DLS dynamics seem surprising in the context of reinforcement learning models. As noted, there is a long tradition of referring to chained S-R associations as the underlying base of habitual behavior, and developments in learning theory have pinpointed conditions under which behavior might be understood as an S-R habit as opposed to an A-O behavior or Pavlovian conditioned response (Dickinson, 1985; Balleine and Dickinson, 1998; Daw et al., 2005a; Yin and Knowlton, 2006; Holland, 2008; Schneck and Vezina, 2012). The DLS is widely regarded as a source, if not *the* source, for such associations in the brain. Yet fitting aspects of the neural recording data within this framework is not straightforward. Concerning our maze task alone, we have highlighted the many dynamic processes that exist in DLS physiology, including those of different types of task-related patterns, different task epochs, and different neuronal types. Notable for this discussion is the finding that the DLS pattern forms prior to habit emergence, correlates with how automatic a run is from early on, and remains stable after devaluation despite major accompanying changes in behavior and valuation of stimuli, responses, and outcomes. Moreover, we, as our colleagues, find relatively few neurons in the DLS that have preferential responses to S-R combinations (Berke et al., 2009; Thorn et al., 2010; Smith and Graybiel, 2013a). One complex way that these findings might be reconciled with an S-R view is if behavior were partly habitual from the moment of DLS pattern formation (which it might be), and if that particular habit memory were stored stably and used selectively during over-training but not used during most runs after reward devaluation.

Contrasts have also arisen in related work showing that as a devaluation-insensitive habit develops, there is a loss of specific head-movement-related activity in DLS neurons in a headbobbing task (Tang et al., 2009). However, a similar weakening of neural activity of tongue-related units in lateral striatum occurs with training on a licking task, but this licking behavior remains devaluation-sensitive (Tang et al., 2009). In another study, actionrelated neural responses that are modulated by the identity of a preceding cue arise in the DLS (Stalnaker et al., 2010). Although this result would appear to offer evidence for an S-R association, the same correlates arise simultaneously in the DMS, a "non-habit" site. The authors speculate that S-R integration was occurring elsewhere, perhaps in sites receiving DLS outputs. All of these incongruities might be good indicators that we should look elsewhere than in the DLS or its outputs for habit-related activity. However, if the neural dynamics that dominate DLS recordings in these studies are considered to reflect important features of ongoing behavior, as they certainly appear to, different conclusions arise. One is that stimulus-specific response plans might be supported more widely in the striatum than we thought. Another is that the increased efficiency of performance-related representations in DLS activity might aid behavior in a wide variety of Pavlovian, goal-directed, and habitual tasks. And as is so often true, the specific demands of tasks shape the patterns of activity found.

Returning to the T-maze, the findings that we review suggest that the DLS can contribute to habits in a way that does not encode a chain of task-related S-R associations, a possibility that has at least some support in the literature. Although S-R associations need not take any particular form in neural activity, the lack of stimulus-evoked firing in the DLS has raised the question of how habits are represented in this region if not by responses to paired Ss and Rs (difficult-to-isolate contextual S's notwithstanding) (Berke et al., 2009; Root et al., 2010; Thorn et al., 2010; Smith and Graybiel, 2013a). Similarly, although abolishing dopamine input to the DLS leads some behaviors to be devaluation-sensitive (i.e., not "habitual") (Faure et al., 2005), a task designed to tap into S-R learning strategies can be performed mostly normally in Parkinsonian patients with similar dopaminergic dysfunction, suggesting to the authors an independence of S-R learning from the DLS (de Wit et al., 2011). From the brain's point of view, the neural dynamics that do occur during habit formation might provide some insight into the role of DLS in mediating behavior, even if it does not lead precisely to the same S-R notions that inspired the research.

## **DORSOLATERAL STRIATUM (DLS) ACTION CHUNKING AS HABIT FORMATION**

The dominant pattern that emerges in the DLS during habit formation, at least on the maze, is the accentuation of salient actions or behavioral boundaries and has a close relationship to run automaticity. If we take this pattern as a starting point, action chunking seems like a plausible underlying DLS mechanism for controlling habits (Graybiel, 1998). In this way, sequences of actions can become linked and can be executed as a unit automatically or semi-automatically. To the extent that associations are driving the chunking process, the links that are made between each action could provide a means for dissociating behavior from the outcome.

The chunking together of the actions and accentuation of start-related activity can be viewed as enhancing the performance of the bounded sequence of actions, once the sequence is initiated. This is an important property, one that could permit actions even close to reward to be conducted in the context of the chunked-together unit. There is much evidence that under normal conditions, actions and cues that are proximal to reward are more closely associated with its specific features, and thus are more sensitive to shifts in its value or contingency. Actions or cues occurring more distally to reward might carry a more diluted reward representation arising from additional associations made with subsequent actions and cues, yielding more outcome-independence. This touches on the problem of credit assignment in reinforcement learning theory (Sutton and Barto, 1998). Empirically, although reward-distal cues appear to make good predictors of impending reward, they are less sensitive to shifts in motivational state or outcome value compared to rewardproximal cues (Tindell et al., 2005; Zhang et al., 2009; Smith et al., 2011). Conditioned behavioral responses to cues, instrumental actions, and species-specific behaviors (e.g., predation in the cat) show a similar sensitivity to state or reward value changes based on proximity to actual reward receipt (Morgan, 1974; Holland and Straub, 1979; Balleine et al., 1995; Holland et al., 2008).

This phenomenon seems to bear out in the maze as well. When we devalued all rewards on our maze after the initial devaluation, the DLS pattern was rapidly lost. The rats showed a progressive backward breakdown effect (Morgan, 1974), first failing to drink the reward and then failing to turn upon instruction, and yet persisting in the initial run initiation (Smith and Graybiel, 2013a). This result raises the possibility that the maze running actions, once released from being expressed as a chunk, could show similar variations in flexibility based on their proximity to reward. By extension, when previously chunked as a sequence, the full run behaved like the most reward-distal action (i.e., run start). This proposal meshes quite well with the finding that devaluation of a well-learned action sequence can lead to the loss of the sequence in full, as opposed to the loss of only the reward-proximal element (Ostlund et al., 2009). Similarly, monkeys trained to press buttons sequentially in order to obtain reward will continue to conduct the full sequence even if the reward is made available to them earlier, an effect that appears to relate to the strength of striatal dopamine input (Matsumoto et al., 1999). Similar continuation of reward-seeking acts despite disinterest in the reward itself has been noted in a variety of older behavioral studies (see Morgan, 1974). This general phenomenon calls to mind the famous "kerplunk" effect (Carr and Watson, 1908), in which animals are trained well on a complex maze and then an experimenter suddenly moves the reward (and end-wall) to a closer position. Well-trained animals continue running right past the reward and contact the wall, as though they had formed a habit of a certain response set that would be carried out in full even if it resulted poorly.

We raise the possibility that part of this action-chunking and outcome-decoupling process might involve a motivational value that is bound to the chunked action sequence itself. Cues, for instance, can grow to acquire incentive value specific to the reward with which they are paired; the cues then become attractive and meaningful, and can pull in behavior (Bolles, 1972; Bindra, 1978; Toates, 1986; Berridge, 2004; Rudebeck and Murray, 2011). Something similar might occur with strongly reinforcement, by which doing them becomes attractive and rewarding in its own right (Berridge, 2009). Glickman and Schiff (1967) suggested that some actions, such as those related to food consumption or copulation, have had value attached to them over the course of evaluation. DLS activity might be one candidate substrate for this value-binding process. Such a view might help to account for several observations: the strong accentuation of reinforced actions by DLS activity in the T-maze task; the fact that this activity pattern was maintained after devaluation of one reward (i.e., when animals mostly ran to the non-devalued goal regardless of instruction, as though it were an immediately valued route); and the fact that the activity pattern was lost after all outcomes were devalued and behavior was extinguished (when values decay). Actions attaining incentive value could in principle drive behavior independently from expected outcome value, and thus function much differently than the "action value" signals of reinforcement learning and behavioral economics (Rangel et al., 2008).

This action-bound value conceptualization of chunked behaviors might ultimately also help link together disparate functions of the DLS, including its contribution to behaviors that are not easily interpreted in the context of prediction-error learning, such as instinctive grooming patterns and Pavlovian-to-instrumental transfer (Aldridge et al., 2004; Corbit and Janak, 2007). Why is there an impetus to perform such actions? There are many possible reasons, but one could be that the behaviors have an intrinsic incentive motivational value, whether innately expressed or acquired through experience. More broadly, too much or too little value might contribute to excessive drive to perform actions (as in obsessive-compulsive spectrum disorders) or loss of the capacity to perform intended actions (e.g., in Parkinson's disease).

Many of the relationships between DLS activity and habitual behavior that we have emphasized are at the level of correlations. Similarly, the lack of detection of robust S-R or other representations in the DLS is not evidence that they are not present; chunking-related activity is a dominant DLS signal, but S-R encoding could well be found in other tasks, in other brain regions, or in other features of DLS physiology. Even how the taskbracketing pattern relates to the motor demands of the behavior has not been fully documented. We especially emphasize that in the DLS, the relatively low activity in-between the pronounced beginning and end of the runs does not mean a lack of activity at these times. Further "expert neurons", with highly specialized functions, remain active even during mid-run when ensemble activity quiets (Barnes et al., 2005; Thorn et al., 2010).This pattern suggests that sparse coding, a feature of many neural systems, could be built into the DLS activity, reflecting the capacity for a few neurons to encode the full sequence, and/or that after a behavioral sequence has become habitual, it is advantageous to free up neurons to participate in other computations. Finally, as originally discovered by Barnes et al. (2005), the chunking pattern is shown by a large population of DLS projection neurons, but many other neurons become quieted throughout the runs as the maze training proceeds. It is still not clear whether the quieted neurons form a special class, for example, the so-called indirect pathway neurons as opposed to direct, as discussed by Thorn et al. (2010). If so, this distinction would parallel the superficial layer-deep layer dichotomy in the bracketing pattern observed in the IL cortex (Smith and Graybiel, 2013a). Clearly further research is needed to characterize the control mechanisms of action chunking in the striatum and elsewhere (Graybiel, 2008; Desmurget and Turner, 2010; Dezfouli and Balleine, 2012). Thus far, however, evidence is compelling: (1) that habits are, in part, chunked action sequences; (2) that this function is reflected, in part, in dynamic patterns of activity in the DLS and DMS; and (3) that action sequence chunking, as reflected by this activity patterning, is a main, but not sole, contribution of the DLS to habits.

# **DUAL OPERATORS FOR HABITS: CONTRASTING DYNAMICS IN THE INFRALIMBIC CORTEX (IL) AND DORSOLATERAL STRIATUM (DLS)**

The prefrontal cortical region known as IL cortex has been identified as having habit-promoting functions similar to those of the DLS, despite its apparent lack of direct connections with the DLS (**Figure 1**). Lesions or inactivation of the IL cortex result in outcome-sensitivity and habit blockade (Coutureau and Killcross, 2003; Killcross and Coutureau, 2003). However, there are specific details of the tasks employed in these studies that are important to consider. Normally, animals trained to the point of exhibiting habitual behavior on a lever press task will continue pressing both a lever paired with a reward that has been devalued and a lever paired with a non-devalued reward. Lesions or GABAergic inhibition of the IL cortex lead animals to press more than normally on the lever for the non-devalued reward, but the animals do not increase pressing on the lever for the devalued reward (Coutureau and Killcross, 2003; Killcross and Coutureau, 2003). These results provided evidence of a more goal-directed form of behavior that would otherwise be habitual were the IL cortex intact. Intra-IL microinjection of dopamine similarly leads to increased pressing on a lever for a non-devalued reward, but it also concomitantly reduces pressing on a lever for a devalued reward (Hitchcott et al., 2007). Collectively, these findings identify the IL cortex as a prefrontal region that is important for maintaining an outcomeinsensitive habit. The distinct result of increased behavior to a valued goal following IL disruption has led to the suggestion that the IL cortex (by way of being affected by dopamine levels) might control changes in the allocation of both goal-directed and habitual strategies when outcome value changes (Hitchcott et al., 2007; Dolan and Dayan, 2013).

To evaluate the extent to which habit formation might be reflected similarly in the firing patterns of the IL cortex and the DLS, we recorded in these regions simultaneously as we trained series of rats in the two-reward maze paradigm (Smith and Graybiel, 2013a). We found that the bracketing pattern developed in the IL cortex as well as in the DLS, but that its formation was delayed until well into the over-training period. This finding, and the differential sensitivity of the IL cortex and DLS patterns to reward devaluation, raised the possibility that the IL cortex and the DLS function as distinct core contributors, or operators, in the development of habits. In this view, habits are promoted by at least two underlying controllers in the brain. The dynamics we recorded suggested a mixed redundancy and distinctiveness of the activities in the two regions, and provided information about the circuit-level patterning of neural activity that occurs as habits are made.

The similar action-bracketing activity in the IL cortex and DLS suggested the existence of at least one common neural signature of habit formation shared by IL-associated prefrontal and DLSassociated neural circuits. The IL pattern was slightly different in terms of makeup, being more broadly activated just prior to run initiation and prior to goal arrival, but in the DLS, as in the IL cortex, activity during the decision period was greatly diminished. The timing of the changes in activity patterns in the two regions was strikingly different. IL activity scarcely changed until the late over-training period, when our behavioral devaluation measures had shown was the time that the maze habit became crystallized, shifting from being devaluation-sensitive to devaluationinsensitive. At this time, the IL task-bracketing pattern formed and persisted until reward devaluation. Although the IL pattern was present during the probe test, at which time behavior was expressed as habitual, it rapidly decayed when the rewards were returned in post-probe training sessions and running choices changed. Later, after a prolonged period of continued post-probe training, the IL pattern returned. This period of further plasticity in IL activity corresponded to a stage at which a second, replacement habit probably had formed on the maze, given the behavior of the rats: initially after devaluation, they began to avoid the devalued goal and, instead, began to run to the non-devalued side. These wrong-way runs increased in frequency over days (despite lack of reward for them), became faster and increasingly similar to the instructed runs to the same location, and lost the deliberative head movements at the turn that had appeared earlier. Thus, the return of the IL task-bracketing pattern appeared to mark the formation of a new habit of simply running to the non-devalued goal. We suggest as a leading possibility that the alignment of the more flexible IL chunking pattern of activity to the more rigid chunking pattern expressed in the DLS might be necessary for expressing behavior as a habit.

Unlike the ensemble activity in the DLS, the IL activity pattern could not be linked consistently to any run-to-run variation of behavior that we assessed. This negative finding was surprising, as it seemed to suggest that the IL cortex was not functioning in our task as an arbiter or habit-selector as had been speculated. If the IL cortex had such a function, its activity ought to reflect, on a given trial, whether behavior was outcome-sensitive or outcomeinsensitive, deliberative or non-deliberative. This form of coding has been observed in other cortical regions (Wunderlich et al., 2012), and is essential for allowing a rapid toggling between more goal-directed and habitual strategies in some ongoing behaviors (Dolan and Dayan, 2013). We also might have expected an outcome value-tracking signal if the IL cortex were, at some level, aiding in goal-directed behavior as prior work suggested (Hitchcott et al., 2007). What we found, instead, was that the IL task-bracketing activity was inversely correlated with the net number of deliberations and level of outcome sensitivity that occurred in entire sessions, each composed of many individual maze runs. The contribution of IL cortex to the maze running habit might therefore be at the level of a state property (Smith and Graybiel, 2013a). Much as the state of stress can lead to either cribbing or pacing in horses, here the state contributed by IL activity could promote habits without specifying their behavioral details.

The task-bracketing ensemble activity that we observed in the IL cortex was mostly restricted to neurons located in the superficial layers of the IL cortex. Neurons recorded from deeper layers exhibited plasticity in firing rate at nearly identical time points, but during runs, the deep layer ensembles became active throughout the run. Thus, there was striking layer-selectivity to the habit-related activity patterns. An interesting possibility is that the chunking pattern in IL cortex is indicative of activity that is communicated across trans-cortical networks; but we lack enough information to test this possibility. We do point out that the bracketing pattern was not general to the prefrontal cortex. Ensemble activity recorded from the overlying prelimbic cortex declined as the habit emerged, consistent with its DMS-like role in promoting behavioral flexibility (Balleine and Dickinson, 1998; Killcross and Coutureau, 2003).

These findings suggested a far more central role for the IL cortex in habit acquisition and expression than formerly appreciated. By some views, the IL cortex was thought to serve as an arbitration or contention-scheduling mechanism for selecting goal-directed vs. habit strategies (Coutureau and Killcross, 2003; Daw et al., 2005a), or to promote habits by acting on learned associations stored elsewhere (Balleine and Killcross, 2006). The dense connectivity that the IL cortex shares with limbic and associative regions, including the amygdala and ventral striatum (Hurley et al., 1991; Vertes, 2004), suggests that IL activation feeds forward to dampen evaluative processes in these sites or to invigorate motivation or prior learned associations in order to promote behavioral persistence. The development or decrease of IL activity—possibly specifically of its task-bracketing activity might ultimately provide a permissive state for habits.

Much like the weakening of DMS activity that we have seen during over-training on the maze (Thorn et al., 2010), the timing of IL plasticity could similarly be critical for habits—when the task-bracketing pattern is expressed alongside the similar ensemble pattern in the DLS, behaviors grow outcome-insensitive. Ultimately, the stages of plasticity in the IL cortex, in the DMS and in their associated circuitries might thus determine or allow the full strength of a habit to be expressed, the essential involvement of DLS in action sequencing or valuation notwithstanding (Killcross and Coutureau, 2003; Yin and Knowlton, 2006; Balleine et al., 2009; Thorn et al., 2010; Smith and Graybiel, 2013a). Research from several laboratories has begun to identify the neurochemical/molecular signals within IL cortex that might aid this privileged function, including transmission involving GABA and dopamine (Coutureau and Killcross, 2003; Hitchcott et al., 2007; Barker et al., 2013).

The neural dynamics and behavioral correlations uncovered in the T-maze study suggest an intimate participation by the IL cortex in sculpting and maintaining habits in addition to a role in selecting them (Hitchcott et al., 2007; Smith and Graybiel, 2013a). If so, it should be possible to test, by means of optogenetic interventions, the view that the IL cortex contributes core components to a habit, just as does the DLS. We have done such testing in two ways.

# **OPTOGENETIC INTERVENTIONS TO TEST THE IMPACT OF REAL-TIME ACTIVITY DYNAMICS IN THE INFRALIMBIC CORTEX (IL)**

# **ROLES OF THE INFRALIMBIC CORTEX (IL) IN HABIT EXPRESSION**

To evaluate a causal contribution of ongoing, real-time activity in the IL cortex to habitual behavior, we incorporated an optogenetic approach (Smith and Graybiel, 2013b). The spatiotemporal resolution provided by optogenetics, now widely noted (Boyden et al., 2005; Bernstein and Boyden, 2011; Fenno et al., 2011; Mei and Zhang, 2012), allowed us to restrict IL disruption to particular time windows of specific populations of neurons. We sought to perturb IL activity only during the maze runs, not before or after the runs, in order to evaluate on-line functions of IL cortex pyramidal neurons during behavior (Smith and Graybiel, 2013b).

We first examined the effect of halorhodopsin-mediated perturbation of IL pyramidal cell activity in over-trained rats, testing during the probe session after they had been given reward devaluation (Smith et al., 2012). Control rats lacking this perturbation behaved like normal over-trained rats: they continued to run, "by habit", to the devalued goal as well as to the non-devalued goal. Rats with IL inhibition, however, exhibited outcome sensitivity: they reduced their running to the devalued goal when so instructed, by about 50%, and thus behaved like rats that had not received the over-training. This effect replicated the ability of IL lesions or chemical treatments to block habitual behavior (Coutureau and Killcross, 2003; Killcross and Coutureau, 2003; Hitchcott et al., 2007), showing again that removing the influence of IL cortex over behavior suppresses actions directed to a devalued goal and shifts them to a more valued one. The optogenetic approach further added a critical fact: that IL cortex exerts powerful on-line influence over ongoing behavior. Avoidance of the devalued goal occurred within a few trials, amounting to just seconds of IL inhibition time. Moreover, rats persisted in avoiding the devalued goal on subsequent days without further inhibition, and did not rebound or have to "learn again" when the inhibition was removed; the habit blockade effect endured. Additional IL perturbation at this stage did nothing to behavior: animals continued to show elevated running to the non-devalued goal and decreased running to the aversive goal, suggesting that IL activity during runs might not be necessary in some contexts for expressing this outcome-appropriate strategy of behavior (Hitchcott et al., 2007; Smith et al., 2012; Dolan and Dayan, 2013).

A major surprise then came when we gave another round of IL perturbation after the prolonged post-devaluation training. At this stage, IL neural activity had begun to exhibit the task-bracketing pattern again, and behavior appeared to reflect the emergence of a second wrong-way running habit. When we applied IL perturbation at this time, but not earlier, this second behavior (putative replacement habit) was blocked with similar immediacy. The frequency of wrong-way runs suddenly decreased, and rats ran back to the devalued goal when so instructed (and drank the reward), whereas control rats continued their pattern of wrong-way runs. This behavior again endured over subsequent sessions, and further sessions of IL perturbation almost entirely returned rats to their originally learned behavior, tested for up to 20 days after the habit reinstatement first occurred.

During these striking optogenetically induced changes in the rats' behavior when instructed to go to the devalued goal, the IL perturbation had no detectable effect on the instructed runs to the non-devalued goal at which the normal reward could be found, as cued. Nor did it affect home-cage intake of the devalued reward where the aversion persisted even in the presence of IL inhibition. These results therefore suggested a highly specific impact of the IL perturbation on the maintenance of acquired habitual strategies of running. The IL cortex thus does appear to have on-line control over habitual behaviors as they occur. This on-line control system exerts a remarkably rapid, robust, and enduring influence over behavior, likely reflecting rapid plasticity in downstream targets of the IL cortex.

Why did the blocked habit come back when the IL cortex was later perturbed? One possibility is that the IL cortex maintains *newly* formed habits, at least for a time. If the task-bracketing pattern, when present, were to contribute a state in which a new response strategy can be executed habitually, as in situations in which it competes with an alternative strategy, then blocking this IL pattern after initial over-training would return rats to a prior strategy, namely, outcome-sensitive or exploratory running. Blocking this pattern again later, after the second habit formed, would similarly strip this second habit away, and then the prior strategy—the initial habit—would be expressed. By this account, the IL does not store initial habits when new ones arise, but does contribute critically to promoting the most situation-appropriate ones that have been learned.

This interpretation could carry implications for considering IL function more broadly as it relates to regulating other learned behaviors. Famously, the IL cortex regulates extinction learning across a range of tasks (Morgan et al., 1993; Rhodes and Killcross, 2004; Peters et al., 2009). The IL cortex also participates directly in maintaining new strategies in tasks requiring animals to shift between using spatial cues vs. response plans to perform (Ragozzino, 2007; Rich and Shapiro, 2009). Depending on conditions, IL inhibition can also lead to a spontaneous recovery of an extinguished drug-seeking behavior, or can conversely prevent the return of drug seeking that would normally be evoked by exposure to a drug context (Peters et al., 2008; Bossert et al., 2011). One possibility is that the IL cortex is specialized for promoting a new response strategy at the expense of an older, prepotent one—be it a new habit, a new response inhibition, or a new mnemonic strategy (Killcross and Coutureau, 2003; Rich and Shapiro, 2009; Smith et al., 2012). The many output connections of IL cortex could support the translation of this general function into different behavioral effects in different situations (Peters et al., 2009). Our finding that IL inhibition could both block and reinstate a particular habit certainly suggests some form of dependency of IL function on context or history (Smith et al., 2012).

## **THE ACTIVITY OF INFRALIMBIC CORTEX (IL) DURING HABIT ACQUISITION**

If IL cortex is critical to the on-line expression of habits, it could be critical also for the acquisition of habits, given a generalized form of on-line monitoring or control by this part of the prefrontal cortex. To test this possibility, we asked whether optogenetic perturbation of IL cortex could prevent the formation of a new habit (Smith and Graybiel, 2013a). An advantage of gene-based targeting approaches, as opposed to lesions or drug microinjections, is that cell populations can be manipulated repeatedly without compromising the integrity of tissue. Leveraging this strength, we tested, albeit with imperfect layer- and cell-type-specific manipulations, whether the IL task-bracketing pattern during the over-training period—the time during which we had found this pattern to form in conjunction with habit formation—was critical to the crystallization of the maze running behavior as a habit. We applied halorhodopsin-mediated perturbation to the IL cortex on each day during over-training on the T-maze task. Animals then underwent reward devaluation and a post-devaluation probe test without further IL inhibition. The behavior of the animals during the probe test clearly showed that they had not formed a habit despite being overtrained. The rats with IL inhibition during the entire over-training period acted like rats with no over-training experience: they avoided the devalued goal during the probe task, behaving as though they had only received initial training to criterion, but had not been over-trained. This finding demonstrates that the IL cortex contributes to more than just habit selection—on-line IL activity during performance is essential for making habits in the first place, as well as for expressing them once they are formed.

This finding also underscores the potential of using temporally precise manipulations for affecting even strongly ingrained and multifaceted behaviors, such as habitual behaviors, as well as for testing causal roles of particular on-line neural dynamics (Boyden et al., 2005; Bernstein and Boyden, 2011; Fenno et al., 2011; Smith and Graybiel, 2013b). Related work on corticostriatal systems and action selection has put optogenetic and pharmacogenetic approaches to use in isolating pathways and cell types necessary or sufficient for goal-directed behaviors (Gremel and Costa, 2013), ritualistic behaviors (Ahmari et al., 2013; Burguiere et al., 2013), behavioral initiation or cessation (Kravitz et al., 2010), and linking rewards or drugs with behavioral plasticity (Witten et al., 2010; Ferguson and Neumaier, 2012; Kravitz et al., 2012; Stuber et al., 2012; Lenz and Lobo, 2013). As we have noted (Smith and Graybiel, 2013b), research on habits will benefit tremendously from continued work with these methods and their successors.

## **CONTINUED REORGANIZATION OF HABIT RESEARCH? UNFINISHED BUSINESS: BRAIN MECHANISMS**

Research is not even close to resolving mechanisms underlying habitual behavior, even work on the DLS, toward which the most attention has been paid. The striatum contains multiple subtypes of neurons (and subtypes of glia) and complex sets of inputs; and only recently have we been able to study the dynamics of these components in relation to behavior. Even aside from behavior, there is rapid progress being made at every level of analysis of striatal activity; conceptually, each discipline is like holographic representation of the larger technological revolution that is affecting neuroscience research directions. In striatal physiology, for example, such technology is producing surprises. For example, co-release of classical neurotransmitters and neuromodulators, such as glutamate, GABA and dopamine, and complex interactions among interneurons and projection neurons have been identified *in vivo* within the striatum (Stuber et al., 2010; Tecuapetla et al., 2010). As a second example, stimulation of cortical excitatory, glutamatergic inputs can lead to predominant suppression of MSNs in a behaviorally relevant manner, due to effects of the cortical afferents on GABAergic interneurons (Burguiere et al., 2013). So much for the plus and minus signs in our diagrams!

Given this rapidly changing face of models of striatal organization and function, it is unclear how the neural recordings that we and others have made of the striatum during habit learning relate to underlying mechanisms. Striatal MSNs are divided into the D1-receptor expressing striatonigral (direct) pathway and D2-expressing striatopallidal (indirect) pathway (Alexander et al., 1986; Albin et al., 1989; Graybiel, 1995). Additional compartmentalization occurs through the striosome vs. matrix organization of MSNs, and in the "matrisome" input-output organization of cortico-basal ganglia circuitry (Graybiel, 1995; Crittenden and Graybiel, 2011). Recorded neurons do not fall cleanly into categories that, as yet, map on to this anatomical organization; there are non-task-related neurons, task-related neurons, rewardrelated neurons, and multiple subtypes within each group. Making sense of this physiological complexity in terms of anatomical connections, as well as in terms of intrastriatal neuronal interaction, will be essential for progress toward understanding habits even just at the level of mechanism in DLS firing dynamics.

Similar points can be made of the IL mechanism driving habitual behavior. Given the heterogeneity of cell types, laminar organization, the multitude of inputs and outputs, and the stunning line of studies now coming out revealing the true complexity of it, much remains to be uncovered. How the IL cortex interfaces with, or does not interface with, the DLS at different phases of habit learning and expression is another major focus of interest. Progress can be now made in comparing stimulations and inhibitions of different pathways or cell classes in either site, or comparing these manipulations at different points in time, as it relates to action and habit learning. What habits are, as the brain sees it, is likely to grow more complex and interesting.

Finally, extending this work to the domain of human behavior will be important. While the DLS shares anatomical homology with primate putamen (or primate DLS) and some evidence indicates a similar functional homology for habits (Tricomi et al., 2009), it is unclear what the primate correspondent of the rodent IL cortex is. The IL cortex shares some functional similarity with the human ventromedial prefrontal cortex in relation to fear extinction (Milad et al., 2006), and to Broadmann Area 25 in relation to aspects of anatomical connectivity and role in depression symptomology (Covington et al., 2010; Holtzheimer and Mayberg, 2011). However, concerning habits, essentially no links have yet been established.

## **UNFINISHED BUSINESS: OUTCOME-SENSITIVITY MEASURES AND THE INSTRUMENTAL/PAVLOVIAN DISTINCTION**

There is similar opportunity for progress in characterizing habits at the behavioral level. The outcome devaluation test has become a standard for defining behaviors as habitual (aside from tests of A-O contingency), and it has become common to think of performance that fails to meet this outcome-sensitivity measure as an S-R habit. However, there are important qualifications to consider. Experiments on Pavlovian learning, for example, have shown that the way in which reward is revalued (e.g., high-speed body rotations vs. taste aversion), and the type of conditioned response being evaluated (e.g., orientation to cues vs. food approach and consumption), can matter greatly for measurements of outcome sensitivity (Morgan, 1974; Holland and Rescorla, 1975; Holland and Straub, 1979; Galarce et al., 2007; Holland and Wheeler, 2009). One fascinating example, sign-tracking or auto-shaping (approaching a cue rather than the source of the reward that it predicts), can in some conditions appear to be insensitive to outcome associations and habit-like (i.e., resistant to an omission contingency) (Stiers and Silberberg, 1974; Hershberger, 1986), whereas in other conditions it can appear as outcome sensitive and quite non-habitual (Locurto et al., 1976; Robinson and Berridge, 2013).

There are also compelling conceptual accounts of behavioral persistence despite changes in outcome value that do not refer to an underlying S-R mechanism. For example, Tolman suggested such persistent behavioral "fixation" resulted from overly strong "sign-gestalt" knowledge acquired in an environment (Tolman, 1932, 1948), Berridge conceives of reward cues as gaining an incentive value that can become "defocused" or detached from moment to moment changes in predicted reward value (Berridge, 2012), and Holland highlights the principle that outcome representations during behavior can become generalized (i.e., something good) vs. being percept or identity specific (i.e., chocolate milk), and that in such former instances they can simply lose their associability with the reward (Konorski, 1967; Holland, 2004; Holland and Wheeler, 2009). Thus, sensory features surrounding major action events on any task could potentially instill devaluation-insensitivity through noninstrumental means, including in our T-maze task. However, the clear action-related DLS/IL activity dynamics support action control as the most parsimonious explanation for our findings. In the broader neuroscience field, strong clues are coming from work on stereotypic or repetitive behaviors, some of which have been shown to result from specific genetic mutations in genes expressed in the striatum and elsewhere (Berridge et al., 2005; Welch et al., 2007; Burguiere et al., 2013) or in specific pathways (Canales and Graybiel, 2000; Hyman et al., 2006; Pascoli et al., 2011; Milad and Rauch, 2012; Ahmari et al., 2013; Burguiere et al., 2013). Such ongoing research at the mechanism level is sure to help us better understand how persistent behaviors arise, and the ways in which they can or cannot be understood as habits. Similar considerations apply to addictive behaviors.

## **COMPLEMENTARY BEHAVIORAL MEASURES**

Additional measurements aside from outcome manipulations can be used to indicate that behavior on a task has become habitual, but it is not always clear how they go together. In the T-maze studies, we observed a progression of behavior toward outcomeinsensitivity, loss of deliberations, and increase in run speed and accuracy (Smith and Graybiel, 2013a). Yet, on a trial-totrial basis, these measures were not necessarily aligned. Animals might deliberate at a turn but then approach a clearly devalued goal. Or they might avoid a devalued goal, yet lack any sign of deliberative movement. Similarly, we noted that animals slowed demonstrably in run speed during the unrewarded probe session, and yet they still displayed total insensitivity to outcome value and performed accurately; thus, they were behaving by "habit", even though their behavior was slow and partly extinguished. Much evidence suggests that outcome sensitivity is a better indicator of habitual behavior than accuracy or speed to define a habitual response (Balleine and Dickinson, 1998). But we need to gain better ways to distinguish habits and skills. For the habits we have studied, defined on the basis of their insensitivity to reward devaluation, multiple behavioral parameters seem to reflect different structural features of the habits. Habits, according to our observations, can be compound behaviors, and in their development can display different components and have mixed characteristics, for example, being deliberative but outcomeinsensitive. The breadth of the activity patterns that we have encountered in making multiple recordings from the striatum and medial prefrontal cortex suggest such compound features as well. Further, we have not touched upon the insistent, extreme habits and repetitive behaviors that can arise from exposure to drugs or appear within the context of neurologic and neuropsychiatric disorders. Searching for a single definition of habitual behaviors may be less productive than searching for the multiple potential circuit-level mechanisms that lead to habits, skills and repetitive behaviors. Finally, we have not approached the mnemonic aspects of habitual representations. These are of the greatest interest to explore.

#### **INTEGRATION WITH COMPUTATIONAL APPROACHES**

Computational work related to habit formation has also moved away from thinking of habits purely in terms of chained S-R reflexes, favoring instead contrasting behaviors based on stored state-based action values learned through prediction-error mechanisms (i.e., model-free behavior) with model-based exploratory behavior (Daw et al., 2005a; McDannald et al., 2012; Dolan and Dayan, 2013). Incorporating the dissociability of outcome and deliberation-related measures of habits together with performance optimization measures might be of particular interest in formulating the underlying rules of behaviors that have modelfree components. Habits, considered as sequences of actions, can also, as argued by Dezfouli and Balleine (2012), be captured by model-*based* learning rules. This kind of debate underscores the reorganization that is occurring in our thinking about habits at many levels.

In the context of our own work, we suspect that the IL cortex and DLS constitute just two of multiple control systems in the brain, and hierarchical computational models need to be, and are being, developed in this domain. Yet, at the heart of the dual-operator notion that we present for these two regions is the idea that they have controlling power in shaping outcomeinsensitive and non-deliberative action sequences, distinct from other co-existing systems important for reward valuation and performance learning. The neural dynamics, in particular the chunking pattern of activity with dissociable time-courses of plasticity and relationships to behavior, suggest that these two regions and their associated circuits together allow the crystallization of behaviors into chunked action-plans that can be executed semi-autonomously despite ongoing changes in the external world.

# **MORE TO COME IN EXPLORATION OF HABITS AS A MODEL FOR STUDYING THE BALANCE BETWEEN FLEXIBLE AND FIXED BEHAVIORS**

We have taken the brain's perspective in this review, and have suggested that habits are composed of multiple operators, two of which are reflected in the neural activity dynamics of the medial prefrontal cortex and dorsolateral part of the striatum. Evidence from this work indicates a relation of the striatal DLS to action chunking and the level of automaticity of the habitual behavior, and a relation of activity in the IL region of the medial prefrontal cortex, especially its upper layers, to promoting the formation and expression of chunked behaviors as outcome-insensitive and nondeliberative. If we set aside the historical notion of S-R chains as the sole defining features of ongoing habits (arguably premature for some habits), we make room for considering flexible dynamics of neural activity across multiple brain circuits and their microcircuits as providing the neural structure underlying habits. We have only touched on the functions of a small set of regions implicated as being necessary for habit formation, or necessary for habits not to be formed, as animals explore their environments. We make the point, however, that the degree of online control over habits by small regions of the medial prefrontal cortex is remarkably strong, and we make the further point that the cooperative activity of regions that were once thought of as acting in opposition actually could be at the heart of the capacity to form enduring habits, behaviors that are of great value in our lives.

# **ACKNOWLEDGMENTS**

We thank Dr. Yasuo Kubota for his comments on the manuscript and acknowledge funding for the work discussed here, provided by NIH grants R01 MH060379 (AMG) and F32 MH085454 (KSS), by Office of Naval Research grant N00014-04-1-0208 (AMG), by European Union grant 201716 (AMG), by the Stanley H. and Sheila G. Sydney Fund (AMG) and by Mr. R. Pourian and Julia Madadi (AMG).

## **REFERENCES**


Toates, F. M. (1986). *Motivational Systems.* Cambridge: Cambridge University Press.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 13 December 2013; accepted: 25 January 2014; published online: 12 February 2014.*

*Citation: Smith KS and Graybiel AM (2014) Investigating habits: strategies, technologies, and models. Front. Behav. Neurosci. 8:39. doi: 10.3389/fnbeh.2014.00039 This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2014 Smith and Graybiel. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Optogenetics, physiology, and emotions

#### *Alexxai V. Kravitz <sup>1</sup> \* and Antonello Bonci <sup>2</sup>*

*<sup>1</sup> Diabetes, Endocrinology, and Obesity Branch, National Institute of Diabetes, Digestive, and Kidney Diseases, Bethesda, MD, USA*

*<sup>2</sup> National Institute of Drug Abuse, Baltimore, MD, USA*

*\*Correspondence: lex.kravitz@nih.gov*

#### *Edited by:*

*Mary Kay Lobo, University of Maryland School of Medicine, USA*

#### *Reviewed by:*

*Melissa R. Warden, Cornell University, USA Dipesh Chaudhury, Mount Sinai School of Medicine, USA*

**Keywords: optogenetics, emotions, anxiety, depression, reward, physiological, synchrony**

Optogenetics is a powerful tool for investigating causal links between neural circuits and behavior. In recent years, optogenetic studies have expanded into the emotional realm, elucidating new facts about the circuits that underlie anxiety, depression, and reward. One caveat with investigating this realm is that emotional responses can be non-linear. Reward is pleasurable to a point, beyond which it can produce mania, an anxious, and unpleasant state. Consistent with this, stimulant drugs that increase dopaminergic function are reinforcing across a limited dose range, above which they are no longer reinforcing, presumably because of anxiogenic effects (Ettenberg and Geist, 1991, 1993; Yang et al., 1992; Deroche et al., 1997). Many published optogenetic studies have not examined potential non-linearities in the relationship between neural activity and the behavior being studied, nor identified where on such a curve their optogenetic manipulation is acting. In addition, the state a neural system achieves during optogenetic stimulation may not necessarily reside on a physiological curve at all, as optogenetics can drive firing parameters outside of physiological ranges (**Figure 1**). We believe that further consideration of this point may lead to more accurate insights into the relationships between neural activity, emotions, and behavior.

Largely for technical reasons, early optogenetic studies did not record neural activity during behavioral manipulations, but instead used slice physiology to validate their manipulations. These studies linked increased activity in specific circuitry to wakefulness (Adamantidis et al., 2007), movement (Aravanis et al., 2007; Gradinaru et al., 2007, 2009; Kravitz et al., 2010), reinforcement (Tsai et al., 2009; Lobo et al., 2010; Stuber et al., 2011; Britt et al., 2012; Kravitz et al., 2012; Liu et al., 2012), anxiety (Tye et al., 2011), and feeding (Aponte et al., 2011), among others. In these studies, slice experiments demonstrated that light could evoke spiking specifically in the target cell types, but could not address how spiking was altered during the behavioral manipulation. The technical barriers to recording from awake animals during optogenetic stimulation have been bridged in recent years, as many researchers have integrated optogenetics with awake *in vivo* recording (Cardin et al., 2010; Royer et al., 2010; Anikeeva et al., 2012; Cohen et al., 2012; Kravitz et al., 2012; Warden et al., 2012; Tye et al., 2013). This allows for comparisons between spiking during spontaneous behavioral and emotional states and optogenetically evoked states. However, defining and quantifying the relevant parameters for such comparisons is still not trivial. Important parameters include firing rates, spatial and temporal firing patterns of individual neurons, and synchrony among populations of neurons. Investigating these parameters in both the spontaneous and stimulated conditions will improve our ability to interpret optogenetic studies, especially when the relationship between firing rates and behavior is not linear.

The efficacy of optogenetically-evoked firing depends on many factors, including viral expression and optical transmission efficiency. In spite of the variability of these factors between laboratories, we believe that many studies are driving firing rates at the high end of, or above, rates achieved under spontaneous conditions. For example, when striatal neurons were tested with a range of stimulation intensities (0.1–3.0 mW), 1 mW was found to achieve nearly maximal firing rates of light-activated neurons within ∼0.5 mm of the stimulation fiber (Kravitz et al., 2012). Other studies have stimulated striatal neurons with higher intensities of light (up to 20 mW) (Lobo et al., 2010; Britt et al., 2012; Tai et al., 2012), likely saturating the firing of many neurons. Excitable structures such as hippocampus and cortex are susceptible to seizure activity, which defines a hard upper limit to usable stimulation intensities. Seizures have been induced with ∼20 mW stimulation of the hippocampus at 10–20 Hz (Osawa et al., 2013), and alluded to with ∼10–20 mW stimulation of the motor cortex at 20 Hz(Gradinaru et al., 2009). These stimulation paradigms are not much stronger than what has been used to stimulate hippocampal cells to re-activate fear memory (∼9 mW at 20 Hz) (Liu et al., 2012), stimulate cortical cells to facilitate movement (∼10–20 mW, constant or pulsed) (Aravanis et al., 2007; Gradinaru et al., 2009), and stimulate amygdala pyramidal cells to facilitate fear conditioning (∼30 mW at 20–50 Hz) (Johansen et al., 2010). Again, without *in vivo* recordings it is difficult to relate different stimulation paradigms to actual firing rates. Still, it is still fair to conclude that most published studies have focused on stimulating near the high end of usable stimulation intensities, which could result in firing outside of that which occurs spontaneously. Moving forward, it may also be worth exploring the low end of stimulation intensity, to examine the effects of more subtle changes in firing which may relate more closely to physiological conditions, or at least reveal a more complete relationship between firing and behavior.

In addition to firing rates, neural systems contain information in spatial (the physical location of the neurons in the brain) and temporal codes (changes in firing rate of these neurons over time) (Pouget et al., 2000; Jortner et al., 2007; Ainsworth et al., 2012). With regard to these spatial and temporal codes, one potentially surprising result of published optogenetic stimulation studies is that most stimulation paradigms have produced relatively normal behavioral responses while presumably saturating and overpowering these codes. This may suggest that rate coding is more important than temporal or spatial coding, although this conclusion is difficult to reconcile with the ubiquity and wealth of information contained in these codes (Pouget et al., 2000; Jortner et al., 2007; Ainsworth et al., 2012). Alternatively, it is possible that stimulation of certain cell groups is "permissive," rather than "informative," in which case the exact pattern or even intensity of firing may not be relevant. As a final possibility, spatial, and temporal codes could be preserved during optogenetic stimulation, riding "along the top" of the stimulated increase in firing rate. While it is not possible to replicate spatial codes with single light sources, new advances with multiple micron-scale light sources (Grossman et al., 2010; Kim et al., 2013a,b), or holographic methods for delivering light (Lutz et al., 2008) may allow for more physiological manipulations of both spatial and temporal coding. This said, such multi-site light sources would need to achieve cellular resolution to truly replicate physiological spatial codes.

Finally, optogenetic stimulation can alter correlations and synchrony among groups of neurons. Most published optogenetic studies have used pulsed stimulation to drive neural activity, which effectively synchronizes populations of neurons at the optical stimulation frequency (Aravanis et al., 2007; Gradinaru et al., 2009; Johansen et al., 2010; Stuber et al., 2011; Tye et al., 2011; Anikeeva et al., 2012; Liu et al., 2012; Burguiere et al., 2013; Osawa et al., 2013). There is growing consensus that aberrant synchronization is a pathological hallmark of many neurological diseases including Parkinson's disease, epilepsy, autism, and schizophrenia (Uhlhaas and Singer, 2006; Brown, 2007). Parkinson's disease is characterized by aberrant synchronization of multiple basal ganglia structures including the subthalamic nucleus (STN) in the beta band (∼10–30 Hz) (Brown, 2007). Optically stimulating fibers innervating the STN at 20 Hz (at ∼10 mW) worsened parkinsonian motor deficits, while stimulating these same fibers at 130 Hz (also at ∼10 mW) alleviated these deficits (Gradinaru et al., 2009). In motor cortex, 130 Hz stimulation again alleviated parkinsonian deficits, while 20 Hz had no effect. In this way, synchronizing a structure at a particular frequency can have effects beyond altering firing rates. One way to mitigate potential issues with synchrony may be to stimulate circuits with a range of frequencies, including irregular patterns of stimulation that avoid synchronizing a structure at a specific frequency (Tai et al., 2012). Alternatively, more subtle stimulation such as low power constant illumination or stepfunction opsins may avoid synchronizing the cells at any particular frequency, while still increasing the output of target cells (Kravitz et al., 2010).

Optogenetic inhibition can side step the above problems of over-stimulation, although it can still induce aberrant synchronization from both optogenetic inhibition and potential rebound spiking. However, as the principal neurons in many brain structures are low firing and optogenetic inhibition is often incomplete *in vivo* (Aravanis et al., 2007; Gradinaru et al., 2009; Royer et al., 2012; Calu et al., 2013; Kim et al., 2013a,b), it may be more difficult to push neurons below their physiological firing rates with optogenetic inhibition. This may be an inherent benefit of optogenetic inhibition for studies that aim to manipulate activity of specific cells to rescue or mimic physiological states. However, other loss-of-function experiments can benefit from inhibiting neurons, irrespective of physiological ranges. Complementary methodologies such as designer receptors exclusively activated by designer drugs (DREADDs) can also investigate the contribution of specific cell types while avoiding aberrant patterns of synchronization (Ferguson et al., 2011; Krashes et al., 2011). Ideally both inhibition and stimulation can be examined in the same system [e.g., (Stuber et al., 2011; Tye et al., 2011, 2013; Britt et al., 2012; Chaudhury et al., 2013; Chen et al., 2013)], in combination with *in vivo* electrophysiology to test whether firing parameters during the manipulations are physiologically relevant. This impressive trifecta has been accomplished by a few studies to date [e.g., (Jennings et al., 2013; Tye et al., 2013)].

The above issues are central for optogenetic experiments that make inferences about neural activity during naturally occurring emotional or behavioral states. However, a second type of experiment sets aside concerns regarding physiological patterns of activity, and aims to stimulate or inhibit with the sole objective of interfering with, and thus mitigating unwanted behavior for therapeutic utility. Here, understanding the relationship between physiological firing of a structure and the emotion or behavior of interest is not as relevant as reducing the unwanted behavior, quite like how deep brain stimulation (DBS) or trans-cranial magnetic stimulation (TMS) can mitigate pathological conditions in neurological disorders without necessarily replicating normal physiological conditions (Miocinovic et al., 2013). In this vein, optogenetic manipulations have been therapeutic for reducing symptoms associated with movement disorders (Gradinaru et al., 2009; Kravitz et al., 2010), depression (Chaudhury et al., 2013; Tye et al., 2013), and compulsive cocaine seeking (Chen et al., 2013).

Regardless of the specific stimulation parameters or methodology, optogenetics has revolutionized neuroscience and offers unprecedented mechanistic and therapeutic opportunities to understand and treat neurological and psychiatric diseases. Quantification of the relationships between physical brain states and perceptual states, as well as firing during optogenetic manipulations will further this progress by facilitating easier quantitative comparisons between studies, reduce the potential for spurious conclusions, and ultimately provide more meaningful insights into the relationships between neural circuits, emotions, and behavior.

## **ACKNOWLEDGMENT**

We acknowledge Drs. Marc Reitman and Mike Krashes for helpful comments.

## **REFERENCES**

Adamantidis, A. R., Zhang, F., Aravanis, A. M., Deisseroth, K., and de Lecea, L. (2007). Neural substrates of awakening probed with optogenetic control of hypocretin neurons. *Nature* 450, 420–424. doi: 10.1038/nature06310


striatal neurons in reinforcement. *Nat. Neurosci.* 15, 816–818. doi: 10.1038/nn.3100


light-assisted perturbation and recording of local neural circuits in the behaving animal. *Eur. J. Neurosci.* 31, 2279–2291. doi: 10.1111/j.1460- 9568.2010.07250.x


*Received: 08 October 2013; accepted: 03 November 2013; published online: 19 November 2013.*

*Citation: Kravitz AV and Bonci A (2013) Optogenetics, physiology, and emotions. Front. Behav. Neurosci. 7:169. doi: 10.3389/fnbeh.2013.00169*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2013 Kravitz and Bonci. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Timing matters: using optogenetics to chronically manipulate neural circuitry and rhythms

# **Michelle M. Sidor and Colleen A. McClung\***

Department of Psychiatry, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA

#### **Edited by:**

Mary K. Lobo, University of Maryland School of Medicine, USA

#### **Reviewed by:**

Deborah Suchecki, Universidade Federal de Sao Paulo, Brazil Ofer Yizhar, Weizmann Institute of Science, Israel

#### **\*Correspondence:**

Colleen A. McClung, Department of Psychiatry, University of Pittsburgh School of Medicine, 3550 Terrace Street, Pittsburgh, PA 15213, USA e-mail: mcclungca@upmc.edu

The ability to probe defined neural circuits with both the spatial and temporal resolution imparted by optogenetics has transformed the field of neuroscience. Although much attention has been paid to the advantages of manipulating neural activity at millisecond timescales in order to elicit time-locked neural responses, little consideration has been given to the manipulation of circuit activity at physiologically relevant times of day, across multiple days. Nearly all biological events are governed by the circadian clock and exhibit 24 h rhythms in activity. Indeed, neural circuit activity itself exhibits a daily rhythm with distinct temporal peaks in activity occurring at specific times of the day. Therefore, experimentally probing circuit function within and across physiologically relevant time windows (minutes to hours) in behaving animals is fundamental to understanding the function of any one particular circuit within the intact brain. Furthermore, understanding how circuit function changes with repeated manipulation is important for modeling the circuit-wide disruptions that occur with chronic disease states. Here, we review recent advances in optogenetic technology that allow for chronic, temporally specific, control of circuit activity and provide examples of chronic optogenetic paradigms that have been utilized in the search for the neural circuit basis of behaviors relevant to human neuropsychiatric disease.

**Keywords: optogenetics, opsins, circadian rhythms, addiction, depression, bipolar disorder, obsessive-compulsive disorder, mouse models**

#### **INTRODUCTION**

Optogenetics has transformed the field of neuroscience with its ability to manipulate neural circuit activity with unprecedented spatial and temporal precision. Indeed, the ability to manipulate neural activity at physiologically relevant millisecond timescales has leveraged an advantage of using optogenetics as a tool to link time-locked changes in neural activity to behavioral and/or physiological events and has been instrumental in our understanding of the neural circuitry driving an array of behavioral states (Tye and Deisseroth, 2012; Nieh et al., 2013). Little consideration, however, has been given to (1) the importance of proper timing for delivery of optogenetic stimulation, (i.e., at physiologically relevant times of day); and (2) to the overall duration of manipulation (acute vs. chronic). Although treated separately for the purpose of this review, these two temporal considerations are ultimately interconnected: in addition to acute optogenetic stimulation, the expanding use of chronic stimulation paradigms, in general, will require attention to the circadian timing of circuit manipulation; the proposed use of optogenetics to alter circuit rhythms will require extended stimulation parameters.

## **CONSIDERATION OF DIURNAL-SPECIFIC CONTROL OF NEURAL ACTIVITY**

The circadian clock is a temporal interface that synchronizes internal physiological and behavioral events to the external environment. A core group of proteins generate biological rhythms in an approximate 24 h cycle (**Figure 1**). The master clock governing these rhythms resides in the suprachiasmatic nucleus (SCN) of the anterior hypothalamus (Ralph et al., 1990) and coordinates the activity of self-sustained peripheral clocks (Yoo et al., 2004) that are expressed almost ubiquitously throughout bodily tissue. Components of the molecular clock are also found in extra-SCN regions in the central nervous system and notably in brain regions implicated in mood-regulation and reward, such as the hippocampus, amygdala, prefrontal cortex, lateral habenula, nucleus accumbens, and ventral tegmental area (VTA) of both rodents (Abe et al., 2002; McClung et al., 2005; Guilding and Piggins, 2007; Webb et al., 2009) and humans (Li et al., 2013). In addition to exhibiting rhythms in circadian gene expression, these brain regions exhibit rhythms in neural firing, neural activity, neurotransmitter levels and receptor expression (Guilding and Piggins, 2007; Sleipness et al., 2007; Hampp et al., 2008; Webb et al., 2009; Baltazar et al., 2013). Perceptible temporal peaks and troughs in neural activity are observed across brain regions with each displaying a unique temporal profile of activity. It is proposed that coordinated neural activity across regions is important for the proper daily timing of behaviors and avoids co-occurrence of conflicting motivational states (i.e., feeding vs. sleeping, for example). Therefore, consideration of not only how neural circuit activity impacts behavior but also how the timing of stimulation influences a

protein 1 (BMAL1). This complex binds to enhancer box (E-box) regulatory element and activates transcription of the Period (Per) and Cryptochrome (Cry) genes. As PERs and CRYs accumulate during the day, they dimerize and translocate back to the nucleus where they interact with CLOCK and BMAL1 to repress their own transcription. As these negative elements are degraded by casein kinases (CK) at night, repression of CLOCK and BMAL1 is removed, and a new cycle of transcription begins the following morning. The CLOCK-BMAL1 complex also binds to an array of clock-controlled genes (ccg) that ultimately regulate biological processes such as sleep-wake cycles, body temperature, hormone secretion, feeding, and activity (modified from an original by Mohawk et al., 2012; Frank et al., 2013).

given behavioral state may yield additional, ethologically relevant information. This is particularly relevant when studying the reward system as there is known daily variation in rewardseeking and drug response in both rodents and humans. For instance, the reinforcing properties of cocaine are greater during the light than dark-phase of the light/dark cycle in rodents (Abarca et al., 2002). Furthermore, nicotine administration and sensitivity in rodents and humans follows a daily rhythm, with more intense periods of administration and sensitivity to nicotine presenting during the light-cycle (Mooney et al., 2006; Mexal et al., 2012). Importantly, the propensity to administer drugs of abuse and the sensitivity to natural reward in rodents has been linked to diurnal changes in dopaminergic activity within the mesolimbic system (Webb et al., 2009). Therefore, neural circuit manipulation during peaks and troughs of activity may yield more pronounced behavioral responses, i.e., inhibition of a given circuit when activity is usually highest may be preferred over inhibiting when activity is at a daily low (**Figure 2A**). Alternatively, for long-term stimulation paradigms (discussed in next section), stimulating a circuit at a time of day that is not in accordance with its natural rhythm may disrupt overall network synchrony, which may or may not be the intended experimental effect.

# **DIURNAL RHYTHM CHANGES WILL NECESSITATE CHRONIC STIMULATION PARADIGMS**

Conversely, one may want to use optogenetic approaches to purposefully disrupt or alter diurnal rhythms. Studies that have employed jet lag paradigms or other types of phase resetting protocols have demonstrated that days to weeks are required for an animal to fully adjust/entrain to a new light/dark cycle (Albrecht, 2012). It stands to reason, therefore, that studies aimed at altering the amplitude, phase and/or period of neural activity rhythms will require long stretches of time. This will necessitate the development of novel stimulation paradigms that permit both chronic excitation and inhibition within the same neurons. The prospect of these studies are exciting as they will directly determine the relevance of diurnal rhythms in select neuronal populations to disease states and basic brain function.

To exemplify this point, our lab uses the *Clock*1*19* mutant mouse to study how chronic alteration in the daily activity of specific neural circuits impacts reward and mood-related behaviors. These mice display a behavioral profile that is strikingly similar to human bipolar-mania (Roybal et al., 2007) that occurs within the overall context of disrupted circadian rhythms, driven by a point mutation in exon19 of the *Clock* gene (Vitaterna et al., 1994; King et al., 1997). Behavioral abnormalities include lower levels of anxiety-like behaviors, decreased depressive-like behaviors, hyperactivity, reduced sleep time, and increased propensity for both natural (i.e., sucrose) and drug rewards (Naylor et al., 2000; Roybal et al., 2007). Lithium treatment, which is used as a moodstabilizing agent to treat human bipolar disorder, is also effective at reversing many of these behavioral abnormalities (Roybal et al., 2007). *Clock*1*19* mutant mice exhibit gross neural circuit abnormalities that relate to improper timing of neural events, including altered phase-coupling within the nucleus accumbens (Dzirasa et al., 2010) and deficits in synchronization of neural activity across limbic brain regions (Dzirasa et al., 2011). Additionally, *Clock*1*19* mice exhibit profound alterations in the diurnal activity of the mesolimbic dopamine system, (McClung et al., 2005; Spencer et al., 2012) which has been shown to underlie components of their manic-related behavioral profile (Coque et al., 2011). Although it is intriguing to speculate that altered circadian clock machinery is the driving force generating these neural circuit timing-deficits, the exact mechanism has yet to be fully elucidated. Regardless, the important point to emphasize is that manipulating the timing, in addition to the direction, of neural circuit activity is instrumental to understanding how altered neural activity contributes to abnormal behaviors. This same principle can be applied to the study of any behavior that exhibits natural daily rhythms and to the host of chronic disorders that have been linked to disruptions in biological rhythms (Bass and Takahashi, 2010; Yu and Weaver, 2011; Albrecht, 2012; McCarthy and Welsh, 2012; Coogan et al., 2013; Frank et al., 2013).

# **CHRONIC OPTOGENETIC MODULATION OF NEURAL CIRCUIT ACTIVITY**

In general, understanding how chronic disruption of neural activity, i.e., that which occurs over the course of many days and weeks, contributes to the development of abnormal behavior

may better model the chronic circuit-wide disruptions underlying neurological disorders, psychiatric disorders, and drug addiction. One can speculate that perturbing neural circuit activity chronically may induce neural plastic changes, such as altered synaptic structuring, altered neural firing properties, and overall network re-wiring, not induced by acute modulation. Chronic neural circuit modulation, therefore, has the potential to provide additional information not revealed by studying acute time-locked neural responses. Chronic optogenetic stimulation paradigms—defined as manipulation that occurs across multiple, consecutive dayshave been used by relatively few studies. Their use, as demonstrated below, can often yield distinct results from those obtained using acute stimulation protocols. It should be noted that although we focus on the *in vivo* use of chronic manipulation, there are a few *in vitro* studies that have emerged which use chronic optogenetic stimulation for various applications (Stroh et al., 2011; Lignani et al., 2013).

In the first study, Lobo et al. (2013) used a chronic optogenetic stimulation protocol to determine the circuit-level mechanisms underlying distinct patterns of protein expression induced in striatal neurons in response to chronic stimuli (**Figure 2B**). Chronic stimuli such as drugs of abuse, stress, and natural reward lead to a stable up-regulation of the transcription factor, 1FosB, in the striatum (McClung et al., 2004). This increase is thought to be important for the long-term adaptive changes associated with chronic stimulus exposure. Previous evidence has suggested that specific chronic stimuli differentially induce 1FosB in specific subsets of striatal GABAergic projection medium spiny neurons (MSNs) (McClung et al., 2004; Pitchers et al., 2013). However, the specific circuits mediating the distinct induction of striatal 1FosB in response to different stimuli were unknown. Lobo et al. (2013) used a chronic optogenetic approach to stimulate specific striatal afferents and examined the resulting pattern of 1FosB induction in dopamine receptor 1 (D1) vs. dopamine receptor 2 (D2) enriched MSNs. To accomplish this, the light-sensitive ion channel, channelrhodopsin (ChR2), was expressed in the VTA and in glutamatergic neurons of the ventral hippocampus, medial prefrontal cortex (mPFC), and amygdala, followed by fiber optic implantation at the respective cell bodies. Mice received phasic pulses of blue light at 20 Hz for 40 ms to stimulate VTA neurons and 20 Hz pulses for 30 s for stimulation of mPFC, amygdala, and hippocampal neurons. Optic stimulation was delivered for 10 min a day for five consecutive days and 1FosB was measured in D1 and D2 enriched MSNs 24 h following the last day of the 5 day stimulation protocol. Chronic optic stimulation of the different striatal inputs lead to a predominate induction of 1FosB in D1- MSNs, consistent with previous studies showing that optogenetic stimulation of these afferents promote reward. Additionally this demonstrated that repeated episodes of 10 min optic stimulation were sufficient to induce neural plasticity similar to repeated exposure to drugs of abuse and natural rewards. The ability to probe cell-defined and projection-specific circuits leverages an advantage for using chronic optogenetic stimulation to delineate the circuit mechanisms driving adaptive neural changes.

In another study, Ahmari et al. (2013) addressed the role of cortico-striatal dysregulation in obsessive-compulsive disorder (OCD). There is compelling evidence to suggest a role for hyperactivity of the orbitofrontal cortex (OFC) and ventromedial striatum (VMS) in OCD pathology (Pittenger et al., 2011), however, a causal role has been difficult to ascertain. Ahmari and colleagues employed a chronic optogenetic stimulation protocol to directly assess the role of repeated cortico-striatal stimulation on OCDlike behaviors in mice (**Figure 2B**). To accomplish this, a credependent virus encoding ChR2 was transduced into glutamatergic neurons of the OFC followed by a fiber optic implant in the VMS to chronically stimulate the OFC-VMS projection pathway. EMX-cre mice (ensures glutamate cell-type specificity for viral transduction) received 10 Hz, 10 ms pulses of blue light for 5 min a day, across multiple days. A sustained, stimulation-independent increase in OCD-like behavior (as indicated by excessive grooming relative to control mice) was observed following 3 days of stimulation. Repeated stimulation also led to a progressive increase in light-evoked VMS neural firing that paralleled the increase in repetitive grooming behavior. This effect persisted for 2 weeks after cessation of a 6 day stimulation protocol, indicating that chronic stimulation induced long-term plastic changes at OFC-VMS synapses. Importantly, changes in grooming behavior were not seen following acute stimulation, defined as grooming measured both immediately and 24 h following a single 5 min stimulation session. This indicates that chronic circuit changes are required for the expression of specific behaviors that extend beyond immediate changes in neural firing.

It should be emphasized that the opsins, or light-sensitive ion channels, used to transduce optic signals to neurons can vastly affect the permitted time scales used for stimulation. For instance, the work described above relied on ChR2 for control of neural activity, which is an opsin with fast channel photokinetics (Nagel et al., 2003; Boyden et al., 2005). This means that the ion channel opens and closes on a millisecond timescale in response to light and that repeated pulses of light are required for prolonged and sustained activation. This can become problematic as ChR2 has the potential to desensitize with repeated stimulation (Nagel et al., 2003), vastly limiting its use for protocols that involve prolonged periods of stimulation (i.e., over tens of minutes). Furthermore, repeated stimulation with light irradiances (mW/mm<sup>2</sup> ) required to activate ChR2 may raise concern over the potential for significant heat generation, which can have deleterious effects on brain tissue (Yizhar et al., 2011a). Moving forward, the use of optogenetics for more prolonged stimulation will require opsins that do not readily desensitize and that are more sensitive to lower powers of light (such as the step-function opsins) (Berndt et al., 2009; Yizhar et al., 2011b).

## **TECHNOLOGICAL ADVANCES PERMITTING CHRONIC OPTOGENETIC STIMULATION**

There are a growing number of hardware options available that are compatible with chronic optogenetic stimulation protocols. For instance, chronic fiber implants have greatly reduced tissue damage associated with the repeated removal and insertion of optical fibers to deep brain structures. These optical neural interfaces include a fiber optic, which transmits light to deep brain structures, and a metal or ceramic ferrule that is secured to the fiber optic (Aravanis et al., 2007). The ferrule protrudes from the head for coupling and tethering to an external light source. These fiber cannulas are relatively simple to make in-house (Sparta et al., 2012) and can be surgically implanted and secured for chronic use (Ung and Arenkiel, 2012).

When the duration of the chronic stimulation protocol extends from minutes to hours and across many days, however, there are other hardware considerations that need to be made. For instance, animals are usually tethered to a light source during acute stimulation, however, more complex chronic stimulation protocols may involve substantially extended durations of time whereby prolonged tethering in the homecage or experimental environment is not possible. Furthermore, for multiple stimulations across a 24 h period, the constant removal and re-tethering of the animal may induce profound handling stress and have a negative impact on behavior. In these cases, wireless optogenetic technology may provide a truly remote and non-invasive system for chronic control of neural activity and will permit more complex chronic stimulation paradigms across a single day or multiple days and weeks.

Although the field of wireless optogenetic technology is still in its infancy, a variety of promising platforms have emerged that use infrared or radiofrequency (RF) signaling to remotely control a head-mounted device. This device can detect and relay wirelessly generated signals to light-emitting diodes (LEDs) that are secured directly on the head-mounted device (Iwai et al., 2011; Wentz et al., 2011; Ameli et al., 2013). This circumvents the need for tethered fiber optic devices that require a physical fiber cable connection between an animal and an external light source. Wireless head-mounted LEDs have been used to stimulate surface brain structures, such as the motor cortex in Thy1-ChR2 transgenic mice (Iwai et al., 2011), and have been coupled to implanted fiber optic cannulas for optic stimulation of deeper brain structures (Wentz et al., 2011). With the advent of redshifted opsins, such as C1V1 (Yizhar et al., 2011b; Mattis et al., 2012) and the newer ReaChR (Lin et al., 2013) designed by Roger Tsien's group, deep brain structures can be targeted wirelessly without the need for a fiber optic insert into the brain. In this case, a head-mounted red LED would be sufficient to activate opsins expressed in deep brain structures, as red light scatters less than blue light and can penetrate into deep tissue more readily.

Micro-LEDs offer a slightly different approach to wireless control of neural activity whereby a µ-LED is implanted directly into the brain rather than being mounted on a head-stage (Kim et al., 2013; McAlinden et al., 2013; McCall et al., 2013). At least one group has combined these cellular-scale injectable µ-LEDs with a head-mounted antenna for remote and wireless control of the chronically implanted µ-LEDs (Kim et al., 2013; McCall et al., 2013). The implanted LEDs were well tolerated by freely moving animals and maintained operational functionality up to the tested maximum of 6 months (Kim et al., 2013). As a proof of principle for wireless control of neural activity, the authors connected the implanted µ-LED probe to a head-stage antenna to detect wirelessly generated RF signals. VTA dopamine neurons transduced with a cre-dependent ChR2 were tonically stimulated using this wireless system. Here, tyrosine hydroxylase (TH): Cre mice received 5 Hz, 5 ms pulses of blue light for 3 min while behaving in the elevated zero maze—a validated behavioral test used to assess anxiety-like behaviors. Acute tonic stimulation of dopamine neurons produced an anxiolytic behavioral response, independent of locomotor changes, as measured while animals were being actively stimulated.

Many of these head-mounted devices, however, are bulky and would benefit from scaling-down to sizes compatible with longterm use in rodents in order to avoid damage to the hardware components. Furthermore, it is essential that steps be taken to ensure adequate detection of wirelessly generated signals as limited detection ranges/distances may present a current limiting factor. We look forward to continued advances in the fields of electrical, mechanical, optical, and biological engineering as this will lead to a vast expansion of technological options permitting novel and more refined ways to probe the nervous system (Deisseroth and Schnitzer, 2013).

#### **SUMMARY**

Here we propose that the time of day and duration (acute vs. chronic) of circuit modulation are important experimental considerations for certain optogenetic applications. Manipulation of neural activity at physiologically-relevant times of day is an important methodological consideration for both acute and chronic stimulation of circuits that display natural daily rhythms. Experiments aimed at altering diurnal rhythms using optogenetics will necessitate the use of chronic stimulation paradigms due to the extended timescale required for the entrainment of rhythms. Chronic stimulation paradigms have begun to emerge with the potential to uncover the neuroplastic events and circuit-wide disruptions associated with chronic disease states. Although these temporal considerations may not apply in all cases, an awareness of their rationale, experimental advantage, and appropriateness to specific experimental methodologies will permit more physiologically relevant paradigms for probing neural circuit function in health and disease.

## **REFERENCES**


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 03 December 2013; accepted: 27 January 2014; published online: 14 February 2014.*

*Citation: Sidor MM and McClung CA (2014) Timing matters: using optogenetics to chronically manipulate neural circuitry and rhythms. Front. Behav. Neurosci. 8:41. doi: 10.3389/fnbeh.2014.00041*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience.*

*Copyright © 2014 Sidor and McClung. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Recombineering strategies for developing next generation BAC transgenic tools for optogenetics and beyond

# *Jonathan T. Ting\*† and Guoping Feng*

*McGovern Institute for Brain Research and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA*

#### *Edited by:*

*Mary Kay Lobo, University of Maryland School of Medicine, USA*

#### *Reviewed by:*

*Michelle Gray, The University of Alabama at Birmingham, USA Joseph D. Dougherty, Washington University School of Medicine, USA*

#### *\*Correspondence:*

*Jonathan T. Ting, McGovern Institute for Brain Research and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, 43 Vassar St., Cambridge, MA 02139, USA e-mail: jtting@mit.edu*

#### *†Present address:*

*Jonathan T. Ting, Human Cell Types Program, Allen Institute for Brain Science, Seattle, WA, USA*

The development and application of diverse BAC transgenic rodent lines has enabled rapid progress for precise molecular targeting of genetically-defined cell types in the mammalian central nervous system. These transgenic tools have played a central role in the optogenetic revolution in neuroscience. Indeed, an overwhelming proportion of studies in this field have made use of BAC transgenic Cre driver lines to achieve targeted expression of optogenetic probes in the brain. In addition, several BAC transgenic mouse lines have been established for direct cell-type specific expression of Channelrhodopsin-2 (ChR2). While the benefits of these new tools largely outweigh any accompanying challenges, many available BAC transgenic lines may suffer from confounds due in part to increased gene dosage of one or more "extra" genes contained within the large BAC DNA sequences. Here we discuss this under-appreciated issue and propose strategies for developing the next generation of BAC transgenic lines that are devoid of extra genes. Furthermore, we provide evidence that these strategies are simple, reproducible, and do not disrupt the intended cell-type specific transgene expression patterns for several distinct BAC clones. These strategies may be widely implemented for improved BAC transgenesis across diverse disciplines.

**Keywords: bacterial artificial chromosome, transgenic mice, BAC recombineering, Drd1a, Drd2, Adora2a, Chat, DAT**

## **INTRODUCTION**

Bacterial Artificial Chromosomes (BACs) are large DNA constructs composed of a small cloning vector backbone ligated to large fragments of restriction-digested genomic DNA that can be stably propagated as well as manipulated in bacterial host cells. Extensive BAC libraries have been constructed with genomic material from a variety of organisms and have served as indispensable tools for large-scale genome sequencing and mapping efforts. One such project culminated in the release of three landmark mouse BAC libraries derived from the C57BL/6J and 129S6/SvEvTac strains with a combined 30-fold coverage of the mouse genome (Osoegawa et al., 2000). This resource was quickly tapped to create the first BAC transgenic mouse line with functional transgene expression driven by successful integration of an engineered Bacterial Artificial Chromosome into the mouse genome (Yang et al., 1997). This pioneering work established the feasibility of diverse targeted manipulations of BAC clones in *E. coli* by homologous recombination, a method now commonly referred to as BAC recombineering (recombinationmediated genetic engineering). Furthermore, this work boldly asserted the incredible potential of BACs for gene therapy, disease modeling, and other basic research applications aimed at deciphering gene function.

The ability to obtain and manipulate BAC clones with genomic DNA spanning hundreds of kilobases is highly advantageous for transgenic applications given that most mouse BAC clones (average insert size 150–200 kb) encompass one or more genes including flanking regions essential to instructing cell-type specific expression patterns *in vivo*. Furthermore, the presence of large spans of insulating genetic material around a transgene expression cassette can curb transgene silencing or mosaic expression *in vivo* due to position effect variegation following random integration into unfavorable sites of the host genome (Bian and Belmont, 2010). Indeed, Yang et al. (1997) proposed that BAC transgenesis would be especially expedient for creating mice with faithful cell-type specific expression of Cre recombinase for gene disruption in mice, an idea that directly catalyzed the immensely successful Gene Expression and Nervous System ATlas (GENSAT) project at Rockefeller University led by Nathaniel Heintz and colleagues (Gong et al., 2003; Heintz, 2004). Importantly, BAC transgenic Cre driver rodents (particularly the extensive collection from the GENSAT project) have played a pivotal role in expanding the utility of emerging optogenetics-based technologies in neuroscience by enabling unprecedented access to monitor and manipulate genetically-defined cell populations in the nervous system when used in combination with Cre-inducible expression strategies for diverse optogenetic probes (Atasoy et al., 2008; Cardin et al., 2009; Petreanu et al., 2009; Witten et al., 2011; Madisen et al., 2012; Saunders et al., 2012; Zariwala et al., 2012). In addition, numerous laboratories have tapped into the knowledge base of the GENSAT project to guide the development of new tools such as the first collections of BAC transgenic mice with direct cell-type specific expression of ChR2 in the central nervous system (Hagglund et al., 2010; Zhao et al., 2011). Collectively, these BAC transgenic tools are now widely implemented for optogenetic deconstruction of complex neural circuits that mediate diverse animal behaviors and brain states (Tsai et al., 2009; Kravitz et al., 2010; Witten et al., 2010; Aponte et al., 2011; Halassa et al., 2011; Yizhar et al., 2011; Tai et al., 2012; Tan et al., 2012; Bock et al., 2013; Chaudhury et al., 2013; Cui et al., 2013; Stamatakis et al., 2013; Steinberg et al., 2013). The broader topic of optogenetic dissection of behavior in mammalian model systems has been extensively reviewed elsewhere (Tye and Deisseroth, 2012; Yizhar, 2012; Lenz and Lobo, 2013; Nieh et al., 2013).

Here we propose strategies for developing the next generation of BAC transgenic lines that are devoid of overexpressed extra genes. We describe multiple BAC recombineering strategies for eliminating undesirable extra genes from BAC clones in order to circumvent confounds due to overexpression of such extra genes in BAC transgenic lines. In addition, we demonstrate that these modification procedures may be performed in parallel or sequentially with routine BAC recombineering steps for introducing a transgene of interest under the control of cell-type specific promoter elements. Together, these procedures effectively result in pure transgenic expression cassettes ≥65 kb in size that can be used in pronuclear injection to produce BAC transgenic animals. These steps are simple, efficient, reproducible, and can be implemented for the modification of any BAC clone. We are applying these methods in developing the next generation of BAC transgenic animals for optogenetics-based research and expect these strategies may be widely adapted across diverse disciplines.

## **MATERIALS AND METHODS**

#### **TARGETING VECTOR CONSTRUCTION**

A pBlueScript-derived vector iTV1 was used to generate BAC targeting vectors. iTV1 contains cloning sites for A and B homology arms which flank a large multiple cloning site, bovine growth hormone polyadenylation (BGHpA) signal, and FRT flanked neomycin resistance cassette (FRT-NEO-FRT). First, iTV1 was modified by addition of the woodchuck post-translational regulatory element (WPRE) upstream of the BGHpA to create iTV1-WPRE. This modification was chosen to improve transgene protein expression level and RNA stability. BAC specific homology arms A and B (400–600 bp each arm) were PCR amplified from BAC template DNA and cloned into iTV1- WPRE. In a separate mammalian expression vector, two copies of ChR2(E123T/H134R), herein referred to simply as ChETATR, and the orange/red emitting tdTomato fluorophore were combined using viral 2A linkers. We selected P2A with a C-terminal GSG linker because this combination was shown to be the most efficiently processed of all viral 2A elements (Kim et al., 2011) and thus was the most likely to enable a complete physical uncoupling of the opsin and fluorophore with no significant unprocessed protein fraction. The ChETATR-P2A-ChETATR-P2A-tdTomato cassette (i.e., 2xChETA-P2A-tdTomato) was then sub-cloned into the large multiple cloning site of iTV1-A/B-WPRE to complete the BAC targeting vectors. In this final cloning step great care was taken to ensure minimal disruptions to the junction between the A Box and the start of the ChETA sequence, except for addition of a Kozak consensus sequence.

#### **BAC TRIMMING CASSETTES**

The pBlueScript-based deletion cassettes were constructed by standard PCR and cloning methods. The order of assembled features is as follows: (1) a 50–350 bp BAC-specific homology arm (selected to target the preferred location for BAC trimming in a BAC clone of interest), (2) AscI and NotI restriction sites (for later removal of the BAC vector during purification of BAC DNA for pronuclear injection), (3) the *bla* gene encoding ampicillin resistance (AmpR), and (4) a "loxP deletion" homology arm (targeting the BAC vector sequence adjacent to the wild-type loxP site in pBACe3.6 or pTARBAC1 vectors).

loxP deletion arm: 5 -gttaacgtgccggcacggcctgggtaaccaggtattttgtc cacataaccgtgcgcaaaatgttgtggataagcaggacacagcagcaatccacagcaggcat acaaccgcacaccgaggttactccgttctacaggttacgacgacatgtcaatacttgcccttgac aggcattgatggaatcgtagtctcacgctgatagtctgatcgacaatacaagtg-3

BAC clone specific deletion arms:

Adora2a BAC (RP24-238K3): 5 -gagctgagtggccagcgacctattgcc taggcatagataaccatatatca-3

Drd2 BAC (RP23-161H15): 5 -gagaccagtgccagcagaagctatggtc attgtggtgataggagcgtggctga-3

Chat BAC (RP24-256F2): 5 -ttcagtccactatactttcctgcttttcttcatg gcttagcaaggttcctggacctcagcagagttaatagaaaatgcaggctgcaactggatggtt agcgatgaaactaagcaactctagacagtgcagtcagacacatactttctctaactggcggga gttactcactcacgcaatcacctctaacacttaaccacagcaggtggaaattgagttagtttaa gaggctaactctgtgctaagcctggggacttgggacaggaaagccttggccccgcccagcag tggccccgcccacctctctgaaggctggactggcggttgcctagcagcagg-3

DAT BAC (RP24-269I17): 5 -agatataacctacctttgcatgttagccagga ttaagatttatattaccg-3

Pvalb BAC (RP24-306A6): 5 -ttccggaaggtgcacagcacgggtgctgtc cgcagttgtttgctgtgcaagcatgcagggccttcctgtcttcctcactgctcaacgttgcacat ttttccctccccgctatttcagtttttagggttcataacatgcttgggctttaaagatggcatttcg attctggacgtgaa-3

Vgat BAC (RP24-246L1): 5 -agttccatccctggaacctattgggtgaagag aacaggcatcttggcacgctcccacacccatcacgcaataaataaacaaatgaacagaaaaa gaagtcaaaggggaaggagattagcttaacacggtcacaaaactaagggctagatctgcaag gctatgcagacagagaccaagggagagtaaggggacagggaggggcggagtccagcccag tgtgggtggagcctttggtcccttagttagagca-3

#### **GENE INACTIVATION CASSETTES**

Sfxn1 homology arms A and B (1 kb each) were PCR amplified and cloned into pBlueScript II. A 29 bp dsOligo containing a three-frame translational stop mutation and diagnostic SalI restriction site (5 -gaattc**tag**a**taa**c**tag***gtcgac*ctgcag-3 ), was synthesized and cloned between the two homology arms. This vector is designed to insert the three-frame translational stop cassette after the 13th codon in the Sfxn1 coding region in the *Drd1a*-spanning BAC clone RP23-47M2 and was thus called "Sfxn1 Target." A second vector "VAChT Target" was designed with the identical strategy except that homology arms A and B directed the insertion of the three-frame translational stop cassette after the 5th codon in the *Slc18a3* coding region (VAChT locus) in the *Chat*spanning BAC clone RP24-256F2. Relatively long 1 kb homology arms were selected in order to ensure a high efficiency of homologous recombination since this targeting cassette does not contain a selection marker for screening and must be used in combination with a separate iTV1-based targeting vector.

Critical regions of all constructs were verified by DNA sequencing. The targeting vectors were linearized by restriction digestion and purified in preparation for subsequent use in BAC recombineering steps. Care was taken to ensure adequate duration of restriction digestion (or in some cases two sequential rounds of digestions were performed) such that there was no contaminating uncut plasmid carried over into the electroporation step.

Additional information including complete DNA sequences of the targeting vectors and modified BAC clones used in this study are available upon request.

## **BAC RECOMBINEERING**

Appropriate BAC clones were selected from the RP23/RP24 C57 mouse BAC library following a detailed analysis using the Esembl Genome Browser (www*.*Ensembl*.*org) and were obtained from the Children's Hospital Oakland Research Institute (CHORI). The intact BAC DNA was isolated from the original host DH10B strain by BAC mini-prep and verified by restriction digestion and pulsed field gel electrophoresis (PFGE). Successfully verified BAC DNA was transferred into the EL250 strain in preparation for recombineering steps and maintained as glycerol stocks stored at −80◦C.

Most often the first strategic recombineering step was BAC trimming to delete unwanted regions of BAC DNA harboring extra genes. 20 mL of LB media plus chloramphenicol were inoculated with EL250 cells containing the target BAC clone and placed in a shaking incubator at 32◦C for 2 h until the cells reached an early log growth phase. The culture was induced for 15 min at 42◦C in a shaking water bath to permit transient expression of genes required for homologous recombination. Cells were rapidly chilled, pelleted, and washed three times with ice-cold MilliQ water to remove salts. Approximately 10–100 ng of linearized BAC trimming vector was added to 50µL of induced cells in a 1 mm gap cuvette and the mixture was immediately electroporated (1.75 kV, 25µF, 200 Ohms). The cells were recovered and plated onto LB plus chloramphenicol and carbenicillin to select for successful homologous recombination events, meaning insertion of the AmpR cassette in place of the targeted region. Double resistant clones were picked for further analysis to verify the deletions.

In a second round of targeting the virtually identical steps were performed to insert the *ChETA*TR transgene expression cassette at the initiating methionine start codon of the targeted gene within the selected BAC clone by homologous recombination. In this case, successfully modified clones were resistant to chloramphenicol, carbenicillin, and kanamycin due to insertion of the FRT-NEO-FRT in the targeting vector. (Note: although it is easily feasible to perform the first and second targeting events in a single step due to the insertion of two different selection cassettes, it was preferable to do these steps in series in order to save a glycerol stock of the successfully trimmed BAC in EL250 cells for future reuse). The NEO cassette was then excised from the modified BAC by arabinose induction of flp recombinase in the EL250 cells. Modified BAC clones were extensively screened for accuracy and the correctly targeted BAC DNA was grown in large scale and purified using the BAC100 kit (Clontech). 10–15µg of BAC DNA was restriction digested with NotI or AscI to liberate the BAC vector from insert and the linear fragments were separated by PFGE. The intact BAC insert band was excised from the PFG, electroeluted, and spot dialyzed against fresh microinjection buffer.

For gene inactivation recombineering experiments the linearized Sfxn1 Target DNA was combined at ∼20-fold molar excess with linearized iTV1-D1-ChETA targeting construct DNA immediately prior to the electroporation step. This was best accomplished by diluting linearized iTV1-D1-ChETA targeting DNA 1:20 in water and then adding approximately equal volumes of linearized Sfxn1 Target and diluted iTV1-D1-ChETA into the cuvette for the electroporation into induced EL250 cells propagating the *Drd1a*-spanning BAC RP23-47M2. Double recombinants were identified by resistance to chloramphenicol and kanamycin and further verified by PCR screening and subsequent restriction digestion and PFGE of isolated BAC DNA to test for the incorporation of the diagnostic *SalI* site. Large scale BAC DNA isolation and processing for pronuclear injection were carried out as indicated above. In additional experiments we used VAChT Target to inactivate *Slc18a3* in the Chat-spanning BAC RP24-256F2. This dual targeting step was carried out together with the iTV1-Chat-ChETA targeting vector following identical methods as for Sfxn1 inactivation.

## **PRONUCLEAR INJECTION AND IDENTIFICATION OF TRANSGENIC FOUNDERS**

Transgenic mice (pure C57BL/6 Taconic) were generated by pronuclear injection of the highly purified intact BAC DNA into fertilized oocytes at a concentration of 0.5–2.0 ng/µL. Genotypes were determined by PCR from mouse tail DNA samples and linespecific primer sets. Mice that had the transgene integrated in the genome were kept as founders to establish distinct lines by mating to C57BL/6J mice. One line each of *Slc6a3-2xChETATR-P2A-tdTomato* (DAT-ChETA), *Drd1a-2xChETATR-P2A-tdTomato* (D1-ChETA), and *Adora2a-2xChETATR-P2A-tdTomato* (A2A-ChETA) were established for further analysis. Only hemizygous mice on a pure C57 background were used for experiments in this study. All research involving mice was conducted according to the Institutional Animal Care and Use Committee guidelines at MIT. All procedures were approved by the Institutional Animal Care and Use Committee at MIT.

## **SLICE IMMUNOSTAINING AND CONFOCAL IMAGING**

Mice were anesthetized with tribromoethanol (Avertin) and transcardially perfused with phosphate buffered saline (137 mM NaCl, 10 mM NaH2PO4, and 2.7 mM KCl, pH = 7.4; PBS) followed by 10% neutral-buffered formalin (Sigma-Aldrich, St Louis, MO). Brains were post-fixed in 10% formalin at 4◦C overnight. Fixed brains were sectioned (50 µm) on a vibratome (Leica Microsystems, Buffalo Grove, IL) and the free-floating sections were washed for 10 min three times with PBS. Sections were then blocked for 1 h at room temperature in PBS containing 5% normal goat serum, 2% bovine serum albumin and 0.2% Triton-X 100. After blocking, the sections were incubated overnight at 4◦C in blocking buffer containing one or more of the following antibodies: anti-ChAT (1:200 dilution; Millipore AB144P), anti-2A (1:500 dilution; Millipore ABS31), anti-red fluorescent protein (1:500 dilution; Rockland 600-401-379), anti-GFP (1:5000 dilution; Abcam AB6566), anti-DAT (1:300 dilution; Millipore MAB369), or anti-DARPP-32 (1:2000 dilution; BD Transduction 611520). Following the overnight incubation, sections were washed for 20 min three times with PBS and then incubated for 1 h at room temperature in Alexa dye-conjugated secondary antibody (1:1500; Invitrogen,Grand Island, NY). Alexa dyes utilized were: Alexa Fluor 488 (A-10667, goat anti-mouse and A-11034, goat anti-rabbit), and Alexa Fluor 555 (A-21429, goat anti-rabbit). Sections were washed for 20 min three times with PBS, dried on a glass microscope slide and mounted with VectaShield (Vector Laboratories, Burlingame, CA). Slides were stored at −20◦C and were thawed to room temperature immediately prior to imaging. Whole brain montage images were acquired with CellSens software on an Olympus BX61 equipped with a motorized stage and epifluorescence illumination. Confocal z-stack images were acquired on an Olympus Fluoview 1000 laser scanning confocal microscope equipped with 488 and 543 nm laser lines.

## **RESULTS**

#### **IMPLEMENTING WELL CHARACTERIZED BAC CLONES TO ACHIEVE REPRODUCIBLE CELL-TYPE SPECIFIC TRANSGENE EXPRESSION**

We have previously developed several BAC transgenic mouse lines for expression of ChR2 in diverse neuronal subsets of the nervous system (Zhao et al., 2011). Our general strategy has been to select BAC clones that have previously been validated in the GENSAT project, as determined by the successful development of mouse lines for cell type-specific expression of Cre or EGFP. With this strategy we have observed that independent BAC transgenic mouse lines created by modification of the same original BAC clone exhibit highly reproducible cell-type specific transgene expression that recapitulates endogenous gene expression patterns. This is exemplified in the case of the *Chat*-spanning BAC clone RP23-246B12, which was successfully used to express both EGFP and ChR2-EYFP in cholinergic neurons of the nervous system (**Figure 1**). Thus, a known BAC clone can be modified for expression of virtually any transgene of interest in a welldefined neuronal subset with a relatively high probability of success following pronuclear injection.

#### **MOST BAC CLONES USED TO MAKE BAC TRANSGENIC LINES SPAN MULTIPLE GENES**

In the course of this work it was noted that the majority of BAC clones selected for making BAC transgenic mice and rats contained multiple extra genes in addition to the explicitly targeted region (**Table 1**). Presumably this relates to the conventional wisdom that larger BAC clones spanning the targeted region are more likely to encompass the necessary regulatory elements for recapitulating the desired transgene expression pattern. Thus, in order to more broadly assess this issue of increased gene dosage for off-target genes we analyzed the entire collection of BAC clones from the GENSAT Cre driver mouse repository (http://www*.* gensat*.*org/cre*.*jsp). In total, 139 BAC clones from the RP23/RP24 C57 library were located and viewed with alignment to the mouse genome using the Ensembl Genome Browser (http://www*.* ensembl*.*org). This enabled us to determine the number of extra genes (i.e., genes other than the targeted gene region) spanned by each unique BAC clone. We report that 75% of the BAC clones used to derive GENSAT Cre driver lines contain at least one or more extra gene (**Figure 2A**), and in the extreme cases

**FIGURE 1 | Independent BAC transgenic mouse lines created by modification of the same original BAC clone exhibit highly reproducible cell-type specific transgene expression that recapitulates endogenous gene expression patterns. (A)** Sagittal brain section showing EYFP expression for *Chat-EYFP* line GH293 mice from the GENSAT project. **(B)** *Upper*, sagittal brain section showing native EYFP fluorescence for *Chat-ChR2(H134R)-EYFP* line 6. *Lower*, high magnification images of anti-GFP and anti-ChAT co-immunostaining demonstrating excellent co-localization. **(B)** Adapted from Zhao et al. (2011). Scale bar: 100 microns. Note the virtually identical pattern of transgene expression between these two independent lines shown in **(A,B)** with strong signal detected in known cholinergic nuclei including brainstem motor nuclui (facial nucleus and motor trigeminal nucleus), striatum, olfactory tubercle, and basal forebrain. Both lines were created by modification of BAC clone RP23-246B12 to insert the transgene of interest in place of the *Chat* gene.

up to 14 independent and complete extra genes were spanned by a single BAC clone (e.g., RP23-368D24 used to create *Thbs*-Cre lines SW48-Cre and SW52-Cre; **Figure 2B**). Only 25% of BAC clones were entirely devoid of extra genes (12%) or only contained incomplete portions of gene coding regions (13%). Notably, many of the analyzed BAC clones used to derive GENSAT Cre driver lines were also previously implemented for making EGFP reporter lines.

### **EXTRA GENES CONTAINED WITHIN BAC CLONES CAN RESULT IN UNINTENDED GENE OVEREXPRESSION IN BAC TRANSGENIC LINES**

The infrequently discussed down side of selecting very large BAC clones for BAC transgenesis is the potential for overexpression of unwanted extra genes and any associated behavioral, biochemical, or electrophysiological confounds. We previously **Table 1 | BAC transgenic lines typically have increased gene dosage of one or more extra gene.**


*\*Partial gene that is missing the first coding exon containing ATG.*

*ˆ Truncated gene.*

developed a ChAT-ChR2-EYFP BAC transgenic mouse line (line 6) that enables optogenetic control of cholinergic neuron firing *in vitro* and *in vivo* (Ren et al., 2011; Zhao et al., 2011; Ma and Luo, 2012). The BAC clone that was used to create our ChR2-EYFP line was the same as the clone used to create independent ChAT-Cre mouse and rat lines (Gong et al., 2007; Witten et al., 2011). This *Chat*-spanning BAC clone contains the additional genes *Ercc6* (partial), *Ogdhl, 1700024G13Rik* and *Slc18a3* (**Figures 3A–C**). Importantly, the *Slc18a3* gene is nested within the first intron of the *Chat* gene and encodes for the vesicular acetylcholine transporter (VAChT), the overexpression of which has profound implications for cholinergic function. This prediction was confirmed in a recent study that provided direct evidence for increased VAChT expression in ChAT-ChR2-EYFP line 6 mice, and linked this VAChT overexpression to increased cholinergic tone, enhanced motor endurance, and attention and memory deficits (Kolisnyk et al., 2013). More specifically, the authors reported a 20-fold increase in VAChT transcript level in striatum and a 5-fold increase in VAChT protein level in hippocampus with no significant elevation of ChAT expression in

either region (summarized in **Figure 3D**). The protein overexpression was associated with a 3-fold increase in stimulus-evoked ACh release in hippocampal slices, consistent with the behavioral data that VAChT overexpression has important functional consequences (Kolisnyk et al., 2013). Furthermore, striatal VAChT overexpression and elevated ACh release are associated with increased severity of psychomotor stimulant-induced repetitive behaviors in ChAT-ChR2-EYFP line 6 BAC transgenic mice (Jill R. Crittenden and Ann Graybiel, pers. commun.).

## **STRATEGY FOR DELETION OF EXTRA GENES BY BAC TRIMMING**

The most straightforward approach for eliminating extra genes contained in BAC clones is to remove segments from either end of the BAC insert, a method known as BAC trimming or shaving. We implemented a versatile method for BAC trimming in which small antibiotic selection cassettes are used to replace precise BAC DNA segments using a recombineering step in *E. coli* (Hill et al., 2000; Testa et al., 2003, 2004). The selection cassettes were PCR amplified using long primers to add ∼50 bp homology arms to either end, and then used for recombineering in EL250 cells. The homology arms define the exact BAC region that will be deleted by the insertion of the selection cassette, and the selection cassette enables subsequent identification of properly modified BAC clones. In addition, many different selection cassettes can be utilized for BAC trimming (e.g., ampicillin resistance-*bla*, kanamycin resistance-*neo*, chloramphenicol resistance-*cam*, blasticidin resistance-*bsd*, and streptomycin resistance-*rpsL*), providing options for sequential rounds of BAC trimming and transgene targeting using different selection markers. To further generalize this strategy for deleting fragments from any BAC clone in the RP23 and RP24 library, in subsequent experiments one of the homology arms was designed on the end of the BAC vector such that only the second homology arm needs to be customized for each unique BAC clone to define the deletion region—the region containing undesirable extra genes. In addition, we added NotI and AscI restriction sites adjacent to the unique homology arm but before the selection cassette. Once the deletion of the BAC segments was confirmed by restriction digestion and pulsed field gel electrophoresis (PFGE) the BAC insert was liberated from the BAC vector and selection cassette by NotI or AscI digestion and subjected to further purification. We have used this method to successfully delete precisely defined BAC DNA segments, as demonstrated for the *Slc6a3*-spanning BAC clone RP24-269I17 in which the extra genes *Clptm1l*, *Tert*, *Slc6a18*, and *Slc6a19* located in the 3 flanking region of the *Slc6a3* (DAT) gene were deleted by homologous recombination in which an ∼84 kb BAC region was replaced with a 1 kb ampicillin resistance (AmpR) cassette (**Figures 4A–C**). The "trimmed" BAC clone can then be used for insertion of a transgene cassette in place of the *Slc6a3* gene by standard BAC recombineering using an iTV-based targeting vector (see Materials and Methods). The successful DAT BAC trimming was verified by diagnostic restriction digestion with SalI and PFGE (**Figure 4D**).

It is worth noting that other methods have been described for removal of extra genes from BAC clones to produce BAC transgenic mouse lines. In one study a modified BAC clone was subjected to restriction digestion and PFGE, and fortuitously, a fragment could be excised that contained suitably large 5 and 3 flanking regions but excluded several extra genes originally present in the BAC clone (Chao et al., 2010). In another study a spontaneous deletion was identified in a single clone during the recombineering steps, and this deletion serendipitously removed the only extra gene present in the BAC clone (Belforte et al., 2010). However, the recombineering-based method we have outlined for

diagram of the *DAT* (*Slc6a3*)-spanning BAC clone RP24-269I17 reveals the presence of many extra genes including *Clptm1l*, *Tert*, *Slc6a18* and the truncated genes *Lpcat1* and *Slc6a3*. **(B)** The extra genes *Clptm1l*, *Tert*, *Slc6a18*, and *Slc6a19* located in the 3' flanking region of the *DAT* gene are deleted by homologous recombination. The recombination event replaces the ∼84 kb BAC region with a 1 kb ampicillin resistance (AmpR) cassette. **(C)** Diagram of the "trimmed" *DAT* BAC clone that can now be used for insertion of a transgene cassette in place of the *DAT* gene by BAC recombineering. **(D)** Verification of DAT BAC trimming by diagnostic restriction digestion with SalI and PFGE. Lane 1 and 2: modified DAT BAC DNA. Lane 3 and 4: trimmed DAT BAC DNA. Lane 5: MidRange I PFG Marker. In successfully trimmed BAC clones the 91 kb DNA fragment (blue arrow) is reduced to 17 kb (red arrow). In addition, a 6 and 4 kb fragment are deleted

contains the extra gene *Sfxn1* in the promoter region of the *Drd1a* gene. In the dual targeting strategy, two simultaneous homologous recombination events must occur. The first targeting event is the insertion of the transgene cassette (including the NEO selection marker) in place of the *Drd1a* gene. The second targeting event is the insertion of a triple frame stop mutation (and SalI restriction site) immediately downstream of the *Sfxn1* start codon. Double recombinant clones are identified by kanamycin resistance. **(F)** Verification of *Sfxn1* gene inactivation in modified *Drd1a* BAC DNA by diagnostic restriction digestion with SalI and PFGE. Lane 1: MidRange I PFG Marker. Lane 2: 2.5 kb marker. Lane 4: *Drd1a* BAC DNA (+*Sfxn1*). Lanes 6–8: *Drd1a* BAC DNA following *Sfxn1* inactivation. For successfully modified BAC clones the ∼95 kb DNA fragment (blue arrow) is cleaved into 60 and 35 kb fragments (red arrows). Note that all other bands remain unaltered.

BAC trimming affords greater reliability and precision for routine deletion of extra genes.

## **INACTIVATION OF POTENTIALLY CONFOUNDING "EXTRA" GENES BY MINIMAL INSERTION MUTATIONS**

Although BAC trimming can be a versatile and powerful method for deleting both small and large segments of BAC DNA, in some cases it is not permissible to apply this method for the removal of extra genes. For example, when an extra gene is located within the critical 5 and 3 flanking regions of the targeted gene, deletion of these flanking regions would likely negatively affect the pattern of gene expression. Thus, we devised a recombineering strategy to inactivate the *Sfxn1* gene nested within the putative promoter region of the *Drd1a* gene by inserting stop codons in all three reading frames to interrupt the coding region of *Sfxn1*. In this manner the inactivation of an extra gene can be carried out with minimal insertions as small as ∼15 bp, but in our case we have designed a 29 bp sequence (see Materials and Methods) for targeted inactivation of *Sfxn1* gene in the *Drd1a*-spanning BAC RP24-47M2 that further introduces unique restriction sites for diagnostic purposes (**Figure 4E**). The targeting vector for gene inactivation does not contain any selection marker and therefore must be used in conjunction with a second targeting vector that does contain a selection marker, in this case either an iTV-based targeting vector or a BAC trimming cassette.

In order to ensure that the antibiotic selection appropriately isolates double recombinant clones, the concentrations of the linearized targeting vectors were carefully proportioned. The linearized Sfxn1 Target and iTV1-D1-ChETA Target DNA were combined at ∼20:1 for the dual targeting step in EL250 cells propagating RP23-47M2 in order to favor a high likelihood of isolating clones with successful inactivation of *Sfxn1*. Initial attempts did not yield correct double recombinant clones, but instead, only the ChETA cassette with selection marker were found to be inserted in recovered BAC clones. The results were markedly improved by using high quality MAXI prep DNA of high concentration, ensuring the linearization was complete with no uncut plasmid remaining, and use of LB Lennox media instead of LB Miller in the recombineering protocol. With these changes the majority of experiments yielded one or more correct double recombinant BAC clones with insertion of the ChETA expression cassette downstream of the *Drd1a* promoter region and inactivation of *Sfxn1* by insertion of the three-frame stop mutation immediately downstream of the *Sfxn1* start codon. The insertion of the inactivation cassette incorporated a new diagnostic SalI restriction site adjacent to the three-frame stop mutation, and thus, it was feasible to verify *Sfxn1* inactivation at the DNA level by SalI digestion and PFGE (**Figure 4F**).

### **PRESERVED CELL-TYPE SPECIFIC EXPRESSION PATTERNS IN BAC TRANSGENIC MICE CREATED WITH ADVANCED RECOMBINEERING STRATEGIES**

An important consideration for the BAC trimming and gene inactivation strategies employed here is whether or not the BAC DNA manipulations ultimately negatively impact the transgene expression pattern in transgenic animals. Although these strategies appear highly effective for removal of unwanted genes from BAC clones, the utility of such manipulations would be greatly diminished if the cell type-specific expression patterns were not preserved. To demonstrate faithful expression we created BAC transgenic mouse lines for expression of the fast kinetic ChR2 variant ChETA under the regulatory elements for *Adora2A* (A2A), *Drd1a* (D1), and *Slc6a3* (DAT). The BAC clones were modified as described using BAC trimming or gene inactivation strategies such that no unwanted extra genes were present. The modified BAC clones were used for pronuclear injection to create BAC transgenic founders. In each case we were able to successfully identify a single founder line for A2A-ChETA, D1-ChETA, and DAT-ChETA with functional transgene expression matching to the respective endogenous expression patterns (**Figure 5**). In order to further validate transgene expression in D1-ChETA mice we crossed these mice to the well characterized D1-EGFP or D2-EGFP GENSAT BAC transgenic mice. Our analysis revealed a near perfect overlap of EGFP and tdTomato (proxy for ChETA) restricted to striatonigral medium spiny neurons (MSNs) in D1- EGFP/D1-ChETA double transgenic mice (**Figures 6A,B**). In contrast, virtually no overlap in EGFP and tdTomato expression was observed in the dorsal striatum region of D2-EGFP/D1-ChETA double transgenic mice indicating exclusion of ChETA expression from striatopallidal MSNs (**Figures 6C,D**).

# **DISCUSSION**

We have designed and implemented BAC recombineering strategies for eliminating undesirable extra genes from BAC clones in order to circumvent confounds due to overexpression of extra genes in BAC transgenic lines. The BAC recombineering strategies are simple, efficient, and reproducible and have been implemented to successfully modify numerous mouse BAC clones from the RP23/RP24 library. Our BAC trimming strategy enables precisely targeted deletions of undesirable BAC DNA regions, such as those stretches harboring off-target extra genes. By judicious selection of BAC clones, in most cases it is feasible to remove all unwanted extra genes with a single deletion step targeting only one end (rather than both ends) of the BAC. We have demonstrated precise deletions of up to 111 kb and spanning as many as four unique genes in a single recombineering event with no evidence for any upper size limit thus far. Notably, excessive removal of large expanses of BAC DNA can be detrimental to the intended transgene expression pattern. Thus, although BAC trimming may be optimal for a wide variety of applications, it may not be feasible to fully remove all unwanted portions of BAC DNA for every BAC clone of interest.

In some BAC clones a gene of interest may occasionally be found to reside in very close proximity to a separate off-target gene, as was the case for the *Drd1a* and *Chat* spanning BAC clones in our study. In such cases a simple BAC trimming strategy is not suitable since deletion of the unwanted extra gene would also eliminate putative 5 or 3 regulatory elements essential for specifying the unique expression pattern of the targeted gene. Thus, a gene inactivation strategy was implemented to insert a three-frame translational stop mutation plus diagnostic restriction site immediately downstream of the initiating methionine start codon in the unwanted extra gene. The size of the inactivation cassette is intentionally minimal (e.g., lacking antibiotic

**FIGURE 5 | Preserved cell-type specific expression patterns in BAC transgenic mice created with advanced recombineering strategies. (A–D)** Anti-2A slice staining in brain slices from DAT-ChETA line 3 mice reveals the distribution of membrane-targeted ChETA protein. Sagittal **(A)** and coronal **(B)** sections showing ChR2 expression in midbrain dopamine neurons. **(C,D)** High magnification images of axon terminal labeling in the dorsal striatum **(C)** and labeled neurons in the ventral tegmental area **(D)**. **(E,F)** Anti-RFP slice staining in brain slices from A2A-ChETA line 13 mice

showing tdTomato expression in striatopallidal medium spiny neurons (MSNs) in coronal **(E)** and sagittal **(F)** sections. Lighter expression can also be detected in putative cortical astrocytes. **(G)** Anti-2A slice staining in a sagittal brain slice from D1-ChETA line 1 mice showing ChETA expression in striatonigral MSNs. Additional expression is apparent in other brain regions, particularly the dentate gyrus, layer VI cortex, and olfactory bulb. **(F)** Anti-DARPP32 slice staining to label both striatonigral and striatopallidal MSNs. Scale bars: 1 mm in **(A,B,E–H)** and 100µm in **(C,D)**.

selection cassette) and serves to eliminate functional gene expression from the unwanted extra gene without additional disruption to important regulatory elements in the region. The BAC recombineering strategy for gene inactivation requires dual targeting together with a second targeting cassette incorporating a functional selection marker. This dual targeting strategy allows for simultaneous insertion of a transgene cassette and inactivation of an extra gene by selecting for clones that have undergone successful homologous recombination at both locations within the BAC DNA sequence—i.e., double recombinant clones. In order to ensure a high likelihood that the antibiotic selection appropriately isolates double recombinant clones, the concentrations of each linearized targeting vector must be carefully proportioned. With optimization, the dual homologous recombination step was efficient enough to routinely identify multiple successfully targeted clones in a single attempt.

The overall goal of our efforts was to remove all extra genes from the selected BAC clones using either or both of these versatile recombineering strategies, leaving only a single targeted gene with sufficient 5 and 3 flanking regions to direct faithful gene expression *in vivo* in transgenic mice derived from pronuclear injection of the modified BAC DNA. In essence, the resultant modified BACs could be considered as designer "mini BACs" that retain only the most essential elements from the parent BAC clone while removing all non-essential portions. In designing our approaches we have had excellent initial success following a general guideline of retaining at least 50 kb of 5 sequence and 15–20 kb of 3 sequence flanking the targeted gene within BAC clones; however, the extent of 5 and 3 regions required must be determined empirically on a case-by-case basis. Once a successfully modified BAC clone is validated for proper cell-type specific expression, the same BAC modification strategy or preferably the same trimmed BAC clone itself can then be re-purposed for subsequent recombineering and expression of virtually any other transgene of interest. We provide a list of several BAC clones that have been modified using these methods and that can be utilized to target transgene expression to defined neuronal subsets (**Table 2**).

The approaches we have described for modifying BAC DNA using recombineering in *E. coli* build upon and extend the work of others. In three earlier studies BAC trimming (sometimes also called BAC shaving) was similarly accomplished by targeted replacement of a DNA region with a selection marker using homologous recombination (Hill et al., 2000; Testa et al., 2003, 2004). The end goal of such BAC DNA manipulation strategies was either to create precise vectors for traditional gene targeting in mouse embryonic stem cells or to reduce redundant effort in genome sequencing projects. In contrast, we applied BAC trimming to eliminate extra off-target genes so as to avoid potentially confounding effects in mice derived from pronuclear BAC DNA injections.

A substantially different method of retrieving mini BAC fragments was previously described. This method utilized PCR and gap repair in *E. coli* to subclone ≤80 kb BAC fragments into the low copy number pBR322 vector (Lee et al., 2001). However, the resulting transgene expression patterns were highly variable and did not fully recapitulate the endogenous gene expression patterns. This stands in contrast to our results demonstrating excellent preservation of the intended cell-type specific transgene expression patterns in transgenic mice derived from BAC DNA modified using our improved strategies. This difference is likely due to the strict size limitation for sub-cloning DNA fragments

into pBR322, whereas no practical size limitation is expected with typical BAC vectors such as pBACe3.6 and pTARBAC1, particularly since BAC DNA is being trimmed rather than retrieved by gap repair. Thus, the approaches that we have described seem to be more flexible when selection of considerably larger 5 or 3 regions surrounding the target expression cassette are required for achieving faithful expression *in vivo,* as is most often the case.

There are now extensive repositories of BAC transgenic mouse lines available, including over a thousand distinct Cre driver and EGFP reporter lines produced by the GENSAT project alone. We analyzed the full collection of BAC clones used for creating the extensive GENSAT Cre driver repository as a representative data set and determined that 75% of the BAC clones contain at least one or more extra genes. Even more surprisingly, one-third of the


**Table 2 | Successfully modified BAC clones using BAC trimming and gene inactivation approaches.**

*\*Partial gene sequence only.*

*ˆ Gene inactivation by insertion of a 3-frame triple STOP cassette.*

BAC clones contained at least three or more extra genes and 5% contained at least nine or more extra genes. The most extreme case was a BAC clone that spanned fourteen different fully intact extra genes, and this clone was used to create both Cre driver and EGFP reporter lines (*Thbs3*-spanning BAC clone RP23-368D24). Despite the extensive proven utility of numerous GENSAT lines, it is an unfortunate possibility that many available BAC transgenic lines may suffer from confounds of varying degrees due in part to overexpression of one or more "extra" genes. In some cases the unintended overexpression of one or more extra genes in various tissues of transgenic animals may have profound consequences and implications for interpreting experimental results. In other cases the extra genes may not be expressed at functionally relevant levels or perhaps there may be no measurable impact despite high level expression (Heiman et al., 2008). In support of the former, recent experimental evidence has emerged claiming significant behavioral and functional alterations in several commonly used BAC transgenic mouse lines (Ade et al., 2011; Bagetta et al., 2011; Kramer et al., 2011; Kolisnyk et al., 2013), thereby raising concerns about some aspects of the findings from the numerous studies that have implemented these lines to date. Although some aspects of the reported defects have been disputed or reconciled (Chan et al., 2012; Nelson et al., 2012), this controversy highlights the need for more rigorous systematic characterization of BAC transgenic lines. In the study by Kolisnyk et al. (2013) it was clearly demonstrated that the measured behavioral and physiological alterations were the direct result of high BAC copy number insertion and unintended overexpression of VAChT in *Chat-ChR2(H134R)-EYFP* line 6 BAC transgenic animals (Kolisnyk et al., 2013). Given this finding, further detailed investigations are likely to reveal significant impacts of overexpressed extra genes in many other widely used BAC transgenic lines. This point underscores the potential benefits of the improved BAC recombineering strategies we have developed for removing or inactivating extra genes for the purpose of creating transgenic tools with fewer potential confounds.

Beyond the limited examples we have cited above, there are undoubtedly many commonly used BAC transgenic lines for which behavioral or physiological confounds are likely but have yet to be directly assessed. As an example, several BAC transgenic lines have been reported for transgene expression in cholinergic neurons using identical or highly similar *Chat*-spanning BAC clones as the one used in developing the *Chat-ChR2(H134R)- EYFP* line 6 mice. These include a *Chat-*Cre driver line (Gong et al., 2007), two distinct *Chat-EGFP* reporter lines (Gong et al., 2003; Tallini et al., 2006), and a *Chat-L10a-EGFP* BacTRAP line (Doyle et al., 2008). Furthermore, a *Chat-*Cre BAC transgenic rat was developed by pronuclear injection of the identical *Chat-*Cre mouse BAC clone used to create the *Chat-*Cre mouse line (Witten et al., 2011). Thus, although no direct assessment of VAChT overexpression has been reported for these five independent lines, it is expected that all will have some degree of overexpression of VAChT (and potentially *Ogdhl* and *1700024G13Rik*) depending on the BAC transgene copy number for each line. However, it is worth noting that high BAC copy number insertions as reported for the *ChAT-ChR2(H134R)-EYFP* line 6 tend to be far less common as opposed to low BAC copy number insertions.

In the long-term, we propose that the most effective strategy will be to re-engineer and develop new BAC transgenic lines using improved BAC recombineering strategies to eliminate nonessential portions of BAC DNA, including (but not limited to), methods described in this study, and to deposit validated mini BACs into an open source repository for widespread use. The versatile new resources we have developed (e.g., BAC modification vectors and successfully modified BAC clones) and the simple and efficient methods we have demonstrated should facilitate progress in this area by providing further options to support diverse recombineering needs. Furthermore, with advances in pronuclear injection-based targeted transgenesis (Ohtsuka et al., 2010; Tasic et al., 2011) and identification of "safe" genomic docking loci permissible for high level expression, it may soon be possible to develop BAC transgenic animals that are free of both confounding extra genes and detrimental positional effects from random transgene integration. Continued technological innovation will help to ensure an ever-expanding collection of highly useful BAC transgenic tools that can pave the way to deciphering the functions of the remarkably diverse cell types in the brain.

## **ACKNOWLEDGMENTS**

We wish to acknowledge Dr. Bernd Gloss for providing the iTV1 plasmid used for constructing BAC targeting vectors and for discussions on the BAC recombineering procedures described in this work. We thank Dr. Karl Deisseroth for providing the original *ChR2(E123T/H134R)-EYFP* DNA construct. The EL250 strain used for BAC recombineering was generously provided by Dr. Neal Copeland. We thank Peimin Qi and the members of the MIT Transgenic Facility for excellent technical support related to pronuclear injection of BAC DNA. We thank Jill R. Crittenden for critical comments on this manuscript. This work was supported in part by a National Alliance for Research on Schizophrenia and Depression: The Brain and Behavior Research Foundation Young Investigator Award to Jonathan T. Ting, and U.S. National Institutes of Health Ruth L. Kirschstein National Research Service Awards to Jonathan T. Ting (F32-MH084460).

## **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 16 January 2014; accepted: 15 March 2014; published online: 03 April 2014. Citation: Ting JT and Feng G (2014) Recombineering strategies for developing next generation BAC transgenic tools for optogenetics and beyond. Front. Behav. Neurosci. 8:111. doi: 10.3389/fnbeh.2014.00111*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience. Copyright © 2014 Ting and Feng. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Optogenetic inhibition of neurons by internal light production

## **Benjamin B. Land, Catherine E. Brayton, Kara E. Furman, Zoe LaPalombara and Ralph J. DiLeone\***

Department of Psychiatry, Yale University School of Medicine, New Haven, CT, USA

#### **Edited by:**

Mary Kay Lobo, University of Maryland School of Medicine, USA

#### **Reviewed by:**

Antoine Adamantidis, McGill University - Douglas Mental Health University Institute, Canada Dennis Ryan Sparta, University of North Carolina at Chapel Hill, USA

#### **\*Correspondence:**

Ralph J. DiLeone, Department of Psychiatry, Yale University School of Medicine, 34 Park Street, New Haven, CT 06519, USA e-mail: ralph.dileone@yale.edu

Optogenetics is an extremely powerful tool for selective neuronal activation/inhibition and dissection of neural circuits. However, a limitation of in vivo optogenetics is that an animal must be tethered to an optical fiber for delivery of light. Here, we describe a new method for in vivo, optogenetic inhibition of neural activity using an internal, animal-generated light source based on firefly luciferase. Two adeno-associated viruses encoding luciferase were tested and both produced concentration-dependent light after administration of the substrate, luciferin. Mice were co-infected with halorhodopsin- and luciferase-expressing viruses in the striatum, and luciferin administration significantly reduced Fos activity compared to control animals infected with halorhodopsin only. Recordings of neuronal activity in behaving animals confirmed that firing was greatly reduced after luciferin administration. Finally, amphetamine-induced locomotor activity was reduced in halorhodopsin/luciferase mice pre-injected with luciferin compared to controls. This demonstrates that virally encoded luciferase is able to generate sufficient light to activate halorhodopsin and suppress neural activity and change behavior. This approach could be used to generate inhibition in response to activation of specific molecular pathways.

**Keywords: optogenetics, dopamine, luciferase, amphetamine, striatum**

#### **INTRODUCTION**

The use of optogenetics has increased dramatically in the last several years, largely due to its applicability to studies spanning from single-cell electrophysiology to whole-animal behavior. *In vivo* techniques are particularly powerful as they allow for control of behavior with manipulation of a single neuron subtype (Witten et al., 2010; Narayanan et al., 2012). However, one restriction to *in vivo* optogenetics is the necessity of an external light source (laser or LED) that must be connected to the animal via indwelling cannula. While optical rotary joints allow for relatively free movement within an open-topped behavioral apparatus, home-cage illumination or any structure with an enclosed top are not feasible. Recently, two groups have introduced wireless optogenetics approaches, where the signal/power is sent to a headstage containing indwelling LEDs (Wentz et al., 2011; Kim et al., 2013). While this solves the problem of tethering, specialized equipment is needed to both send the signal and to receive it, which could add a significant cost to the setup and may require significant expertise to implement.

One potential solution is to eliminate the need for an external light source by creating a system in which the animal produces its own light. This can be accomplished by leveraging light-producing proteins that exist in nature, in a similar fashion to the opsins that are sensitive to light. These proteins can be expressed using viral systems in a way similar to the opsins, to internally generate light. Firefly luciferase is one such protein, and it produces yellow-red light in the presence of its substrate, luciferin (Nakatsu et al., 2006). Mammalian systems readily tolerate luciferase and luciferin, and this system has been used *in vivo* to measure gene activity and track labeled cells (Contag, 2007). Notably, the emission spectrum of firefly luciferase overlaps highly with the action spectrum of halorhodopsin (Zhang et al., 2007), a bacterially derived, amber light photoreceptive chloride pump that inhibits neural activity. This suggests that with sufficient light output, luciferase could serve as a light source that would activate halorhodopsin and inhibit neural activity, eliminating the need for an external light source, and establishing a genetically-encoded light generating system. While this reduces control over timing, it would allow for production of light in response to specific cellular signals. Here, we show that luciferin/luciferase and halorhodopsin are sufficient to reduce neural activity *in vivo* and disrupt basic behaviors, establishing a proof-of-principle for luciferase-based inhibition.

## **MATERIALS AND METHODS**

#### **ANIMALS**

Eighteen male C57Bl/6 mice were used for these studies. Animals were group housed, with the exception of the two mice used for electrophysiology that were single-housed after surgery. Animals were kept on a 12 h light/dark cycle and provided standard chow and water *ad libitum*, and all animal procedures were performed in accordance with the protocol approved by the Institutional Animal Care and Use Committee (IACUC).

## **VIRAL PREPARATION AND SURGERY**

Viral production for the EF1a-luciferase construct was accomplished using a triple-transfection, helper-free method, and purified as described in detail previously (Hommel et al., 2003). To generate the EF1a-luciferase construct, a flox-ChR2 construct (AAV2-DIO-ChR2-eYFP, UNC viral core) was restriction digested outside of the asymmetric loxP sites using NheI and AscI restriction enzymes, and a cassette containing luciferase was inserted. AAV5-hSyn-eNpHR3.0-eYFP (halorhodopsin) and AAV2-CMVluciferase virus were purchased from University of North Carolina viral core.

For surgery, animals were anesthetized with 10% ketamine/ 1% xylazine (10 ml/kg body weight) and placed in a stereotaxic frame (Stoelting). After craniotomy, mice were injected with AAV2-luciferase (0.5 uL, EF1a or CMV) and/or AAV2 halorhodopsin (0.5 uL) into the dorsal striatum (AP: +0.7, ML ± 1.75, DV −3.5 from bregma). For the electrophysiological studies, animals were injected with virus into the ventral striatum (AP: +1.4, ML −0.7, DV −4.5 from bregma) before placement of the electrode. A 16-channel array of microwire electrodes (Tucker-Davis Technologies, TDT) was placed in the same craniotomy. A ground wire attached to the array was placed above the dura through a burr hole in the opposite hemisphere. Arrays were composed of 16 Teflon-coated, 50 µm tungsten wires arranged in an 8 × 2 configuration with each electrode spaced by 250 µm. *In vitro* impedance of the electrodes is 100–300 kohms. Electrodes were affixed to the skull with cyanoacrylate ("Metabond") and methyl methacrylate (dental cement) and two screws placed over the parietal bones. In total, headstages weighed approximately 2.5 g and were well-tolerated by animals.

#### **IN VITRO LIGHT ASSAY**

HT-1080 cells were cultured using standard media, and were infected with either no virus or increasing dilutions of the EF1a or CMV luciferase virus (1:100, 1:500, 1:2500). Twenty four hours after infection, cells were harvested and combined with a luciferase based light detection kit (Stratagene). After adding luciferin, the cells were placed in a luminometer (Berthold) and light was detected, expressed as Relative Light Units per second (RLU/s).

## **IN VIVO ELECTROPHYSIOLOGY**

After approximately 2–3 weeks of recovery to ensure optimal viral expression and neuronal signal, freely moving animals were connected to the recording system while in a behavioral box (Med Associates). After 1 h of habituation, neurons were recorded for a 10 min baseline and then given an IP injection of either saline or luciferin (150 mg/kg). Recordings were continued for 1 h postinjection. Plugging and unplugging from the electrode cables was performed under isofluroane anesthesia to minimize stress on the animals.

Neural ensemble recordings were made using a multichannel acquisition processor (MAP) from TDT. Putative single neural units were identified on-line using an oscilloscope and an audio monitor. The TDT off-line sorter was used to analyze the signals and to remove artifacts due to cable noise. Principal component analysis (PCA) and waveform shape were used for spike sorting (Sears et al., 2010). Single units were identified as having: (1) consistent waveform shape; (2) separable clusters in PCA space; (3) average amplitude estimated at least three times larger than background activity; and (4) a consistent refractory period of at least 2 ms in interspike interval histograms. Those units identified on-line as potential single units that did not meet these criteria off-line were not included in this analysis. Graphical exploratory analysis of neural activity and quantitative analysis of basic firing properties (firing rate, interspike intervals) were analyzed using custom routines for MATLAB. For each well isolated neuron, post-injection firing rates were normalized to mean pre-injection firing rates (in the 10 min immediately preceding drug injection) and binned (60 s bins). Activity was then compared between neurons recorded in luciferin and saline conditions.

## **LOCOMOTOR ACTIVITY**

After 2–3 weeks of recovery after viral surgery, animals were prehabituated for 90 min to locomotor chambers that consisted of a mouse cage-bottom, without bedding, inside of an infrared onedimension beam-break array (Med Associates). On test day, animals were again habituated to the chambers for 90 min. Animals were then briefly removed and injected with luciferin (150 mg/kg) and returned to the chamber. Twenty minutes later animals were injected with amphetamine (2.5 mg/kg) and returned to the chamber for 90 min. Locomotor counts were calculated as the number of consecutive beam-breaks in the chamber, and these counts were made throughout the entire testing period.

## **IMMUNOFLUORESCENCE**

Animals that performed the locomotor behavioral tests were allowed to recover for at least 2 weeks before sacrifice. On the day of sacrifice, animals were injected with luciferin (150 mg/kg), followed 20 min later by injection of pentylenetetrazol (PTZ, 45 mg/kg) to induce seizure activity. Animals were observed for signs of seizure (as confirmation of a general increase in neural activity) and 90 min after PTZ injection were deeply anesthetized and intracardially perfused with 4% paraformaldehyde. The brain was removed and post-fixed overnight in paraformaldehyde, and after immersion in sucrose for cryoprotection, 40 µm sections were made on a freezing microtome, and stored in 1 × PBS with 0.01% sodium azide to prevent bacterial growth. Immunohistochemistry was performed according to methods described previously (Sears et al., 2010). Staining for Fos (Rabbit anti-cFos; Santa Cruz; 1:500) or Luciferase (Goat anti-luciferase; Promega; 1:500) with secondary antibodies (Alexa or 555 and 633; Invitrogen/Life Sciences) was performed in 3% normal donkey serum and 0.3% Triton-X 100. Tissue was visualized and images were captured using a fluorescent microscope (Zeiss) using standard FITC and TRITC filters or using a confocal microscope (Olympus). Fos labeling was quantified by standard threshold settings in ImageJ (NIH) over matched areas on one or more (averaged) sections per animal.

# **STATISTICS**

Comparisons were made using one-way and two-way ANOVA, and unpaired, two-tailed *t*-tests where appropriate. Differences in means were considered significant if *p*-values were <0.05, and were calculated using Graphpad Prism 5.0 and 6.0. Error bars represent s.e.m.

## **RESULTS**

Because the emission spectrum of firefly luciferase overlaps considerably with the action spectrum of halorhodopsin (**Figure 1A**), we reasoned that light generation from luciferase should activate these chloride pumps to drive hyperpolarization of membranes. Towards this goal, we designed an adenoassociated viral construct containing the firefly luciferase gene with an EF1a promoter, in addition to testing a virus containing the CMV promoter (UNC viral core, **Figure 1B**). To assess the ability of these constructs to generate light *in vitro*, we infected HT-1080 cells with varying dilutions. After 24 h of infection, cells were assayed for light production after treatment with luciferin. With both viruses, light production was dilution-dependent, with higher concentration of virus producing more light, as expected (**Figure 1C**, CMV, *F*(3,12) = 113.7, *P* < 0.0001; EF1a, *F*(3,19) = 12.01, *P* = 0.0001). The CMV-luciferase generated approximately 6-fold more light (0.78 order of magnitude) than the EF1a-luciferase, likely due

activation spectra for firefly luciferase (green) and halorhodopsin (amber), respectively. Note the high degree of overlap between the two. **(B)** Schematics of the firefly luciferase viral constructs used. **(C)** Light output (in infection at various dilutions of virus in HT-1080 cells (n = 4 per group for CMV, n = 5,6,5,7 for 1:100, 1:500, 1:2500, no virus respectively for EF1a). \* P < 0.05, \*\* P < 0.01.

**FIGURE 2 | Luciferase and halorhodopsin inhibit cFos activity. (A)** Micrographs depicting the dorsal striatum of an animal infected with luciferase (red) and halorhodopsin (green, lower), or halorhodopsin alone (upper). cFos immunofluorescence 90 min after PTZ injection is in blue. **(B)** Quantification of cFos in the dorsal striatum of luciferase/halorhodopsin and halo only animals 90 min after PTZ injection (n = 6 striatal samples from four animals). **(C)** Schematic drawing of the rostro-caudal distribution of luciferase, halorhodopsin, and cFos in one animal. Note that in general, there is very little cFos immunofluorescence where both luciferase and halorhodopsin are expressed. \*\* P < 0.01.

to differences in titer between the viruses. Nonetheless, these results suggest that both constructs are capable of producing light.

To test whether luciferase-induced light was sufficient to inhibit neural activity *in vivo* using biochemical markers, we infected mice in the dorsal striatum with either a combination of the EF1a-luciferase and halorhodopsin, or halorhodopsin alone. After recovery and behavioral testing (described below), animals were pre-treated with luciferin (150 mg/kg) 20 min prior to injection with PTZ, a drug known to induce seizures and cFos expression throughout the striatum. This dose of luciferin was derived from literature showing luciferase activity for 1–2 h after injection (Burgos et al., 2003; Cordeau and Kriz, 2012). Ninety minutes after PTZ injection, animals were sacrificed and processed for cFos immunofluorescence. Control animals injected with halorhodopsin alone showed a robust Fos induction in the dorsal striatum, and this response was reduced when luciferase was co-expressed with halorhodopsin (**Figures 2A, B**, *t*<sup>10</sup> = 5.78, *P* = 0.0002). We mapped the extent of both viral expression and cFos induction (**Figure 2C**), and found a large volume of

components. Right, the two waveforms corresponding to the PCA. **(C)** Top, rastors depicting action potentials for neurons in the saline (gray, n = luciferin-injected condition (n = 6). Bottom, normalized firing rate over the

course of the experiment. \*\* P < 0.01.

viral spread and a striking reduction in cFos in regions with expression of both viruses. These results suggest that luciferase is able to produce light *in vivo* in at sufficient power to activate halorhodopsin and affect neural activity.

To further demonstrate the efficacy of this inhibitory strategy in real time, we implanted mice with 16-channel electrodes for *in vivo* electrophysiology, in addition to the luciferase/halorhodopsin dual infection, into the ventral striatum (**Figure 3A**). This allowed us to directly monitor neural activity as a function of luciferin/luciferase-driven light. Following acclimation to the chamber, animals were given a 10 min baseline period, after which they were injected with either luciferin (150 mg/kg), or saline on alternating, counterbalanced days. Units were sorted by PCA based on waveform (**Figure 3B**), and firing was normalized to the baseline period. Compared to saline injection, luciferin produced a marked decrease in firing that began soon after injection (**Figure 3C**, significantly different from normalized firing = 1 at 9 min, one-sample *t*-test, *P* < 0.01) and lasted at least 60 min (*F*(69,1449) = 3.61, *P* < 0.0001, interaction of time by treatment). Summing total action potentials shows that during the baseline period (−10–0 min) there was no difference in spikes (*t*<sup>21</sup> = 1.47, *P* = 0.16), while there was a dramatic decrease in spiking in the 60 min after luciferin injection compared to saline (**Figure 3D**; *t*<sup>21</sup> = 3.03, *P* = 0.006). Because we did not see a return to baseline, a third animal was implanted as described above and injected with a 10-fold lower dose of luciferin (15 mg/kg). Again, we saw a characteristic decrease in firing, but at this dose the normalized firing returned to baseline at 60 min (**Figure 3E**). This shows that neural activity can be attenuated rapidly and in a sustained fashion using this luciferase-based approach.

Finally, to demonstrate that these decreases in activity have functional behavioral consequences, animals with bilateral, dorsal striatal injections of either luciferase/halorhodopsin or halorhodopsin alone were tested in a locomotor activity paradigm after amphetamine treatment. Animals were habituated to the locomotor boxes (90 min) on the testing day, and were then injected with luciferin (150 mg/kg) 20 min before injection of amphetamine (2.5 mg/kg, **Figure 4A**). While there were no differences in the last 20 min of the habituation period or the 20 min following luciferin injection, animals with luciferase/halorhodopsin expression were unaffected by amphetamine for 30 min after amphetamine injection, compared to controls whose activity increased as expected (**Figure 4B**; *F*(5,50) = 2.83, *P* = 0.025). After 30 min, the experimental animals' activity increased slightly and matched control activity for the remainder of the test period. These findings corroborate the neural activity data, and show that neural inhibition using this approach can have profound effects on behavior.

## **DISCUSSION**

In the present study, we have demonstrated that a combination of luciferase and halorhodopsin viruses can be used together to suppress neural activity without an external light source. This hybrid, chemical-optogenetics approach requires administration of luciferin, but is otherwise noninvasive and could be easily adapted to any testing apparatus or environment. Potentially, luciferin could be delivered via minipump for chronic exposure.

**FIGURE 4 | Amphetamine induced locomotor activity is attenuated with luciferase-based illumination. (A)** Timeline of the behavioral test. **(B)** Locomotor activity during the last 20 min of baseline, 20 min of luciferin alone, and 90 min after amphetamine (n = 6 per group). \* P < 0.05 for 30 min following amphetamine injection, two-way ANOVA, interaction of time and treatment.

Our initial studies have shown that a single injection of luciferin can inhibit neurons for at least 1 h *in vivo*, and this correlates strongly with our behavioral measures. Further, this approach is adaptable to cell-type specific targeting using Cre-lox systems with halorhodopsin.

In principle, this technique resembles the DREADD (Designer Receptors Exclusively Activated by Designer Drugs) system for inactivating neurons (Armbruster et al., 2007), but there are at least two important differences. First, because this uses a combination of chemogenetics and optogenetics, one could also use traditional optogenetic illumination in these animals if needed. This makes the present system more adaptable to current optogenetic approaches and allows a single set of animals to be used with both external and internal light delivery. Also, because the halorhodopsin is a chloride pump, its inhibitory effects are rapid and direct with behavioral effects lasting <1 h. This is in contrast to the DREADD system, which uses G-protein coupled receptors whose behavioral effects last ∼8 h after agonist (clozapine N-oxide) injection (Alexander et al., 2009; Carter et al., 2013). While the onset time for DREADD stimulation can also be rapid (e.g., Sasaki et al., 2011; Garner et al., 2012), the present data suggests that the luciferase system may provide a shorter time window of effect.

The use of a genetically encoded luciferase also allows potential application of this system as part of an inhibitory feedback loop in response to activation of a specific signaling pathway. Luciferase could be put under direct transcriptional regulation of a known pathway, or potentially modified to allow for post-translational activation, allowing for dynamic response that reflects activation of a signaling cascade. Although this approach would still require injection of luciferin, it could be useful for functional identification of pathways associated with addiction, or other diseases where aberrant neuronal activation contributes to disease pathology.

In sum, this endogenous light-production technique adds to the expanding toolkit that can be applied to optogenetic control of behavior. Uniquely, this approach allows neuronal inhibition without tethering or specialized equipment, expanding the breadth of behavioral experiments that can be performed.

## **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 09 January 2014; accepted: 13 March 2014; published online: 01 April 2014. Citation: Land BB, Brayton CE, Furman KE, LaPalombara Z and DiLeone RJ (2014) Optogenetic inhibition of neurons by internal light production. Front. Behav. Neurosci. 8:108. doi: 10.3389/fnbeh.2014.00108*

*This article was submitted to the journal Frontiers in Behavioral Neuroscience*. *Copyright © 2014 Land, Brayton, Furman, LaPalombara and DiLeone. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

# ADVANTAGES OF PUBLISHING IN FRONTIERS

FAST PUBLICATION Average 90 days from submission to publication

COLLABORATIVE PEER-REVIEW

Designed to be rigorous – yet also collaborative, fair and constructive

RESEARCH NETWORK Our network increases readership for your article

## OPEN ACCESS

Articles are free to read, for greatest visibility

## TRANSPARENT

Editors and reviewers acknowledged by name on published articles

GLOBAL SPREAD Six million monthly page views worldwide

## COPYRIGHT TO AUTHORS

No limit to article distribution and re-use

IMPACT METRICS Advanced metrics track your article's impact

SUPPORT By our Swiss-based editorial team

EPFL Innovation Park · Building I · 1015 Lausanne · Switzerland T +41 21 510 17 00 · info@frontiersin.org · frontiersin.org