Mechanistic Resolution Required to Mediate Operant Learned Behaviors: Insights from Neuronal Ensemble-Specific Inactivation
Brandon L. Warren
Nobuyoshi Suto
Bruce T. Hope- 1Behavioral Neuroscience Branch, Intramural Research Program (IRP), National Institute on Drug Abuse (NIDA), National Institutes of Health (NIH), Department of Health and Human Services (DHHS), Baltimore, MD, USA
- 2Department of Molecular and Cellular Neuroscience, The Scripps Research Institute, La Jolla, CA, USA
Many learned behaviors are directed by complex sets of highly specific stimuli or cues. The neural mechanisms mediating learned associations in these behaviors must be capable of storing complex cue information and distinguishing among different learned associations—we call this general concept “mechanistic resolution”. For many years, our understanding of the circuitry of these learned behaviors has been based primarily on inactivation of specific cell types or whole brain areas regardless of which neurons were activated during the cue-specific behaviors. However, activation of all cells or specific cell types in a brain area do not have enough mechanistic resolution to encode or distinguish high-resolution learned associations in these behaviors. Instead, these learned associations are likely encoded within specific patterns of sparsely distributed neurons called neuronal ensembles that are selectively activated by the cues. This review article focuses on studies of neuronal ensembles in operant learned responding to obtain food or drug rewards. These studies suggest that the circuitry of operant learned behaviors may need to be re-examined using ensemble-specific manipulations that have the requisite level of mechanistic resolution.
Introduction
Learned associations between reinforcers and predictive stimuli are critical for survival. Unfortunately, these learned associations are also implicated in addictive behaviors. Indeed, part of what makes addictive behaviors so intractable is that craving and relapse are often triggered by exposure to closely associated stimuli (Wikler, 1973; Goldberg, 1975; O’Brien et al., 1984; Stewart et al., 1984; Siegel, 1999). Likewise, exposure to predictive stimuli causes reinstatement of both drug and natural reward seeking in animal models (Davis and Smith, 1976; de Wit and Stewart, 1981; Weingarten, 1983; Petrovich et al., 2007; Yager and Robinson, 2010). Furthermore, the predictive stimuli that trigger relapse can be very specific (Powell et al., 1990). Thus, memories underlying this type of high-resolution behavior must be represented within the brain with equally high-fidelity mechanistic resolution. Mechanistic resolution describes the physical properties of the brain that underlie its resolving power. The brain’s ability to encode and distinguish among different sets of highly specific stimuli necessitates high mechanistic resolution. Because the brain can convert compound cues and contexts into highly specific memories, this conversion process must be able to translate highly specific information into physiological changes within the brain.
Currently, the majority of studies attempting to investigate the neurobiology of memory assess “global” alterations and their functions within brain regions, neurons of a specific cell-type, or afferent/efferent connections between two brain areas, regardless of the specific neural activity patterns during behavior (Nestler et al., 1993; Kalivas et al., 2005; Hyman et al., 2006; Koob, 2006; Bowers et al., 2010; Jennings et al., 2013; Otis et al., 2017). These experiments have traditionally relied upon pharmacological inactivation of discrete regions or receptor subtypes. More recently, optogenetic and chemogenetic approaches have allowed for more precise targeting of brain regions, circuits, or cell types and have given us greater insight into their role in memory than ever before. However, the view that global alterations within a given brain area can represent each unique learned association between a distinct environmental cue and a specific behavioral response is incompatible with the characteristics of mechanistic resolution. An alteration that effects all cells of a given type or within a given brain area or circuit can be thought of as a binary change. In this case, each cell would be affected similarly, which means there are only two possible states (on or off). Thus, the mechanistic resolution of a global alteration is limited to two behavioral states, making it an unlikely mechanism for storing and distinguishing among specific memories. Instead, high-resolution information is thought to be encoded within specific patterns of neurons, called neuronal ensembles, that are selectively activated in response to specific cues and reinforcers during behavior.
Fos-Expressing Neuronal Ensembles are Capable of High Mechanistic Resolution
Immediate early gene (IEG) expression is an indirect marker used to assess neuronal activity (Cruz et al., 2015). The most commonly used IEG, Fos, can be detected within 10–30 min of strong neuronal activity. The Fos promoter is activated when strong and persistent calcium influx coincides with high levels of excitatory input, leading to expression of Fos mRNA and protein (Morgan and Curran, 1991; Deisseroth et al., 2003; Cohen and Greenberg, 2008; Brami-Cherrier et al., 2009; Cahill et al., 2014; Kawashima et al., 2014); the extensive literature is described in more detail in Cruz et al. (2015). It is thought that the neurons receiving highest glutamatergic input are selected by the cues and contexts present during self-administration (Cruz et al., 2015). In aggregate, these activated neurons make up the group of neurons that form a neuronal ensemble. Fos-expressing neuronal ensembles meet the requirements for mechanistic resolution. Specifically, Fos-expressing neuronal ensembles are capable of representing highly specific associations based on patterns of activation and connectivity. Studies indicate about 5% or less of neurons in a histochemical section are Fos-expressing following most operant learned behaviors (Bossert et al., 2011; Fanous et al., 2013; Warren et al., 2016; Caprioli et al., 2017). These percentages of Fos-expressing neurons, quantified in two-dimensional histochemical slices, translate to less than 1% of neurons in three-dimensional brain volumes. Since less than 1% of neurons in a brain volume are activated enough to express Fos, there is an immense number of possible patterns for Fos-expressing neuronal ensembles (see Figure 1). Furthermore, each learned association is thought to be represented by a specific pattern of Fos-expressing neurons in each brain region. The immense number of possible configurations (or neuronal ensembles) allows for many distinct associations to be made and stored within a single brain region, which provides the high degree of mechanistic resolution required to encode learned associations among complex sets of high resolution cues, contexts and rewards in operant learned behaviors.
Figure 1. Formula for calculating number of possible combinations of activated neurons. A hypothetical brain region containing 300 neurons is shown on the left, if 1% of the neurons are active, 4,455,100 possible combinations are possible. With more neurons available in real brain areas, the number of possible combinations is much greater.
Work using single cell electrophysiology appears to support this Fos-expressing neuronal ensemble hypothesis. Within the nucleus accumbens, single unit recordings during operant responding have demonstrated that different rewards cause firing in different neurons (Carelli and Deadwyler, 1994; Chang et al., 1998; Carelli, 2002a; Carelli and Wondolowski, 2003; Deadwyler et al., 2004; Opris et al., 2009; Cameron and Carelli, 2012). Neurons activated by one reinforcer (cocaine) are often not activated by a second reinforcer (food; Carelli and Ijames, 2001; Carelli, 2002b). This suggests that the two learned associations activate largely different populations of neurons, likely different neuronal ensembles. However, evidence that distinct Fos-expressing neuronal ensembles co-mingle within the same brain area and play causal roles in distinct learned associations has been lacking.
Fos-Expressing Neuronal Ensembles Mediate Operant Learned Behaviors
Current methods for demonstrating whether a brain region or circuit is necessary for behavior rely on technologies that inactivate either a whole brain region, or specific phenotypes of neurons (e.g., glutamatergic vs. GABAergic projecting neurons), without accounting for differences in neuronal activity. While these technologies have provided a strong framework for our understanding of the neuroanatomy of behavior, they are not congruent with the processing, storage, or discrimination of high-resolution information in learned behaviors (see Figure 2). Several technologies have recently emerged that enable neuronal ensembles to be labeled and manipulated in vivo; these technologies have previously been described in Mayford et al. (1996); Cruz et al. (2013); Kawashima et al. (2013, 2014); Ramirez et al. (2013); Liu et al. (2014); and Sørensen et al. (2016). This review, however, focuses on the biological question of how Fos-expressing neuronal ensembles can mediate operant learned responding for food and drug rewards with high mechanistic resolution. We provide evidence that two of these ensembles can intermingle in the same brain area to encode separate learned associations that drive behavior in opposing directions. We also describe how low resolution global manipulations of neural activity (excitation or inhibition) cannot distinguish these ensembles, and even mask the multiple roles of a brain area in the behavior of interest.
Figure 2. Comparing global to ensemble-specific inactivation. The left column shows hypothetical neuronal ensembles. The middle column illustrates that global inactivation inactivates all neurons (both ensemble A and B). The right column shows ensemble-specific inactivation of ensemble A, leaving ensemble B unaffected.
Inactivating Fos-Expressing Ensembles using the Daun02 Chemogenetic Procedure
The earliest technology used to demonstrate a causal role for behaviorally activated Fos-expressing neuronal ensembles in a learned behavior was the Daun02 inactivation procedure using Fos-LacZ transgenic rats (Bossert et al., 2011; Cruz et al., 2013, 2014, 2015; Koya et al., 2016). Indeed, the Daun02 inactivation procedure has been used in the vast majority of studies that examined Fos-expressing ensembles in operant learned behaviors. In Fos-lacZ rats, activation of the Fos promoter leads to expression of the LacZ coding sequence, resulting in translation of Fos as well as the enzyme β-galactosidase (βgal) only in strongly activated neurons. Following βgal induction, the inactive prodrug Daun02 is infused into the brain region of interest. Daun02 is catalyzed by βgal into daunorubicin, a toxic compound that leads to early inactivation of calcium-dependent action potentials (Engeln et al., 2016), followed by apoptosis and cell death only in the strongly activated βgal-expressing neurons (Pfarr et al., 2015). After a couple of days, the behavioral effects of the targeted neural inactivation by Daun02 can be assessed under a drug free condition (Bossert et al., 2011; Fanous et al., 2012; Koya et al., 2016) to demonstrate causal roles for neuronal ensembles within multiple nodes of the corticostriatal circuit, including prefrontal cortex, striatum and amygdala (Bossert et al., 2011; Cruz et al., 2013, 2014, 2015; Pfarr et al., 2015; de Guglielmo et al., 2016; Funk et al., 2016; Koya et al., 2016; Suto et al., 2016; Caprioli et al., 2017).
In 2009, our group published a series of experiments designed to assess a role for neuronal ensembles in the environmental control of sensitized locomotor responding for cocaine (Koya et al., 2009). We administered cocaine to rats in a specific environmental context in daily sessions for 7 days. In a subsequent test, rats are given a threshold dose of cocaine and placed in either a novel context, or into the context that was previously paired with cocaine. Only rats placed in the context previously paired with cocaine showed hyperlocomotion (Koya et al., 2009). This difference represents a learned association between the effects of cocaine and the environmental context. Rats that were re-exposed to the cocaine-paired context also have increased Fos-expression within the nucleus accumbens. We then used the Daun02 inactivation procedure to test whether these Fos-expressing neuronal ensembles played a causal role in maintaining the learned association between cocaine’s effects and the context. Inactivation of neuronal ensembles associated with the cocaine administration context significantly blunted sensitized cocaine-induced locomotion. This finding suggested that Fos-expressing neuronal ensembles do play a causal role in cocaine sensitization, despite representing less than 1% of neurons. Unlike ensemble-specific lesions of behaviorally activated Fos-expressing neurons, lesions of the nucleus accumbens brain region paradoxically enhanced locomotor sensitization for both nicotine and cocaine (Kelsey and Willmore, 2006).
Subsequent studies also indicate very different effects on operant learned behavior when using global inactivation methods vs. ensemble-specific inactivation methods. Bossert et al. (2011) demonstrated that Fos-expressing neuronal ensembles mediate context-induced reinstatement of heroin seeking. Rats were first trained to lever press for heroin infusions in one context (context A), then the behavior was extinguished in a second context (context B). Under normal conditions, re-exposure to context A induces reinstatement of heroin seeking. The authors show that re-exposure to context A induced robust Fos-expression within the vmPFC. Daun02 inactivation of neurons associated with exposure to the heroin context significantly reduced context-induced reinstatement of heroin seeking (Bossert et al., 2011). This strongly suggests that neuronal ensembles within the vmPFC mediate learned associations about the heroin-context. An ancillary finding was that a separate group of rats, exposed to the extinction context prior to Daun02 administration had somewhat higher responding in a subsequent test of extinction recall. Although this increase was not significant (likely due to limited numbers of rats in each group), it suggested that the infralimbic cortex might also be involved in extinction. This is particularly interesting because global inactivation of the infralimbic cortex paradoxically increased cocaine seeking behavior in extinguished rats (Peters et al., 2008), suggesting that this brain area is involved in extinction (although others have found contradictory findings in cue-induced reinstatement of heroin seeking (Rogers et al., 2008), and methamphetamine seeking (Rocha and Kalivas, 2010)). Nevertheless, these results provided the first hint that two distinct memories (self-administration and extinction) could be represented separately in the vmPFC.
To more directly test this hypothesis, we used self-administration of palatable food pellets to show that self-administration and extinction memories are represented separately within the vmPFC (Warren et al., 2016). We first trained rats to self-administer palatable food pellets in daily sessions. Once reliable responding was achieved, we exposed rats to three different extinction conditions (0, 2, or 7 daily extinction sessions). Fos expression in the vmPFC was highest following 2 days of extinction, suggesting a period of enhanced neuronal ensemble formation, likely corresponding to formation of the extinction memory. We hypothesized that rats undergoing 0 days of extinction would retain the intact self-administration memory, while rats undergoing 2 days of extinction would have formed an extinction memory in addition to the self-administration memory. Global inhibition of the vmPFC did not influence responding for palatable food pellets in either group. This contrasts with earlier work that used global inactivation of vmPFC to inhibit context-induced reinstatement for food-seeking and induce reinstatement of food-seeking in the extinction context (Eddy et al., 2016). Despite the finding that global inactivation of the vmPFC had no effect on behavior, we found that selective inactivation of neuronal ensembles associated with self-administration impaired self-administration recall. Conversely, inactivation of neuronal ensembles associated with extinction impaired extinction recall. This suggests that both self-administration and extinction ensembles intermingle within the vmPFC. Perhaps more astonishing was the finding that region-wide inactivation did not influence behavior, while ensemble-specific inactivation with the Daun02 procedure did. This disparity underscores a fundamental flaw present in global inactivation studies. In this experiment, region-wide inactivation simply did not have the technical resolution required to disentangle these competing ensembles. This may also be one reason that earlier experiments using region-wide manipulations did not see differences in food-seeking. The key is that the difference is in which cells are inactivated, rather than how many or what kind of cells are inactivated.
The distinction between which cells and how many cells influence behavior was recently tested using a new line of transgenic rats (Pfarr et al., 2015). The pCAG-LacZ line of transgenic rats express βgal ubiquitously under the control of the pCAG promoter. Thus, by comparing results of Daun02 inactivation in Fos-LacZ (activity dependent inactivation) and pCAG-LacZ (non-selective inactivation) transgenic rats it is possible to directly compare the effect of region-wide inactivation with selective activity-dependent inactivation. The first study using this methodology exposed a key difference between non-selective and selective activity-dependent inactivation of the vmPFC in cue-induced reinstatement of alcohol seeking (Pfarr et al., 2015). Inactivation of neuronal ensembles associated with alcohol-seeking using the Daun02 procedure in Fos-LacZ rats resulted in a profound increase in cue-induced alcohol seeking. These results appear to demonstrate that neuronal ensembles associated with alcohol cues inhibit alcohol seeking. While this study seems to contradict findings from our lab, it is likely that several methodological and pharmacological differences might explain this discrepancy. While resolving these issues is likely outside the scope of this review, there is one clear take-away that unites these studies. Importantly, the authors found that inactivation of the entire vmPFC in pCAG-LacZ rats had no effect on cue-induced alcohol seeking. Thus, activity-dependent inactivation of a small number of neurons had a significant effect on behavior, while global inactivation had no effect.
Perhaps the most compelling evidence for the importance of selectively inactivating which rather than how many or what kind of cells comes from another study (Suto et al., 2016). In this study, the rats were first operant conditioned to self-administer a sweet solution (containing both glucose and saccharine). Lever insertion signaled onset of each self-administration session and each lever press led to cue-light illumination. The rats were then trained to recognize two distinct auditory cues (white noise and beeping tone) as the discriminative stimulus predictive of reward availability (S+) or reward omission (S−). Subsequent tests revealed that S+ and S−, respectively, potentiated and suppressed basal reward seeking induced by lever and cue-light.
Daun02 was then used to disrupt infralimbic cortical neurons selectively reactive to S+ or S−. Following recovery, rats were tested again for the bidirectional modulation of reward seeking by S+ and S−. Inactivation of S+ associated neurons blocked the promotion of reward seeking by S+, but spared the suppression by S−. In contrast, Daun02 lesion of S− associated neurons blocked the suppression of reward seeking by S− but spared the promotion by S+ (Suto et al., 2016). These findings provide the causal evidence for the concurrent existence of two distinct ensembles, each mediating opposing environmental actions on appetitive behavior, in the same vmPFC brain area. Like other studies reviewed here, these findings raise a caution to the use of non-selective techniques that manipulate neural activity irrespective of intrinsic neural activity for determining the brain-behavioral functions (Suto et al., 2016). Overall, these Daun02 inactivation studies indicate different effects on learned behaviors when using a high-resolution ensemble-specific manipulation as compared with low-resolution manipulations of whole brain areas or cell types.
Inactivating Fos-Expressing Ensembles using Fos-tTA Transgenic Mice
As mentioned before, several other technologies have been developed to manipulate behaviorally activated neuronal ensembles in learned behaviors. These techniques have been used primarily to examine Pavlovian conditioned behaviors such as cue and/or context-induced fear conditioning (Mayford et al., 1996; Cruz et al., 2013; Kawashima et al., 2013, 2014; Ramirez et al., 2013; Liu et al., 2014; Sørensen et al., 2016). One of the most powerful approaches for identifying and manipulating these neuronal ensembles utilizes transgenic Fos-tTA (tetracycline(tet)-off transcriptional activator) mice from the Mayford lab (Reijmers et al., 2007; Wiltgen et al., 2007), and has been used to assess Fos-expressing neuronal ensembles in a number of learned behaviors, particularly fear conditioning (Reijmers et al., 2007; Garner et al., 2012; Liu et al., 2012, 2014; Ramirez et al., 2013; Redondo et al., 2014; Tonegawa et al., 2015; Yokose et al., 2017). Fos-tTA mice express tTA protein under the control of the Fos promoter so that only activated neurons express tTA protein. A second transgene with a tetracycline-responsive element promoter (TRE) allows tTA to drive expression of the second transgene only in activated neurons. The second transgene can be a fluorescent marker to identify activated neurons, a light-sensitive rhodopsin in optogenetic studies, or a DREADD receptor for chemogenetic studies. Importantly, the expression of the second transgene can be repressed by introducing doxycycline to the mouse’s diet, since doxycycline binds and represses tTA. Therefore, expression of the second transgene can be limited to a particular time window by maintaining doxycycline in the mouse’s diet, but removing doxycycline from the mouse’s diet prior to neuronal activation.
Fos-tTA mice have also been used to demonstrate that one can manipulate different Fos-expressing ensembles in the same brain area and induce opposing effects on behavior (Redondo et al., 2014). Different intermingling neuronal ensembles in the basolateral amygdala were activated during aversive or rewarding experience in conditioned place avoidance and preference assays. In these experiments, optogenetic activation of basolateral amygdala ensembles encoding the aversive or rewarding experience could be linked to previously neutral contextual stimuli stored in the hippocampus to induce conditioned place avoidance or preference, respectively. Separate intermingling neuronal ensembles in the basolateral amygdala have also been shown to encode both aversive and rewarding information and drive opposing effects on behavior (Kim et al., 2016). Overall, in animals that encode both aversive and rewarding unconditioned stimuli or learned associations in the same basolateral amygdala brain area, global inactivation (or activation) of both ensembles would have unpredictable net effects when these ensembles control the same behavior.
Virus-Based Methods for Manipulating Neuronal Ensembles In Vivo
Viral based methods for manipulating neurons based on activity level represent another promising approach for identifying and manipulating neuronal ensembles. Several approaches have been proposed, although most have been limited to labeling activated neurons (Kawashima et al., 2014). Several of these methods now offer the ability to manipulate ensembles (Gore et al., 2015; Obenhaus et al., 2016; Sørensen et al., 2016). In particular, Gore et al. (2015) used a novel lentiviral system that employed a Fos promoter to drive expression of channelrhodopsin in Fos-expressing ensembles within the same basolateral amygdala brain area that encoded either aversive or rewarding unconditioned stimuli. Photoactivation of these two ensembles in the presence of previously neutral stimuli induced learned associations that induced opposing effects on behavior in response to the same cue, depending on which basolateral amygdala ensemble was activated during learning. Again, in animals that encode both aversive and rewarding unconditioned stimuli or learned associations in the same basolateral amygdala brain area, global inactivation (or activation) of both ensembles would have unpredictable net effects when these ensembles control the same behavior.
Future Perspectives
Taken together, these findings raise questions about the utility of studies involving global manipulations—that alter (excite or inhibit) the activity of specific brain areas or circuits regardless of intrinsic neural activity—in directly mediating high-resolution operant learned behaviors. As shown, the low resolution of global manipulations can obscure a brain region’s role in behavior. Since environmental information and memory storage in the brain are more likely mediated by high-resolution mechanisms such as neuronal ensembles rather than by global alterations in whole brain areas, the ensemble-specific manipulations are much more congruent with how the brain processes, stores and distinguishes among high-resolution information in learned behaviors. Indeed, data is emerging that indicate selective molecular and cellular alterations in Fos-expressing neuronal ensembles in a variety of systems and behaviors. Pavlovian conditioned behaviors also appear to share a requirement for high mechanistic resolution mediated by Fos-expressing neuronal ensembles. As the technical resolution of our manipulations continues to advance, it is likely that we will gain an even more clear perspective on the role of neuronal ensembles in a variety of behaviors. In this way, technical resolution can more closely match the mechanistic resolution of the behavioral process being studied.
Author Contributions
This is a mini review article. All authors researched and wrote the manuscript.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
This research was supported by the National Institute on Drug Abuse, Intramural Research Program, NIH.
References
Bossert, J. M., Stern, A. L., Theberge, F. R., Cifani, C., Koya, E., Hope, B. T., et al. (2011). Ventral medial prefrontal cortex neuronal ensembles mediate context-induced relapse to heroin. Nat. Neurosci. 14, 420–422. doi: 10.1038/nn.2758
Bowers, M. S., Chen, B. T., and Bonci, A. (2010). AMPA receptor synaptic plasticity induced by psychostimulants: the past, present, and therapeutic future. Neuron 67, 11–24. doi: 10.1016/j.neuron.2010.06.004
Brami-Cherrier, K., Roze, E., Girault, J. A., Betuing, S., and Caboche, J. (2009). Role of the ERK/MSK1 signalling pathway in chromatin remodelling and brain responses to drugs of abuse. J. Neurochem. 108, 1323–1335. doi: 10.1111/j.1471-4159.2009.05879.x
Cahill, E., Salery, M., Vanhoutte, P., and Caboche, J. (2014). Convergence of dopamine and glutamate signaling onto striatal ERK activation in response to drugs of abuse. Front. Pharmacol. 4:172. doi: 10.3389/fphar.2013.00172
Cameron, C. M., and Carelli, R. M. (2012). Cocaine abstinence alters nucleus accumbens firing dynamics during goal-directed behaviors for cocaine and sucrose. Eur. J. Neurosci. 35, 940–951. doi: 10.1111/j.1460-9568.2012.08024.x
Caprioli, D., Venniro, M., Zhang, M., Bossert, J. M., Warren, B. L., Hope, B. T., et al. (2017). Role of dorsomedial striatum neuronal ensembles in incubation of methamphetamine craving after voluntary abstinence. J. Neurosci. 37, 1014–1027. doi: 10.1523/JNEUROSCI.3091-16.2017
Carelli, R. M. (2002a). The nucleus accumbens and reward: neurophysiological investigations in behaving animals. Behav. Cogn. Neurosci. Rev. 1, 281–296. doi: 10.1177/1534582302238338
Carelli, R. M. (2002b). Nucleus accumbens cell firing during goal-directed behaviors for cocaine vs. ‘natural’ reinforcement. Physiol. Behav. 76, 379–387. doi: 10.1016/s0031-9384(02)00760-6
Carelli, R. M., and Deadwyler, S. A. (1994). A comparison of nucleus accumbens neuronal firing patterns during cocaine self-administration and water reinforcement in rats. J. Neurosci. 14, 7735–7746.
Carelli, R. M., and Ijames, S. G. (2001). Selective activation of accumbens neurons by cocaine-associated stimuli during a water/cocaine multiple schedule. Brain Res. 907, 156–161. doi: 10.1016/s0006-8993(01)02604-x
Carelli, R. M., and Wondolowski, J. (2003). Selective encoding of cocaine versus natural rewards by nucleus accumbens neurons is not related to chronic drug exposure. J. Neurosci. 23, 11214–11223.
Chang, J. Y., Janak, P. H., and Woodward, D. J. (1998). Comparison of mesocorticolimbic neuronal responses during cocaine and heroin self-administration in freely moving rats. J. Neurosci. 18, 3098–3115.
Cohen, S., and Greenberg, M. E. (2008). Communication between the synapse and the nucleus in neuronal development, plasticity, and disease. Annu. Rev. Cell Dev. Biol. 24, 183–209. doi: 10.1146/annurev.cellbio.24.110707.175235
Cruz, F. C., Babin, K. R., Leao, R. M., Goldart, E. M., Bossert, J. M., Shaham, Y., et al. (2014). Role of nucleus accumbens shell neuronal ensembles in context-induced reinstatement of cocaine-seeking. J. Neurosci. 34, 7437–7446. doi: 10.1523/jneurosci.0238-14.2014
Cruz, F. C., Javier Rubio, F., and Hope, B. T. (2015). Using c-fos to study neuronal ensembles in corticostriatal circuitry of addiction. Brain Res. 1628, 157–173. doi: 10.1016/j.brainres.2014.11.005
Cruz, F. C., Koya, E., Guez-Barber, D. H., Bossert, J. M., Lupica, C. R., Shaham, Y., et al. (2013). New technologies for examining the role of neuronal ensembles in drug addiction and fear. Nat. Rev. Neurosci. 14, 743–754. doi: 10.1038/nrn3597
Davis, W. M., and Smith, S. G. (1976). Role of conditioned reinforcers in the initiation, maintenance and extinction of drug-seeking behavior. Pavlov. J. Biol. Sci. 11, 222–236.
de Guglielmo, G., Crawford, E., Kim, S., Vendruscolo, L. F., Hope, B. T., Brennan, M., et al. (2016). Recruitment of a neuronal ensemble in the central nucleus of the amygdala is required for alcohol dependence. J. Neurosci. 36, 9446–9453. doi: 10.1523/JNEUROSCI.1395-16.2016
Deadwyler, S. A., Hayashizaki, S., Cheer, J., and Hampson, R. E. (2004). Reward, memory and substance abuse: functional neuronal circuits in the nucleus accumbens. Neurosci. Biobehav. Rev. 27, 703–711. doi: 10.1016/j.neubiorev.2003.11.011
Deisseroth, K., Mermelstein, P. G., Xia, H., and Tsien, R. W. (2003). Signaling from synapse to nucleus: the logic behind the mechanisms. Curr. Opin. Neurobiol. 13, 354–365. doi: 10.1016/s0959-4388(03)00076-x
de Wit, H., and Stewart, J. (1981). Reinstatement of cocaine-reinforced responding in the rat. Psychopharmacology 75, 134–143. doi: 10.1007/bf00432175
Eddy, M. C., Todd, T. P., Bouton, M. E., and Green, J. T. (2016). Medial prefrontal cortex involvement in the expression of extinction and ABA renewal of instrumental behavior for a food reinforcer. Neurobiol. Learn. Mem. 128, 33–39. doi: 10.1016/j.nlm.2015.12.003
Engeln, M., Bastide, M. F., Toulmé, E., Dehay, B., Bourdenx, M., Doudnikoff, E., et al. (2016). Selective inactivation of striatal FosB/ΔFosB-expressing neurons alleviates L-dopa-induced dyskinesia. Biol. Psychiatry 79, 354–361. doi: 10.1016/j.biopsych.2014.07.007
Fanous, S., Goldart, E. M., Theberge, F. R., Bossert, J. M., Shaham, Y., and Hope, B. T. (2012). Role of orbitofrontal cortex neuronal ensembles in the expression of incubation of heroin craving. J. Neurosci. 32, 11600–11609. doi: 10.1523/JNEUROSCI.1914-12.2012
Fanous, S., Guez-Barber, D. H., Goldart, E. M., Schrama, R., Theberge, F. R., Shaham, Y., et al. (2013). Unique gene alterations are induced in FACS-purified Fos-positive neurons activated during cue-induced relapse to heroin seeking. J. Neurochem. 124, 100–108. doi: 10.1111/jnc.12074
Funk, D., Coen, K., Tamadon, S., Hope, B. T., Shaham, Y., and Le, A. D. (2016). Role of central amygdala neuronal ensembles in incubation of nicotine craving. J. Neurosci. 36, 8612–8623. doi: 10.1523/JNEUROSCI.1505-16.2016
Garner, A. R., Rowland, D. C., Hwang, S. Y., Baumgaertel, K., Roth, B. L., Kentros, C., et al. (2012). Generation of a synthetic memory trace. Science 335, 1513–1516. doi: 10.1126/science.1214985
Goldberg, S. R. (1975). Stimuli associated with drug injections as events that control behavior. Pharmacol. Rev. 27, 325–340.
Gore, F., Schwartz, E. C., Brangers, B. C., Aladi, S., Stujenske, J. M., Likhtik, E., et al. (2015). Neural representations of unconditioned stimuli in basolateral amygdala mediate innate and learned responses. Cell 162, 134–145. doi: 10.1016/j.cell.2015.06.027
Hyman, S. E., Malenka, R. C., and Nestler, E. J. (2006). Neural mechanisms of addiction: the role of reward-related learning and memory. Annu. Rev. Neurosci. 29, 565–598. doi: 10.1146/annurev.neuro.29.051605.113009
Jennings, J. H., Sparta, D. R., Stamatakis, A. M., Ung, R. L., Pleil, K. E., Kash, T. L., et al. (2013). Distinct extended amygdala circuits for divergent motivational states. Nature 496, 224–228. doi: 10.1038/nature12041
Kalivas, P. W., Volkow, N., and Seamans, J. (2005). Unmanageable motivation in addiction: a pathology in prefrontal-accumbens glutamate transmission. Neuron 45, 647–650. doi: 10.1016/j.neuron.2005.02.005
Kawashima, T., Kitamura, K., Suzuki, K., Nonaka, M., Kamijo, S., Takemoto-Kimura, S., et al. (2013). Functional labeling of neurons and their projections using the synthetic activity-dependent promoter E-SARE. Nat. Methods 10, 889–895. doi: 10.1038/nmeth.2559
Kawashima, T., Okuno, H., and Bito, H. (2014). A new era for functional labeling of neurons: activity-dependent promoters have come of age. Front. Neural Circuits 8:37. doi: 10.3389/fncir.2014.00037
Kelsey, J. E., and Willmore, E. J. (2006). Electrolytic lesions of the nucleus accumbens enhance locomotor sensitization to nicotine in rats. Behav. Neurosci. 120, 600–611. doi: 10.1037/0735-7044.120.3.600
Kim, J., Pignatelli, M., Xu, S., Itohara, S., and Tonegawa, S. (2016). Antagonistic negative and positive neurons of the basolateral amygdala. Nat. Neurosci. 19, 1636–1646. doi: 10.1038/nn.4414
Koob, G. F. (2006). The neurobiology of addiction: a neuroadaptational view relevant for diagnosis. Addiction 101, 23–30. doi: 10.1111/j.1360-0443.2006.01586.x
Koya, E., Margetts-Smith, G., and Hope, B. T. (2016). Daun02 inactivation of behaviorally activated fos-expressing neuronal ensembles. Curr. Protoc. Neurosci. 76, 8.36.1–8.36.17. doi: 10.1002/cpns.2
Koya, E., Uejima, J. L., Wihbey, K. A., Bossert, J. M., Hope, B. T., and Shaham, Y. (2009). Role of ventral medial prefrontal cortex in incubation of cocaine craving. Neuropharmacology 56, 177–185. doi: 10.1016/j.neuropharm.2008.04.022
Liu, X., Ramirez, S., Pang, P. T., Puryear, C. B., Govindarajan, A., Deisseroth, K., et al. (2012). Optogenetic stimulation of a hippocampal engram activates fear memory recall. Nature 484, 381–385. doi: 10.1038/nature11028
Liu, X., Ramirez, S., and Tonegawa, S. (2014). Inception of a false memory by optogenetic manipulation of a hippocampal memory engram. Philos. Trans. R. Soc. Lond. B Biol. Sci. 369:20130142. doi: 10.1098/rstb.2013.0142
Mayford, M., Bach, M. E., Huang, Y. Y., Wang, L., Hawkins, R. D., and Kandel, E. R. (1996). Control of memory formation through regulated expression of a CaMKII transgene. Science 274, 1678–1683. doi: 10.1126/science.274.5293.1678
Morgan, J. I., and Curran, T. (1991). Proto-oncogene transcription factors and epilepsy. Trends Pharmacol. Sci. 12, 343–349. doi: 10.1016/0165-6147(91)90594-i
Nestler, E. J., Hope, B. T., and Widnell, K. L. (1993). Drug addiction: a model for the molecular basis of neural plasticity. Neuron 11, 995–1006. doi: 10.1016/0896-6273(93)90213-b
Obenhaus, H. A., Rozov, A., Bertocchi, I., Tang, W., Kirsch, J., Betz, H., et al. (2016). Causal interrogation of neuronal networks and behavior through virally transduced ivermectin receptors. Front. Mol. Neurosci. 9:75. doi: 10.3389/fnmol.2016.00075
O’Brien, C. P., Ehrman, R. N., and Ternes, J. W. (1984). Classical conditioning in opiate dependence. NIDA Res. Monogr. 49, 35–46.
Opris, I., Hampson, R. E., and Deadwyler, S. A. (2009). The encoding of cocaine vs. natural rewards in the striatum of nonhuman primates: categories with different activations. Neuroscience 163, 40–54. doi: 10.1016/j.neuroscience.2009.06.002
Otis, J. M., Namboodiri, V. M., Matan, A. M., Voets, E. S., Mohorn, E. P., Kosyk, O., et al. (2017). Prefrontal cortex output circuits guide reward seeking through divergent cue encoding. Nature 543, 103–107. doi: 10.1038/nature21376
Peters, J., Lalumiere, R. T., and Kalivas, P. W. (2008). Infralimbic prefrontal cortex is responsible for inhibiting cocaine seeking in extinguished rats. J. Neurosci. 28, 6046–6053. doi: 10.1523/JNEUROSCI.1045-08.2008
Petrovich, G. D., Ross, C. A., Gallagher, M., and Holland, P. C. (2007). Learned contextual cue potentiates eating in rats. Physiol. Behav. 90, 362–367. doi: 10.1016/j.physbeh.2006.09.031
Pfarr, S., Meinhardt, M. W., Klee, M. L., Hansson, A. C., Vengeliene, V., Schonig, K., et al. (2015). Losing control: excessive alcohol seeking after selective inactivation of cue-responsive neurons in the infralimbic cortex. J. Neurosci. 35, 10750–10761. doi: 10.1523/JNEUROSCI.0684-15.2015
Powell, J., Gray, J. A., Bradley, B. P., Kasvikis, Y., Strang, J., Barratt, L., et al. (1990). The effects of exposure to drug-related cues in detoxified opiate addicts: a theoretical review and some new data. Addict. Behav. 15, 339–354. doi: 10.1016/0306-4603(90)90044-x
Ramirez, S., Liu, X., Lin, P. A., Suh, J., Pignatelli, M., Redondo, R. L., et al. (2013). Creating a false memory in the hippocampus. Science 341, 387–391. doi: 10.1126/science.1239073
Redondo, R. L., Kim, J., Arons, A. L., Ramirez, S., Liu, X., and Tonegawa, S. (2014). Bidirectional switch of the valence associated with a hippocampal contextual memory engram. Nature 513, 426–430. doi: 10.1038/nature13725
Reijmers, L. G., Perkins, B. L., Matsuo, N., and Mayford, M. (2007). Localization of a stable neural correlate of associative memory. Science 317, 1230–1233. doi: 10.1126/science.1143839
Rocha, A., and Kalivas, P. W. (2010). Role of the prefrontal cortex and nucleus accumbens in reinstating methamphetamine seeking. Eur. J. Neurosci. 31, 903–909. doi: 10.1111/j.1460-9568.2010.07134.x
Rogers, J. L., Ghee, S., and See, R. E. (2008). The neural circuitry underlying reinstatement of heroin-seeking behavior in an animal model of relapse. Neuroscience 151, 579–588. doi: 10.1016/j.neuroscience.2007.10.012
Siegel, S. (1999). Drug anticipation and drug addiction. The 1998 H. David Archibald Lecture. Addiction 94, 1113–1124. doi: 10.1046/j.1360-0443.1999.94811132.x
Sørensen, A. T., Cooper, Y. A., Baratta, M. V., Weng, F. J., Zhang, Y., Ramamoorthi, K., et al. (2016). A robust activity marking system for exploring active neuronal ensembles. Elife 5:e13918. doi: 10.7554/eLife.13918
Stewart, J., de Wit, H., and Eikelboom, R. (1984). Role of unconditioned and conditioned drug effects in the self-administration of opiates and stimulants. Psychol. Rev. 91, 251–268. doi: 10.1037/0033-295x.91.2.251
Suto, N., Laque, A., De Ness, G. L., Wagner, G. E., Watry, D., Kerr, T., et al. (2016). Distinct memory engrams in the infralimbic cortex of rats control opposing environmental actions on a learned behavior. Elife 5:e21920. doi: 10.7554/eLife.21920
Tonegawa, S., Pignatelli, M., Roy, D. S., and Ryan, T. J. (2015). Memory engram storage and retrieval. Curr. Opin. Neurobiol. 35, 101–109. doi: 10.1016/j.conb.2015.07.009
Warren, B. L., Mendoza, M. P., Cruz, F. C., Leao, R. M., Caprioli, D., Rubio, F. J., et al. (2016). Distinct fos-expressing neuronal ensembles in the ventromedial prefrontal cortex mediate food reward and extinction memories. J. Neurosci. 36, 6691–6703. doi: 10.1523/JNEUROSCI.0140-16.2016
Weingarten, H. P. (1983). Conditioned cues elicit feeding in sated rats: a role for learning in meal initiation. Science 220, 431–433. doi: 10.1126/science.6836286
Wikler, A. (1973). Dynamics of drug dependence. Implications of a conditioning theory for research and treatment. Arch. Gen. Psychiatry 28, 611–616. doi: 10.1001/archpsyc.1973.01750350005001
Wiltgen, B. J., Law, M., Ostlund, S., Mayford, M., and Balleine, B. W. (2007). The influence of Pavlovian cues on instrumental performance is mediated by CaMKII activity in the striatum. Eur. J. Neurosci. 25, 2491–2497. doi: 10.1111/j.1460-9568.2007.05487.x
Yager, L. M., and Robinson, T. E. (2010). Cue-induced reinstatement of food seeking in rats that differ in their propensity to attribute incentive salience to food cues. Behav. Brain Res. 214, 30–34. doi: 10.1016/j.bbr.2010.04.021
Keywords: learned associations, neuronal ensembles, mechanistic resolution, operant learning
Citation: Warren BL, Suto N and Hope BT (2017) Mechanistic Resolution Required to Mediate Operant Learned Behaviors: Insights from Neuronal Ensemble-Specific Inactivation. Front. Neural Circuit. 11:28. doi: 10.3389/fncir.2017.00028
Received: 01 December 2016; Accepted: 05 April 2017;
Published: 24 April 2017.
Edited by:
Rui M. Costa, Champalimaud Foundation, PortugalReviewed by:
Ya-tang Li, California Institute of Technology, USAYing Zhang, Dalhousie University, Canada
Copyright © 2017 Warren, Suto and Hope. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Bruce T. Hope, bhope@intra.nida.nih.gov