The role of the dorsal raphé nucleus in reward-seeking behavior
- 1Department of Physiology, Kansai Medical University, Hirakata, Japan
- 2Precursory Research for Embryonic Science and Technology, Japan Science and Technology Agency, Kawaguchi, Japan
Pharmacological experiments have shown that the modulation of brain serotonin levels has a strong impact on value-based decision making. Anatomical and physiological evidence also revealed that the dorsal raphé nucleus (DRN), a major source of serotonin, and the dopamine system receive common inputs from brain regions associated with appetitive and aversive information processing. The serotonin and dopamine systems also have reciprocal functional influences on each other. However, the specific mechanism by which serotonin affects value-based decision making is not clear. To understand the information carried by the DRN for reward-seeking behavior, we measured single neuron activity in the primate DRN during the performance of saccade tasks to obtain different amounts of a reward. We found that DRN neuronal activity was characterized by tonic modulation that was altered by the expected and received reward value. Consistent reward-dependent modulation across different task periods suggested that DRN activity kept track of the reward value throughout a trial. The DRN was also characterized by modulation of its activity in the opposite direction by different neuronal subgroups, one firing strongly for the prediction and receipt of large rewards, with the other firing strongly for small rewards. Conversely, putative dopamine neurons showed positive phasic responses to reward-indicating cues and the receipt of an unexpected reward amount, which supports the reward prediction error signal hypothesis of dopamine. I suggest that the tonic reward monitoring signal of the DRN, possibly together with its interaction with the dopamine system, reports a continuous level of motivation throughout the performance of a task. Such a signal may provide “reward context” information to the targets of DRN projections, where it may be integrated further with incoming motivationally salient information.
Introduction
Serotonin (5-hydroxytryptamine, 5-HT) is present in almost all organisms from plants to vertebrates. In mammals, 5-HT has been found in all organs, such as the brain, gut, lung, liver, kidney, and skin, as well as platelets. Such a wide distribution indicates that 5-HT is an essential chemical for all living animals. In the brain, the distribution of 5-HT projections is widespread, regulating the activity of almost all brain regions. Thus, it is no surprise that 5-HT has been implicated in a variety of brain functions, such as the sleep-wake cycle, appetite, locomotion, emotion, hormonal regulation, and as a trophic factor.
In addition to the “basic” brain functions described above, the role of 5-HT in cognitive functions, including attention, control of impulsivity, coping with stress, social behavior, value-based decision making, and learning and memory, has also captured a great deal of attention. The breakdown of the 5-HT system is often associated with neuropsychiatric diseases including depression, schizophrenia, drug abuse, autism, and Parkinson's disease. However, the specific mechanisms by which 5-HT is involved in these cognitive processes are not yet clear.
Among the possible functions of 5-HT, this review will focus on its role in reward-seeking behavior. There are already good reviews about the role of 5-HT in value-based decision making, often being compared with dopamine function. For example, it has been proposed that the tonic and phasic dopamine and 5-HT systems represent value and action, which are not independent, in an opposite manner. Thus, dopamine may be involved in behavioral activation to obtain rewards and 5-HT may be involved in inhibition in the face of punishment (Boureau and Dayan, 2010; Cools et al., 2011). This unified model can account for the variety of aspects of decision making, including response vigor, time discounting, switching, and risk sensitivity, observed in behavioral-pharmacological experiments in animals and humans. The aim of this review is to further focus on the anatomical and physiological evidence of the 5-HT system and link it with the above findings. I will first review the anatomical evidence that supports the involvement of the raphé nuclei, the origin of 5-HT, in reward-dependent behavior. Among the raphé nuclei, I will focus on the dorsal raphé nucleus (DRN) because it has strong anatomical and physiological connections with the brain areas that are related to reward processing. Second, I will introduce pharmacological studies that examined the impact of changes in the brain levels of 5-HT on reward-seeking behavior. Although the results are mixed, depending on the affected brain regions and the type of 5-HT receptors examined, these studies generally support the inhibitory effect of 5-HT on reward-seeking behavior.
The behavioral pharmacological studies examined how 5-HT is utilized at the projection targets. On the other hand, it is also critical to reveal when and in which situations 5-HT is secreted or when DRN neurons are activated in real time. Recently, several research groups measured the activity of single DRN neurons while animals performed behavioral tasks. I will review the results of single unit recordings from the DRN, including our recent experiments in monkeys. The results show that DRN neuronal activity continuously keeps track of the expected and received reward value throughout the trials.
Finally, I will discuss the possible mechanisms by which 5-HT modulates value-based decision making, together with dopamine and other brain structures, such as the lateral habenula, amygdala, frontal cortex, and basal ganglia.
Anatomical Implication of the Role of 5-HT in Motivational Behavior
There is a great amount of evidence demonstrating tight anatomical connections between the raphé nuclei and the brain areas that are related to reward (Azmitia and Gannon, 1986; Molliver, 1987; Jacobs and Azmitia, 1992; Michelsen et al., 2007).
Among the 9 raphé nuclei B1–B9 (Dahlstroem and Fuxe, 1964), those that are often discussed in relation to reward-related behavior are the DRN, which is the largest group (B7), lumped together with B6, and the median raphé nucleus (MRN), which consists of B8 and B5.
Input to the DRN that May be Involved in Reward Processing (Figure 1, Left)
The DRN receives projections from many brain areas that have been associated with reward and punishment. These areas tend to project to distinct divisions of the DRN (Aghajanian and Wang, 1977; Sakai et al., 1977; Behzadi et al., 1990; Peyron et al., 1998).
Cortical areas projecting to the DRN include the medial prefrontal (Arnsten and Goldman-Rakic, 1984), lateral and medial orbital, cingulate, infralimbic, and insular cortices (Arnsten and Goldman-Rakic, 1984; Sesack et al., 1989; Amat et al., 2005). At least a part of the projection from the medial frontal cortex is via GABA interneurons in the raphé nuclei (Arnsten and Goldman-Rakic, 1984; Hajos et al., 1998; Varga et al., 2001, 2003; Jankowski and Sesack, 2004), which in turn project to 5-HT neurons.
Subcortical areas projecting to the DRN include the amygdala (Peyron et al., 1998; Lee et al., 2007), substantia nigra pars reticulata (SNr), ventral pallidum, preoptic area, claustrum, bed nucleus of the stria terminalis, zona incerta, medial and lateral preoptic areas, hypothalamus, and, most prominently, the lateral habenula nucleus (Pasquier et al., 1976; Aghajanian and Wang, 1977; Wang and Aghajanian, 1977c; Herkenham and Nauta, 1979; Stern et al., 1979; Kalen et al., 1989; Peyron et al., 1998; Varga et al., 2003), whose projection is through the fasciculus retroflexus. The lateral habenula is a brain region that represents negative motivational values, such as reward omission and aversive stimuli (Matsumoto and Hikosaka, 2007, 2009; Hong and Hikosaka, 2008) and transmits these signals to midbrain dopamine neurons and the DRN. Many studies have reported an inhibitory effect from the habenula to the DRN via the rostromedial tegmental nucleus (RMTg). Stimulation of the habenula suppresses the activity of DRN 5-HT neurons (Aghajanian and Wang, 1977; Wang and Aghajanian, 1977c; Stern et al., 1979; Nishikawa and Scatton, 1984, 1985, 1986; Scatton et al., 1984; Nishikawa et al., 1986; Ferraro et al., 1997; Varga et al., 2003) and decreases 5-HT release in the caudate nucleus and substantia nigra (Reisine et al., 1982; but see Kalen et al., 1989).
The hypothalamus is also an important source of reward information for the DRN (Celada et al., 2002). Hypothalamic orexin neurons are activated by arousal, feeding, and rewarding stimuli (Mieda and Yanagisawa, 2002; Lee et al., 2005; Harris and Aston-Jones, 2006) and facilitate 5-HT release (Tao et al., 2006). The amygdala, in which neurons encode positive or negative motivational values (Ledoux, 2000; Belova et al., 2008), also sends projections to the DRN.
The dopamine neurons in the ventral tegmental area (VTA) and substantia nigra pars compacta (SNc) also project to the DRN and MRN (Kalen et al., 1988; Mansour et al., 1990; Peyron et al., 1995; Kitahama et al., 2000), which may exert facilitatory effects on putative 5-HT neurons in the DRN by D2-like dopamine receptor activation (Ferre and Artigas, 1993; Mendlin et al., 1999; Haj-Dahmane, 2001).
Finally, the activity of neurons in the raphé nuclei is regulated by 5-HT via the 5-HT1A receptor found on the somata and dendrites of neurons in the raphé nuclei, where it functions as a somato-dendritic auto-receptor (Wang and Aghajanian, 1977a; Gozlan et al., 1983; Verge et al., 1985; Carey et al., 2004).
Figure 1. Simplified diagram of input to (left) and output from (right) the dorsal raphé nucleus. DRN, dorsal raphé nucleus; Dopamine, dopamine neurons; l-Habenula, lateral habenula; RMTg, rostromedial tegmental nucleus; and GPb, globus pallidus external and internal border (anatomically called the internal medullary lamina). The open and filled rectangles correspond to excitatory and inhibitory connections, respectively. The arrows indicate that the effect is unknown or excitatory and inhibitory effects have been reported.
Output from the Raphé Nuclei (Figure 1, Right)
Efferent projections from the raphé nuclei are widespread, but constitute a topographic organization along the rostrocaudal and medial-lateral axes (Imai et al., 1986; Abrams et al., 2004; Lee et al., 2008). Separate ascending pathways have been described in rats and primates. In the rat, the largest pathway is the medial forebrain bundle, which carries fibers from the MRN and DRN to a wide range of target areas in the forebrain. In primates, a significant number of these fibers (~25%) are heavily myelinated (Azmitia and Gannon, 1986), and the largest pathway appears to be the dorsal raphé-cortical tract, which enters the cortex through the internal capsule (Azmitia and Segal, 1978; Azmitia and Gannon, 1986). Many projection sites include areas that are associated with reward processing, such as the neocortex, nuclei in the basal ganglia, nucleus accumbens, amygdala, septum, hippocampus, and hypothalamus (Azmitia and Segal, 1978; Azmitia and Gannon, 1986; Molliver, 1987; Vertes, 1991; Peyron et al., 1998).
The innervations from the DRN have several characteristics. First, individual DRN neurons give rise to several sets of collateral (branched) projections to distinct, but functionally related, targets. Single DRN 5-HT neurons project to the septum and the entorhinal area, both of which are essential for normal hippocampal function (Kohler et al., 1982), to various combinations of the olfactory cortex, septum, and medial thalamus (De Olmos and Heimer, 1980), to the prefrontal cortex and nucleus accumbens (Van Bockstaele et al., 1993), and to the central nucleus of the amygdala and paraventricular nucleus in the hypothalamus, both of which are involved in central autonomic control, anxiety, and conditional fear (Petrov et al., 1994; Lowry, 2002). This branching is also observed in the DRN projections to the sensory-motor areas, such as the lateral geniculate body and superior colliculus (Villar et al., 1988), which are important for visual information processing, and the substantia nigra subthalamic nucleus and caudate-putamen (Van Der Kooy and Hattori, 1980; Imai et al., 1986), which are involved in the execution of movement. These serotonergic collateral projections to functionally and anatomically related targets could facilitate the integrated and temporally coordinated modulation of multiple brain regions.
Second, 5-HT acts on all major dopaminergic pathways, i.e., nigrostriatal, mesocortical, mesolimbic, and tuberoinfundibular. The interaction of the 5-HT system with the dopamine system has been documented in the frontal cortex and basal ganglia nuclei, which form part of the nigrostriatal, mesocortical, and mesolimbic dopamine pathways. The fourth dopamine pathway, the tuberoinfundibular pathway, projects from the arcuate nucleus to the median eminence in the hypothalamus. Here, dopamine inhibits the secretion of prolactin from the anterior pituitary gland during the resting state. It is also known that stressful events that evoke prolactin release seem to rely, at least partially, on central serotonin function (Bregonzio et al., 1998). In terms of receptor types, although with some exceptions, receptor types such as 5-HT1A, 5-HT1B, 5-HT2A, 5-HT3, and 5-HT4 facilitate dopamine release, while 5-HT2C exerts tonic inhibition on dopamine release (for review, Alex and Pehek, 2007).
Here, in the first section, we discuss primarily the anatomical connections of the DRN to the reward-related brain areas. In the second section, we will focus more on the differential functional effects of 5-HT.
Projections to the striatum and SNr
Among the widespread efferent projections of the DRN, those to the basal ganglia structures, especially the striatum and substantia nigra, may be particularly important for the control of the reward-dependent modulation of action (monkey, Lavoie and Parent, 1990; rat Van Der Kooy and Hattori, 1980; Imai et al., 1986). In monkeys (Lavoie and Parent, 1990; Haber, 2003), 5-HT terminals are particularly abundant in the ventral striatum, including the nucleus accumbens, ventrolateral region of the putamen, and ventromedial region of the caudate nucleus. The influence of 5-HT depends on the type and location of its receptors. High levels of 5-HT1B, 2A, and 2C receptors are reported in the striatum (Wright et al., 1995; Eberle-Wang et al., 1996).
Many reports examining the function of 5-HT in the striatum have focused on its effect on dopamine release. Electrical stimulation of the DRN enhanced dopamine release in the nucleus accumbens, but reduced it in the dorsal striatum; however, the specific effect depends on the type of receptors present. The facilitatory effect of endogenous 5-HT on dopamine release in the nucleus accumbens depends on the presence of 5-HT2A and 5-HT3 receptors, and not on 2B/2C receptors. Conversely, 5-HT2C receptors tonically inhibit dopamine release in the dorsal and ventral striatum (Jiang et al., 1990; Chen et al., 1991; De Deurwaerdere et al., 1998). The activation of 5-HT1B in the nucleus accumbens reportedly attenuated dopamine-dependent responses to a conditioned reward (Fletcher and Korth, 1999; but see Galloway et al., 1993).
The SNr is one of the major targets of the DRN in rats (Dray et al., 1976; Fibiger and Miller, 1977; Azmitia and Segal, 1978; Van Der Kooy and Hattori, 1980; Wirtshafter et al., 1987; Corvaja et al., 1993; Van Bockstaele et al., 1994; Moukhles et al., 1997), cats (Mori et al., 1987), and monkeys (Lavoie and Parent, 1990). In monkeys, 5-HT innervations are particularly dense in the SNr, but much less so in the SNc (Lavoie and Parent, 1990). Coexpression of 5-HT2C receptor mRNA with glutamic acid decarboxylase, but not with tyrosine hydroxylase mRNA, indicates that 5-HT2C receptors are restricted to GABAergic neurons (Eberle-Wang et al., 1997). The functional significance of 5-HT in the SNr, however, is not well understood.
Projections to the SNc and VTA
Electron microscopy studies have shown that 5-HT neurons make direct synaptic contacts with dopaminergic and non-dopaminergic neurons in the VTA (Herve et al., 1987; Van Bockstaele et al., 1994), indicating the direct and indirect influence of the raphé nuclei on the midbrain dopamine system. Electrical stimulation of the MRN (Dray et al., 1976) and DRN (Trent and Tepper, 1991; Gervais and Rouillard, 2000) inhibits the majority of (but not all) the activity of dopamine neurons. Further studies showed that the effect of 5-HT on midbrain dopamine neurons depends on the subtypes of 5-HT receptors present and the location of the dopamine neurons (Alex and Pehek, 2007). The systemic application of a 5-HT2C agonist decreased the baseline activity of dopamine neurons in a dose-dependent manner (Di Giovanni et al., 2000; Gobert et al., 2000), while the application of a 5-HT2C/2B antagonist caused a dose-dependent increase in the baseline and burst activity of dopamine neurons (Ugedo et al., 1989; Di Giovanni et al., 1999). As 5-HT2C receptors are mainly localized in GABAergic neurons in the SNr and VTA, which in turn inhibit dopamine neurons, the inhibitory effect of a 5-HT2C agonist on dopamine function is, at least in part, due to the GABA-mediated tonic inhibitory effect of 5-HT on mesolimbic and nigrostriatal dopamine function. On the other hand, the activation of VTA 5-HT1B receptors increases mesolimbic dopamine release, probably by inhibiting GABA release (Yan and Yan, 2001; Yan et al., 2004). Some authors have reported direct facilitatory effects of 5-HT on dopamine neurons in vitro (Nedergaard et al., 1988). In addition, 5-HT receptors located presynaptically on dopamine terminals or postsynaptically in dopamine projection areas could activate feedback loops, such as the striato-nigral, nucleus accumbens-VTA, or frontal-VTA pathways, thus indirectly altering the excitability of dopamine neurons in the SNc or VTA, resulting in changes in their baseline firing rates (Di Giovanni et al., 2010).
Projections to the amygdala
Several nuclei of the amygdala receive rich serotonergic innervations (Steinbusch, 1981). In rats, the rostral and medial subregions are dense projection sites of 5-HT neurons. In monkeys, 5-HT projections are found widely in the amygdala, with the highest concentration in the lateral division of the central nucleus and lateral-dorsal part of the bed nucleus of the stria terminalis (Sadikot and Parent, 1990; Freedman and Shi, 2001). The effect of DRN on neurons in the amygdala is reportedly inhibitory and mediated by direct DRN-amygdala serotonergic projections (Wang and Aghajanian, 1977b).
Projections to the hypothalamus
The hypothalamus plays a significant role in the processing of natural rewards, such as food and sex (Harris et al., 2005; Muschamp et al., 2007), and it receives strong inputs from the DRN (Nambu et al., 1999). Extracellular 5-HT levels increased in the medial and lateral hypothalamus during the anticipation and intake of food, but not after its consumption (Schwartz et al., 1990). Interestingly, this finding was interpreted in line with the reward-inhibiting and satiety-facilitating functions of 5-HT in the hypothalamus (Hoebel et al., 1989).
Projections to the cortex
The DRN also projects to virtually all cortical areas, and its effect can be excitatory and inhibitory, depending on which layers it projects to and the presence of different receptor types. Electrical stimulation of the DRN and MRN inhibits the majority of medial prefrontal cortex neurons via 5-HT1A (Hajos et al., 2003; Puig et al., 2005) or 5-HT2 (Mantz et al., 1990) receptors. Among several receptor types, 5-HT2A receptors are particularly dense in the prefrontal and anterior cingulate cortices (Pazos et al., 1985), and they are primarily located on the apical dendrites of pyramidal neurons (Jakab and Goldman-Rakic, 1998; Cornea-Hebert et al., 1999). Prefrontal 5-HT2A receptors may activate cortico-tegmental projection neurons, which in turn facilitate VTA dopamine neurons (Pehek et al., 2006). On the other hand, 5-HT2A/2C receptors are also present in the GABAergic interneurons of the cortex and may regulate glutamatergic output (Abi-Saab et al., 1999). 5-HT2C activation in the medial frontal cortex suppresses cocaine-seeking behavior (Pentkowski et al., 2010).
5-HT and the Reward Circuit
5-HT has long been implicated in a wide variety of motivational process; however, contrasting effects have been reported, many indicate a positive reward effect, but some others indicate a negative effect.
The positive reward effects of 5-HT have been described mainly in relation to brain self-stimulation experiments where animals perform operant responses such as pressing a bar to receive electrical stimulation of the brain. The majority of self-stimulation studies have focused on the medial forebrain bundle, which contains ascending dopaminergic fibers; however, several studies have also shown that stimulation of the raphé nuclei and their vicinity is equally effective (Miliaressis et al., 1975; Miliaressis, 1977; Rompre and Miliaressis, 1985). In addition, some pharmacological experiments using the systemic reduction of 5-HT reported attenuated cocaine-seeking behavior (Tran-Nguyen et al., 1999, 2001).
However, many lines of evidence indicate the inhibitory effects of 5-HT on the reward circuitry. The systemic injection of the 5-HT releaser d-fenfluramine (Fletcher, 1995) and the injection of 5-HT into the accumbens (Fletcher, 1996; Fletcher and Korth, 1999) attenuated conditioned responses to obtain amphetamine. The systemic reduction of 5-HT also reportedly enhanced reward-related behavior (Leccese and Lyness, 1984; Tran-Nguyen et al., 2001), while the findings of others depended on the type of reinforcement and the method used to reduce the function of 5-HT.
Experiments with the local injection of 5-HT inhibitors in the raphé nuclei support an inhibitory role of the raphé nuclei in motivational behavior. The local injection of a low dose of the 5-HT1A agonist 8-hydroxy-2-(di-n-propylamino)tetralin (8-OH-DPAT), which selectively inhibits serotonergic neurons in the MRN or DRN (Fletcher et al., 1993, 1995), and muscimol (Liu and Ikemoto, 2007) into the MRN induces conditioned place preference. It is of particular interest that these effects were reversed when the dopamine antagonists were administered systematically (Fletcher et al., 1999; Liu and Ikemoto, 2007) or directly into the nucleus accumbens (Muscat et al., 1989) or striatum (Fletcher and Davies, 1990; Fletcher, 1991), indicating that the reward effect of 5-HT antagonists may depend, at least partly, on the removal of the inhibitory influence of 5-HT on the mesolimbic dopamine system. Indeed, the systemic administration of 8-OH-DPAT increased the firing rate of the majority (75%) of dopamine cells studied and stimulated their bursting activity (Prisco et al., 1994).
The role of 5-HT in reward is complicated by the fact that it binds to a large number of receptor types that have different effects on reward-oriented behavior (Higgins and Fletcher, 2003). One of the principal receptor types involved in reward-oriented behavior may be the 5-HT2C receptor. This receptor's mRNA is expressed in the anterior olfactory nucleus, olfactory tubercle, claustrum, piriform and entorhinal cortices, lateral septal nucleus, amygdala, subiculum and ventral part of CA3, lateral habenula, subthalamic nucleus, SNr, VTA (Molineaux et al., 1989; Pompeiano et al., 1994; Wright et al., 1995; Eberle-Wang et al., 1997; Clemett et al., 2000), and dorsal and ventral (including nucleus accumbens) striatum, all of which are important parts of the reward-related circuitry. Another functional characteristic of the 5-HT2C receptor is that it possesses a high level of constitutive activity, even in the absence of agonist stimulation (Berg et al., 2008). It has been reported that neurons with 5-HT2C receptors in the nucleus accumbens and striatum are probably GABAergic projection neurons (Eberle-Wang et al., 1997). It was also suggested that all 5-HT2C mRNA-containing cells in the SNr and VTA are GABAergic, not dopaminergic, neurons. Thus, the tonic suppressive influence of 5-HT on dopamine neurons would be by 5-HT2C receptors acting on GABAergic neurons, which in turn suppress dopaminergic neurons in the VTA. This mechanism would allow 5-HT2C to exert a tonic influence on the activity of the mesocortical and mesolimbic dopaminergic pathways. Note, however, that a recent study provided anatomical and behavioral support for the localization of 5-HT2C receptors on dopamine neurons in the VTA (Ji et al., 2006). Altogether, 5-HT2C receptors tonically regulate, mainly by inhibition, dopamine release from the terminal regions of the nigrostriatal and mesolimbic pathways (Di Giovanni et al., 1999; Gobert et al., 2000).
As described above, many behavioral-pharmacological studies have reported the effects of 5-HT on the reward circuitry. However, the direction (positive or negative) of its effects should be analyzed carefully because it may vary depending on the method used to modulate 5-HT levels (e.g., systemic or local), the location of self-stimulation (Ahn et al., 2005), or the kinds of behavioral test used (Mosher et al., 2005; Hayes et al., 2009).
Another hypothesis for the role of the 5-HT system in reward-seeking behavior is that 5-HT regulates the timescale of reward prediction, such as the balance between immediate and delayed rewards. In reinforcement learning theory, the state value is discounted when the delivery of the reward is delayed, and Doya et al. suggested that 5-HT regulates this reward discounting rate (Doya, 2002; Tanaka et al., 2004). Indeed, the 5-HT level and firing rate in the DRN increased when rats waited to obtain rewards, and the level of neuronal firing was correlated with successful waiting (Miyazaki et al., 2010, 2011). Such “wait to obtain a reward” behavior might be originally initiated by the reward signal that activates the dopamine system, which then promotes behavioral vigor or activation, and at the same time, the subsequent activation of the DRN is necessary for the successful withholding of responses to obtain rewards.
5-HT and Aversive Information Processing
The participation of 5-HT in aversive information processing has also been reported repeatedly. Strong evidence that 5-HT is involved in aversive information processing comes from the observation that there is a change in neuronal activity in the raphé nuclei or an increase in 5-HT levels in response to aversive stimuli. Stress-related stimuli activate immediate-early gene expression within the DRN (Pezzone et al., 1993). In the DRN of anesthetized rats, the majority of neurochemically identified 5-HT neurons with a clock-like firing pattern were phasically excited, whereas the majority of bursting 5-HT neurons were inhibited by noxious footshocks (Schweimer and Ungless, 2010). Activity level of the raphe nuclei is also modulated; it is increased under inescapable shocks (Grahn et al., 1999; Takase et al., 2004). Forced swimming induced an increase or decrease in 5-HT levels, as measured by microdialysis, depending on the brain region examined; its levels increased in the striatum, but decreased in the amygdala and lateral septum (Kirby et al., 1995).
The role of 5-HT in aversive information processing has multiple facets. First, several lines of evidence suggest that 5-HT modulates sensitivity to threat-related stimuli and punishment (for review, Deakin, 1991; Cools et al., 2008). A negative correlation between 5-HT levels and aversion has been demonstrated repeatedly, indicating the analgesic effect of 5-HT. Low levels of 5-HT in human subjects, achieved by acute tryptophan (the precursor of 5-HT) depletion, enhanced the responsiveness of several brain regions, especially the amygdala, to aversive stimuli, such as fearful faces and negative words (Hariri et al., 2002; Cools et al., 2005; Hariri and Holmes, 2006; Roiser et al., 2008). Low levels of 5-HT also alter the performance of a probabilistic reversal learning task by abnormally enhancing the impact of punishment, such as the inappropriate avoidance of less frequent punishment (Evers et al., 2005; Chamberlain et al., 2006). Note, however, that the role of 5-HT in a probabilistic reversal task may come from the changes in the processing of negative feedback signals per se, rather than changes in sensitivity to the error, because the changes in medial frontal activity did not differ between errors that were or were not followed by behavioral correction (Evers et al., 2005).
Just as decreased 5-HT function causes punishment processing to be enhanced, animal studies have shown that an increase in 5-HT levels inhibits responses to punishment. A well-known example is that increasing 5-HT levels via selective 5-HT reuptake inhibitors produces a potent reduction in the levels of anxiety, an effect underlying many anxiolytic drugs. Conditioned fear stress increases extracellular 5-HT levels in the rat medial prefrontal cortex, followed by a reduction of freezing behavior (Hashimoto et al., 1999). 5-HT also suppresses panic or defensive reactions (Maier and Watkins, 2005) and aggression (Marsh et al., 2002; Miczek et al., 2007). The DRN itself and the projection sites of 5-HT, such as the prefrontal cortex and amygdala, may be involved in this process (Graeff et al., 1996). The amygdala has an essential role in the learning and expression of conditioned fear to unconditional and conditional stimuli (Bechara et al., 1995; Ledoux, 2007), and the injection of the 5-HT reuptake blocker citalopram to the amygdala, which presumably enhances 5-HT levels, impairs fear conditioning (Inoue et al., 2004). Amygdala neurons that are excited by the electrical stimulation of glutamate-releasing inputs from the frontal cortex are inhibited by the concurrent iontophoresis of 5-HT, probably by the activation of GABA-releasing neurons through excitatory 5-HT receptors in the amygdala (Stutzmann et al., 1998; Stutzmann and Ledoux, 1999). Thus, deficient 5-HT function might result in the enhanced processing of harmful stimuli because of the diminished inhibitory modulation of excitatory sensory afferents, thereby enabling innocuous sensory signals to be processed by the amygdala as being emotionally salient.
Secondly, recent theoretical and experimental studies suggest that 5-HT does not operate solely as an affective (i.e., aversive) factor. Instead, the influence of 5-HT on aversive processing is evident on the junction of affective and activational factors; specifically, behavioral inhibition in the face of aversive predictions (Dayan and Huys, 2008, 2009; Boureau and Dayan, 2010). For example, in a task in which healthy human subjects decided to respond or not to obtain a reward or to avoid punishment, temporarily lowering 5-HT levels abolished the punishment-induced slowing of their response, but it did not affect the general inhibition of their motor response or sensitivity to aversive outcomes (Crockett et al., 2009). However, aversive predictions can be an instrumental process that links stimuli, responses, and outcomes, or they can be a Pavlovian process that links stimuli and outcomes. Here, further study revealed that 5-HT is involved in reflexive, Pavlovian aversive predictions because the latencies for the punished and non-punished responses were prolonged in the presence of punishment stimuli under acute tryptophan depletion (Crockett et al., 2012).
The third aspect of 5-HT-dependent neuronal processes associated with aversive experiences is behavioral control over a stressor. Generally, the emotional consequences related to aversive events are less severe if the subjects have control over the aversive events, and a lack of control of stress leads to mood and anxiety disorders. Experimentally, animals exposed to inescapable stressors subsequently exhibit “learned helplessness,” a set of behavioral changes that include an impaired ability to escape from aversive events, increased fear conditioning and anxiety, a potentiated response to addictive drugs, and altered pain sensitivity. It has been suggested that 5-HT is involved in this “reduction of action” after a stressful, uncontrollable situation. Indeed, the activity of DRN 5-HT neurons, as measured by Fos expression (Grahn et al., 1999), and 5-HT levels, as measured by in vivo microdialyzis (Maswood et al., 1998), in the DRN or its projection sites (Amat et al., 1998; Bland et al., 2003a,b) were enhanced under an inescapable stress, such as tailshock, but not under an escapable stress. Further, the intense activation of DRN 5-HT neurons by an uncontrollable stress sensitizes these neurons for a period of time (Amat et al., 1998). The inactivation of 5-HT blocks the occurrence of these behavioral changes (Maier et al., 1994, 1995).
One possible mechanism for the activation of the DRN under inescapable stress is input to the DRN from the habenula. Lesions of the habenula severely attenuate the rise in 5-HT levels in the DRN under both escapable and inescapable stress, thus eliminating the difference between them and producing behavioral indifference (Amat et al., 2001). The frontal cortex may also be involved in this process. When a stressor is controllable, the DRN is no longer activated by the stressor due to inhibitory signals from the ventral medial prefrontal cortex (Amat et al., 2005). The role of the ventral medial prefrontal cortex may be to detect the fact that the stressor is controllable rather than to escape learning per se. If controllable, the prefrontal cortex inhibits DRN activation and thus prevents learned helplessness. A recent study used an optogenetic approach to reveal more detailed neuronal circuits that support such behavioral changes; activation of the prefrontal-DRN pathway is causally involved in an increase in effortful movement during the forced swim test, which is a challenging and inescapable situation, whereas activation of the prefrontal-habenula pathway caused the opposite effect (Warden et al., 2012). Note that such a situation or state-dependence is also documented for single neuronal activity. For example, DRN neurons in the rat responded to a tone differently, depending on the reward and no-reward context (Li et al., 2013).
Fourth, an increase of 5-HT may regulate the processing of stress via the activation of pituitary and adrenal functions (Vernikos-Danellis et al., 1977), which have bi-directional interactions with the 5-HT system. There is a dense projection of corticotropin-releasing factor (CRF) neurons to the raphé nuclei in rats (Cummings et al., 1983; Lowry et al., 2008) and humans (Austin et al., 1997). A subpopulation of CRF-containing neurons is present in the dorsomedial part of the DRN, and dual-labeling immunohistochemistry revealed that almost all CRF-containing neurons are serotonergic (Commons et al., 2003). Intracerebroventricular injections of the selective CRF2 receptor agonist urocortin 2 increased the activity of serotonergic neurons (Abrams et al., 2004; Staub et al., 2005, 2006). However, the effects of CRF on the DRN appeared to be either excitatory or inhibitory, probably depending on the location of the recorded neurons within the DRN, e.g., neurons in the ventromedial region were inhibited, whereas neurons in the dorsomedial and lateral wings had variable responses (Kirby et al., 2000). In addition, CRF-containing axons from the dorsomedial DRN project to CRF-containing neurons of the central nucleus of the amygdala, a stress related area and a part of the central autonomic system (Petrov et al., 1994). There is also a dense projection of 5-HT neurons to the suprachiasmatic nucleus, which in turn regulates the secretion of CRF from the hypothalamus and, consequently, adrenocorticotropic hormone (ACTH) release. Thus, it is important to emphasize the two roles of 5-HT in the mammalian brain, i.e., as a neurotransmitter and a hormonal factor. These two aspects may be related to each other as a recent study showed that the negative prediction error signal in the ventral striatum is strengthened under stress (Robinson et al., 2013).
Single Unit Recordings from the DRN
The anatomical and pharmacological evidence reviewed above suggests that 5-HT has potent effects on reward and punishment, and that its effects are tightly regulated by the neural circuitry interacting with the DRN. A missing piece of this puzzle is precisely how DRN neurons behave while reward-oriented behavior unfolds in real time. The studies reviewed above typically manipulated DRN function over long timescales, such as hours, and over a wide spatial extent, altering 5-HT function in multiple brain regions simultaneously. Yet the reward-related processes that the DRN regulates, including seeking, consuming, and learning about rewards, are performed during natural behavior within the span of minutes or seconds. In addition, while the behavioral-pharmacological experiments examined how 5-HT is utilized at the projection sites, much less is understood about in which situations DRN neurons secrete 5-HT.
To understand which aspects of cognitive behavior are encoded by the activity of DRN neurons in real time, several research groups have measured the activity of single DRN neurons while animals performed behavioral tasks. In the following section, I will introduce our studies in primates performing “biased reward saccade tasks” (Figures 2A,B) (Bromberg-Martin et al., 2010). Using saccades as a behavioral measure is advantageous for several reasons. First, the measurement and assessment of changes in behavior, i.e., eye movement, are relatively simple. Second, the neuronal circuit for the generation of eye movements is well established.
Figure 2. (A) One direction rewarded memory guided saccade (1DR-MGS) task. After the monkey fixated on the central fixation point for 1200 ms, one of the two target positions was flashed for 100 ms. After the fixation point disappeared, the monkey made a saccade to the cued position to receive a liquid reward. The white arrows indicate the direction of gaze. In a block of 20–28 trials (e.g., left-large block), one target position (e.g., left) was associated with a large reward and the other position (e.g., right) was associated with a small reward. The position-reward contingency was then reversed (e.g., right-large block). (B) Left-large and right-large conditions were alternated between blocks with no external cue. The location of the target was determined pseudo-randomly. (C,D) Examples of the activity of two DRN neurons in the 1DR-MGS task. The activity in the large- and small-reward trials is shown in red and blue, respectively. The histograms and raster plots are shown in three sections: the left section is aligned to the time of fixation point onset (FPon), the middle section is aligned to target onset (TGon) and fixation point offset (FPoff), and the right section is aligned to reward onset (RWon). The black dots indicate saccade onset (SACon); the blue dots indicate reward onset and offset. Note that reward offset (RWoff) applies only to the large-reward trials. (E) Population activity of DRN neurons in the 1DR-MGS task (n = 84). The activity of each neuron is presented as a row of pixels. Left and center: changes in the neuronal firing rate from baseline are compared in the large- and small-reward trials. The color of each pixel indicates the ROC value based on the comparison of the firing rate between a control period just before fixation onset (400 ms duration) and a test window centered on the pixel (100 ms duration). This analysis was repeated by moving the test window in 20-ms steps. The warm colors (ROC > 0.5) indicate increases in the firing rate relative to the control period, while the cool colors (ROC < 0.5) indicate decreases in the firing rate. Right: changes in reward-dependent modulation. The ROC value of each pixel was based on the comparison of the firing rate between the large- and small-reward trials. The warm colors (ROC > 0.5) indicate higher firing rates in the large-reward trials than in the small-reward trials. Modified from (Nakamura et al., 2008).
While the activity of DRN neurons was found to be correlated with a variety of events, including movements, stimulus identity, and response direction (Ranade and Mainen, 2009), we found that reward information is one of the most influential factors for the modulation of DRN neuronal activity. A comparison of DRN activity with that of midbrain dopamine neurons also highlighted the distinct aspects of reward coding by different monoamine neurotransmitters.
Single Neuronal Activity of the Primate DRN in a Biased Reward Saccade Task
Nakamura et al. recorded DRN neuronal activity while monkeys performed memory-guided saccade tasks with a biased reward schedule (Nakamura et al., 2008). After fixation on a central fixation point, a target flashed briefly to either the left or right. After a delay of 800 ms, the animal made a saccade in the direction where the target was previously presented (Figure 2A). The main feature of the task was the block design of the reward schedule (Figure 2B). For every 20–28 consecutive trials, called a block, one direction was always associated with a large reward, while the other direction was always associated with a small reward (e.g., right-large, left-small). Thus, we can measure the effect of the expectation and receipt of a certain reward size on neuronal activity. In addition, this target location-reward size contingency was switched between blocks (e.g., right-large, left-small to right-small, left-large) without an explicit signal, which caused the receipt of an unexpectedly large or small reward on the very first trial of each block. This feature enabled us to measure the effect of the positive and negative reward prediction error.
DRN Neurons Encode the Expected and Received Reward Value
We found that many DRN neurons exhibited task-related activity that was modulated by the expected and received reward value. Figure 2C shows a representative example. This neuron exhibited an increase in activity after the onset of the fixation point (FPon) followed by regular and tonic firing until reward onset (RWon). The activity further increased after the onset of a large reward, but ceased after the onset of a small reward, and this trend lasted tonically after reward onset. Another example neuron in Figure 2D showed an opposite modulation pattern. This neuron exhibited a decrease in activity after the onset of the fixation point followed by a tonic increase for small reward trials and suppression for large reward trials.
Reward-dependent modulation in activity was commonly observed in the population of DRN neurons. Figure 2E illustrates the time course of activity modulation using receiver operating characteristic (ROC) analysis by comparing the firing rate of each neuron for large (Figure 2E, left) and small (Figure 2E, middle) reward conditions to their baseline activity during 400 ms before fixation onset. During both periods before and after reward delivery, called the pre- and post-reward periods, respectively, many DRN neurons exhibited tonic increases (shown in warm colors) or decreases (cool colors) in activity. Figure 2E, right, compares the activity of each neuron between the large- and small-reward trials. The tonic reward effect was present in many neurons during both the pre- and post-reward periods.
There was a notable difference in reward-dependent modulation between the pre- and post-reward periods, indicating a different source of information. For each neuron, the change in activity during the pre-reward period, compared with baseline activity, tended to be in the same direction in both the large- and small-reward trials. On the contrary, the change in activity during the post-reward period, compared with baseline activity, tended to be in the opposite direction. For example, for the neuron shown in Figure 2A, the pre-reward activity increased compared with the baseline in both the large- and small-reward trials. On the other hand, its post-reward activity increased in the large-reward trials, but was inhibited in the small-reward trials relative to its baseline activity before fixation point onset. Thus, the main cause of the reward effect during the pre-reward period was that the change in activity tended to be stronger in the large-reward trials than in the small-reward trials. Conversely, the reward-dependent modulation of post-reward activity was caused by the modulation of activity in the opposite direction, depending on the reward value.
DRN Neurons Keep Track of the Expected and Received Reward Value
DRN neurons exhibited a tonic increase or decrease in activity that was modulated by the expected and received reward value. What do these tonic changes encode? One possibility is that this tonic modulation of activity encodes sustained aspects of motivated behavior, such as the state of expectation of future rewards for each moment. If so, the activity during the fixation period may represent the expected value of the performance of the task itself. This is because the animal did not know the exact value of the upcoming reward during the fixation period, but knew the averaged expected reward value, which should be a value between the large and small rewards. After target presentation, the exact expected/received reward value was known. If the neurons encoded the behavioral tasks primarily in terms of their reward value throughout a trial, then the neurons excited during the fixation period should be preferentially excited by the reward cues (i.e., carrying positive reward signals), whereas the neurons inhibited during the fixation period should be preferentially inhibited by the reward cues (i.e., carrying negative reward signals). Conversely, if the neurons encoded the fixation period and reward value in an independent manner, then there should be no systematic relationship between fixation- and reward-related activity.
Analysis revealed that there was indeed a strong correlation between the tonic activity level of a neuron during the fixation period and its encoding of reward-related cues and outcomes. For example, neurons like the one presented in Figure 2C showed a sustained elevation in activity during the fixation period. After reward delivery, these neurons responded with a positive reward signal, with higher activity in response to the large- than to the small-reward trials. Other neurons, like the one presented in Figure 2D, showed a sustained suppression in activity during the fixation period, with higher activity in unrewarded trials than in rewarded trials. The population average of normalized activity was computed separately for neurons with positive, negative, or no significant reward signals in response to the outcome (Figures 3A–C). Neurons with positive reward signals for the outcome had elevated activity during the early period of the task (Figure 3A); if the rewarded target appeared, their activity was elevated further, whereas if the unrewarded target appeared, they returned to near baseline. Neurons with negative reward signals had suppressed activity during the early period of the task (Figure 3B); if the rewarded target appeared, their activity was suppressed further, whereas if the unrewarded target appeared, they returned to near baseline. Neurons with no significant reward signals had a tendency for small phasic responses to the fixation point and targets and slightly elevated activity during the task (Figure 3C).
Figure 3. Population average activity of dorsal raphé neurons separated by their reward signals in response to the outcome. (A–C), Normalized activity is shown for the 1DR-MGS task, separately for positive-reward cells (A), negative-reward cells (B), and non-outcome responsive cells (C) The neurons were sorted into these categories based on significant reward discrimination during a 150–450-ms window after outcome onset (gray bar on the x-axis; p < 0.05, Wilcoxon rank-sum test). Thick lines, mean normalized activity; light shaded areas, 1 SEM. (D) Neural activity during the fixation period was positively correlated with reward coding during the target and outcome periods. The x-axis indicates the fixation period response, which was measured as the ROC area for each neuron for discriminating between its firing rate at 500–900 ms after fixation point onset vs. the pre-fixation period at 0–400 ms before fixation point onset. The y-axis indicates reward discrimination, which was the difference in reward responses between the large- and small-reward trials. The text indicates rank correlation (rho) and its p-value. The dark dots indicate neurons with a significant excitation or inhibition during the fixation period. The colored dots indicate neurons with significantly higher activity during the rewarded trials (red) or during the unrewarded trials (blue) (p < 0.05, Mann-Whitney U-test). The black lines indicate the line of best fit calculated using type 2 least-squares regression. (E,F) The first (E) and second (F) principal components of dorsal raphé neural activity profiles during the memory-guided saccade. Curves represent the normalized firing rate of the principal component during the fixation period (black) and after the onset of the rewarded (red) and unrewarded (blue) target, separately for the contralateral-rewarded block (dark colors) and ipsilateral-rewarded block (light colors). The first principal component indicated tonically increased activity during the fixation period and positive-reward coding during the target, memory, and outcome periods. The second component indicated tonically increased activity in response to reward delivery. Modified from (Bromberg-Martin et al., 2010).
The activity of a neuron during the fixation period was strongly positively correlated with its degree of reward discrimination during the post-target and post-reward periods (Figure 3D). If the elevation of activity during the fixation period was stronger, the neuron had higher discrimination of a positive-reward signal; with stronger activity for large- than small-reward trials during the post-target and post-reward periods. If the inhibition of fixation activity was stronger, the neuron had higher discrimination of a negative-reward signal; with stronger activity for small- than large-reward trials during the post-target and post-reward periods. Thus, most DRN neurons responded to the initiation of a behavioral task in the same direction as they responded to the reward cues and outcomes, and those neurons with stronger task coding also had stronger reward coding. These two signals combined so that the level of DRN activity tracked progress throughout the task toward obtaining future rewards. This form of correlated task and reward coding had a dominant influence on DRN neurons, and it was not simply one of many systematic forms of task and reward encoding.
So far, the analysis showed that DRN neurons encoded the information of the task (i.e., fixation period activity) and reward outcome in a correlated manner. However, it did not analyze whether this correlation had a dominant tendency or it was merely one of many systematic forms of task and reward encoding. The analysis was also performed only for restricted periods of the task, which were determined tentatively. To characterize the activity patterns of neurons during all task phases in an unbiased manner, we applied principal component analysis (Richmond and Optican, 1987; Paz et al., 2005). In this analysis, the activity of each neuron is described as a linear combination of the major components of activity that varied systematically with the task variables; the first principal component represents the most common pattern of neural activity with the greatest amount of variance, the second principal component explains the second most common pattern of neural activity, and so on. Then, the activity profile of every neuron may be reconstructed as the sum of its mean neural activity profile plus a weighted combination of the principal components. If a neuron is assigned a component positive weight, then its activity is positively related to the time series of that component. Conversely, if a neuron is assigned a component negative weight, then its activity is negatively related to the time series of that component.
In the DRN population of neurons, the first principal component (Figure 3E) indicated a positive correlation between task onset-related activity and reward coding. It consisted of a gradual increase in tonic activity during the inter-trial interval and after fixation point onset, followed by an additional increase in tonic activity in response to the rewarded target. The second principal component (Figure 3F) had a prolonged tonic change in activity after a reward was delivered. Thus, whereas the first component resembled “task-reward value coding,” the second component resembled “reward delivery coding.”
Note that principal component analysis treats neural activity as a linear combination of orthogonal components; if the “true” components underlying neural activity are combined non-linearly or are not orthogonal, the principal components may not represent them perfectly. Nevertheless, further analysis indicated that only the first two principal components explained significantly more variance in activity than would be expected under the null hypothesis that there were no systematic patterns in the data using shuffled datasets (Bromberg-Martin et al., 2010). Thus, these principal components explained most of the systematic variation in neuronal activity that was related to task events.
Difference from Dopamine Neurons
Reward-dependent modulations of the activity of DRN neurons were distinctively different from those observed in putative dopamine neurons for the same task (the visually guided version of the biased-reward saccade task, Figure 4A). First, whereas DRN neurons responded to both the reward-predicting stimulus and the reward itself (TGon and RWon, respectively, Figure 4B), dopamine neurons predominantly responded to the reward-predicting sensory stimulus (TGon). Second, whereas the DRN contains neurons that preferred larger rewards and neurons that preferred smaller rewards, dopamine neurons invariably preferred larger rewards (i.e., are excited by larger rewards). Third, whereas DRN neurons reliably coded the value of the received reward, whether or not it was expected, dopamine neurons responded to a reward only when it was larger or smaller than expected. Figure 4C shows the changes in neuronal activity during the pre- and post-reward periods when the target location-reward value contingency was switched. The activity of positive and negative reward-coding DRN neurons exhibited the expected (pre-reward) and received (post-reward) reward values. The changes in the activity of dopamine neurons during the pre-reward period were similar to those of DRN neurons. However, unlike DRN neurons, dopamine neurons responded to reward delivery only when the cue position-reward contingency was switched so that the reward was unexpectedly small or large, consistent with the prediction error hypothesis (Schultz, 1998; Kawagoe et al., 2004). Finally, whereas DRN neurons typically exhibited tonic responses, dopamine neurons exhibited phasic responses. Thus, DRN neurons provide tonic signals related to the expected and received reward values, unlike dopamine neurons that provide phasic signals related to the reward prediction error.
Figure 4. DRN neurons and dopamine neurons encode different aspects of the reward. (A) Visually guided version of the one direction rewarded saccade (1DR-VGS) task. (B) Activity of 167 DRN neurons and 64 dopamine neurons for the 1DR-VGS task. The same format is used as in Figure 2E. (C) Changes in neuronal activity with the reversal of position-reward contingency. Top and middle: DRN neurons with large- and small-reward preferences, respectively. Bottom: dopamine neurons. For each group, the activity during the pre-reward period (400 ms after target onset) is shown on the left, and the activity during the post-reward period (400–800 ms after reward onset for DRN neurons; 0–400 ms after reward onset for dopamine neurons) is shown on the right. For each graph, the left panel shows large-to-small reward reversal; the right panel shows small-to-large reward reversal. The large-reward trials are indicated by dark gray; the small-reward trials are indicated by clear areas (as in the top). Shown are the mean and SE of the normalized neuronal activity for the n-th trial after contingency reversal. The asterisks (*) indicate activity that was significantly different from the activity in the last five trials of the block with the reversed contingency (p < 0.01, Mann-Whitney U-test). Modified from (Nakamura et al., 2008).
Discussion
The characteristic features of the activity of DRN neurons observed in the biased-reward saccade tasks were a tonic response pattern and stronger modulation for the most valuable option in either a positive or negative manner. The tonic activity underlying the expected reward value indicates its role in subjective motivation to obtain a reward or “wanting;” the response to the received reward value indicates its role in a subjective hedonic experience or “liking” (Berridge and Kringelbach, 2008). Correlated fixation-period activity, which represents the task value, and post-outcome activity, which represents the value of the received reward, indicate that DRN activity encodes behavioral tasks primarily in terms of their reward value throughout a trial. The principal components, which explained the majority of activity patterns, indicate that this reward coding, aside from other possible sensory-motor coding, is the major component of DRN activity. Conversely, DRN neurons do not appear to encode the prediction error signal of appetitive or aversive events.
Possible sources of the pre-reward activity (i.e., the response to fixation and target) may be dopamine neurons in the SNc, VTA, and lateral habenula (Figure 5). Since dopamine neurons are excited by a large reward-predicting cue, DRN neurons would also be excited by the same cue (Kawagoe et al., 2004). Indeed, during the pre-reward period, a large-reward preference was more common (~20% of all task-related DRN neurons) than a small-reward preference (~5%). The main projection from the lateral habenula to the DRN is, on the other hand, inhibitory. Using the same biased-reward saccade tasks, Matsumoto et al. showed that lateral habenula neurons were excited by stimuli that predict small rewards and were inhibited by a large-reward predicting cue (Matsumoto and Hikosaka, 2007). Such modulation of habenula activity would then be inversely translated into the large-reward preference of DRN neurons via inhibitory neurons in the RMTg.
Figure 5. Schematics of the activity of neurons in the different brain regions that project to the DRN. Red lines, large reward, blue lines, small reward. Activity for the biased-reward saccade task in dopamine (Kawagoe et al., 2004), lateral habenula (Matsumoto and Hikosaka, 2007, 2009), and DRN neurons (Nakamura et al., 2008), and for the Pavlovian conditioning task in the amygdala (Belova et al., 2007, 2008) is shown.
The post-reward responses of DRN neurons are unlikely to be derived from dopamine or habenula neurons because neither of them exhibit post-reward responses, except on the first trial after the block was switched. Possible origins of the post-reward activity include the amygdala, hypothalamus, and medial prefrontal cortex. In the post-reward period, unlike the pre-reward period, the direction of modulation relative to the baseline was often opposite between the large- and small-reward trials. This observation indicates different sources of activity for the large- and small-reward trials. It was also found that one population of DRN neurons showed a large-reward preference and another population showed a small-reward preference. One possible interpretation would be that the source of the two kinds of reward-related signals (small > large and large > small) are represented in other brain areas, such as the anterior cingulate cortex (Niki and Watanabe, 1979; Amiez et al., 2006), and these signals are transmitted to the DRN (Arnsten and Goldman-Rakic, 1984). Another possible source is the amygdala. Amygdala neurons, like DRN neurons, tracked progress throughout a behavioral task, such that the response of a neuron to the start of the task was strongly correlated with its response to the reward cue and outcome. They also include both positive and negative coding neurons (Belova et al., 2008). Another possibility is that the reward information originated from the same group of neurons, but was transmitted to the DRN by different mechanisms; one directly, the other indirectly, via inhibitory connections. For example, the ventral medial prefrontal cortex inhibits 5-HT neurons in the DRN by targeting local GABAergic interneurons (Varga et al., 2001). Such multi-channeled inputs would enable the DRN to integrate positive and negative reward values independently over time.
Possible Functions of the DRN in Reward Processing and the Direction of Future Research
The tonic activity of DRN neurons may be ideal to signal a continuous level of motivation and hedonic experience throughout the performance of a task. Such a signal may provide a “reward context” signal to the targets of DRN projections, where the signal may be used differently depending on the type of 5-HT receptor present.
First, the sustained reward signals in the DRN could be used to track the value of the current behavioral state. Such estimated values have an important role in theories of reinforcement learning, which suggest that the prediction error signal of dopamine neurons is calculated as the difference between the actual and expected reward values. Thus, DRN activity could contribute to the computation of prediction errors by providing the current state of the expected reward value.
Second, DRN activity may report the long-term averaged reward, rather than immediate, phasic reward information (Daw et al., 2002). In real life, one needs to integrate flows of information, including both appetitive and aversive events and situations, to achieve better decision making to adapt to external changes. The tonic activation patterns of DRN neurons may be useful in integrating appetitive and aversive information coming from different sources (as in Figure 1, left) over a substantial period of time.
The activity of DRN neurons observed in behaving monkeys is characterized by a mirror-image pattern of reward coding by different subsets of neurons, namely, positive and negative reward coding (Figure 5). The current theoretical account of 5-HT function is that it may be involved in behavioral inhibition in the face of punishment (Cools et al., 2011). Thus, the neuronal activity data in the DRN of behaving animals appears partially unexpected because some neurons showed stronger activity in the expectation of large rewards. This seemingly inconsistent finding may be because different groups of neurons might map onto neurochemically or anatomically different subgroups. In the neurochemical account, it is possible that the negative coding DRN neurons could be serotonergic projection neurons, while the positive coding ones may be GABAergic interneurons. Clarifying the underlying cell properties is essential for further understanding of the function of 5-HT (Schweimer and Ungless, 2010). In the anatomical account, neurons may respond differently depending on the circuit in which they are involved. For example, a recent single unit recording study in primates (Inaba et al., 2013) reported that neurons that prefer rewards tend to be distributed more rostrally, while neurons that prefer no rewards were distributed more caudally. It is possible that these different types of neurons may be involved in different anatomical circuits in the brain.
The mirror-image activity of different sets of DRN neurons also suggests that their function may be highly context-dependent. The DRN is anatomically and functionally linked to different circuits involving different brain structures, such as the frontal cortex, amygdala, basal ganglia, and dopamine neurons, and context here may depend on which circuit is mainly involved. Indeed, Warden et al. showed that stimulation of specific projections from the medial frontal cortex to the DRN caused changes in animals' movement in a challenging situation (the forced swim test), while stimulation of the overall DRN caused, in addition to the usual effects observed in a challenging situation, a general increase in movement (the open field test) (Warden et al., 2012). The activation of a specific pathway of DRN neurons with specific task-related activity may support the context-dependent selection of value-based decision making.
Another possible function of the seemingly opposite signals might be the interaction between the 5-HT and other systems, including dopamine systems, to compute appetitive and aversive information in a balanced manner (Figure 5). As in Solomon and Corbit's affective dynamics model (Solomon and Corbit, 1974), the value of rewards is treated as a continuous signal rather than the pulsatile pattern of the value signal, and the tonic DRN activity we observed may correspond to this signal. With the normal level of DRN activity and 5-HT, the baseline activity of dopamine neurons may be tonically suppressed. In addition, phasic appetitive and aversive event-indicating cues would drive both dopamine and DRN neurons which inhibit, at least partly, dopamine neurons, simultaneously. Thus, the DRN would attenuate the strength of responses of the dopamine system to appetitive and aversive events. This process might have the advantage of maintaining equilibrium in terms of reward to prevent excessive positive or negative value coding. Given the variety of 5-HT receptors and their functions, this scheme is, of course, simplistic. It should also be clarified whether the regulation of the reward circuit by 5-HT is always dopamine-dependent, like the proposed scheme, or it can act independently and directly. Combined research of circuit-specific manipulation such as the optogenetic approach and detailed analyses of neural activity in relation to changes in behavior would lead to a clear understanding of the role of 5-HT.
Conflict of Interest Statement
The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
Kae Nakamura was supported by Precursory Research for Embryonic Science and Technology, a Grant-in-Aid for Scientific Research B, a Grant-in-Aid for Scientific Research on Priority Areas, and Human Frontier Science Program. I thank Dr. Ethan S. Bromberg-Martin for helpful comments.
References
Abi-Saab, W. M., Bubser, M., Roth, R. H., and Deutch, A. Y. (1999). 5-HT2 receptor regulation of extracellular GABA levels in the prefrontal cortex. Neuropsychopharmacology 20, 92–96. doi: 10.1016/S0893-133X(98)00046-3
Abrams, J. K., Johnson, P. L., Hollis, J. H., and Lowry, C. A. (2004). Anatomic and functional topography of the dorsal raphe nucleus. Ann. N.Y. Acad. Sci. 1018, 46–57. doi: 10.1196/annals.1296.005
Aghajanian, G. K., and Wang, R. Y. (1977). Habenular and other midbrain raphe afferents demonstrated by a modified retrograde tracing technique. Brain Res. 122, 229–242. doi: 10.1016/0006-8993(77)90291-8
Ahn, K. C., Pazderka-Robinson, H., Clements, R., Ashcroft, R., Ali, T., Morse, C., et al. (2005). Differential effects of intra-midbrain raphe and systemic 8-OH-DPAT on VTA self-stimulation thresholds in rats. Psychopharmacology (Berl.) 178, 381–388. doi: 10.1007/s00213-004-2031-3
Alex, K. D., and Pehek, E. A. (2007). Pharmacologic mechanisms of serotonergic regulation of dopamine neurotransmission. Pharmacol. Ther. 113, 296–320. doi: 10.1016/j.pharmthera.2006.08.004
Amat, J., Baratta, M. V., Paul, E., Bland, S. T., Watkins, L. R., and Maier, S. F. (2005). Medial prefrontal cortex determines how stressor controllability affects behavior and dorsal raphe nucleus. Nat. Neurosci. 8, 365–371. doi: 10.1038/nn1399
Amat, J., Matus-Amat, P., Watkins, L. R., and Maier, S. F. (1998). Escapable and inescapable stress differentially and selectively alter extracellular levels of 5-HT in the ventral hippocampus and dorsal periaqueductal gray of the rat. Brain Res. 797, 12–22. doi: 10.1016/S0006-8993(98)00368-0
Amat, J., Sparks, P. D., Matus-Amat, P., Griggs, J., Watkins, L. R., and Maier, S. F. (2001). The role of the habenular complex in the elevation of dorsal raphe nucleus serotonin and the changes in the behavioral responses produced by uncontrollable stress. Brain Res. 917, 118–126. doi: 10.1016/S0006-8993(01)02934-1
Amiez, C., Joseph, J. P., and Procyk, E. (2006). Reward encoding in the monkey anterior cingulate cortex. Cereb. Cortex 16, 1040–1055. doi: 10.1093/cercor/bhj046
Arnsten, A. F., and Goldman-Rakic, P. S. (1984). Selective prefrontal cortical projections to the region of the locus coeruleus and raphe nuclei in the rhesus monkey. Brain Res. 306, 9–18. doi: 10.1016/0006-8993(84)90351-2
Austin, M. C., Rhodes, J. L., and Lewis, D. A. (1997). Differential distribution of corticotropin-releasing hormone immunoreactive axons in monoaminergic nuclei of the human brainstem. Neuropsychopharmacology 17, 326–341. doi: 10.1016/S0893-133X(97)00083-3
Azmitia, E. C., and Gannon, P. J. (1986). The primate serotonergic system: a review of human and animal studies and a report on Macaca fascicularis. Adv. Neurol. 43, 407–468.
Azmitia, E. C., and Segal, M. (1978). An autoradiographic analysis of the differential ascending projections of the dorsal and median raphe nuclei in the rat. J. Comp. Neurol. 179, 641–667. doi: 10.1002/cne.901790311
Bechara, A., Tranel, D., Damasio, H., Adolphs, R., Rockland, C., and Damasio, A. R. (1995). Double dissociation of conditioning and declarative knowledge relative to the amygdala and hippocampus in humans. Science 269, 1115–1118. doi: 10.1126/science.7652558
Behzadi, G., Kalen, P., Parvopassu, F., and Wiklund, L. (1990). Afferents to the median raphe nucleus of the rat: retrograde cholera toxin and wheat germ conjugated horseradish peroxidase tracing, and selective D-[3H]aspartate labelling of possible excitatory amino acid inputs. Neuroscience 37, 77–100. doi: 10.1016/0306-4522(90)90194-9
Belova, M. A., Paton, J. J., and Salzman, C. D. (2008). Moment-to-moment tracking of state value in the amygdala. J. Neurosci. 28, 10023–10030. doi: 10.1523/JNEUROSCI.1400-08.2008
Belova, M. A., Paton, J. J., Morrison, S. E., and Salzman, C. D. (2007). Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala. Neuron 55, 970–984. doi: 10.1016/j.neuron.2007.08.004
Berg, K. A., Harvey, J. A., Spampinato, U., and Clarke, W. P. (2008). Physiological and therapeutic relevance of constitutive activity of 5-HT 2A and 5-HT 2C receptors for the treatment of depression. Prog. Brain Res. 172, 287–305. doi: 10.1016/S0079-6123(08)00914-X
Berridge, K. C., and Kringelbach, M. L. (2008). Affective neuroscience of pleasure: reward in humans and animals. Psychopharmacology (Berl.) 199, 457–480. doi: 10.1007/s00213-008-1099-6
Bland, S. T., Hargrave, D., Pepin, J. L., Amat, J., Watkins, L. R., and Maier, S. F. (2003a). Stressor controllability modulates stress-induced dopamine and serotonin efflux and morphine-induced serotonin efflux in the medial prefrontal cortex. Neuropsychopharmacology 28, 1589–1596. doi: 10.1038/sj.npp.1300206
Bland, S. T., Twining, C., Watkins, L. R., and Maier, S. F. (2003b). Stressor controllability modulates stress-induced serotonin but not dopamine efflux in the nucleus accumbens shell. Synapse 49, 206–208. doi: 10.1002/syn.10229
Boureau, Y. L., and Dayan, P. (2010). Opponency revisited: competition and cooperation between dopamine and serotonin. Neuropsychopharmacology 36, 74–97. doi: 10.1038/npp.2010.151
Bregonzio, C., Navarro, C. E., and Donoso, A. O. (1998). NMDA receptor antagonists block stress-induced prolactin release in female rats at estrus. Eur. J. Pharmacol. 350, 259–265. doi: 10.1016/S0014-2999(98)00251-9
Bromberg-Martin, E. S., Hikosaka, O., and Nakamura, K. (2010). Coding of task reward value in the dorsal raphe nucleus. J. Neurosci. 30, 6262–6272. doi: 10.1523/JNEUROSCI.0015-10.2010
Carey, R. J., Depalma, G., Damianopoulos, E., Muller, C. P., and Huston, J. P. (2004). The 5-HT1A receptor and behavioral stimulation in the rat: effects of 8-OHDPAT on spontaneous and cocaine-induced behavior. Psychopharmacology (Berl.) 177, 46–54. doi: 10.1007/s00213-004-1917-4
Celada, P., Casanovas, J. M., Paez, X., and Artigas, F. (2002). Control of serotonergic neurons in the dorsal raphe nucleus by the lateral hypothalamus. Brain Res. 932, 79–90. doi: 10.1016/S0006-8993(02)02284-9
Chamberlain, S. R., Muller, U., Blackwell, A. D., Clark, L., Robbins, T. W., and Sahakian, B. J. (2006). Neurochemical modulation of response inhibition and probabilistic learning in humans. Science 311, 861–863. doi: 10.1126/science.1121218
Chen, J. P., Van Praag, H. M., and Gardner, E. L. (1991). Activation of 5-HT3 receptor by 1-phenylbiguanide increases dopamine release in the rat nucleus accumbens. Brain Res. 543, 354–357. doi: 10.1016/0006-8993(91)90050-6
Clemett, D. A., Punhani, T., Duxon, M. S., Blackburn, T. P., and Fone, K. C. (2000). Immunohistochemical localisation of the 5-HT2C receptor protein in the rat CNS. Neuropharmacology 39, 123–132. doi: 10.1016/S0028-3908(99)00086-6
Commons, K. G., Connolley, K. R., and Valentino, R. J. (2003). A neurochemically distinct dorsal raphe-limbic circuit with a potential role in affective disorders. Neuropsychopharmacology 28, 206–215. doi: 10.1038/sj.npp.1300045
Cools, R., Calder, A. J., Lawrence, A. D., Clark, L., Bullmore, E., and Robbins, T. W. (2005). Individual differences in threat sensitivity predict serotonergic modulation of amygdala response to fearful faces. Psychopharmacology (Berl.) 180, 670–679. doi: 10.1007/s00213-005-2215-5
Cools, R., Nakamura, K., and Daw, N. D. (2011). Serotonin and dopamine: unifying affective, activational, and decision functions. Neuropsychopharmacology 36, 98–113. doi: 10.1038/npp.2010.121
Cools, R., Roberts, A. C., and Robbins, T. W. (2008). Serotoninergic regulation of emotional and behavioural control processes. Trends Cogn. Sci. 12, 31–40. doi: 10.1016/j.tics.2007.10.011
Cornea-Hebert, V., Riad, M., Wu, C., Singh, S. K., and Descarries, L. (1999). Cellular and subcellular distribution of the serotonin 5-HT2A receptor in the central nervous system of adult rat. J. Comp. Neurol. 409, 187–209.
Corvaja, N., Doucet, G., and Bolam, J. P. (1993). Ultrastructure and synaptic targets of the raphe-nigral projection in the rat. Neuroscience 55, 417–427. doi: 10.1016/0306-4522(93)90510-M
Crockett, M. J., Clark, L., and Robbins, T. W. (2009). Reconciling the role of serotonin in behavioral inhibition and aversion: acute tryptophan depletion abolishes punishment-induced inhibition in humans. J. Neurosci. 29, 11993–11999. doi: 10.1523/JNEUROSCI.2513-09.2009
Crockett, M. J., Clark, L., Apergis-Schoute, A. M., Morein-Zamir, S., and Robbins, T. W. (2012). Serotonin modulates the effects of Pavlovian aversive predictions on response vigor. Neuropsycho pharmacology 37, 2244–2252. doi: 10.1038/npp.2012.75
Cummings, S., Elde, R., Ells, J., and Lindall, A. (1983). Corticotropin-releasing factor immunoreactivity is widely distributed within the central nervous system of the rat: an immunohistochemical study. J. Neurosci. 3, 1355–1368.
Dahlstroem, A., and Fuxe, K. (1964). Evidence for the existence of monoamine-containing neurons in the central nervous system. I. demonstration of monoamines in the cell bodies of brain stem neurons. Acta Physiol. Scand. Suppl. SUPPL 232:231–255.
Daw, N. D., Kakade, S., and Dayan, P. (2002). Opponent interactions between serotonin and dopamine. Neural Netw. 15, 603–616. doi: 10.1016/S0893-6080(02)00052-7
Dayan, P., and Huys, Q. J. (2008). Serotonin, inhibition, and negative mood. PLoS Comput. Biol. 4:e4. doi: 10.1371/journal.pcbi.0040004
Dayan, P., and Huys, Q. J. (2009). Serotonin in affective control. Annu. Rev. Neurosci. 32, 95–126. doi: 10.1146/annurev.neuro.051508.135607
De Deurwaerdere, P., Stinus, L., and Spampinato, U. (1998). Opposite change of in vivo dopamine release in the rat nucleus accumbens and striatum that follows electrical stimulation of dorsal raphe nucleus: role of 5-HT3 receptors. J. Neurosci. 18, 6528–6538.
De Olmos, J., and Heimer, L. (1980). Double and triple labeling of neurons with fluorescent substances; the study of collateral pathways in the ascending raphe system. Neurosci. Lett. 19, 7–12. doi: 10.1016/0304-3940(80)90247-5
Deakin, J. F. (1991). Depression and 5HT. Int. Clin. Psychopharmacol. 6(Suppl 3), 23–28. discussion: 29–31. doi: 10.1097/00004850-199112003-00002
Di Giovanni, G., De Deurwaerdere, P., Di Mascio, M., Di Matteo, V., Esposito, E., and Spampinato, U. (1999). Selective blockade of serotonin-2C/2B receptors enhances mesolimbic and mesostriatal dopaminergic function: a combined in vivo electrophysiological and microdialysis study. Neuroscience 91, 587–597. doi: 10.1016/S0306-4522(98)00655-1
Di Giovanni, G., Di Matteo, V., Di Mascio, M., and Esposito, E. (2000). Preferential modulation of mesolimbic vs. nigrostriatal dopaminergic function by serotonin(2C/2B) receptor agonists: a combined in vivo electrophysiological and microdialysis study. Synapse 35, 53–61.
Di Giovanni, G., Esposito, E., and Di Matteo, V. (2010). Role of serotonin in central dopamine dysfunction. CNS Neurosci. Ther. 16, 179–194. doi: 10.1111/j.1755-5949.2010.00135.x
Doya, K. (2002). Metalearning and neuromodulation. Neural Netw. 15, 495–506. doi: 10.1016/S0893-6080(02)00044-8
Dray, A., Gonye, T. J., Oakley, N. R., and Tanner, T. (1976). Evidence for the existence of a raphe projection to the substantia nigra in rat. Brain Res. 113, 45–57. doi: 10.1016/0006-8993(76)90005-6
Eberle-Wang, K., Lucki, I., and Chesselet, M. F. (1996). A role for the subthalamic nucleus in 5-HT2C-induced oral dyskinesia. Neuroscience 72, 117–128. doi: 10.1016/0306-4522(95)00548-X
Eberle-Wang, K., Mikeladze, Z., Uryu, K., and Chesselet, M. F. (1997). Pattern of expression of the serotonin2C receptor messenger RNA in the basal ganglia of adult rats. J. Comp. Neurol. 384, 233–247.
Evers, E. A., Cools, R., Clark, L., Van Der Veen, F. M., Jolles, J., Sahakian, B. J., et al. (2005). Serotonergic modulation of prefrontal cortex during negative feedback in probabilistic reversal learning. Neuropsychopharmacology 30, 1138–1147. doi: 10.1038/sj.npp.1300663
Ferraro, G., Montalbano, M. E., Sardo, P., and La Grutta, V. (1997). Lateral habenula and hippocampus: a complex interaction raphe cells-mediated. J. Neural. Transm. 104, 615–631. doi: 10.1007/BF01291880
Ferre, S., and Artigas, F. (1993). Dopamine D2 receptor-mediated regulation of serotonin extracellular concentration in the dorsal raphe nucleus of freely moving rats. J. Neurochem. 61, 772–775. doi: 10.1111/j.1471-4159.1993.tb02187.x
Fibiger, H. C., and Miller, J. J. (1977). An anatomical and electrophysiological investigation of the serotonergic projection from the dorsal raphe nuclues to the substantia nigra in the rat. Neuroscience 2, 975–987. doi: 10.1016/0306-4522(77)90120-8
Fletcher, P. J. (1991). Dopamine receptor blockade in nucleus accumbens or caudate nucleus differentially affects feeding induced by 8-OH-DPAT injected into dorsal or median raphe. Brain Res. 552, 181–189. doi: 10.1016/0006-8993(91)90082-7
Fletcher, P. J. (1995). Effects of d-fenfluramine and metergoline on responding for conditioned reward and the response potentiating effect of nucleus accumbens d-amphetamine. Psychopharmacology (Berl.) 118, 155–163. doi: 10.1007/BF02245834
Fletcher, P. J. (1996). Injection of 5-HT into the nucleus accumbens reduces the effects of d-amphetamine on responding for conditioned reward. Psychopharmacology (Berl.) 126, 62–69. doi: 10.1007/BF02246412
Fletcher, P. J., and Davies, M. (1990). Dorsal raphe microinjection of 5-HT and indirect 5-HT agonists induces feeding in rats. Eur. J. Pharmacol. 184, 265–271. doi: 10.1016/0014-2999(90)90618-G
Fletcher, P. J., and Korth, K. M. (1999). Activation of 5-HT1B receptors in the nucleus accumbens reduces amphetamine-induced enhancement of responding for conditioned reward. Psychopharmacology (Berl.) 142, 165–174. doi: 10.1007/s002130050876
Fletcher, P. J., Korth, K. M., and Chambers, J. W. (1999). Selective destruction of brain serotonin neurons by 5,7-dihydroxytryptamine increases responding for a conditioned reward. Psychopharmacology (Berl.) 147, 291–299. doi: 10.1007/s002130051170
Fletcher, P. J., Ming, Z. H., and Higgins, G. A. (1993). Conditioned place preference induced by microinjection of 8-OH-DPAT into the dorsal or median raphe nucleus. Psychopharmacology (Berl.) 113, 31–36. doi: 10.1007/BF02244330
Fletcher, P. J., Tampakeras, M., and Yeomans, J. S. (1995). Median raphe injections of 8-OH-DPAT lower frequency thresholds for lateral hypothalamic self-stimulation. Pharmacol. Biochem. Behav. 52, 65–71. doi: 10.1016/0091-3057(94)00441-K
Freedman, L. J., and Shi, C. (2001). Monoaminergic innervation of the macaque extended amygdala. Neuroscience 104, 1067–1084. doi: 10.1016/S0306-4522(01)00157-9
Galloway, M. P., Suchowski, C. S., Keegan, M. J., and Hjorth, S. (1993). Local infusion of the selective 5HT-1b agonist CP-93,129 facilitates striatal dopamine release in vivo. Synapse 15, 90–92. doi: 10.1002/syn.890150109
Gervais, J., and Rouillard, C. (2000). Dorsal raphe stimulation differentially modulates dopaminergic neurons in the ventral tegmental area and substantia nigra. Synapse 35, 281–291.
Gobert, A., Rivet, J. M., Lejeune, F., Newman-Tancredi, A., Adhumeau-Auclair, A., Nicolas, J. P., et al. (2000). Serotonin(2C) receptors tonically suppress the activity of mesocortical dopaminergic and adrenergic, but not serotonergic, pathways: a combined dialysis and electrophysiological analysis in the rat. Synapse 36, 205–221.
Gozlan, H., El Mestikawy, S., Pichat, L., Glowinski, J., and Hamon, M. (1983). Identification of presynaptic serotonin autoreceptors using a new ligand: 3H-PAT. Nature 305, 140–142. doi: 10.1038/305140a0
Graeff, F. G., Guimaraes, F. S., De Andrade, T. G., and Deakin, J. F. (1996). Role of 5-HT in stress, anxiety, and depression. Pharmacol. Biochem. Behav. 54, 129–141. doi: 10.1016/0091-3057(95)02135-3
Grahn, R. E., Will, M. J., Hammack, S. E., Maswood, S., Mcqueen, M. B., Watkins, L. R., et al. (1999). Activation of serotonin-immunoreactive cells in the dorsal raphe nucleus in rats exposed to an uncontrollable stressor. Brain Res. 826, 35–43. doi: 10.1016/S0006-8993(99)01208-1
Haber, S. N. (2003). The primate basal ganglia: parallel and integrative networks. J. Chem. Neuroanat. 26, 317–330. doi: 10.1016/j.jchemneu.2003.10.003
Haj-Dahmane, S. (2001). D2-like dopamine receptor activation excites rat dorsal raphe 5-HT neurons in vitro. Eur. J. Neurosci. 14, 125–134. doi: 10.1046/j.0953-816x.2001.01616.x
Hajos, M., Gartside, S. E., Varga, V., and Sharp, T. (2003). In vivo inhibition of neuronal activity in the rat ventromedial prefrontal cortex by midbrain-raphe nuclei: role of 5-HT1A receptors. Neuropharmacology 45, 72–81. doi: 10.1016/S0028-3908(03)00139-4
Hajos, M., Richards, C. D., Szekely, A. D., and Sharp, T. (1998). An electrophysiological and neuroanatomical study of the medial prefrontal cortical projection to the midbrain raphe nuclei in the rat. Neuroscience 87, 95–108. doi: 10.1016/S0306-4522(98)00157-2
Hariri, A. R., and Holmes, A. (2006). Genetics of emotional regulation: the role of the serotonin transporter in neural function. Trends Cogn. Sci. 10, 182–191. doi: 10.1016/j.tics.2006.02.011
Hariri, A. R., Mattay, V. S., Tessitore, A., Kolachana, B., Fera, F., Goldman, D., et al. (2002). Serotonin transporter genetic variation and the response of the human amygdala. Science 297, 400–403. doi: 10.1126/science.1071829
Harris, G. C., and Aston-Jones, G. (2006). Arousal and reward: a dichotomy in orexin function. Trends Neurosci. 29, 571–577. doi: 10.1016/j.tins.2006.08.002
Harris, G. C., Wimmer, M., and Aston-Jones, G. (2005). A role for lateral hypothalamic orexin neurons in reward seeking. Nature 437, 556–559. doi: 10.1038/nature04071
Hashimoto, S., Inoue, T., and Koyama, T. (1999). Effects of conditioned fear stress on serotonin neurotransmission and freezing behavior in rats. Eur. J. Pharmacol. 378, 23–30. doi: 10.1016/S0014-2999(99)00441-0
Hayes, D. J., Mosher, T. M., and Greenshaw, A. J. (2009). Differential effects of 5-HT2C receptor activation by WAY 161503 on nicotine-induced place conditioning and locomotor activity in rats. Behav. Brain Res. 197, 323–330. doi: 10.1016/j.bbr.2008.08.034
Herkenham, M., and Nauta, W. J. (1979). Efferent connections of the habenular nuclei in the rat. J. Comp. Neurol. 187, 19–47. doi: 10.1002/cne.901870103
Herve, D., Pickel, V. M., Joh, T. H., and Beaudet, A. (1987). Serotonin axon terminals in the ventral tegmental area of the rat: fine structure and synaptic input to dopaminergic neurons. Brain Res. 435, 71–83. doi: 10.1016/0006-8993(87)91588-5
Higgins, G. A., and Fletcher, P. J. (2003). Serotonin and drug reward: focus on 5-HT2C receptors. Eur. J. Pharmacol. 480, 151–162. doi: 10.1016/j.ejphar.2003.08.102
Hoebel, B. G., Hernandez, L., Schwartz, D. H., Mark, G. P., and Hunter, G. A. (1989). Microdialysis studies of brain norepinephrine, serotonin, and dopamine release during ingestive behavior. Theoretical and clinical implications. Ann. N.Y. Acad. Sci. 575, 171–191. discussion: 192–173. doi: 10.1111/j.1749-6632.1989.tb53242.x
Hong, S., and Hikosaka, O. (2008). The globus pallidus sends reward-related signals to the lateral habenula. Neuron 60, 720–729. doi: 10.1016/j.neuron.2008.09.035
Imai, H., Steindler, D. A., and Kitai, S. T. (1986). The organization of divergent axonal projections from the midbrain raphe nuclei in the rat. J. Comp. Neurol. 243, 363–380. doi: 10.1002/cne.902430307
Inaba, K., Mizuhiki, T., Setogawa, T., Toda, K., Richmond, B. J., and Shidara, M. (2013). Neurons in monkey dorsal raphe nucleus code beginning and progress of step-by-step schedule, reward expectation, and amount of reward outcome in the reward schedule task. J. Neurosci. 33, 3477–3491. doi: 10.1523/JNEUROSCI.4388-12.2013
Inoue, T., Li, X. B., Abekawa, T., Kitaichi, Y., Izumi, T., Nakagawa, S., et al. (2004). Selective serotonin reuptake inhibitor reduces conditioned fear through its effect in the amygdala. Eur. J. Pharmacol. 497, 311–316. doi: 10.1016/j.ejphar.2004.06.061
Jacobs, B. L., and Azmitia, E. C. (1992). Structure and function of the brain serotonin system. Physiol. Rev. 72, 165–229.
Jakab, R. L., and Goldman-Rakic, P. S. (1998). 5-Hydroxytryptamine2A serotonin receptors in the primate cerebral cortex: possible site of action of hallucinogenic and antipsychotic drugs in pyramidal cell apical dendrites. Proc. Natl. Acad. Sci. U.S.A. 95, 735–740. doi: 10.1073/pnas.95.2.735
Jankowski, M. P., and Sesack, S. R. (2004). Prefrontal cortical projections to the rat dorsal raphe nucleus: ultrastructural features and associations with serotonin and gamma-aminobutyric acid neurons. J. Comp. Neurol. 468, 518–529. doi: 10.1002/cne.10976
Ji, S. P., Zhang, Y., Van Cleemput, J., Jiang, W., Liao, M., Li, L., et al. (2006). Disruption of PTEN coupling with 5-HT2C receptors suppresses behavioral responses induced by drugs of abuse. Nat. Med. 12, 324–329. doi: 10.1038/nm1349
Jiang, L. H., Ashby, C. R. Jr., Kasser, R. J., and Wang, R. Y. (1990). The effect of intraventricular administration of the 5-HT3 receptor agonist 2-methylserotonin on the release of dopamine in the nucleus accumbens: an in vivo chronocoulometric study. Brain Res. 513, 156–160. doi: 10.1016/0006-8993(90)91103-N
Kalen, P., Strecker, R. E., Rosengren, E., and Bjorklund, A. (1988). Endogenous release of neuronal serotonin and 5-hydroxyindoleacetic acid in the caudate-putamen of the rat as revealed by intracerebral dialysis coupled to high-performance liquid chromatography with fluorimetric detection. J. Neurochem. 51, 1422–1435. doi: 10.1111/j.1471-4159.1988.tb01107.x
Kalen, P., Strecker, R. E., Rosengren, E., and Bjorklund, A. (1989). Regulation of striatal serotonin release by the lateral habenula-dorsal raphe pathway in the rat as demonstrated by in vivo microdialysis: role of excitatory amino acids and GABA. Brain Res. 492, 187–202. doi: 10.1016/0006-8993(89)90901-3
Kawagoe, R., Takikawa, Y., and Hikosaka, O. (2004). Reward-predicting activity of dopamine and caudate neurons—a possible mechanism of motivational control of saccadic eye movement. J. Neurophysiol. 91, 1013–1024. doi: 10.1152/jn.00721.2003
Kirby, L. G., Allen, A. R., and Lucki, I. (1995). Regional differences in the effects of forced swimming on extracellular levels of 5-hydroxytryptamine and 5-hydroxyindoleacetic acid. Brain Res. 682, 189–196. doi: 10.1016/0006-8993(95)00349-U
Kirby, L. G., Rice, K. C., and Valentino, R. J. (2000). Effects of corticotropin-releasing factor on neuronal activity in the serotonergic dorsal raphe nucleus. Neuropsychopharmacology 22, 148–162. doi: 10.1016/S0893-133X(99)00093-7
Kitahama, K., Nagatsu, I., Geffard, M., and Maeda, T. (2000). Distribution of dopamine-immunoreactive fibers in the rat brainstem. J. Chem. Neuroanat. 18, 1–9. doi: 10.1016/S0891-0618(99)00047-2
Kohler, C., Chan-Palay, V., and Steinbusch, H. (1982). The distribution and origin of serotonin-containing fibers in the septal area: a combined immunohistochemical and fluorescent retrograde tracing study in the rat. J. Comp. Neurol. 209, 91–111. doi: 10.1002/cne.902090109
Lavoie, B., and Parent, A. (1990). Immunohistochemical study of the serotoninergic innervation of the basal ganglia in the squirrel monkey. J. Comp. Neurol. 299, 1–16. doi: 10.1002/cne.902990102
Leccese, A. P., and Lyness, W. H. (1984). The effects of putative 5-hydroxytryptamine receptor active agents on D-amphetamine self-administration in controls and rats with 5,7-dihydroxytryptamine median forebrain bundle lesions. Brain Res. 303, 153–162. doi: 10.1016/0006-8993(84)90223-3
Ledoux, J. E. (2000). Emotion circuits in the brain. Annu. Rev. Neurosci. 23, 155–184. doi: 10.1146/annurev.neuro.23.1.155
Lee, H. S., Eum, Y. J., Jo, S. M., and Waterhouse, B. D. (2007). Projection patterns from the amygdaloid nuclear complex to subdivisions of the dorsal raphe nucleus in the rat. Brain Res. 1143, 116–125. doi: 10.1016/j.brainres.2007.01.081
Lee, H. S., Park, S. H., Song, W. C., and Waterhouse, B. D. (2005). Retrograde study of hypocretin-1 (orexin-A) projections to subdivisions of the dorsal raphe nucleus in the rat. Brain Res. 1059, 35–45. doi: 10.1016/j.brainres.2005.08.016
Lee, S. B., Lee, H. S., and Waterhouse, B. D. (2008). The collateral projection from the dorsal raphe nucleus to whisker-related, trigeminal sensory and facial motor systems in the rat. Brain Res. 1214, 11–22. doi: 10.1016/j.brainres.2008.04.003
Li, Y., Dalphin, N., and Hyland, B. I. (2013). Association with reward negatively modulates short latency phasic conditioned responses of dorsal raphe nucleus neurons in freely moving rats. J. Neurosci. 33, 5065–5078. doi: 10.1523/JNEUROSCI.5679-12.2013
Liu, Z. H., and Ikemoto, S. (2007). The midbrain raphe nuclei mediate primary reinforcement via GABA(A) receptors. Eur. J. Neurosci. 25, 735–743. doi: 10.1111/j.1460-9568.2007.05319.x
Lowry, C. A. (2002). Functional subsets of serotonergic neurones: implications for control of the hypothalamic-pituitary-adrenal axis. J. Neuroendocrinol. 14, 911–923. doi: 10.1046/j.1365-2826.2002.00861.x
Lowry, C. A., Hale, M. W., Evans, A. K., Heerkens, J., Staub, D. R., Gasser, P. J., et al. (2008). Serotonergic systems, anxiety, and affective disorder: focus on the dorsomedial part of the dorsal raphe nucleus. Ann. N.Y. Acad. Sci. 1148, 86–94. doi: 10.1196/annals.1410.004
Maier, S. F., and Watkins, L. R. (2005). Stressor controllability and learned helplessness: the roles of the dorsal raphe nucleus, serotonin, and corticotropin-releasing factor. Neurosci. Biobehav. Rev. 29, 829–841. doi: 10.1016/j.neubiorev.2005.03.021
Maier, S. F., Grahn, R. E., and Watkins, L. R. (1995). 8-OH-DPAT microinjected in the region of the dorsal raphe nucleus blocks and reverses the enhancement of fear conditioning and interference with escape produced by exposure to inescapable shock. Behav. Neurosci. 109, 404–412. doi: 10.1037/0735-7044.109.3.404
Maier, S. F., Kalman, B. A., and Grahn, R. E. (1994). Chlordiazepoxide microinjected into the region of the dorsal raphe nucleus eliminates the interference with escape responding produced by inescapable shock whether administered before inescapable shock or escape testing. Behav. Neurosci. 108, 121–130. doi: 10.1037/0735-7044.108.1.121
Mansour, A., Meador-Woodruff, J. H., Bunzow, J. R., Civelli, O., Akil, H., and Watson, S. J. (1990). Localization of dopamine D2 receptor mRNA and D1 and D2 receptor binding in the rat brain and pituitary: an in situ hybridization-receptor autoradiographic analysis. J. Neurosci. 10, 2587–2600.
Mantz, J., Godbout, R., Tassin, J. P., Glowinski, J., and Thierry, A. M. (1990). Inhibition of spontaneous and evoked unit activity in the rat medial prefrontal cortex by mesencephalic raphe nuclei. Brain Res. 524, 22–30. doi: 10.1016/0006-8993(90)90487-V
Marsh, D. M., Dougherty, D. M., Moeller, F. G., Swann, A. C., and Spiga, R. (2002). Laboratory-measured aggressive behavior of women: acute tryptophan depletion and augmentation. Neuropsychopharmacology 26, 660–671. doi: 10.1016/S0893-133X(01)00369-4
Maswood, S., Barter, J. E., Watkins, L. R., and Maier, S. F. (1998). Exposure to inescapable but not escapable shock increases extracellular levels of 5-HT in the dorsal raphe nucleus of the rat. Brain Res. 783, 115–120. doi: 10.1016/S0006-8993(97)01313-9
Matsumoto, M., and Hikosaka, O. (2007). Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447, 1111–1115. doi: 10.1038/nature05860
Matsumoto, M., and Hikosaka, O. (2009). Representation of negative motivational value in the primate lateral habenula. Nat. Neurosci. 12, 77–84. doi: 10.1038/nn.2233
Mendlin, A., Martin, F. J., and Jacobs, B. L. (1999). Dopaminergic input is required for increases in serotonin output produced by behavioral activation: an in vivo microdialysis study in rat forebrain. Neuroscience 93, 897–905. doi: 10.1016/S0306-4522(99)00213-4
Michelsen, K. A., Schmitz, C., and Steinbusch, H. W. (2007). The dorsal raphe nucleus—from silver stainings to a role in depression. Brain Res. Rev. 55, 329–342. doi: 10.1016/j.brainresrev.2007.01.002
Miczek, K. A., De Almeida, R. M., Kravitz, E. A., Rissman, E. F., De Boer, S. F., and Raine, A. (2007). Neurobiology of escalated aggression and violence. J. Neurosci. 27, 11803–11806. doi: 10.1523/JNEUROSCI.3500-07.2007
Mieda, M., and Yanagisawa, M. (2002). Sleep, feeding, and neuropeptides: roles of orexins and orexin receptors. Curr. Opin. Neurobiol. 12, 339–345. doi: 10.1016/S0959-4388(02)00331-8
Miliaressis, E. (1977). Serotonergic basis of reward in median raphe of the rat. Pharmacol. Biochem. Behav. 7, 177–180. doi: 10.1016/0091-3057(77)90204-0
Miliaressis, E., Bouchard, A., and Jacobowitz, D. M. (1975). Strong positive reward in median raphe: specific inhibition by para-chlorophenylalanine. Brain Res. 98, 194–201. doi: 10.1016/0006-8993(75)90521-1
Miyazaki, K. W., Miyazaki, K., and Doya, K. (2010). Activation of the central serotonergic system in response to delayed but not omitted rewards. Eur. J. Neurosci. 33, 153–160. doi: 10.1111/j.1460-9568.2010.07480.x
Miyazaki, K., Miyazaki, K. W., and Doya, K. (2011). Activation of dorsal raphe serotonin neurons underlies waiting for delayed rewards. J. Neurosci. 31, 469–479. doi: 10.1523/JNEUROSCI.3714-10.2011
Molineaux, S. M., Jessell, T. M., Axel, R., and Julius, D. (1989). 5-HT1c receptor is a prominent serotonin receptor subtype in the central nervous system. Proc. Natl. Acad. Sci. U.S.A. 86, 6793–6797. doi: 10.1073/pnas.86.17.6793
Molliver, M. E. (1987). Serotonergic neuronal systems: what their anatomic organization tells us about function. J. Clin. Psychopharmacol. 7, S3–S23. doi: 10.1097/00004714-198712001-00002
Mori, S., Matsuura, T., Takino, T., and Sano, Y. (1987). Light and electron microscopic immunohistochemical studies of serotonin nerve fibers in the substantia nigra of the rat, cat and monkey. Anat. Embryol. (Berl.) 176, 13–18. doi: 10.1007/BF00309747
Mosher, T., Hayes, D., and Greenshaw, A. (2005). Differential effects of 5-HT2C receptor ligands on place conditioning and locomotor activity in rats. Eur. J. Pharmacol. 515, 107–116. doi: 10.1016/j.ejphar.2005.03.041
Moukhles, H., Bosler, O., Bolam, J. P., Vallee, A., Umbriaco, D., Geffard, M., et al. (1997). Quantitative and morphometric data indicate precise cellular interactions between serotonin terminals and postsynaptic targets in rat substantia nigra. Neuroscience 76, 1159–1171. doi: 10.1016/S0306-4522(96)00452-6
Muscat, R., Montgomery, A. M., and Willner, P. (1989). Blockade of 8-OH-DPAT-induced feeding by dopamine antagonists. Psychopharmacology (Berl.) 99, 402–408. doi: 10.1007/BF00445567
Muschamp, J. W., Dominguez, J. M., Sato, S. M., Shen, R. Y., and Hull, E. M. (2007). A role for hypocretin (orexin) in male sexual behavior. J. Neurosci. 27, 2837–2845. doi: 10.1523/JNEUROSCI.4121-06.2007
Nakamura, K., Matsumoto, M., and Hikosaka, O. (2008). Reward-dependent modulation of neuronal activity in the primate dorsal raphe nucleus. J. Neurosci. 28, 5331–5343. doi: 10.1523/JNEUROSCI.0021-08.2008
Nambu, T., Sakurai, T., Mizukami, K., Hosoya, Y., Yanagisawa, M., and Goto, K. (1999). Distribution of orexin neurons in the adult rat brain. Brain Res. 827, 243–260. doi: 10.1016/S0006-8993(99)01336-0
Nedergaard, S., Bolam, J. P., and Greenfield, S. A. (1988). Facilitation of a dendritic calcium conductance by 5-hydroxytryptamine in the substantia nigra. Nature 333, 174–177. doi: 10.1038/333174a0
Niki, H., and Watanabe, M. (1979). Prefrontal and cingulate unit activity during timing behavior in the monkey. Brain Res. 171, 213–224. doi: 10.1016/0006-8993(79)90328-7
Nishikawa, T., and Scatton, B. (1984). The inhibitory GABAergic influence on striatal serotonergic neurons depends upon the habenulo-raphe pathways. Brain Res. 304, 157–161. doi: 10.1016/0006-8993(84)90873-4
Nishikawa, T., and Scatton, B. (1985). Inhibitory influence of GABA on central serotonergic transmission. Involvement of the habenulo-raphe pathways in the GABAergic inhibition of ascending cerebral serotonergic neurons. Brain Res. 331, 81–90. doi: 10.1016/0006-8993(85)90717-6
Nishikawa, T., and Scatton, B. (1986). Neuroanatomical site of the inhibitory influence of anxiolytic drugs on central serotonergic transmission. Brain Res. 371, 123–132. doi: 10.1016/0006-8993(86)90817-6
Nishikawa, T., Fage, D., and Scatton, B. (1986). Evidence for, and nature of, the tonic inhibitory influence of habenulointerpeduncular pathways upon cerebral dopaminergic transmission in the rat. Brain Res. 373, 324–336. doi: 10.1016/0006-8993(86)90347-1
Pasquier, D. A., Anderson, C., Forbes, W. B., and Morgane, P. J. (1976). Horseradish peroxidase tracing of the lateral habenular-midbrain raphe nuclei connections in the rat. Brain Res. Bull. 1, 443–451. doi: 10.1016/0361-9230(76)90114-3
Paz, R., Natan, C., Boraud, T., Bergman, H., and Vaadia, E. (2005). Emerging patterns of neuronal responses in supplementary and primary motor areas during sensorimotor adaptation. J Neurosci 25, 10941–10951. doi: 10.1523/JNEUROSCI.0164-05.2005
Pazos, A., Cortes, R., and Palacios, J. M. (1985). Quantitative autoradiographic mapping of serotonin receptors in the rat brain. II. Serotonin-2 receptors. Brain Res. 346, 231–249. doi: 10.1016/0006-8993(85)90857-1
Pehek, E. A., Nocjar, C., Roth, B. L., Byrd, T. A., and Mabrouk, O. S. (2006). Evidence for the preferential involvement of 5-HT2A serotonin receptors in stress- and drug-induced dopamine release in the rat medial prefrontal cortex. Neuropsychopharmacology 31, 265–277. doi: 10.1038/sj.npp.1300819
Pentkowski, N. S., Duke, F. D., Weber, S. M., Pockros, L. A., Teer, A. P., Hamilton, E. C., et al. (2010). Stimulation of medial prefrontal cortex serotonin 2C [5-HT(2C)] receptors attenuates cocaine-seeking behavior. Neuropsychopharmacology 35, 2037–2048. doi: 10.1038/npp.2010.72
Petrov, T., Krukoff, T. L., and Jhamandas, J. H. (1994). Chemically defined collateral projections from the pons to the central nucleus of the amygdala and hypothalamic paraventricular nucleus in the rat. Cell Tissue Res. 277, 289–295. doi: 10.1007/BF00327776
Peyron, C., Luppi, P. H., Kitahama, K., Fort, P., Hermann, D. M., and Jouvet, M. (1995). Origin of the dopaminergic innervation of the rat dorsal raphe nucleus. Neuroreport 6, 2527–2531. doi: 10.1097/00001756-199512150-00019
Peyron, C., Petit, J. M., Rampon, C., Jouvet, M., and Luppi, P. H. (1998). Forebrain afferents to the rat dorsal raphe nucleus demonstrated by retrograde and anterograde tracing methods. Neuroscience 82, 443–468. doi: 10.1016/S0306-4522(97)00268-6
Pezzone, M. A., Lee, W. S., Hoffman, G. E., Pezzone, K. M., and Rabin, B. S. (1993). Activation of brainstem catecholaminergic neurons by conditioned and unconditioned aversive stimuli as revealed by c-Fos immunoreactivity. Brain Res. 608, 310–318. doi: 10.1016/0006-8993(93)91472-5
Pompeiano, M., Palacios, J. M., and Mengod, G. (1994). Distribution of the serotonin 5-HT2 receptor family mRNAs: comparison between 5-HT2A and 5-HT2C receptors. Brain Res. Mol. Brain Res. 23, 163–178. doi: 10.1016/0169-328X(94)90223-2
Prisco, S., Pagannone, S., and Esposito, E. (1994). Serotonin-dopamine interaction in the rat ventral tegmental area: an electrophysiological study in vivo. J. Pharmacol. Exp. Ther. 271, 83–90.
Puig, M. V., Artigas, F., and Celada, P. (2005). Modulation of the activity of pyramidal neurons in rat prefrontal cortex by raphe stimulation in vivo: involvement of serotonin and GABA. Cereb. Cortex 15, 1–14. doi: 10.1093/cercor/bhh104
Ranade, S. P., and Mainen, Z. F. (2009). Transient firing of dorsal raphe neurons encodes diverse and specific sensory, motor, and reward events. J. Neurophysiol. 102, 3026–3037. doi: 10.1152/jn.00507.2009
Reisine, T. D., Soubrie, P., Artaud, F., and Glowinski, J. (1982). Involvement of lateral habenula-dorsal raphe neurons in the differential regulation of striatal and nigral serotonergic transmission cats. J. Neurosci. 2, 1062–1071.
Richmond, B. J., and Optican, L. M. (1987). Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex. II. Quantification of response waveform. J. Neurophysiol. 57, 147–161.
Robinson, O. J., Overstreet, C., Charney, D. R., Vytal, K., and Grillon, C. (2013). Stress increases aversive prediction error signal in the ventral striatum. Proc. Natl. Acad. Sci. U.S.A. 110, 4129–4133. doi: 10.1073/pnas.1213923110
Roiser, J. P., Levy, J., Fromm, S. J., Wang, H., Hasler, G., Sahakian, B. J., et al. (2008). The effect of acute tryptophan depletion on the neural correlates of emotional processing in healthy volunteers. Neuropsychopharmacology 33, 1992–2006. doi: 10.1038/sj.npp.1301581
Rompre, P. P., and Miliaressis, E. (1985). Pontine and mesencephalic substrates of self-stimulation. Brain Res. 359, 246–259. doi: 10.1016/0006-8993(85)91435-0
Sadikot, A. F., and Parent, A. (1990). The monoaminergic innervation of the amygdala in the squirrel monkey: an immunohistochemical study. Neuroscience 36, 431–447. doi: 10.1016/0306-4522(90)90439-B
Sakai, K., Salvert, D., Touret, M., and Jouvet, M. (1977). Afferent connections of the nucleus raphe dorsalis in the cat as visualized by the horseradish peroxidase technique. Brain Res. 137, 11–35. doi: 10.1016/0006-8993(77)91010-1
Scatton, B., Serrano, A., Rivot, J. P., and Nishikawa, T. (1984). Inhibitory GABAergic influence on striatal serotonergic transmission exerted in the dorsal raphe as revealed by in vivo voltammetry. Brain Res. 305, 343–352. doi: 10.1016/0006-8993(84)90440-2
Schultz, W. (1998). Predictive reward signal of dopamine neurons. J. Neurophysiol. Brain Res. Bull. 80, 1–27.
Schwartz, D. H., Hernandez, L., and Hoebel, B. G. (1990). Serotonin release in lateral and medial hypothalamus during feeding and its anticipation. Brain Res. Bull. 25, 797–802. doi: 10.1016/0361-9230(90)90173-W
Schweimer, J. V., and Ungless, M. A. (2010). Phasic responses in dorsal raphe serotonin neurons to noxious stimuli. Neuroscience 171, 1209–1215. doi: 10.1016/j.neuroscience.2010.09.058
Sesack, S. R., Deutch, A. Y., Roth, R. H., and Bunney, B. S. (1989). Topographical organization of the efferent projections of the medial prefrontal cortex in the rat: an anterograde tract-tracing study with Phaseolus vulgaris leucoagglutinin. J. Comp. Neurol. 290, 213–242. doi: 10.1002/cne.902900205
Solomon, R. L., and Corbit, J. D. (1974). An opponent-process theory of motivation. I. Temporal dynamics of affect. Psychol. Rev. 81, 119–145. doi: 10.1037/h0036128
Staub, D. R., Evans, A. K., and Lowry, C. A. (2006). Evidence supporting a role for corticotropin-releasing factor type 2 (CRF2) receptors in the regulation of subpopulations of serotonergic neurons. Brain Res. 1070, 77–89. doi: 10.1016/j.brainres.2005.10.096
Staub, D. R., Spiga, F., and Lowry, C. A. (2005). Urocortin 2 increases c-Fos expression in topographically organized subpopulations of serotonergic neurons in the rat dorsal raphe nucleus. Brain Res. 1044, 176–189. doi: 10.1016/j.brainres.2005.02.080
Steinbusch, H. W. (1981). Distribution of serotonin-immunoreactivity in the central nervous system of the rat-cell bodies and terminals. Neuroscience 6, 557–618. doi: 10.1016/0306-4522(81)90146-9
Stern, W. C., Johnson, A., Bronzino, J. D., and Morgane, P. J. (1979). Effects of electrical stimulation of the lateral habenula on single-unit activity of raphe neurons. Exp. Neurol. 65, 326–342. doi: 10.1016/0014-4886(79)90102-X
Stutzmann, G. E., and Ledoux, J. E. (1999). GABAergic antagonists block the inhibitory effects of serotonin in the lateral amygdala: a mechanism for modulation of sensory inputs related to fear conditioning. J. Neurosci. 19, RC8.
Stutzmann, G. E., Mcewen, B. S., and Ledoux, J. E. (1998). Serotonin modulation of sensory inputs to the lateral amygdala: dependency on corticosterone. J. Neurosci. 18, 9529–9538.
Takase, L. F., Nogueira, M. I., Baratta, M., Bland, S. T., Watkins, L. R., Maier, S. F., et al. (2004). Inescapable shock activates serotonergic neurons in all raphe nuclei of rat. Behav. Brain Res. 153, 233–239. doi: 10.1016/j.bbr.2003.12.020
Tanaka, S. C., Doya, K., Okada, G., Ueda, K., Okamoto, Y., and Yamawaki, S. (2004). Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat. Neurosci. 7, 887–893. doi: 10.1038/nn1279
Tao, R., Ma, Z., Mckenna, J. T., Thakkar, M. M., Winston, S., Strecker, R. E., et al. (2006). Differential effect of orexins (hypocretins) on serotonin release in the dorsal and median raphe nuclei of freely behaving rats. Neuroscience 141, 1101–1105. doi: 10.1016/j.neuroscience.2006.05.027
Tran-Nguyen, L. T., Baker, D. A., Grote, K. A., Solano, J., and Neisewander, J. L. (1999). Serotonin depletion attenuates cocaine-seeking behavior in rats. Psychopharmacology (Berl.) 146, 60–66. doi: 10.1007/s002130051088
Tran-Nguyen, L. T., Bellew, J. G., Grote, K. A., and Neisewander, J. L. (2001). Serotonin depletion attenuates cocaine seeking but enhances sucrose seeking and the effects of cocaine priming on reinstatement of cocaine seeking in rats. Psychopharmacology (Berl.) 157, 340–348. doi: 10.1007/s002130100822
Trent, F., and Tepper, J. M. (1991). Dorsal raphe stimulation modifies striatal-evoked antidromic invasion of nigral dopaminergic neurons in vivo. Exp. Brain Res. 84, 620–630. doi: 10.1007/BF00230974
Ugedo, L., Grenhoff, J., and Svensson, T. H. (1989). Ritanserin, a 5-HT2 receptor antagonist, activates midbrain dopamine neurons by blocking serotonergic inhibition. Psychopharmacology (Berl.) 98, 45–50. doi: 10.1007/BF00442004
Van Bockstaele, E. J., Biswas, A., and Pickel, V. M. (1993). Topography of serotonin neurons in the dorsal raphe nucleus that send axon collaterals to the rat prefrontal cortex and nucleus accumbens. Brain Res. 624, 188–198. doi: 10.1016/0006-8993(93)90077-Z
Van Bockstaele, E. J., Cestari, D. M., and Pickel, V. M. (1994). Synaptic structure and connectivity of serotonin terminals in the ventral tegmental area: potential sites for modulation of mesolimbic dopamine neurons. Brain Res. 647, 307–322. doi: 10.1016/0006-8993(94)91330-7
Van Der Kooy, D., and Hattori, T. (1980). Dorsal raphe cells with collateral projections to the caudate-putamen and substantia nigra: a fluorescent retrograde double labeling study in the rat. Brain Res. 186, 1–7. doi: 10.1016/0006-8993(80)90250-4
Varga, V., Kocsis, B., and Sharp, T. (2003). Electrophysiological evidence for convergence of inputs from the medial prefrontal cortex and lateral habenula on single neurons in the dorsal raphe nucleus. Eur. J. Neurosci. 17, 280–286. doi: 10.1046/j.1460-9568.2003.02465.x
Varga, V., Szekely, A. D., Csillag, A., Sharp, T., and Hajos, M. (2001). Evidence for a role of GABA interneurones in the cortical modulation of midbrain 5-hydroxytryptamine neurones. Neuroscience 106, 783–792. doi: 10.1016/S0306-4522(01)00294-9
Verge, D., Daval, G., Patey, A., Gozlan, H., El Mestikawy, S., and Hamon, M. (1985). Presynaptic 5-HT autoreceptors on serotonergic cell bodies and/or dendrites but not terminals are of the 5-HT1A subtype. Eur. J. Pharmacol. 113, 463–464. doi: 10.1016/0014-2999(85)90099-8
Vernikos-Danellis, J., Kellar, K. J., Kent, D., Gonzales, C., Berger, P. A., and Barchas, J. D. (1977). Serotonin involvement in pituitary-adrenal function. Ann. N.Y. Acad. Sci. 297, 518–526. doi: 10.1111/j.1749-6632.1977.tb41879.x
Vertes, R. P. (1991). A PHA-L analysis of ascending projections of the dorsal raphe nucleus in the rat. J. Comp. Neurol. 313, 643–668. doi: 10.1002/cne.903130409
Villar, M. J., Vitale, M. L., Hokfelt, T., and Verhofstad, A. A. (1988). Dorsal raphe serotoninergic branching neurons projecting both to the lateral geniculate body and superior colliculus: a combined retrograde tracing-immunohistochemical study in the rat. J. Comp. Neurol. 277, 126–140. doi: 10.1002/cne.902770109
Wang, R. Y., and Aghajanian, G. K. (1977a). Antidromically identified serotonergic neurons in the rat midbrain raphe: evidence for collateral inhibition. Brain Res. 132, 186–193. doi: 10.1016/0006-8993(77)90719-3
Wang, R. Y., and Aghajanian, G. K. (1977b). Inhibiton of neurons in the amygdala by dorsal raphe stimulation: mediation through a direct serotonergic pathway. Brain Res. 120, 85–102. doi: 10.1016/0006-8993(77)90499-1
Wang, R. Y., and Aghajanian, G. K. (1977c). Physiological evidence for habenula as major link between forebrain and midbrain raphe. Science 197, 89–91. doi: 10.1126/science.194312
Warden, M. R., Selimbeyoglu, A., Mirzabekov, J. J., Lo, M., Thompson, K. R., Kim, S. Y., et al. (2012). A prefrontal cortex-brainstem neuronal projection that controls response to behavioural challenge. Nature 492, 428–432.
Wirtshafter, D., Stratford, T. R., and Asin, K. E. (1987). Evidence that serotonergic projections to the substantia nigra in the rat arise in the dorsal, but not the median, raphe nucleus. Neurosci. Lett. 77, 261–266. doi: 10.1016/0304-3940(87)90509-X
Wright, D. E., Seroogy, K. B., Lundgren, K. H., Davis, B. M., and Jennes, L. (1995). Comparative localization of serotonin1A, 1C, and 2 receptor subtype mRNAs in rat brain. J. Comp. Neurol. 351, 357–373. doi: 10.1002/cne.903510304
Yan, Q. S., and Yan, S. E. (2001). Activation of 5-HT(1B/1D) receptors in the mesolimbic dopamine system increases dopamine release from the nucleus accumbens: a microdialysis study. Eur. J. Pharmacol. 418, 55–64. doi: 10.1016/S0014-2999(01)00913-X
Keywords: 5-HT, dopamine, raphé, saccade, primate, reinforcement, reward
Citation: Nakamura K (2013) The role of the dorsal raphé nucleus in reward-seeking behavior. Front. Integr. Neurosci. 7:60. doi: 10.3389/fnint.2013.00060
Received: 10 February 2013; Accepted: 30 July 2013;
Published online: 27 August 2013.
Edited by:
KongFatt Wong-Lin, University of Ulster, UKReviewed by:
Quentin Huys, University College London, UKAndrew J. Greenshaw, University of Alberta, Canada
Copyright © 2013 Nakamura. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Kae Nakamura, Department of Physiology, Kansai Medical University, 2-5-1 Shin-machi, Hirakata-city, Osaka, 570-1010, Japan e-mail: nakamkae@hirakata.kmu.ac.jp