# INDIVIDUAL DIFFERENCES: FROM NEUROBIOLOGICAL BASES TO NEW INSIGHT ON APPROACH AND AVOIDANCE BEHAVIOR

EDITED BY: Daniela Laricchiuta PUBLISHED IN: Frontiers in Systems Neuroscience

#### *Frontiers Copyright Statement*

*© Copyright 2007-2016 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.*

*The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.*

*Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.*

*Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.*

*As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.*

> *All copyright, and all rights therein, are protected by national and international copyright laws.*

*The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.*

ISSN 1664-8714 ISBN 978-2-88919-715-6 DOI 10.3389/978-2-88919-715-6

# About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

# Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

# Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

# What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# **INDIVIDUAL DIFFERENCES: FROM NEUROBIOLOGICAL BASES TO NEW INSIGHT ON APPROACH AND AVOIDANCE BEHAVIOR**

#### Topic Editor:

**Daniela Laricchiuta,** IRCCS Fondazione Santa Lucia and University "Sapienza" of Rome, Italy

Brain circuitries that mediate approach and avoidance behaviors

Figure modified from: Laricchiuta D and Petrosini L (2014) Individual differences in response to positive and negative stimuli: endocannabinoid-based insight on approach and avoidance behaviors. Front. Syst. Neurosci. 8:238. doi: 10.3389/fnsys.2014.00238

The superordinate division of emotions is distributed along a bipolar dimension of affective valence, from approaching rewarding situations to avoiding punitive situations. Avoiding and approaching behaviors determine the disposition to the primary emotions of fear and attachment and the behavioral responses to the environmental stimuli of danger, novelty and reward. Approach or avoidance behaviors are associated with the brain pathways controlling cognitive and attentional function, reward sensitivity and emotional expression, involving prefrontal cortex, amygdala, striatum and cerebellum. Individual differences in approach and avoidance behavior might be modulated by normal variance in the level of functioning of different neurotransmitter systems, such as dopaminergic, serotoninergic, noradrenergic and endocannabinoid systems as well as many peptides such as corticotropin releasing hormone. These substances act at various central target areas to increase intensity of appetitive or defensive motivation.

Physiologically, personality temperaments of approach and avoidance are viewed as instiga-

tors of propensity. They produce immediate affective, cognitive and behavioral inclinations in response to stimuli and orient individuals across domains and situations in a consistent fashion. Although the action undoubtedly emerges directly from these temperamental proclivities, ultimate behavioral outcomes are often a function of the integration among goal pursuit, self-regulation, and temperament trait.

Defective coping strategies to aversive or rewarding stimuli characterize the patho-physiology of anxiety- and stress-related disorders or compulsive and addiction behaviors, respectively. Individuals with neuropsychiatric symptoms such as depression, suicidal behavior, bipolar mania, schizophrenia, substance use disorders, pathological gambling and anxiety disorders have scores which fall at the extreme tails of the normal distribution for a specific temperamental trait.

The present Research Topic on the individual differences in emotional and motivational processing emphasizes the link between neuronal pattern and behavioral expression. The Topic includes experimental and clinical researches addressing the individual differences related to approach and avoidance and their behavioral characterization, structural and neurochemical profiles, synaptic connections, and receptor expressions. Studies are organized in a framework that puts in evidence the phenotypic expression and neurobiological patterns characterizing the individual differences and their biological variance.

**Citation:** Laricchiuta, D., ed. (2016). Individual Differences: From Neurobiological Bases to New Insight on Approach and Avoidance Behavior. Lausanne: Frontiers Media. doi: 10.3389/978-2-88919-715-6

# Table of Contents


Isaac R. Galatzer-Levy, Justin Moscarello, Esther M. Blessing, JoAnna Klein, Christopher K. Cain and Joseph E. LeDoux

*24 Strain-dependent differences in corticolimbic processing of aversive or rewarding stimuli*

Diego Andolina, Stefano Puglisi-Allegra and Rossella Ventura


Martin Reuter, Andrew J. Cooper, Luke D. Smillie, Sebastian Markett and Christian Montag


*65 Individual differences in approach-avoidance aptitude: some clues from research on Parkinson's disease* Alberto Costa and Carlo Caltagirone

*74 Truth, control, and value motivations: the "what," "how," and "why" of approach and avoidance*

James F. M. Cornwell, Becca Franks and E. Tory Higgins

*89 Individual differences in response to positive and negative stimuli: endocannabinoid-based insight on approach and avoidance behaviors* Daniela Laricchiuta and Laura Petrosini

# Editorial: Individual differences: from neurobiological bases to new insight on approach and avoidance behavior

Daniela Laricchiuta1, 2 \*

1 IRCCS Fondazione Santa Lucia, Rome, Italy, <sup>2</sup> Department of Dynamic and Clinical Psychology, Faculty of Medicine and Psychology, University "Sapienza" of Rome, Rome, Italy

Keywords: affective and emotional neuroscience, dopaminergic and endocannabinoid systems, fear system, motivational disorders, personality traits, reinforcement sensitivity theory, resilience, rewarding and aversive stimuli

Many different labels have been proposed over the years to cover the definition of approach and avoidance. Initially, an Approach-Avoidance distinction was conceptualized in terms of valencebased processes, rather than over behavior. In 1960s, an Approach-Withdrawal distinction was introduced arguing that in all organisms the motivation is grounded in overt behavioral actions toward or away from stimuli. Subsequently, it was presumed that action and emotional tendencies are grounded in specific brain systems. Only recently, it was preferred the Approach-Avoidance distinction that expands the previous Approach-Withdrawal distinction in terms of energization of the behavior by (motivation), or direction of the action toward (behavior), positive stimuli in the case of the approach, and in parallel, energization of the behavior by, or direction of the action away from, negative stimuli in the case of the avoidance (Laricchiuta and Petrosini, 2014).

The approach and avoidance behaviors appear to be the primary reactions to novel, rewarding, or dangerous stimuli on which all successive responses are based in order to gain successful adaptation. Thus, the positive or negative valence of the stimulus is considered the core of Approach-Avoidance distinction. Further, the hedonic principle to approach pleasure and avoid pain is frequently presumed to be the fundamental principle upon which motivation is built (Cornwell et al., 2014). In this framework, the approach system is considered a motivational system that activates reward-seeking behavior associated with impulsivity/exploration, whereas the avoidance system is considered an attentional system that promotes appetitive response inhibition or active overt withdrawal.

The approach and avoidance behaviors are biologically based and constitutionally ingrained, since all organisms are "pre-programmed" to approach or avoid particular classes of stimuli. Approach and avoidance behaviors are anchored to the brain networks implicated in action and reaction to salient stimuli and controlling cognitive and attentional functions, reward sensitivity and emotional expression. These networks involve cerebral nodes interconnected as prefrontal cortex, amygdala, hypothalamus, striatum and cerebellum. By acting on them the neurotrasmitter systems increase the intensity of appetitive or defensive motivation. In fact, individual differences in approach and avoidance behaviors might be modulated by normal variance at the level of functioning of different neurotransmitter systems, such as dopaminergic, serotoninergic, noradrenergic and endocannabinoid systems as well as many peptides such as corticotropin releasing hormone. Experimental findings collected over the years show how the genetic background may play a critical role in modulating aminergic and GABAergic neurotransmission in prefrontal-accumbal-amygdaloid system in response to different rewarding or aversive experiences. Further, important results highlight the modulatory role for genetic

#### Edited and reviewed by:

Maria V. Sanchez-Vives, Institució Catalana de Recerca i Estudis Avançats - Institut d'Investigacions Biomèdiques August Pi i Sunyer, Spain

> \*Correspondence: Daniela Laricchiuta,

daniela.laricchiuta@uniroma1.it

Received: 14 July 2015 Accepted: 26 August 2015 Published: 15 September 2015

#### Citation:

Laricchiuta D (2015) Editorial: Individual differences: from neurobiological bases to new insight on approach and avoidance behavior. Front. Syst. Neurosci. 9:125. doi: 10.3389/fnsys.2015.00125 variability of the dopaminergic system in individual differences in action-valence interaction (Richter et al., 2014).

Physiologically, human temperaments of approach and avoidance are viewed as instigators of propensity. They produce immediate cognitive, affective, and behavioral inclinations in response to stimuli and orient individuals across domains and situations in a consistent fashion. Although the action undoubtedly emerges directly from the temperamental proclivities, ultimate behavioral outcomes are often function of the integration among goal pursuit, self-regulation, and temperament traits. Also the motivational salience plays an important role in shaping behavior. Individuals regulate their emotions in a wide variety of ways. The aberrations in the elaboration of aversive or rewarding stimuli as well as defective coping strategies characterize many psychopathological disorders, as attention-deficit/hyperactivity disorders, depression and substance abuse on one hand, or anxiety and post-traumatic stress disorder on the other hand. Thus, individual differences in approach and avoidance may represent predictors of vulnerability (or resilience) to neuropsychiatric diseases.

The present Research Topic deals with the hot issue of individual differences in emotional and motivational processing, attempting to clarify "what," "how," and "why" of human and animal approach and avoidance behaviors, emphasizing the link between neuronal pattern and behavioral expression (McNaughton and Corr, 2014). The Topic includes experimental and clinical researches on the individual differences focusing behavioral characterization, structural and neurochemical profiles, synaptic connections, and receptor expression of approach and avoidance (Andolina et al., 2015). The translational models included in the present Research Topic consider the neurobiological mechanisms that give rise to outliers in approach and avoidance behaviors (Galatzer-Levy et al., 2014). Using the central tendency that assumes population homogeneity potentially overlooks the individual differences that explain responses to positive or negative stimuli. Crucial findings indicate that the heterogeneous approach and avoidance responses may be informative for understanding both resilience and impaired coping strategy.

Further, great importance has been given to the researches facing the clarification of diseases associated with inappropriate responses to aversive or rewarding situations. An interesting contribution to the Research Topic has been given from a literature revision on Parkinson's disease to understand whether neurobiological (dopaminergic dysfunction) and neuropsychological (executive function alteration) modifications due to Parkinson's disease are associated to changes in approach-avoidance related personality features (Costa and Caltagirone, 2015). Parkinson's disease patients may show approach-avoidance imbalance as documented by lower novelty-seeking and higher harm-avoidance temperamental traits.

It has been also addressed the issue of whether some forms of emotional regulation are healthier than others by focusing on two commonly used strategies: cognitive reappraisal (changing the way one thinks about potentially emotion-eliciting events) and expressive suppression (changing the way one behaviorally responds to emotion-eliciting events) (Cutuli, 2014). Findings on individual differences have been reviewed showing that using cognitive reappraisal to regulate emotions is associated with healthier patterns of affect, social functioning, and well-being in comparison to using expressive suppression. Once more, brain structural basis and functional activation linked to the habitual use of cognitive reappraisal and expressive suppression are discussed in detail.

Given the growing need for standardized paradigms (Markett et al., 2014) and self-report inventories measuring individual differences in approach and avoidance, a new questionnaire measuring the revised constructs of behavioral inhibition and activation systems and fight-flight-freezing system has evidenced that a functional genetic polymorphism on the arginine vasopressin receptor 1a gene is associated with individual differences in the behavioral inhibition dimensions (Reuter et al., 2015).

Considered as a whole, the present Research Topic calls attention on individual differences related to approach and avoidance behaviors as resilience or risk factors to disease and inefficient coping strategies, in response to environmental challenges.

# References


and neural correlates. Front. Syst. Neurosci. 8:175. doi: 10.3389/fnsys.2014. 00175


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Laricchiuta. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Approach, avoidance, and their conflict: the problem of anchoring

#### *Neil McNaughton1 \* and Philip J. Corr <sup>2</sup>*

*<sup>1</sup> Department of Psychology and Brain Health Research Centre, University of Otago, Dunedin, New Zealand*

*<sup>2</sup> Department of Psychology, City University London, London, UK*

*\*Correspondence: nmcn@psy.otago.ac.nz*

#### *Edited by:*

*Daniela Laricchiuta, IRCCS Santa Lucia Foundation, Italy*

*Reviewed by: Luke D. Smillie, The University of Melbourne, Australia*

**Keywords: approach, avoidance, conflict, personality, biomarkers, anchoring**

To understand the neurobiology of individual differences in approach and avoidance behavior, we must anchor constructs at the behavioral level to the long-term global sensitivities of the neural systems that give rise to the observed stable patterns of behavior. We will argue that this requires not only appropriate data at both the neural and behavioral levels but also appropriate account to be taken of interactions at the intervening level of the conceptual nervous system (Hebb, 1949; Gray, 1975). In particular, in accounting for approach and avoidance behavior we must include consideration of the distinction between valuation and motivation (Corr and McNaughton, 2012), of interactions between the approach system and the avoidance system (Gray and Smith, 1969), and of their interaction with a distinct additional system that is activated by approach-avoidance conflict (Gray, 1977; summarized in Corr, 2013).

But first we need to ask why would we expect there to be traits linked to global approach and avoidance systems? Simple animals (with little or no brain) can produce approach and avoidance behavior (toward benefits and ultimately reproduction; and away from dangers and ultimately failure to reproduce) via multiple independent rules of thumb (Krebs et al., 1983). But we can expect more complex brains to have largely integrated these simple elements into systems more generally dedicated to approach or avoidance "because this is how [a few] genes can build a complex system that will produce appropriate but flexible behavior to increase fitness. . . . Rather than just pre-programmed movements such as tropisms and taxes, . . . if the genes are efficiently to control behavior . . . they must specify the goals for action" (Rolls, 2000, pp. 183, 190). Together with the evolution of general approach and avoidance systems that are not tied to any specific motivating stimulus (reinforcer), we would expect evolution of the long-term adaptive control of their overall sensitivity to adequate inputs. Such stable sensitivity would be the neurobiological basis of approach and avoidance personality traits.

Determining the appropriate neurobiological measure for the sensitivity of a highly evolved approach or avoidance system is not simple. These systems have hierarchically organized neural levels with processing ranging from "quick and dirty" to "slow and sophisticated" for both perception (LeDoux, 1994) and action (Graeff, 1994, 2010). Sensitivity to input determines which level of the system is activated and so sensitivity cannot reside in any one of the modules within the system (McNaughton and Corr, 2004). The source of any sensitivity must, therefore, be identified independently—in essence requiring at least a preliminary surface level description of traits.

Existing theories of personality provide a number of competing surface level, lexically-derived, systems with trait measures that relate to approach and avoidance either indirectly via constructs such as Extraversion and Neuroticism (Eysenck, 1957) or directly via constructs such as Harm Avoidance (Cloninger et al., 1993). Each system is stable, with links to mental disorder (Strelau and Zawadzki, 2011; Gomez et al., 2012; Mullins-Sweatt and Lengel, 2012; Trull, 2012) and brain structure (Gardini et al., 2009; DeYoung et al., 2010). But even when starting with approach and avoidance as primary constructs, they are derived "top-down" from pools of lexically-chosen questionnaire items (Carver and White, 1994; Elliot and Thrash, 2010) not from biological anchors. They also depend on factor analysis, which determines the number of dimensions, *but not location of trait axes* of the personality "space" that items occupy (Lykken, 1971; Corr and McNaughton, 2008). It is little more than an act of faith to believe that the causal structure of personality is isomorphic with its lexical factor structure. So, even if we knew for certain that there were only two dimensions within a particular measured personality space, one questionnaire system could have a single simple trait anxiety dimension (orthogonal to, say, impulsiveness) that was a combination of neuroticism and introversion in another (Gray, 1970)—the two systems differing only on which items from an original pool were used to create scales. Factor analytically derived trait measures can also easily meet the criterion of having "simple structure" (in the sense that a set of items loads highly on only one factor so factors can be clearly identified by unique item loadings) while implying improbable causation (Lykken, 1971). Further, not only is there no reason to suppose that biologically accurate scales should have simple structure but also current scale systems, even though designed to have this, often do not (DeYoung, 2006, 2010).

McNaughton and Corr Anchoring approach and avoidance sensitivity

The plethora of competing trait scales can to some extent be encompassed by just five major trait dimensions that include both normal people and those with psychiatric disorders (Markon et al., 2005; Revelle et al., 2011; Krueger et al., 2012). However, the traits of the competing systems have complex relations to these five large scale dimensions and it is open to question whether there are five fundamental dimensions or whether these are complex facets riding on two or even just one major dimension of personality (Markon et al., 2005; DeYoung et al., 2007; Rushton and Irwing, 2009). These large scale dimensions have "facets" that potentially represent the true underlying sources of personality; and different "approaches differ substantially in the number and nature of the facets they propose, indicating that further conceptual and empirical work is needed to achieve a consensual specification of the Big Five factors at lower levels of abstraction. [Further], given that the Big Five were derived initially from analyses of the personality lexicon, one might wonder whether they merely represent linguistic artifacts" (John et al., 2008, p 141). With no "bottom up" neural anchor to definitely locate the correct rotation of any true biological trait/facet axis, there is no unequivocal way to unify the various systems currently in use.

A related problem, on which we focus below, is that the bulk of personality research has required statistical independence (orthogonality) of the extracted factors. To do otherwise would greatly increase the already large number of alternative trait solutions for any particular item space. However, as we will see, there is good reason to see surface level behavior as being determined interactively even if the biological control of the underlying sensitivities is independent. Likewise, even if the control of factors is neurally independent, when one, e.g., neuroticism, is a risk factor for another, e.g., anxiety (Andrews et al., 1990), then they will become statistically linked in the population as a result.

The solution for approach/avoidance traits is to anchor their factor spaces to measures derived from existing neural state theory. **Figure 1** is derived from one particular detailed neuropsychological theory (Gray and Smith, 1969;

**FIGURE 1 | Overall relation of approach (BAS), avoidance (FFFS = fight, freeze, flee), and conflict (BIS = behavioral inhibition) systems—an updated model.** The inputs to the system are classified in terms of the delivery (+) or omission (−) of primary positive reinforcers (PosR) or primary negative reinforcers (NegR) or conditional stimuli (CS) or innate stimuli (IS) that predict such primary events. The BIS is activated when it detects approach-avoidance conflict—suppressing prepotent responses and eliciting risk assessment and displacement behaviors. The systems interact in a variety of ways to generate behavior, see text. The shaded areas are all points at which traits appear to operate. Figure and legend modified from Gray and McNaughton (2000) and Corr and McNaughton (2012).

Gray, 1982; Gray and McNaughton, 2000; McNaughton and Corr, 2004; Corr and McNaughton, 2012) but its system level description captures issues that must be taken into account by any approach/avoidance account of personality. Adequate stimuli (reinforcers) must first be valued and, importantly, negative stimuli (e.g., losses) have a higher exchange rate that positive ones (e.g., gains); that is, people usually show loss aversion (Kahneman and Tversky, 1979).

Any specific positive or negative reinforcer can produce approach or avoidance depending on its contingency (presentation or omission) with responding. For any given reinforcer, the motivational sensitivity of approach activation is different from avoidance activation; and these are separate from the distinct valuation sensitivities of gain to loss (Hall et al., 2011). The strength of response output for any given level of approach activation also depends on distance from the goal (not shown in **Figure 1**) and does so to a lesser extent than does avoidance (Miller, 1944).

Even with independent trait sensitivities, state approach output depends on the level of avoidance activation, and vice versa: their activations sum to generate arousal, while subtracting to determine choice—giving rise to phenomena such as behavioral contrast and peak shift (Gray and Smith, 1969). As a result, when approach and avoidance are strongly and equally activated, arousal is high but the probability of both approach and avoidance is low; in addition, the approachavoidance conflict is detected by a third system (with its own trait sensitivity) that is unlike either pure approach or pure avoidance (withdrawal) in being affected by anxiolytic drugs (Gray, 1977). Both approach and avoidance are then inhibited and replaced by behaviors such as risk assessment (Gray and McNaughton, 2000) and displacement (Hinde, 1998), while arousal and negative bias (risk aversion) are increased. With this plethora of interactions, it will be difficult to extract true approach and avoidance traits from the surface structure of behavior—especially if orthogonal factors such as gain and approach have been conflated in a single construct such as reward (Corr and McNaughton, 2012).

However, neural measures should be able to target the internal representations of the specific elements depicted in **Figure 1**; challenge their response with appropriate combinations of stimuli; and so dissect out the specific contribution of a particular trait sensitivity. These neural measures can then be used to anchor traits within the conventional factor spaces and determine non-orthogonalities. Paradoxically, we are closest to achieving this with the most embedded neural construct: sensitivity to conflict. The argument for the use of primarily neural rather than questionnaire measures of approach and avoidance sensitivities has been made in detail previously—coupled with arguments for combining bottom up neural analysis with top down behavioral analysis (Smillie, 2008a,b; DeYoung, 2010). Here, we would emphasize, in addition, that the choice of neural measures should be strongly theoretically based and behaviorally and or pharmacologically validated in relation to the theory. Otherwise a plethora of questionnaires becomes a plethora of putative neural measures.

The conflict system is defined by the action of anxiolytic drugs (Gray, 1977) acting on receptors for endogenous compounds (Guidotti et al., 1978; Polc, 1995) that could mediate the system's trait sensitivity. Anxiolytic action is specifically linked to hippocampal rhythmicity in rodents (Woodnorth and McNaughton, 2002; McNaughton et al., 2006, 2007) and this has led to development of a human scalp EEG homolog (McNaughton et al., 2013) that provides a biomarker for conflict sensitivity in humans. This biomarker appears to be linked to the shared variance in neuroticism and trait anxiety much more than either of their unique variances (Neo et al., 2011).

In summary, we believe that approach and avoidance systems have evolved in such a way that global control of sensitivities to gain, loss, approach, avoidance and conflict can underlie human personality traits (Corr and McNaughton, 2012). While each of these long-term sensitivities is likely to be controlled independently, under normal ecological circumstances short-term behavioral output will be the result of complex interactions between them (**Figure 1**). However, the combination of appropriate neural measures with designs that dissect these interactions should provide the means to anchor trait measures in the data spaces that personality research has already shown have long term stability and important behavioral, and particularly psychiatric, consequences. Critically, the factor analysis of lexically-derived variables at the surface level of description cannot be assumed to reflect the deeper construct processes that are giving rise to surface descriptions; and no adjustment of the basic factor analysis method can avoid the problem created when there is no neural anchor to ensure inclusion of correct items and unique rotational solution after initial factoring.

## **REFERENCES**


a maladaptive personality trait model and inventory for DSM-5. *Psychol. Med.* 42, 1879–1890. doi: 10.1017/S0033291711002674


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 03 April 2014; accepted: 16 June 2014; published online: 03 July 2014.*

*Citation: McNaughton N and Corr PJ (2014) Approach, avoidance, and their conflict: the problem of anchoring. Front. Syst. Neurosci. 8:124. doi: 10.3389/fnsys. 2014.00124*

*This article was submitted to the journal Frontiers in Systems Neuroscience.*

*Copyright © 2014 McNaughton and Corr. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Heterogeneity in signaled active avoidance learning: substantive and methodological relevance of diversity in instrumental defensive responses to threat cues

*Isaac R. Galatzer-Levy1 \*, Justin Moscarello2, Esther M. Blessing1, JoAnna Klein2, Christopher K. Cain1,2,3 and Joseph E. LeDoux2,3*

*<sup>1</sup> Department of Psychiatry, New York University School of Medicine, New York, NY, USA*

*<sup>2</sup> Department of Arts and Sciences, Center for Neural Science, New York University, New York, NY, USA*

*<sup>3</sup> Nathan Klein Institute, Orangeburg, SC, USA*

#### *Edited by:*

*Daniela Laricchiuta, IRCCS Santa Lucia Foundation, Italy*

#### *Reviewed by:*

*Jose L. Pardo-Vazquez, Fundaçao Champalimaud, Portugal Silvia Middei, Consiglio Nazionale delle Ricerche, Italy*

#### *\*Correspondence:*

*Isaac R. Galatzer-Levy, New York University School of Medicine, 1 Park Ave., New York, NY 10016, USA e-mail: isaac.galatzer-levy@ nyumc.org*

Individuals exposed to traumatic stressors follow divergent patterns including resilience and chronic stress. However, researchers utilizing animal models that examine learned or instrumental threat responses thought to have translational relevance for Posttraumatic Stress Disorder (PTSD) and resilience typically use central tendency statistics that assume population homogeneity. This approach potentially overlooks fundamental differences that can explain human diversity in response to traumatic stressors. The current study tests this assumption by identifying and replicating common heterogeneous patterns of response to signaled active avoidance (AA) training. In this paradigm, rats are trained to prevent an aversive outcome (shock) by performing a learned instrumental behavior (shuttling between chambers) during the presentation of a conditioned threat cue (tone). We test the hypothesis that heterogeneous trajectories of threat avoidance provide more accurate model fit compared to a single mean trajectory in two separate studies. Study 1 conducted 3 days of signaled AA training (*n* = 81 animals) and study 2 conducted 5 days of training (*n* = 186 animals). We found that four trajectories in both samples provided the strongest model fit. Identified populations included animals that acquired and retained avoidance behavior on the first day (Rapid Avoiders: 22 and 25%); those who never successfully acquired avoidance (Non-Avoiders; 20 and 16%); a modal class who acquired avoidance over 3 days (Modal Avoiders; 37 and 50%); and a population who demonstrated a slow pattern of avoidance, failed to fully acquire avoidance in study 1 and did acquire avoidance on days 4 and 5 in study 2 (Slow Avoiders; 22.0 and 9%). With the exception of the Slow Avoiders in Study 1, populations that acquired demonstrated rapid step-like increases leading to asymptotic levels of avoidance. These findings indicate that avoidance responses are heterogeneous in a way that may be informative for understanding both resilience and PTSD as well as the nature of instrumental behavior acquisition. Characterizing heterogeneous populations based on their response to threat cues would increase the accuracy and translatability of such models and potentially lead to new discoveries that explain diversity in instrumental defensive responses.

**Keywords: signaled active avoidance, fear conditioning, threat conditioning, Posttraumatic Stress Disorder (PTSD), heterogeneity, latent growth mixture modeling, resilience**

# **INTRODUCTION**

Animal models of stress are thought to provide information about the course and etiology of stress psychopathology such as Posttraumatic Stress Disorder (PTSD) (Yehuda and Antelman, 1993). Animal studies typically examine the mean response to threat challenge paradigms. However, a key feature of stress and trauma responses in humans is marked heterogeneity where only a minority of individuals develops significant and prolonged symptomatology (Bonanno, 2004; Yehuda and Ledoux, 2007; Galatzer-Levy et al., 2013a). Recently, researchers have begun to disaggregate heterogeneous populations of animals based on their behavioral response to acquired aversive cues. To date studies have identified distinct populations based on their rate and ability to extinguish learned threat (fear) cues (Bush et al., 2007; Cowansage et al., 2013; Galatzer-Levy et al., 2013b) as well as their ability to initiate instrumental behaviors to terminate such cues (Choi et al., 2010). Importantly distinct neurobiological mechanisms have been identified that differentiate subpopulations both in studies of threat extinction (Cowansage et al., 2013) and instrumental responses (Choi et al., 2010). Such an approach has significant promise for the identification social and neurobiological characteristics that influence the development of qualitatively distinct behavioral phenotypes.

Studies of Pavlovian conditioning have been formative in elucidating the neural mechanisms underlying conditioned defensive reactions (e.g., Johansen et al., 2011; Ledoux, 2014). Abnormal functioning of these neural mechanisms may underlie stress psychopathology (Yehuda and Ledoux, 2007). The initiation of situation-specific instrumental behaviors can ameliorate conditioned defensive reaction to threat cues such as freezing (Cain and LeDoux, 2008; Choi et al., 2010; Moscarello and Ledoux, 2013), just as the initiation of situation-specific active coping behaviors ameliorate the potential negative psychological effects of dangerous or harmful traumatic events (Gross and Thompson, 2007; Hartley and Phelps, 2010; Bonanno and Burton, 2013). Such behavioral outputs are thought to result from a complex interplay between afferent and efferent neurocircuitry governing arousal, threat learning, motivation, and habit formation. Characterizing individual differences in instrumental behaviors in response to threat and harm among animals exposed to identical experimental conditions can provide information about normal and abnormal functioning of this circuitry. Ultimately, this can facilitate neurobiology research of abnormal stress responses such as PTSD and healthy responses such as resilience.

Signaled active avoidance (AA), which combines sequential Pavlovian and instrumental conditioning (Mowrer and Lamoreaux, 1946), involves an active behavioral response to conditioned threat. In a typical experiment, rats are trained to shuttle across a divided chamber during auditory CS presentation, causing termination of the CS and omission of the footshock US (Choi et al., 2010; Ledoux, 2014). Importantly, while AA requires Pavlovian learning to encode threat, the transition to successful instrumental avoidance requires active suppression of freezing (Lázaro-Muñoz et al., 2010; Moscarello and Ledoux, 2013), an innate defensive response to a Pavlovian CS (Blanchard and Blanchard, 1969; Fanselow and Poulos, 2005).

Animals do not uniformly learn signaled AA. A subset of animals will not acquire the avoidance response, referred to as "Poor Avoiders" (Choi et al., 2010). Several studies have exploited this heterogeneity to investigate neural mechanisms that mediate competition between Pavlovian and instrumental memories during AA training (Choi et al., 2010; Lázaro-Muñoz et al., 2010; Martinez et al., 2013). Poor Avoiders also show increased Pavlovian freezing, and AA performance is restored in these animals by lesions of the central amygdala (CeA), a region that is essential for conditioned threat reactions (Choi et al., 2010; Lázaro-Muñoz et al., 2010). In a recent study, Martinez et al. (2013) compared brain c-Fos expression following AA training between good vs. poor AA avoiders, and found differences in amygdala-PFC circuits (Martinez et al., 2013) similar to those identified in good vs. poor extinguishers (Hefner et al., 2008).

These findings demonstrate that distinct subpopulations can be disaggregated to identify neurobiological mechanisms mediating distinctive profiles of avoidance-related behavior (Martinez et al., 2013). This approach may be particularly relevant to PTSD and other anxiety disorders, which occur in a minority of individuals (Kessler et al., 2005), and involve persistent and maladaptive threat responses, leading to increased vigilance and intrusive fear memories (Mahan and Ressler, 2012). Animal models of threat extinction have revealed neural mechanisms that translate to clinical findings, and are now central to current concepts of PTSD diagnosis and treatment (Parsons and Ressler, 2013). Active avoidance holds significant promise for understanding the functional interactions between circuits governing defense, arousal, reinforcement, motivation and control, which together instantiate behavior.

While distinct sub-populations can be disaggregated using cutoff scores based on behavioral responses, this does not provide evidence that such populations are truly present in the data. Only one study to date has attempted to empirically determine if such threat challenges produce distinct behavioral phenotypes (Galatzer-Levy et al., 2013b). This work utilized Latent Growth Mixture Modeling (LGMM) to statistically test for population heterogeneity in threat extinction learning over successive trials. LGMM provides a method to empirically identify heterogeneous latent classes distinguished by their pattern of change over time. Results of this study indicated that multiple homogeneous subpopulations in an overall heterogeneous population better fit the data than a single population. Identified populations included those who rapidly extinguished, those who slowly extinguished, and those who failed to extinguish Pavlovian reactions to the CS (Galatzer-Levy et al., 2013b), a pattern consistent with the heterogeneity in response trajectories following human trauma exposure, both in shape and proportion (Galatzer-Levy et al., 2013a). In clinical studies, LGMM techniques have been used to identify heterogeneous trajectories of symptom and stress response among individuals exposed to significant life stressors and traumatic events (Galatzer-Levy et al., 2011, 2012, 2013c, 2014; Bonanno et al., 2012a,b; Galatzer-Levy and Bonanno, 2012). Importantly, LGMM does not require *a priori* hypotheses of bimodal good vs. poor avoiders, but provides a statistical method for empirically determining the number and shape of trajectories that best fit the data, and a framework for testing hypotheses related to that heterogeneity (Del Boca, 2004). Thus LGM provides the opportunity to empirically identify and characterize those trajectories that best fit the data.

While discernable populations may be identified using cluster analytic techniques such as LGMM, it is important to determine if these populations are distinct behavioral phenotypes, or simply statistical anomalies. If the trajectories are valid, other behaviors typically associated with good or poor performance should also be differentiated by AA population membership. Previous evidence also indicates that animals that perform poorly during signaled AA training also demonstrate decreased inter- trial exploratory behavior freezing (Vicens-Costa et al., 2011). Thus, animals that demonstrate greater active avoidance will likely demonstrate greater inter- trial exploratory behavior.

In the current investigation, we test the hypothesis that heterogeneous patterns of signaled AA, as measured in an auditory two-way shuttling paradigm, can be identified and replicated using an LGMM approach. We also apply LGMM to simultaneously collected data on inter-trial crossing responses (ITRs), i.e., the number of times animals cross between divided chambers in between AA trials, which is a measure of inter-trial exploratory behavior. We predict that rapid avoiders identified by LGM should also show increased ITRs.

# **METHODS**

## **ANIMALS**

Subjects were 267 naïve male Sprague-Dawley rats (Hilltop Laboratories) weighing 250–300 g at the time of arrival. Animals were used in two separate AA studies. Study 1 (*n* = 81 rats) consisted of 3 days of signaled AA training while Study 2 (*n* = 186 rats) utilized identical procedures with training extended through 5 days. Rats were individually housed in plastic tubs with *ad libitum* access to food and water, and kept on a 12 h light/dark cycle (lights on at 8 AM). All procedures were approved by the NYU University Animal Welfare Committee. Animals in Study 1 had intracranial guide cannula implants as previously described (Moscarello and Ledoux, 2013) while animals in Study 2 did not.

## **APPARATUS**

## *Signaled active avoidance apparatus*

Signaled active avoidance training occurred in 6 identical Plexiglas and metal rectangular shuttle boxes (50.8 × 25.4 × 30.5 cm, LWH) separated into two equal compartments by a metal divider placed halfway along the length of the chamber (Coulbourn Instruments). A passage in the divider (8 × 9 cm, WH) allowed animals to move freely between compartments. The floor was comprised of conductive stainless steel bars. The CS was a 5 kHz, 70 db tone delivered via two speakers mounted on opposite walls of the chamber. The US was a 0.7 mA footshock administered via the floor by a scrambled shocker. The chamber was lit by two 0.5 W light bulbs, one in each compartment. The shuttle box was housed within a larger sound-attenuating cubicle.

Shuttling (movement from one compartment to the other) was monitored by two infrared arrays, each comprised of 5 emitter-detector pairs, located on either side of the metal divider. Sessions were also recorded on DVD by a pair of black and white infrared cameras, one in each compartment.

## *Signaled active avoidance training*

On the day prior to the initiation of training, all animals were habituated to the shuttle box for 1 h. Shuttling between compartments was recorded as a measure of baseline activity. Twenty-four hours later the first of 3 or 5 consecutive daily avoidance-training sessions began. Each session started with a 5 min acclimation period in which no stimuli were presented. The 1st trial of the 1st session was a Pavlovian trial—a 15 s tone CS preceded a 1 s foot shock US regardless of whether the animal performed the avoidance response (shuttling) during CS presentation. This allowed all animals to acquire the Pavlovian contingency at the same point in training. All subsequent trials were avoidance trials. The CS lasted a maximum of 15 s and was followed immediately by a US lasting a maximum of 15 s. The shock begins immediately after the 15 s CS (if the rat fails to shuttle during the CS) and remains on until an escape shuttle occurs or 15 additional seconds elapse. If the animal shuttled during CS presentation, the tone terminated immediately, and the US was not delivered; this was scored as an avoidance response. Each session was comprised of 30 avoidance CSs with a varying inter-trial interval that was, on average, 120 s. Both CS and US duration depended on the behavior of the animals. If the rat shuttled during the CS presentation, it immediately terminated and thus the CS was less than 15 s. If no shuttle occurred during the CS, then the shock was presented until an escape shuttle occurred or until an additional 15 s elapsed. Thus, if the rat escaped the US then the US presentation was less than 15 s. There were no minimum CS or US durations programmed, only maximums. Avoidance responses were recorded as number of successful shuttles in response to the CS. Inter-trial shuttling responses (ITRs) were recorded simply as the number of non-CS shuttles per session.

# **DATA ANALYTIC APPROACH**

The current study attempts to identify latent (not directly observable) classes of active avoidance acquisition using LGMM. LGMM allows for the empirical exploration of underlying heterogeneity that may otherwise be treated as error when assuming population homogeneity across the sample under study (Del Boca, 2004). An advantage of the LGMM framework is that it provides tests to statistically compare the number of classes and other parameters (Muthen, 2003). LGMM uses a nested model approach where progressively complex models are compared statistically. In this context, the null model is a single linear trajectory characterized by the population mean. Key criteria for model selection are reductions across nested comparisons in the Information Criteria (IC) [the Bayesian Information Criterion (BIC), samplesize adjusted Bayesian Information Criterion (SSBIC), Aikaike Information Criterion (AIC) (Schwartz, 1978; Bozdogan, 1987; Sclove, 1987)], fit statistics [the Lo-Mendell-Rubin likelihood test (LRT), Bootstrap Likelihood Ratio Test (BLRT)], as well as parsimony and interpretability (Nylund, 2007). The IC specifically provides an index for model selection from a finite set of nested models. During model identification one can increase the likelihood function simply by adding more parameters. However, this can result in overfitting. The various IC's are indices that balance model complexity with fit by adding a penalty for added parameters to prevent selection of overfit models. The different IC's are closely related mathematically with small distinctions that result in better performance under different circumstances. As such, it is recommended to attend to all IC indices (Nylund, 2007).

LGMM methods, as well as related methods that utilize fixed effects such as including Latent Class Growth Analysis (LCGA), are useful for identifying and studying homogeneous stress response patterns without reliance on a priori assumptions to define cutoff scores for populations (Galatzer-Levy and Bryant, 2013).

Specifically LGMM methods test whether the population under study is composed of a mixture of discrete distributions characterized as classes of individuals who share profile of growth across measurement point, with class membership determined by parameters including the intercept, slope and other model specific parameters (Curran and Hussong, 2003). Consistent with other methods that utilize maximum likelihood estimation to identify models (though other methods such as Markov Chain Monte Carlo estimation can also be utilized), models may be identified that are only "local solutions" meaning that the model is only accurate in part of the data or a subset of the data. To guard against this, large numbers of random starting values are utilized to identify a solution that replicates across subjects resulting in a "global solution" (Duncan et al., 2006; Kline, 2011). This is conceptually similar to cross-validation methods that identify a solution in one random portion of the data and validate it in a different random portion of the data though it is not explicitly a cross-validation technique.

LGMM has been applied to a wide variety of behaviors in which it is not parsimonious to assume one population defined by a single continuous distribution, including drinking behavior among college students (Greenbaum et al., 2005), childhood aggression (Schaeffer et al., 2003), developmental learning trajectories (Boscardin, 2008), as well as posttraumatic stress in response to military combat (Bonanno et al., 2012b) trauma exposure among police (Galatzer-Levy et al., 2011, 2013c, 2014), emergency medical interventions (Deroon-Cassini, 2010; Galatzer-Levy et al., 2013a) and recently patterns of fear extinction in rats (Galatzer-Levy et al., 2013b).

In the current study, a piecewise modeling approach was utilized so that unique avoidance slopes for each day by class could be identified. Each piece covers three time points, separately capturing each day of training, and with a single intercept representing the number of successful escapes on the first set of trials. Within a piecewise model, multiple progressive linear slopes are modeled in the place of a single slope across time points allowing for information about the time frame of change to be captured without adding significant model complexity (Flora, 2008). We sought a model that was parsimonious, interpretable and that demonstrated lower values on the information criterion indices, and a significant p value for the BLRT and LMRT. We also examined entropy to assess the likelihood that individual rats were conforming to the modeled trajectories. Entropy in this context is a measure of correct classification into modeled parameters. As identified classes are modeled parameters, not cases per say, it is possible that individuals do not conform well to the parameters that are identified. Entropy provides an estimate of how well the data conforms to the modeled parameters. Entropy ranges from 0 to 1 with 1 indicating perfect classification. When entropy is low, it is inappropriate to save probable class assignment for analysis outside of the LGMM because significant error is introduced do to misclassification. Study 1 utilized fixed effects for the intercept and slopes because the relatively small sample size limits the ability to examine free parameters. Study 2 utilized fixed effects only for slopes 2 through 5. Fixing effects aids in model convergence but precludes analysis of covariates that can explain the random variability in these parameters. In the current study, we were primarily interested in identifying classes rather than testing predictors associated with the parameters so fixing these effects did not limit the current study.

## **DATA ANALYSIS**

First, the 30 consecutive daily trials were binned so that each consecutive score represents the mean number of avoidance responses across 10 trials. As such, Study 1 consists of 9 consecutive avoidance scores and Study 2 consists of 15 avoidance scores per rat. Data for ITRs came from study 2 and consisted of 5 measurement points capturing total number of ITRs per session. Using Mplus 6.12 (Muthen and Muthen, 2006), LGMM was employed to identify heterogeneous trajectories of active avoidance learning with the best fitting solution determined using the methods described above. Finally, after the best fitting models were identified, individual animal's probable class assignment was saved for further analysis in SPSS. ITRs were compared using repeated measures ANOVA with probable class assignment used as a fixed factor and total scores for ITRs for each of 5 days of training modeled as the within subjects factor.

# **RESULTS**

## **STUDY 1**

Models with progressive numbers of classes were tested and compared based on the model selection criterion. Consistent improvements in fit were observed based on the information criteria (AIC, BIC, SSBIC), with diminishing reductions in scores through five classes. Class solutions demonstrated marginally significant differences in model fit through four classes based on the LMRT and significant improvement through five classes based on the BLRT compared to the four class solution. Entropy values remained in the high range across solutions. The addition of a fifth class produced an additional small class (3.3%) following an erratic pattern without significantly altering the other classes. Based on this evidence and evidence from the literature that fixed effects can lead to over-identification (Nylund et al., 2007), a four class model was retained as the most parsimonious and interpretable solution (see Table 1 in Supplementary Materials).

The model solution identified *four* classes with substantively distinct patterns of growth in avoidance behavior over 3 days. The best log likelihood estimates were replicated in this model indicating a global solution. Class 1 (Non-Avoiders: 20%) demonstrated an initial intercept that was significantly different from 0 (Est = 1.22; *SE* = 0.45; *p* ≤ 0.01), a non-significant slope on Day 1 (Est = 0.29; *SE* = 0.22; *p* = 0.18), a negative slope on Day 2 (Est = −0.43; *SE* = 0.18; *p* < 0.05), and a marginally significant negative slope on Day 3 (Est = −0.27; *SE* = 0.14; *p* = 0.06). Class 2 (Slow Avoiders: 21%) demonstrated an initial intercept that was significantly different from 0 (Est = 1.91; *SE* = 0.53; *p* < 0.001), and a significant increase in avoidance behavior on Day 1 (Est = 0.97; *SE* = 0.20; p < 0.001) but no increase in learning on Day 2 (Est = 0.28; *SE* = 0.37; *p* = 0.46) or Day 3 (Est = 0.29; *SE* = 0.50; *p* = 0.56). Class 3 (Modal Avoiders: 37%) demonstrated an initial intercept that is significantly different from 0 (Est = 1.47; *SE* = 0.29; *p* < 0.001) and growth in avoidance learning across all 3 days of training (Day 1: Est = 1.39; *SE* = 0.22; *p* < 0.001; Day 2: Est = 1.69; *SE* = 0.21; *p* < 0.001; Day 3: Est = 0.85; *SE* = 0.15; *p* < 0.001). Finally, Class 4 (Rapid Avoiders: 22%) demonstrated an initial intercept that was significantly different from 0 and elevated compared to the other classes (Est = 3.93; *SE* = 0.80; *p* < 0.001) along with significant positive growth on Day 1 (Est = 2.12; *SE* = 0.36; *p* < 0.001) and flat slopes for Day 2 (Est = 0.33; *SE* = 0.22; *p* = 0.13) and Day 3 (Est = 0.13; *SE* = 0.22; *p* = 0.56; see **Figures 1**, **2** for graphical representations of the population mean, individual trajectory means, and distribution within those trajectories). Importantly, as animals in this study were canalized, results

further indicate these phenotypes could be identified even when potentially intrusive recording equipment was employed.

## **STUDY 2**

Model selection for Study 2 was identical to Study 1. The increase in sample size allowed for the estimation of free effects for the intercept and slope for day 1. All other parameters were fixed to aid in model identification. While the information criteria continued to demonstrate reductions through a five class solution, large decreases were only observed through four classes. The LMRT favored a four class solution. The BLRT continued to demonstrate improvements in fit through five classes, but once again, a five class solution revealed an additional small class (3.0%), which in this case was not substantively distinct from another identified class. Entropy values remained high across solutions. Based on these observations, a four class solution was retained (see Table 1 in Supplementary Materials).

The *four* identified classes were similar to those identified in Study 1, with the noticeable exception of the Slow Avoiders, who once again demonstrated a trajectory that was distinctly higher in number of successful avoidances compared to the Non-Avoiders without large gains in avoidance learning. The best log likelihood estimates were replicated in this model indicating a global solution. In the current sample this population demonstrated rapid growth in avoidance learning through Day 4 and Day 5. Specifically, Slow Avoiders (9%) demonstrated an initial intercept that was marginally significantly different from 0 (Est = 0.43; *SE* = 0.25; *p* = 0.09), significant positive growth on Day 1 (Est = 0.70; *SE* = 0.21; *p* ≤ 0.001), non-significant growth on

**FIGURE 2 | Estimated means for and observed individual values for all animals (1) and animals grouped by class (2–5).**

Day 2 (Est = 0.29; *SE* = 0.21; *p* = 0.26) and Day 3 (Est = 0.04; *SE* = 0.26; *p* = 0.87), and significant positive growth on Day 4 (Est = 2.11; *SE* = 0.14; *p* < 0.001) and Day 5 (Est = 1.04; *SE* = 0.34; *p* < 0.01). Other classes included Non-Avoiders (16%) who demonstrated an intercept that was significantly different from 0 (Est = 0.50; *SE* = 0.20; *p* = 0.01), non-significant growth on Day 1 (Est = 0.15; *SE* = 0.14; *p* = 0.29), positive growth on Day 2 (Est = 0.57; *SE* = 0.23; *p* = 0.01), and non-significant growth on Day 3 (Est = −0.23; *SE* = 0.20; *p* = 0.26), Day 4 (Est = −0.03; *SE* = 0.12; *p* = 0.79), and Day 5 (Est = −0.26; *SE* = 0.19; *p* = 0.16). Modal Avoiders (50%) demonstrated a significant intercept (Est = 1.39; *SE* = 0.16; *p* < 0.001), significant positive growth across Day 1 (Est = 0.43; *SE* = 0.12; *p* < 0.001), Day 2 (Est = 2.14 *SE* = 0.13; *p* < 0.001), and Day 3 (Est = 0.70; *SE* = 0.13; *p* < 0.001) and non-significant growth on Day 4 (Est = 0.13; *SE* = 0.09; *p* = 0.15) and Day 5 (Est = 0.01; *SE* = 0.09; *p* = 0.90). Rapid Avoiders (25%) once again demonstrated an elevated intercept compared to the other classes (Est = 4.15; *SE* = 0.39; *p* < 0.001), significant growth on Day 1 (Est = 1.62; *SE* = 0.17; *p* < 0.001), and non-significant growth on Day 2 (Est = 0.29; *SE* = 0.24; *p* = 0.23), Day 3 (Est = 0.09; *SE* = 0.11; *p* = 0.41), Day 4 (Est = 0.11; *SE* = 0.13; *p* = 0.39), and Day 5 (Est = −0.07; *SE* = 0.16; *p* = 0.68; see **Figures 3**, **4** for graphical representations of the population mean, individual trajectory means, and distribution within those trajectories).

## *Post-hoc analysis of inter-trial crossing*

The probable class membership for individual animals from Study 2 was saved to SPSS 20 to assess differences in ITR behavior. A repeated measures ANOVA was utilized to assess the trajectory of ITRs across the 5 days, with each time point being the average ITRs for each day of training. Class membership was utilized as a between-subjects factor. This model revealed an overall effect for time (*Wilks'* λ = 0.43; *F*(4, 172) = 57.94; *p* < 0.001) and a significant interaction between time and class membership (*Wilks'* λ = 0.66; *F*(12, <sup>455</sup>.36) = 6.45; *p* < 0.001). *Post-hoc* analyses using Least Squared Differences correction for multiple comparisons revealed significant differences by class on the course of ITRs that closely resembled trajectories of avoidance response (Table 2 in Supplementary Materials; **Figure 5**). Rapid Avoiders made significantly more ITRs early in training, whereas Non-Avoiders only began making ITRs on Day 3 of training; the other two classes were intermediate.

Finally, because the identified trajectories indicate step-like learning, means, standard deviations and confidence intervals by class for the final time point of each day of training were generated using Study 2 data. This was done to determine the distribution of the percentage of successful avoidance trials that characterizes successful AA acquisition (see Table 3 in Supplementary Materials). Results based on the mean per class and confidence intervals indicate that AA acquisition during a single day of training can be identified by successfully avoiding ≥50% of the last 10 trials and maintained through subsequent trials. Based on this observation, animals can be classified as acquiring AA in smaller samples without reliance on LGMM which comes with heavy sample size burden to identified patterns of response.

#### *Post-hoc analysis of crossings during acclimation*

For study 1, the number of crossings when animals were habituated to the shuttle box for 1 h prior to training was recorded. Classes were compared on baseline habituation behavior quantified as the number of crossings occurring in the 1 h prior to the initiation of training. Mean number of crossings was compared by class using a One-Way ANOVA with individual

comparisons conducted using Least Squared Differences correction for multiple comparisons. Neither the overall model nor the individual comparisons approached significance. Habituation behavior was not recorded in study 2 precluding these analyses on the study 2 animals.

# **DISCUSSION**

The current study identified, characterized, and replicated distinct phenotypic patterns of signaled active avoidance (AA) behavior in response to a two-way shuttling paradigm. Previous studies drawing on distinct AA populations (i.e., good vs. poor avoiders) have made important advances in understanding competition in brain circuits mediating threat and avoidance conditioning formation resulting in individual differences in instrumental behavior (Choi et al., 2010; Lázaro-Muñoz et al., 2010; Martinez et al., 2013; Moscarello and Ledoux, 2013). The current study provides empirical evidence that disaggregating distinct populations provides better model fit then a single population estimate. This indicates that disaggregating distinct populations provides more accurate estimates of animal's behavior then using the population mean which assumes a single homogenous population. Further, identifying such populations through advanced modeling methods provides substantive information about the nature of signaled AA acquisition that is of key relevance for understanding the phenomenon on all levels of investigation.

The current study presents with limitations that are relevant to the interpretation of the current results. First, while we present the weight of animals upon arrival, of key interest is their weight upon the initiation of the experimental procedures as weight may be a proxy for age and may explain heterogeneity in trajectory membership. Second, ITRs were only characterized in Study 2 data and as such the findings related to heterogeneity in ITRs were not replicated across studies. This occurred because ITRs were not consistently recorded in Study 1 limiting our ability to analyze this data. Finally, as **Figures 2**, **4** demonstrate, there is significant, and likely meaningful variability around the population means. The current study utilized some fixed effects to aid in model convergence though **Figures 2**, **4** clearly demonstrate that there is significant variability around the trajectory means. Using free effects allows for the analysis of this variability which may be relevant for addressing key questions about the identified phenotypes. For example, recent work examining trajectories of PTSD symptomatology from days after trauma exposure through 15 months found that the receipt of early exposure psychotherapy affected variability in the slope rather than predicting trajectory membership (Galatzer-Levy et al., 2013a). Such analysis can only be conducted if the slope parameter is not fixed. Despite these limitations, the current findings provide unique information about heterogeneity in behavioral responses to threat cues.

# **POPULATION CHARACTERISTICS OF ACTIVE AVOIDANCE LEARNING**

Specifically, *three* phenotypes were replicated both in terms of behavioral trajectory and relative proportion of the total population. These included Rapid Avoiders (Study 1 = 22%; Study 2 = 25%) who acquired AA within 10 trials and achieved asymptotic levels of avoidance within the *first* day of training, Modal Avoiders (Study 1 = 37%; Study 2 = 50%), who demonstrated step-like increases in AA acquisition for roughly 3 days before converging with Rapid Avoiders, and a class of animals (Non-Avoiders) that failed to acquire AA learning through both 3 and 5 days of training (Study 1 = 20%; Study 2 = 16%). In data from both

class membership are the same as **Figure 2**.

studies, a Slow Avoider group was identified (Study 1 = 21%; Study 2 = 9%) that demonstrated slight gains through the *first* 3 days of training and in Study 2 demonstrated a step-like increase in avoidance acquisition across days 4 and 5.

The use of LGMM provides statistical evidence for consistent population heterogeneity in AA acquisition. Comparing the identified phenotypes to the population mean provides important insights into the limitations of assuming population homogeneity. Characterizing these populations comes with a number of benefits for future research.

The population mean provides the appearance that acquisition is linear and gradual over time, while acquisition among disaggregated phenotypes follows a step-like pattern where animals learn rapidly during some days of training and demonstrate little or no increase on other days. The use of a piecewise modeling approach with a separate slope each day of training allows for the identification of non-linear patterns of acquisition. Disaggregating populations provides evidence that the mean pattern of learning is an artifact resulting from collapsing multiple qualitatively distinct populations, including those who very rapidly acquire AA and those who consistently fail to do so. Further, related to both of the above limitations, the population mean obfuscates important information about the characteristics of signaled AA acquisition. By disaggregating populations, we can observe and characterize asymptotic levels of avoidance both prior to and following acquisition. By identifying latent populations we observe that animals en route to asymptotic levels of active avoidance successfully avoid in ≥to 50% of trials. Further, rapid acquisition appears to occur following the initiation of a new day of training indicating that memory consolidation is likely occurring between training sessions, possibly during sleep. The population mean, however, provides no clear evidence in this regard, as the mean for early trials is influenced by those who rapidly acquire and the mean of the latter trials is influenced by those who fail to acquire (see **Figures 1**, **2**).

Analysis of inter-trial responses (ITRs) supported LGMM results in finding distinct differences between classes, with a consistent direction indicating that increased ITRs early in training facilitate AA acquisition. In particular, Rapid Avoiders were distinguished from Modal Avoiders by markedly more ITRs on Day 1 of training. Increased ITRs on Day 3 also distinguished Slow- from Non-Avoiders. This result is consistent with previous reports of decreased ITRs in an extreme group selected for poor AA and frequent freezing (Vicens-Costa et al., 2011). Given that ITRs appear to be an important predictive correlate of AA, this result also demonstrates the utility of LGMM modeling for tracking relationships between two functionally related behaviors across classes.

An alternative explanation for the identified heterogeneous trajectories of acquisition is that the Rapid, Modal, and Slow Avoider classes merely correspond to subpopulations that acquired the task on variable days of training and that number of days it takes to acquire simply reflects normal variability. Rather, what is interesting about the current results in this conceptualization is that animals have a steep curve for acquisition of active avoidance in a single day leading to asymptotic behavior, but that this acquisition can occur at different times during the training process.

## **THE LIMITS OF ASSUMING AND STUDYING NORMALITY IN ANIMAL BEHAVIOR**

The most common approach to hypothesis testing in behavioral research is *one* developed by Ronald Fischer for the analysis of shared variance between independent and dependent variables in experimental data. This approach examines the mean of the population under the assumption of normal distribution of error terms (Hald, 2007). This method and its underlying assumptions are typically adopted without significant consideration of its implications about the nature of the phenomenon under study (i.e., is it normally distributed and homogenous). These assumptions, which garner their initial evidence from the study of astronomy and botany, are based on the characteristics of the central limit theorem which state that the sum of approximately normal random variables will be a single normal distributed random variable (Stigler, 1986; Hald, 2007). However, a random variable that is the multiplicative product of multiple independent random variables, such as interactions between individual neurobiological characteristics, will not result in a Gaussian normal distribution in the dependent variable (behavior or performance) (Buzsáki and Mizuseki, 2014). This may explain evidence that learning curves for individual subjects follow abrupt step-like increases, though these effects are obfuscated by the use of a single grand population mean which provide the illusion of continuous linear patterns of learning (Gallistel et al., 2004). Steplike behavioral changes are identified using the current approach. The ability to identify these steps can facilitate the examination of time-dependent shifts in circuit functioning, which is not accessible by examining linear relationships with the grand mean or by separating a priori populations.

# **MERITS OF UTILIZING STATISTICAL MODELING APPROACHES TO IDENTIFYING BEHAVIORAL PHENOTYPES**

Studying characteristics of distinct subpopulations within animal data sets has proven highly informative, leading to identification of differences in regional neural activation (Martinez et al., 2013), and hypothalamic-pituitary axis functioning (Cohen et al., 2006). This relatively new approach has several advantages over alternative strategies for creating behavioral models, such as the use of mutant or inbred strains, which involves neurobiological differences unrelated to the behavior, rather than populations characterized solely by behavior. Utilizing population differences for modeling behavior also permits examination of multiple behaviors within each defined population—for example, threat extinction and instrumental avoidance—which, as demonstrated (Choi et al., 2010) can reveal complex latent interactions otherwise not evident from one observable behavior.

In contrast to LGMM methods used in the current and former work (Galatzer-Levy et al., 2013b), previous studies of heterogeneity in animal threat response behaviors have commonly selected subpopulations based on subjective behavioral extremes, or in some cases, used using cut-point values (Bush et al., 2007; Vicens-Costa et al., 2011; Martinez et al., 2013; Ferreira and Nobre, 2014). This approach, while still valuable, has several drawbacks. First, ignoring the relationship of extremes to the modal population obscures identification of important population distinctions that may yield useful information. For example, we were able to identify a minority population of Rapid AA avoiders as distinct from Modal Avoiders. Neglecting the modal population may also cause misinterpretation of populations as resilient or pathological; given that modal behavior is assumed to be advantageous, or at least not maladaptive (Bonanno, 2004), populations should be interpreted in this context. By identifying phenotypic populations and their parameters, such as successful acquisition being characterized ≥50% of avoidance trials or rapid avoidance being characterized by avoidance acquisition in the first day of training, LGMM results can be utilized to identify acquisition or populations in smaller datasets. This is important, as it is often impractical to conduct research on large populations of animals.

Modeling statistical heterogeneity is also valuable when analyzing the results of experimental stress interventions, such as immobilization, which are used to studying the neurobiology of stress pathology e.g., Andero et al. (2013). While examining neurobiological correlates on the aggregate following a stress induction can be informative of mechanisms underlying behavior overall, this approach also assume population homogeneity in neurobiological changes following stress induction. Given that the response to threat is generally an adaptive process (Ledoux, 2012), changes *per se* following stress are not necessarily informative of pathology. Only by identifying those who have an abnormal response can we identify mechanisms associated with stress pathology (i.e., excessive and prolonged swelling following injury, not just swelling which is normative and adaptive). More generally, any strategy that assumes homogeneity and uniformity of variance (Fox, 2008) fails to observe that heterogeneous populations may relate to the same variables in different ways, which may be highly informative—for example, thirsty people will respond differently to the presentation of a glass of water compared to those who are satiated.

The current findings and approach can significantly aid in the discovery of behavioral and neurobiological differences associated with clinically relevant learning and coping strategies. Further, the approach can be generalized to other behaviors that are of clinical importance such as appetitive responses to addictive substances. Identifying heterogeneous populations allows for the exploration of circuit functioning that differentiates populations, the identification of time dependent changes that cause shifts in behavior, and examination of manipulations that alter the proportions of the identified populations. For example, naturally occurring differences in basal CREB, a key protein required for memory formation, has been shown to be associated with distinct behavioral responses to threat conditioning training. Further, direct augmentation of amygdala CREB causes shifts from one extreme to the other (Cowansage et al., 2013).

# **NEURAL CIRCUITRY OF PAVLOVIAN AND INSTRUMENTAL LEARNING IN AA SUBPOPULATIONS**

Several lines of evidence suggest that impaired AA in Slowor Non Avoiders may owe to a reduced capacity to suppress Pavlovian defensive reactions. Slow AA avoiders show increased rates of freezing (Choi et al., 2010; Vicens-Costa et al., 2011; Martinez et al., 2013), and amygdala CeA lesions both restore AA and reduce freezing. Furthermore, lesions of the infralimbic prefrontal cortex (ilPFC), which functions to inhibit CeA activation during threat extinction learning, impair AA, associated with both increased CeA activity and increased Pavlovian freezing (Moscarello and Ledoux, 2013). Therefore, as discussed by Martinez et al., naturally occurring variations in amygdala–prefrontal circuitry could underpin both poor AA acquisition and poor threat extinction. We previously reported significant heterogeneity in Pavlovian threat extinction learning, including populations of Rapid Avoiders, Slow Avoiders, and also Non-Avoiders who ultimately fail to extinguish threat (Galatzer-Levy et al., 2013b). The relationship between extinction and AA has not yet been directly determined and future studies correlating these behaviors in a common population may be highly informative.

Rapid Avoiders constituted a robustly distinct minority. It is of interest to consider neural circuits that may mediate enhanced AA acquisition. It is plausible this subpopulation has a markedly superior capacity to suppress freezing, compared to Modal Avoiders. Alternatively, or additionally, Rapid Avoiders may have enhanced motivational control of instrumental performance, or enhanced reward processing, perhaps due to more efficient neural interactions between the amygdala and nucleus accumbens (Cain and LeDoux, 2008; Boschen et al., 2011). Rapid Avoiders may also more rapidly link defensive organismic states to action, which could stem from more efficient amygdala CeA connections with cholinergic forebrain targets, previously shown to contribute to active responses to threats (Gozzi et al., 2010). It is also possible that in extremely rapid AA acquisition, where escape is reflexively elicited as soon as a route is provided, without initial freezing, escape may more closely approximate an innate survival circuit response. Finally, an equally plausible explanation is that Rapid Avoiders simply accidently crossed chambers during one of the early trials in the first session and as a result may be suppressing a freezing response without having received much exposure to the US. As such, they may have less profound Pavlovian memories to overcome and as such may not have any innate differences compared to Modal or Slow Avoiders. Thus, by identifying heterogeneous populations, hypotheses about the individual differences in experience as well as neurobiology can be examined.

## **CLINICAL RELEVANCE OF ACTIVE AVOIDANCE**

Identifying distinct trajectories of AA acquisition provides an inroad for translational research through comparison to quantitatively and qualitatively related patterns in clinical populations. Naturalistic studies of the course of stress pathology identify a similar heterogeneity in trajectories of symptom nonremission following trauma exposure (Bonanno et al., 2012a; Galatzer-Levy et al., 2013a). Active avoidance learning leads to robust suppression of Pavlovian defensive states, and this reduction in aversive state may further enhance instrumental learning (Choi et al., 2010; Moscarello and Ledoux, 2013). Furthermore, given that learning to terminate aversive experiences has been shown to prevent spontaneous recovery of threat response (Cain and LeDoux, 2007), learning active coping strategies may produce long term symptom management (Ledoux and Gorman, 2001). Thus, successfully acquiring AA may capture adaptive active coping responses to threat with direct relevance for understanding such responses to traumatic events (Lázaro-Muñoz et al., 2010; Martinez et al., 2013; Moscarello and Ledoux, 2013). Conversely, impaired AA may capture undermodulation of defensive states leading to the inability to instantiate behaviors that will ameliorate the threatening situation. The ability to characterize such populations in animals provides an opportunity to study neurobiological features associated with behavioral responses to manipulations such as signaled AA in a way that is not accessible in humans. The translational nature of such models can be greatly improved by identifying analogous behavioral phenotypes in both animals and humans.

# **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at: http://www.frontiersin.org/journal/10.3389/fnsys.2014. 00179/abstract

# **REFERENCES**


with startle reactions: a dissection of anxiety and fear in genetically heterogeneous rats. *Physiol. Behav.* 103, 148–156. doi: 10.1016/j.physbeh.2010. 12.009


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 09 May 2014; accepted: 05 September 2014; published online: 24 September 2014.*

*Citation: Galatzer-Levy IR, Moscarello J, Blessing EM, Klein J, Cain CK and LeDoux JE (2014) Heterogeneity in signaled active avoidance learning: substantive and methodological relevance of diversity in instrumental defensive responses to threat cues. Front. Syst. Neurosci. 8:179. doi: 10.3389/fnsys.2014.00179*

*This article was submitted to the journal Frontiers in Systems Neuroscience.*

*Copyright © 2014 Galatzer-Levy, Moscarello, Blessing, Klein, Cain and LeDoux. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Strain-dependent differences in corticolimbic processing of aversive or rewarding stimuli

# **Diego Andolina1,2\*, Stefano Puglisi-Allegra2,3 and Rossella Ventura2,3**

<sup>1</sup> Dipartimento di Scienze Cliniche Applicate e Biotecnologie, Università degli Studi dell'Aquila, L'Aquila, Italy

<sup>2</sup> Santa Lucia Foundation, Rome, Italy

<sup>3</sup> Dipartimento di Psicologia and Centro 'Daniel Bovet', Sapienza Università di Roma, Rome, Italy

#### **Edited by:**

Maria V. Sanchez-Vives, ICREA-IDIBAPS, Spain

#### **Reviewed by:**

Antonella Gasbarri, University of l'Aquila, Italy Carmen Agustín-Pavón, Imperial College London, UK

#### **\*Correspondence:**

Diego Andolina, Dipartimento di Scienze Cliniche Applicate e Biotecnologie, Università degli Studi dell'Aquila, Via Vetoio, L'Aquila 67010, Italy e-mail: diego.andolina@uniroma1.it; diego.andolina@gmail.com

Aberrations in the elaboration of both aversive and rewarding stimuli characterize several psychopathologies including anxiety, depression and addiction. Several studies suggest that different neurotrasmitters, within the corticolimbic system, are critically involved in the processing of positive and negative stimuli. Individual differences in this system, depending on genotype, have been shown to act as a liability factor for different psychopathologies. Inbred mouse strains are commonly used in preclinical studies of normal and pathological behaviors. In particular, C57BL/6J (C57) and DBA/2J (DBA) strains have permitted to disclose the impact of different genetic backgrounds over the corticolimbic system functions. Here, we summarize the main findings collected over the years in our laboratory, showing how the genetic background plays a critical role in modulating amminergic and GABAergic neurotransmission in prefrontal-accumbal-amygdala system response to different rewarding and aversive experiences, as well as to stress response. Finally, we propose a top-down model for the response to rewarding and aversive stimuli in which amminergic transmission in prefrontal cortex (PFC) controls accumbal and amygdala neurotransmitter response.

**Keywords: strain, neurotransmission, corticolimbic, rewarding stimuli, stress**

# **INTRODUCTION**

Adaptive behavior involves the ability to represent the value of positive or negative stimuli, establish predictions about them, and use these predictions to guide behavior (O'Doherty, 2004). Animals and humans have a propensity to seek out rewards, avoid punishments, and cope with negative situations, such as stressful events. Aberrations in the elaboration of aversive and rewarding stimuli characterize several psychopathologies, including anxiety, depression, and addiction. For instance, disorders of mood and motivation are frequently associated with anhedonia (reduced ability to experience pleasure), and alterations in neural processing of rewarding and aversive stimuli have been recently proposed as an endophenotype of depression (Hasler et al., 2004; McCabe et al., 2009; Ventura et al., 2013). Thus, understanding the neural mechanisms by which positive and aversive stimuli are elaborated is critical for the development of therapeutic approaches for several psychopathologies.

Mood and motivation disorders and other psychiatric conditions have complicated etiologies and result from complex interactions between genetic and environmental precipitating factors. In psychobiology studies, inbred strains are a useful tool to investigate the role of genetic factors, in interaction with aversive and rewarding experiences, in susceptibility to development and expression of psychopathology. In particular, data from C57 and DBA have provided information on how the response of specific neuronal systems is related to genetic background.

## **THE USE OF THE INBRED STRAINS**

The use of inbred strains of mice offers great advantages to studies aimed to determine the function of neurotransmitter systems with regard to the effects of psychotropic drugs, stressful events, and various psychopathologies.

An inbred strain is a set of animals that is produced by at least 20 consecutive generations of sister-brother or parent-offspring matings and that can be traced to a single ancestral pair in the 20th or subsequent generations. Inbred animals are nearly entirely homozygous, providing a well-defined and consistent genotype for analysis. The genetic stability of inbred strains over the years and through laboratories has allowed myriad relevant information for several commonly used strains to be accumulated. Thus, comparative studies on neurotransmitter activity in various regions of the brain in inbred mouse strains, which have differences of behavioral outcomes, is one approach to investigate the neurochemical bases of behavioral expression. In any experimental procedure that involves laboratory-bred stocks, the results might reflect the strain and species that are used. There is a significant amount of data on differences in the effects of various experimental conditions against which the findings from inbred strains can be referenced, controlling the influence of this source of variability. Moreover, behavioral, pharmacological, physiological, and biochemical comparisons between inbred strains constitute a preliminary stage for more extensive genetic research, such as quantitative trait loci (QTL) analysis, to identify and map genes in mice. Such a strategy can facilitate extrapolation of the results to the human genome, due to the significant extent of the linkage homology between human and mouse (Plomin et al., 1991; Crabbe et al., 1994).

### **C57BL/6 AND DBA/2 STRAINS**

C57BL/6 (C57) and DBA/2 (DBA) mice are among the most frequently studied inbred strains with regard to psychobiology because their behavioral responses have strain-dependent differences. Moreover, the functional and anatomical characteristics of their brain neurotransmitter systems have been examined extensively in these strains. A wealth of data on various parameters, such as neurotransmitter metabolism and release, receptor density and distribution, and activity of second messengers, has been accumulated. Consequently, data collected in these strains can offer important indications about the relationship between the behavioral and central effects of different neurotransmitters and, more generally, the involvement of brain neurotransmitters in the control of behavior.

Clinical and preclinical studies suggest that the prefrontal cortex (PFC), striatum (including the nucleus accumbens (NAc)) and amygdala are activated by natural positive or negative salient stimuli, constituting a common substrate for processing rewarding and aversive stimuli (Berridge and Robinson, 1998; Becerra et al., 2001; Jensen et al., 2003; O'Doherty, 2004; Borsook et al., 2007). Aminergic and amino acid transmission are the principal modulatory mechanisms of the corticolimbic system, and the dysregulation of these systems is linked to alterations in the elaboration of aversive and rewarding stimuli underlying various psychopathologies.

The main findings collected over the years in our lab, by microdialysis experiments, have identified the role of several prefrontal cortex-accumbal-amygdala neurotransmitter systems in the elaboration of rewarding or aversive stimuli. Here we report findings from two experimental paradigms: place conditioning and forced swimming test (FST). The place conditioning paradigm permits to investigate the motivational salience attribution process to conditioned stimuli that are associated with primary rewarding and aversive events (Tzschentke, 1998; Reynolds and Berridge, 2002). The FST is one of paradigms most widely used to measure antidepressant activity of new drugs. Moreover, FST allows to assess alterations in depressionlike behavior and coping response to stress in both normal and genetically modified animals (Porsolt et al., 1977; Borsini and Meli, 1988). The behavioral responses in the FST are thought to engage a coping strategy (Thierry et al., 1984), in which immobility behavior is an index of higher perceived motivational impact of a stressful experience. Finally, we used restraint stress to evaluate the time-dependent response induced by stress on different neurotransmitters in specific brain areas by intracerebral *in vivo* microdialysis.

# **BIOLOGICAL BASIS OF PROCESSING REWARDING/AVERSIVE STIMULI: PREFRONTAL-AMYGDALA-ACCUMBAL SYSTEM**

The NAc, together with the PFC and amygdala, can be considered a component of the brain network that regulates effort-related functions (Salamone and Correa, 2012). The prefrontal-accumbal cathecolamine and system has been demonstrated to play a critical role in processing both rewarding and aversive stimuli (Ventura et al., 2007). Moreover, the amygdala is involved in Pavlovian conditioning of emotional responses and modulates memory for arousing experiences (Balleine, 2005; Balleine and Killcross, 2006; McGaugh, 2006), and a complex anatomical and functional connection between the amygdala, medial prefrontal cortex (mpFC), and NAc has been reported (Del Arco and Mora, 2009 for review). Studies showed a crucial role of mpFC/amygdala system in both processing of rewarding and coping to aversive stimuli, including stress conditions (Robinson and Berridge, 1993; Becerra et al., 2001; Gottfried et al., 2002; Jensen et al., 2003; Borsook et al., 2007; Andolina et al., 2013; Rudebeck et al., 2013). A growing body of evidence indicates that the prefrontal aminergic system controls both dopamine (DA) release in the NAc and GABA release in the amygdala, sub-cortical areas that mediate the elaboration of rewarding and aversive stimuli. Moreover, an alteration of this process seems to characterize several psychopathologies, including anxiety, depression, and addiction. Twin and adoption studies have demonstrated a gene-environment interaction in the development of psychiatric disorders, indicating that genetic background modulates the capacity of an environmental risk factor to give rise to mental illness (Caspi and Moffitt, 2006). In preclinical studies, inbred strains are a useful tool to investigate the role of genetic factors, in interaction with aversive and rewarding experiences, in susceptibility to development and expression of psychopathology. Indeed, comparative study of brain neurotransmitter activity and behavior in different genetic backgrounds is a major strategy for determining the neural basis of rewarding and aversive effects in relation to individual differences. In particular, the C57 and DBA strains have allowed us to determine the impact of genetic background on corticolimbic system function.

## **REWARDING STIMULI**

The principal function of DA in motivational salience processes and in the elaboration of rewarding stimuli has been widely reported (Robinson and Berridge, 2001). Thus, increased DA transmission in the NAc mediates the rewarding/reinforcing effects of addictive drugs (Di Chiara and Imperato, 1988; Wise and Rompre, 1989; Pontieri et al., 1995; Koob et al., 1998; Robbins and Everitt, 1999; Ventura et al., 2003, 2005, 2007). However, recent evidence suggests major involvement of brain norepinephrine (NE) in the behavioral and central effects of rewarding pharmacological and natural stimuli (Darracq et al., 1998; Tassin, 1998; Drouin et al., 2001; Zarrindast et al., 2002; Ventura et al., 2007; Latagliata et al., 2010; Puglisi-Allegra and Ventura, 2012). Ventura et al. demonstrated that selective prefrontal NE depletion in mice abolished the increase of DA in the NAc induced by various classes of drugs of abuse and food (Ventura et al., 2003, 2005, 2007, 2008; Latagliata et al., 2010). Moreover, these studies reported that an intact prefrontal cortical NE is necessary for Conditioned Place Preference (CPP) induced by amphetamine, morphine, cocaine, ethanol, and chocolate as well as for reinstatement (relapse) of extinguished morphineinduced CPP and for ethanol intake in a choice test (Ventura et al., 2003, 2005, 2006, 2007; Latagliata et al., 2010). Thus, they demonstrate that prefrontal NE transmission is crucial for accumbal DA release induced by pharmacological and natural rewarding stimuli and processing the rewarding/reinforcing effects of these stimuli. Finally, in addition to the prefrontal noradrenergic/accumbal dopaminergic neuronal circuit, different studies showed a crucial role of mpFC/amygdala system in processing of rewarding stimuli (Robinson and Berridge, 1993; Becerra et al., 2001; Gottfried et al., 2002; Borsook et al., 2007; Rudebeck et al., 2013).

Inter-individual differences have been frequently reported in the elaboration of both rewarding and aversive stimuli. Genotypedependent control of corticolimbic neurotransmission could be responsible for individual differences in the elaboration of positive and aversive stimuli and thus be linked to different susceptibility to psychopathologies. Our findings in C57 and DBA mice support this hypothesis. Concerning the elaboration of rewarding stimuli, we showed differential effects of drugs of abuse (amphetamine), depending on genotype (**Table 1**). For instance, mice of DBA background are hyporesponsive to the behavioral effects of D-amphetamine, whereas C57 mice are highly responsive to the stimulating/reinforcing effects of amphetamine, as evidenced by increased locomotor activity, and amphetamineinduced CPP (Zocchi et al., 1998; Cabib et al., 2000; Ventura et al., 2004). Amphetamine produces lower prefrontal and higher accumbal DA levels, as well as higher locomotor activity, in C57 mice in comparison with DBA mice (Ventura et al., 2004). Moreover, selective prefrontal DA depletion in DBA mice leads to high DA outflow in the NAc and hyperlocomotion, comparable to those observed in C57 mice (Ventura et al., 2004). This evidence demonstrates that mesocortical DA controls the genotype-dependent effects of systemic amphetamine on mesoaccumbens DA release and locomotion. Nevertheless, as we have stressed, noradrenergic transmission in the mpFC has significant modulatory function on accumbal dopaminergic transmission and mediates the rewarding/reinforcing effects of addictive drugs. Because prefrontal DA inhibits NAc DA, whereas NE has been suggested to be enabling (Darracq et al., 1998), we hypothesized that an imbalance in NE/DA in the mpFC regulates DA in the NAc and the related behavioral outcomes, rendering the C57 strain more responsive than DBA. This hypothesis was confirmed by experiments that demonstrated that selective prefrontal cortical NE depletion abolishes the effects of amphetamine on DA in the accumbens and CPP in C57 mice (Ventura et al., 2003), whereas selective prefrontal DA depletion (sparing NE) leads to DA outflow in the NAc and behavioral outcomes in DBA mice, similar to those of C57 (Ventura et al., 2004, 2005). Thus, these evidences demonstrated that genotype-dependent susceptibility to the addictive properties of drugs of abuse (amphetamine) involves imbalanced DA and NE transmission in the mesocorticolimbic system.

# **AVERSIVE STIMULI**

Aversive pharmacological and natural experiences, such as lithium and stress administration, have been shown to activate the same prefrontal cortical-subcortical network affected by rewarding stimuli (Pascucci et al., 2007; Ventura et al., 2007, 2008). In fact, the authors observed that natural and pharmacological aversive stimuli induce a clear-cut increase of NE in the mpFC and DA in the NAcs that was abolished by selective prefrontal NE depletion (Ventura et al., 2007, 2008). Results concerning natural, non-pharmacological aversive experiences (restraint stress) have demonstrated the same prefrontal noradrenergic control on accumbal DA outflow in rats (Pascucci et al., 2007). This study showed that exposure to a novel stressor (restraint) promotes a rapid, massive, and transient increase in NE release in the mpFC, paralleling the rise in mesoaccumbens DA release (Pascucci et al., 2007). Selective prefrontal NE depletion prevents both the cortical NE response and the increase in accumbens DA release, thus confirming the modulatory function of prefrontal NE transmission in accumbal DA transmission induced also by aversive stimuli. All together, these data demonstrate that catecholaminergic transmission in the neural circuit comprising NAc and mpFC is crucial in both stress response and processing of negative and positive stimuli (Cabib et al., 1988; Le Moal and Simon, 1991; Pascucci et al., 2007; Ventura et al., 2007, 2013; Cabib and Puglisi-Allegra, 2012).

In addition to the mpFC noradrenergic/accumbal dopaminergic neuronal circuit, other brain areas and neurotransmitters, such amygdala GABAergic transmission, are likely to be engaged in these processes. Concerning the prefrontal-amygdala system, we have recently showed that amygdalar GABA regulation by prefrontal 5-HT is critical for processing stressful experiences and for determining passive coping outcomes, as measured by FST in mice (Andolina et al., 2013). We have demonstrated that a stressful experience, such as restraint, increases 5-HT levels in the mpFC and GABA levels in the amygdala and that selective depletion of cortical 5-HT canceled out these stress-induced responses, implicating prefrontal 5-HT in the control of GABAergic transmission in the amygdala during stress exposure. Sustained stress-induced 5-HT outflow in the mpFC and GABA outflow in the basolateral amygdala (BLA) lead to sustained immobility. However, a disconnection between prefrontal 5-HT and amygdalar (BLA) GABAergic transmission leads to low immobility in the FST (Andolina et al., 2013). These results highlight other critical neural mechanisms in the perceived motivational impact of stressful experiences in which the prefrontal/amygdala connectivity mediated by 5-HT and GABA transmission has a significant function. Concerning data from inbred strains of mice, our restraint and FST results in C57 and DBA mice showed a genotype control of corticolimbic neurotransmission (**Table 1**). We found that restraint stress inhibited mesoaccumbens DA release, which was accompanied by rapid and strong activation of mesocortical DA metabolism in C57 mice; the opposite pattern occurred in DBA mice, thus demonstrating genetic control over the balance between mesocortical and mesoaccumbens DA responses to stress (Ventura et al., 2001). Moreover, C57 but not DBA mice experienced high immobility in their first session of the FST and immediate and robust activation of mesocortical DA metabolism and inhibition of mesoaccumbens DA metabolism and release. In addition, the behavioral and mesoaccumbens DA responses to FST in C57 mice were reduced and reversed, respectively, by selective dopamine DA depletion in the mpFC (Ventura et al., 2002). These studies showed that, as with rewarding stimuli, the genetic background governs the susceptibility


**Table 1 | Summary of behavioral response and neurotrasmitter release to rewarding and aversive stimuli shown by C57 and DBA mice**.

↑ *increase or* ↓ *decrease in comparison with DBA. Abbrevation: CPP, Conditioned Place Preference; Loc Act, Locomotor activity; FST, Forced swimming test; mpFC, medial prefrontal cortex; NAc, nucleus accumbens; Amy, Amygdala; DA, Dopamine; 5-HT, Serotonin.*

to stressful experiences through the mesocortical-limbic DA response.

Another important neural network mediating stress responses is the mpFC-amygdala circuit that has been shown to be influenced by genotype (Holmes, 2008 for review). Specifically, consistent with the evidence that we have discussed, C57 mice have been reported to display greater immobility in the FST compared to DBA mice (Alcaro et al., 2002; Ventura et al., 2002). DBA and C57 mice are characterized by a different prefrontal 5-HT (Calcagno et al., 2007; Andolina et al., in press). In particular, DBA mice present lower 5-HT transporter binding and lower immobility in the FST than C57 (Sugimoto et al., 2008; Popova et al., 2009). Moreover, DBA mice are homozygous for the 1473G allele TPH-2, linked to low 5-HT synthesis rate, while C57BL/6 mice are homozygous for the 1473C allele. This allelic variant in DBA causes lower brain 5-HT synthesis than in C57BL/6 mice carrying the "C" allele (Zhang et al., 2004; Cervo et al., 2005). Moreover, differences between C57 and DBA mice have been reported for amygdala functioning which have been linked to strain-dependent difference in stress responsiveness (DuBois et al., 2006; Yang et al., 2008; Mozhui et al., 2010). Consistent with the evidences that genetic variation in cortico-amygdala system contributes to individual differences in stress response and to stress-related behavior, recently we reported that C57 mice show higher 5-HT outflow in the mpFC and higher GABA outflow in the BLA induced by stress (restraint) compared with the DBA strain. (Andolina et al., in press).

All together, these data indicate that strain-dependent prefrontal corticolimbic regulation, probably through different neurotransmitter systems including NE, DA, 5-HT, and GABA determines the differences in stress-coping behaviors in the FST in C57 and DBA mice.

# **CONCLUSION**

The evidences that we have discussed demonstrate that neurotransmission in the neural circuit comprising PFC, NAcs, and amygdala, is crucial for processing rewarding and aversive stimuli. Moreover, data on C57 and DBA strains demonstrate how the genetic background determines the intensity and effects of the response to positive and negative stimuli. Most of the experiments that we have discussed above describe how selective NE, DA, or 5- HT depletion in mpFC modifies the neurotransmitter response of subcortical structures, such as the NAc and amygdala. PFC sends glutamatergic outputs to subcortical areas, including the NAc and amygdala, that mediate motor, emotional, and mnemonic function. In a top-down model, alterations in PFC neurotransmission could modify the function of specific PFC cellular networks (Yang and Chen, 2005; Del Arco and Mora, 2009) and, consequently, the function of subcortical structures, including the NAc and amygdala. This could lead to the development of abnormal behaviors associated with psychiatric disorders, such as depression, anxiety, and addiction. Our data support this model, wherein the aminergic system in the mpFC has a central role in dopaminergic and GABAergic neurotransmission in the NAc and amygdala, respectively. Furthermore, our findings support a model of genotypedependent control of the prefrontal-accumbal-amygdala neural circuit, which could mediate the differential behavioral responses to many natural and pharmacological rewarding and aversive stimuli.

## **ACKNOWLEDGMENTS**

We thank Dr Sergio Papalia for his skillful assistance.

This research was supported by "Ricerca Corrente',' Italian Ministry of Health and Ateneo 2012, Sapienza University of Rome and by Ministero della Ricerca Scientifica e Tecnologica (FIRB 2010).

# **REFERENCES**


**Conflict of Interest Statement**: The Review Editor Antonella Gasbarri declares that, despite being affiliated to the same institution as author Diego Andolina, the review process was handled objectively and no conflict of interest exists. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 18 June 2014; accepted: 02 October 2014; published online: 04 February 2015*.

*Citation: Andolina D, Puglisi-Allegra S and Ventura R (2015) Strain-dependent differences in corticolimbic processing of aversive or rewarding stimuli. Front. Syst. Neurosci. 8:207. doi: 10.3389/fnsys.2014.00207*

*This article was submitted to the journal Frontiers in Systems Neuroscience*.

*Copyright © 2015 Andolina, Puglisi-Allegra and Ventura. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

# Valenced action/inhibition learning in humans is modulated by a genetic variant linked to dopamine D2 receptor expression

*Anni Richter 1, Marc Guitart-Masip2,3, Adriana Barman1, Catherine Libeau1, Gusalija Behnisch1, Sophia Czerney1, Denny Schanze4, Anne Assmann1, Marieke Klein1†, Emrah Düzel 5,6,7,8, Martin Zenker 4, Constanze I. Seidenbecher 1,8 and Björn H. Schott 1,8,9,10\**

*<sup>1</sup> Department of Neurochemistry and Molecular Biology, Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Magdeburg, Germany*

*<sup>2</sup> Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, London, UK*

*<sup>3</sup> Ageing Research Centre, Karolinska Institute, Stockholm, Sweden*


#### *Edited by:*

*Daniela Laricchiuta, IRCCS Santa Lucia Foundation, Italy*

#### *Reviewed by:*

*C. Nico Boehler, Ghent University, Belgium*

*Alexander Strobel, Technische Universitaet Dresden, Germany*

#### *\*Correspondence:*

*Björn H. Schott, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118 Magdeburg, Germany e-mail: bschott@neuro2. med.uni-magdeburg.de*

#### *†Present address:*

*Marieke Klein, Department of Human Genetics, Donders Institute for Brain, Cognition and Behaviour, Radboud University Medical Centre, Nijmegen, Netherlands*

Motivational salience plays an important role in shaping human behavior, but recent studies demonstrate that human performance is not uniformly improved by motivation. Instead, action has been shown to dominate valence in motivated tasks, and it is particularly difficult for humans to learn the inhibition of an action to obtain a reward, but the neural mechanism behind this behavioral specificity is yet unclear. In all mammals, including humans, the monoamine neurotransmitter dopamine is particularly important in the neural manifestation of appetitively motivated behavior, and the human dopamine system is subject to considerable genetic variability. The well-studied TaqIA restriction fragment length polymorphism (rs1800497) has previously been shown to affect striatal dopamine metabolism. In this study we investigated a potential effect of this genetic variation on motivated action/inhibition learning. Two independent cohorts consisting of 87 and 95 healthy participants, respectively, were tested using the previously described valenced go/no-go learning paradigm in which participants learned the reward-associated no-go condition significantly worse than all other conditions. This effect was modulated by the TaqIA polymorphism, with carriers of the A1 allele showing a diminished learning-related performance enhancement in the rewarded no-go condition compared to the A2 homozygotes. This result highlights a modulatory role for genetic variability of the dopaminergic system in individual learning differences of action-valence interaction.

#### **Keywords: dopamine D2 receptor, TaqIA, reward learning, motivated learning, action bias**

# **INTRODUCTION**

Efficient decision making requires an individual to select responses that maximize reward and minimize punishment or loss. Such motivated behavior involves two fundamental axes of control, namely valence—spanning reward and punishment, and action—spanning invigoration and inhibition. Previous studies have shown that these two axes are not independent (Guitart-Masip et al., 2012b, 2013; Cavanagh et al., 2013; Chowdhury et al., 2013; for review see Guitart-Masip et al., 2014) and that decision making is not only influenced by an instrumental controller that learns to optimize choices on the basis of their contingent consequences, but also on a Pavlovian controller that generates stereotyped, "hard-wired" behavioral responses to the occurrence of motivationally salient outcomes or learned predictions of such outcomes (Dickinson and Balleine, 2002; Guitart-Masip et al., 2013). The presence of such "hard-wired" response patterns may be an evolutionarily beneficial adaptation to an environment world in which obtaining a reward typically requires some sort of overt behavioral response (*go to win*) whereas avoiding a punishment rather requires an avoidance of those actions that may lead to it (*no-go to avoid losing*). On the other hand, such a response bias may also be a source of suboptimal behavior when Pavlovian and instrumental controllers are in opposition (Breland and Breland, 1961; Dayan et al., 2006; Boureau and Dayan, 2011).

In order to manipulate action and valence orthogonally, Guitart-Masip et al. (2012b) designed a go/no-go learning task that involves besides the commonly investigated conditions *go to win* and *no-go to avoid losing* also the *vice versa* conditions where

*<sup>4</sup> Institute of Human Genetics, Otto von Guericke University of Magdeburg, Magdeburg, Germany*

the participant needs to perform an action to avoid a punishment (*go to avoid losing*) or to inhibit an action to obtain a reward (*nogo to win*). Studies employing this task have repeatedly shown that while active choices in rewarded conditions and passive choices in punished conditions can be learned easily, it is significantly harder to learn an approach behavior to avoid a punishment and yet even more difficult to inhibit an action to obtain a reward. This asymmetry indicates that signals that predict reward are prepotently associated with behavioral activation, whereas signals that predict punishment are intrinsically coupled to behavioral inhibition.

In search for neural mechanisms underlying this behavioral asymmetry in the coupling between action and valence, monoaminergic, particularly dopaminergic, neuromodulation is a prime candidate (Gray and McNaughton, 2000; Boureau and Dayan, 2011; Cools et al., 2011). Dopamine (DA) is believed to enable or enhance the generation of active motivated behavior (Berridge and Robinson, 1998; Niv et al., 2007; Salamone et al., 2007; Beierholm et al., 2013) and to support instrumental learning (Frank et al., 2004; Daw and Doya, 2006; Wickens et al., 2007). It has been observed that DA depletion leads to decreased motor activity and decreased motivated behavior (Ungerstedt, 1971; Palmiter, 2008), along with decreased vigor or motivation to work for rewards in demanding reinforcement schedules (Salamone et al., 2005; Niv et al., 2007). Conversely, boosting DA levels with levodopa invigorates motor responses in healthy humans (Guitart-Masip et al., 2012a) and DA promotes "go" and impairs "no-go" learning, for example in patients with Parkinson's disease (Frank et al., 2004). However, contrary to the expectations suggested by this evidence, administration of levodopa reduced the learning disadvantage of the *no-go to win* condition when compared to the *no-go to avoid losing* (Guitart-Masip et al., 2013). These effects suggested that DA is involved in decreasing the coupling between action and valence, supposedly via DA's actions on neural functions implemented in prefrontal cortex (Hitchcott et al., 2007). It is therefore unclear how striatal DA modulates the coupling between action and valence uncovered in this task.

The aim of the present study was to test whether naturally occurring differences in healthy humans in this valenced action/inhibition learning might arise from dopaminergic mechanisms and how striatal DA effects the action/valence interaction. To address this issue, we used the valenced go/no-go learning paradigm in a cohort of young, healthy subjects, and tested them for the TaqIA restriction length polymorphism (rs1800497), a common genetic variation of the dopamine D2 receptor (DRD2) gene known to affect D2 receptor expression and striatal DA metabolism. Although the underlying molecular mechanisms are yet not fully understood, the TaqIA polymorphism has been repeatedly associated with reduced striatal DRD2 density in A1 carriers as evident from three *post mortem* studies (Noble et al., 1991; Thompson et al., 1997; Ritchie and Noble, 2003) and two out of three conducted *in vivo* binding studies (Laruelle et al., 1998; Pohjalainen et al., 1998; Jonsson et al., 1999). Laakso et al. (2005) suggested that the lower D2 receptor expression leads to decreased autoreceptor function, thereby increasing the DA and/or trace amine synthesis rate in the brains of A1 allele carriers. Moreover, Kirsch et al. (2006) observed an increase of striatal BOLD signal in response to the dopamine D2 receptor agonist bromocriptine in subjects carrying the A1 allele, but not in subjects without the A1 allele, and Stelzel et al. (2010) reported a generally increased striatal BOLD signal in A1 carriers. As striatal BOLD signal has been shown to correlate with DA release (Schott et al., 2008), the increased striatal activation in A1 carriers might be related to higher presynaptic dopaminergic activity (Richter et al., 2013). Because striatal DA is associated with linking action with reward (Berridge and Robinson, 1998; Frank et al., 2004; Daw and Doya, 2006; Niv et al., 2007; Salamone et al., 2007; Wickens et al., 2007; Beierholm et al., 2013), we hypothesized that A1 carriers might show increased coupling between action and valence.

# **MATERIALS AND METHODS**

## **PARTICIPANTS**

Participants were recruited from a cohort of 719 young healthy volunteers of Caucasian ethnicity of a large-scale behavioral genetic study conducted at the Leibniz Institute for Neurobiology, Magdeburg. Given our hypothesis regarding differential performance in the valenced go/no-go task as a function of striatal D2 receptor availability, we selected participants a priori as a function of DRD2 TaqIA genotype. To control for confounding effects of genetic influences on prefrontal DA availability, we also ensured a balanced distribution of the COMT Val108/158 Met polymorphism that is known to affect prefrontal DA levels and D1 receptor binding (Gogos et al., 1998; Matsumoto et al., 2003; Meyer-Lindenberg et al., 2005; Slifstein et al., 2008). All participants were right-handed according to self-report, not genetically related, and had obtained at least a university entrance diploma (Abitur) as educational certificate. Importantly, all participants had undergone routine clinical interview to exclude present or past neurological or psychiatric illness, alcohol, or drug abuse, use of centrally acting medication, the presence of psychosis or bipolar disorder in a first-degree relative, and additionally, given the design of the experiment, regular gambling. Two independent cohorts of healthy participants were tested (cohort 1: 43 females and 44 males; age: range 19–36 years, mean 24.6 years, *SD* = 3*.*1 years; cohort 2: 48 females and 47 males; age: range 20–33 years, mean 24.6 years, *SD* = 2*.*8 years). Because of a previously reported potential association of the A1 allele with nicotine consumption (Verde et al., 2011; for reviews see Comings and Blum, 2000; Lerman et al., 2007), smoking status was assessed from the participants. All participants gave written informed consent in accordance with the Declaration of Helsinki and received financial compensation for participation. The work was approved by the Ethics Committee of the University of Magdeburg, Faculty of Medicine.

# **GENOTYPING**

The DRD2/ANKK1 TaqIA restriction length polymorphism (NCBI accession number: rs1800497) was genotyped using a protocol previously described in Richter et al. (2013). Genomic DNA was extracted from blood leukocytes using the GeneMole® automated system (Mole Genetics AS, Lysaker, Norway) according to the manufacturer's protocol. Genotyping was performed using PCR followed by allele-specific restriction analysis using previously described primers (Grandy et al., 1989). Genotyping was also performed for several additional polymorphisms, including COMT Val108/158 Met (see **Table 1**), to control for confounding effects of other genetic variants and to reduce the risk of population stratification.

# **PARADIGM**

We used a previously employed go/no-go learning task with orthogonalized action requirements and outcome valence (Guitart-Masip et al., 2012b, 2013; Chowdhury et al., 2013). The trial timing is displayed in **Figure 1**. Each trial consisted of presentation of a fractal cue, a target detection task, and a probabilistic outcome. First, one out of four abstract fractal cues was displayed for 1000 ms. Participants were informed that a fractal indicated whether they would subsequently be required to perform a target detection task by pressing a button (go) or not (no-go) and that the cue also indicated the possible valence of the outcome of the subjects' behavior (reward/no reward or punishment/no punishment). However, subjects were not instructed about the contingencies for each fractal image and had to learn them by trial and error. The meaning of the fractal images was randomized across participants. Following a variable interval (250–3500 ms) after offset of the fractal image, the target detection task started: participants had the opportunity to press a button within a time limit of 2000 ms to indicate the side of a circle for go trials, or not to press for no-go trials. After the offset of the circle after 1500 and 1000 ms of fixation, subjects were presented with the outcome. The outcome remained on screen for 2000 ms and after a variable intertrial interval (ITI; 750–1500 ms) a new trial started. Participants were informed that the outcome was probabilistic: in *win* trials 80% of correct choices and 20% of incorrect choices were rewarded with 0.50 C (the remaining 20% of correct and 80% of incorrect choices leading to no outcome), while in *avoid losing* trials 80% of correct choices and 20% of incorrect choices avoided a loss of 0.50 C (the remaining 20% of correct and 80% of incorrect choices leading to a punishment). Thus, there were four trial types depending on the nature of the fractal cue presented at the beginning of the trial: press the correct button in the target detection task to gain a reward (*go to win*); press the correct button in the target detection task to avoid punishment (*go to avoid losing*); do not press a button in the target detection task to gain a reward (*no-go to win*); do not press a button in the target detection task to avoid punishment (*no-go to avoid losing*). The task included 240 trials, 60 trials per condition and was divided into four sessions 9 min each (15 trials per condition in randomized order). Subjects were told that they would be paid their earnings of the task up to a total of 25 C and a minimum of 7 C. Before starting with the learning task, subjects performed 10 trials of the target detection task in order to get familiarized with the speed requirements.

## **STATISTICAL ANALYSIS**

The percentage of correct choices in the target detection task (correct button press for go conditions and correct omission of responses in no-go trials) was collapsed across time bins of 30 trials per condition and analyzed with a mixed ANOVA with time (1st/2nd half), action (go/no-go), and valence (win/lose) as within-subject factors and TaqIA genotype (A1+/A1−) as between-subject factor. Additionally reaction times of correct


valence (reward or loss). On go trials, subjects press a button for the side of a circle. On no-go trials they withhold a response. Arrows indicate rewards (green) or losses (red). Horizontal bars (yellow) symbolize the absence of a win or a loss. ("go"; middle), and the possible outcomes and their probability after withholding a response to the target ("no-go"; right). gw, go to win; gal, go to avoid losing; ngw, no-go to win; ngal, no-go to avoid losing; ITI, intertrial interval.

go responses (RTs) were analyzed using a mixed ANOVA with valence (win/lose) and TaqIA genotype (A1+/A1−) as factors. When appropriate, paired *t*-test, independent sample *t*-test or Mann-Whitney *U*-test were used as *post-hoc* tests.

The analysis of the behavioral data was done in two stages. In cohort 1 we included the TaqIA and the COMT Val108/158 Met polymorphism as between-subject factors. In the second we specifically aimed to replicate the significant effect of TaqIA. The following statistics include TaqIA as the only between-subject factor.

# **RESULTS**

# **GENOTYPING**

Genotyping was performed in the entire cohort of 719 subjects, and two sub-cohorts were recruited based on the DRD2/ANKK1 TaqIA genotype. The data of 87 participants in cohort 1 and 95 participants in cohort 2 were analyzed. In cohort 1, we identified 4 A1 homozygotes, 33 heterozygotes and 50 A2 homozygotes. In cohort 2, genotyping revealed 4 A1 homozygotes, 30 heterozygotes and 61 A2 homozygotes. The distributions in both groups were at Hardy-Weinberg equilibrium (cohort 1: <sup>χ</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>*.*24, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*621; cohort 2: <sup>χ</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>*.*02, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*898). A1 carriers (A1+: A1/A1 and A1/A2) were grouped together for all subsequent analyses as in previous behavioral and imaging studies of the TaqIA polymorphism (Stelzel et al., 2010; Richter et al., 2013). The groups A1+ and A1− (A2/A2) did not differ in gender, in age or in the number of smokers and nonsmokers (**Table 2**).

To control for effects of prefrontal DA availability, participants were also selected regarding the COMT Val108/158 Met (NCBI accession number: rs4680) polymorphism. Genotyping revealed 31 Met/Met, 29 Val/Met, and 27 Val/Val carriers in cohort 1 and 30 Met/Met, 41 Val/Met, and 24 Val/Val carriers in cohort 2. Allelic distribution for the COMT Val108/158 Met polymorphism did not differ significantly for either TaqIA A1 carriers or A2 homozygotes (**Table 2**). The experimenters who performed the behavioral task were blinded regarding DRD2/ANKK1 and COMT genotypes.

To further control for effects of population stratification and potential effects of putatively functional genetic variations in the dopamine system, genotyping was also performed for the

#### **Table 2 | Demographic data.**


*Gender distribution, age (means* ± *standard deviations), number of smokers and nonsmokers. Allelic distributions for following polymorphisms: COMT Val108/158 Met (mm, met homozygotes; vm, val/met heterozygotes; mm, met homozygotes), DAT1-VNTR (9*+*, carriers of the 9-repeat allele 9/9 and 9/10; 9*−*, 10-repeat homozygous subjects 10/10), C957T (CC/CT/TT carriers) and DARPP-32 (CC/CT/TT carriers). A1*+*; carriers of the A1 allele. A1*−*; A2 homozygotes.*

DAT1-VNTR (NCBI accession number: rs28363170), the C957T polymorphism within the DRD2 gene (NCBI accession number: rs6277) and the DARPP-32 polymorphism (NCBI accession number: rs907094) (see **Table 1**). Allelic distributions for the DAT1-VNTR polymorphism did not differ significantly for either TaqIA A1 carriers or A2 homozygotes (**Table 2**). However, because of differences for the C957T and the DARPP-32 polymorphism, we additionally calculated an ANCOVA including these two polymorphisms as covariates (see below).

#### **BEHAVIORAL RESULTS**

We initially performed an omnibus mixed-design ANOVA to test for effects of both DRD2/ANKK1 and COMT genotypes. There was a significant four-fold interaction of DRD2/ANKK1 TaqIA with action, time and valence [*F*(1*,*81) = 5*.*11, *p* = 0*.*027], but no effect of COMT Val108/158 Met polymorphism (all *p >* 0*.*120). All further analyses were therefore focused on the DRD2/ANKK1 TaqIA polymorphism. We computed as ANOVA for repeated measures on the percentage of correct (optimal) choices with action (go/no-go), valence (win/lose) and time (1st/2nd half) as within-subject factors and genotype (A1+/A1−) as betweensubject factor. See **Table 3** for statistics.

Our study reproduced a main effect of action [cohort 1: *F*(1*,* 85) = 62*.*56, *p <* 0*.*001; cohort 2: *F*(1*,* 93) = 50*.*87, *p <* 0*.*001] and an action by valence interaction [cohort 1: *F*(1*,* 85) = 44*.*41, *p <* 0*.*001; cohort 2: *F*(1*,* 93) = 37*.*72, *p <* 0*.*001], as demonstrated in previous studies (Guitart-Masip et al., 2012b, 2013; Cavanagh et al., 2013; Chowdhury et al., 2013). Subjects showed better performance in conditions requiring a go choice than in trials requiring a no-go choice [cohort 1: *t*(86) = 7*.*97, *p <* 0*.*001; cohort 2: *t*(94) = 7*.*68, *p <* 0*.*001], and while they were better at learning from reward as compared to punishment in the go condition [cohort 1: *t*(86) = 6*.*28, *p <* 0*.*001; cohort 2: *t*(94) = 5*.*74, *p <* 0*.*001], this relation reversed in the no-go condition [cohort 1: *t*(86) = 4*.*99, *p <* 0*.*001; cohort 2: *t*(94) = 4*.*63, *p <* 0*.*001]. As Guitart-Masip et al. (2012b, 2013) we also observed a main effect of time [cohort 1: *F*(1*,* 85) = 135*.*92, *p <* 0*.*001; cohort 2: *F*(1*,* 93) = 189*.*21, *p* =*<* 0*.*001] and additionally an action by time interaction [cohort 1: *F*(1*,* 85) = 19*.*09, *p <* 0*.*001; cohort 2: *F*(1*,* 93) = 59*.*77, *p <* 0*.*001], indicating a preponderant initial bias toward go responses [cohort 1: *t*(86) = 4*.*62, *p <* 0*.*001; cohort 2: *t*(94) = 8*.*46, *p <* 0*.*001].

Most interestingly for the current study, we observed a fourfold interaction of action by valence by time by genotype [cohort 1: *F*(1*,* 85) = 5*.*24, *p* = 0*.*025; cohort 2: *F*(1*,* 93) = 4*.*59, *p* = 0*.*035]. This effect was observed in the absence of an action by valence by genotype effect (cohort 1: *p* = 0*.*811; cohort 2: *p* = 0*.*087). While the genotype groups did not differ significantly in their mean performance in the first and second time bin in any condition (cohort 1: *p >* 0*.*143; cohort 2: *p >* 0*.*167), they showed a different degree of improvement from the first to the second time interval (learning gain: mean performance 2nd half—mean performance 1st half; see **Figure 2**). Performance of the A2 homozygotes in the *no-go to win* condition showed increased improvement from the first to the second half of the experiment compared to the A1 carriers [cohort 1: *t*(85) = 2*.*78, *p* = 0*.*007]. In the second cohort this result was replicated [cohort 2: *t*(93) = 2*.*16, *p* = 0*.*033], and A1 carriers showed lower performance in the *go to avoid losing* condition [cohort 2: *t*(93) = 2*.*26, *p* = 0*.*026]. Because performance in the *no-go to win* condition during early trials differed between the two cohorts, we tested whether the observed interaction, which would likely reflect a difference in learning rate, remained significant when combining both datasets. A Three-Way ANCOVA across both cohorts (including cohort as a covariate of no interest; see **Figure 2**) revealed the same three-way interaction revealed by the analyses


*Means* ± *standard deviations are shown. Only effects that were significant in both cohorts are reported. ANOVA was computed with percent correct responses as dependent variable and action, valence, time and genotype as independent variables. Paired t-tests and t-tests for independent samples were performed as post-hoc tests. gw, go to win; gal, go to avoid losing; ngw, nogo to win; ngal, no-go to avoid losing. A1*+*; carriers of the A1 allele. A1*−*; A2 homozygotes.*

in the separate cohorts [*F*(1*,* 179) = 9*.*87, *p* = 0*.*002]. Only in one cohort there was a statistically significant three-way interaction [action by valence by time; cohort 1: *F*(1*,* 85) = 0*.*42, *p* = 0*.*517; cohort 2: *F*(1*,* 93) = 10*.*98, *p* = 0*.*001] and a time by genotype interaction [cohort 1: *F*(1*,* 85) = 3*.*77, *p* = 0*.*055; cohort 2: *F*(1*,* 93) = 6*.*31, *p* = 0*.*014].

Statistics regarding reaction times (RTs) of the go responses are summarized in **Table 4**. We computed an ANOVA with valence (win/lose) as within-subject factor and genotype as betweensubject factor. Irrespective of genotype, RTs in the *go to win* condition were shorter than in the *go to avoid losing* condition [cohort 1: *F*(1*,* 85) = 14*.*06, *p <* 0*.*001; cohort 2: *F*(1*,* 93) = 11*.*21, *p* = 0*.*001]. Regarding DRD2/ANKK1 TaqIA genotype, there was only a trendwise interaction with valence [*F*(1*,* 93) = 3*.*38, *p* = 0*.*069] and a trend for a main effect [*F*(1*,* 93) = 3*.*67, *p* = 0*.*058] in cohort 2, with the A1 carriers being slower in avoiding punishment as compared to the A2 homozygotes [*t*(93) = 2*.*04, *p* = 0*.*046]. Although this nominal effect together with the worse accuracy of the A1 carriers in the *go to avoid losing* condition (**Figure 2**) hints at a worse performance of the A1 carriers in this condition, the interpretation of this result warrants caution as the effects were only apparent in cohort 2 and, moreover, participants were explicitly instructed to respond accurately, while speed was not emphasized.

To rule out that the genotype effects are not simply explained by differences in target detection performance the percentage of trials in which subjects responded incorrectly in the target detection task (i.e., left when the target was on the right side of the display or vice versa) was measured and did not differ significantly between genotype groups (Mann-Whitney *U*-test: cohort 1: A1+: *M* ± *SD* = 1± 3%, A1−: *M* ± *SD* = 1± 2%, *z* = −0.334, *p* = 0*.*738; cohort 2: A1+: *M* ± *SD* = 1± 3%, A1−: *M* ± *SD* = 0± 1%, *z* = −0.428, *p* = 0*.*668).

Because the TaqIA polymorphism is located downstream of the DRD2 gene, the observed genotype effects might putatively result from linkage disequilibrium with other DRD2 polymorphisms, including the C957T. We indeed observed an imbalanced distribution of the C957T polymorphism (rs6277) among TaqIA A1 carriers vs. A2 homozygotes numerically in the first cohort (χ<sup>2</sup> <sup>=</sup> <sup>4</sup>*.*04, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*132) and significantly in the second cohort (χ<sup>2</sup> <sup>=</sup> <sup>25</sup>*.*49, *<sup>p</sup> <sup>&</sup>lt;* <sup>0</sup>*.*001). Moreover, the DARPP-32 polymorphism (rs907094) was unequally distributed in the second cohort only (χ<sup>2</sup> <sup>=</sup> <sup>8</sup>*.*53, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*014). In order to rule out confounding effects, we included the polymorphisms as covariates in an additional ANCOVA. The same was done for COMT Val108/158 Met (rs4680), because the cohorts were stratified with respect to that polymorphism. Importantly, the four-fold action by valence by time by genotype interaction for the TaqIA polymorphism remained significant [cohort 1: *F*(1*,* 82) = 4*.*63, *p* = 0*.*034, cohort 2: *F*(1*,* 90) = 5*.*07, *p* = 0*.*027], while there was no effect for C957T (cohort 1: *p* = 0*.*472, cohort 2: *p* = 0*.*810), DARPP-32 (cohort 1: *p* = 0*.*578, cohort 2: *p* = 0*.*148) or COMT Val108/158 Met polymorphism (cohort 1: *p* = 0*.*161, cohort 2: *p* = 0*.*856).

# **DISCUSSION**

The goal of this study was to investigate how a genetic variant linked to striatal DA responsivity affects the action/valence interaction. To this end, two independent cohorts consisting of 87 and 95 healthy participants were genotyped for the well-characterized DRD2/ANKK1 TaqIA polymorphism (Grandy et al., 1989; Dubertret et al., 2004; Neville et al., 2004) and performed the previously described valenced go/no-go task (Guitart-Masip et al., 2012b, 2013, 2014; Cavanagh et al., 2013; Chowdhury et al., 2013). Our results show differential learning performance in the carriers of the less common A1 allele of the TaqIA polymorphism, which has previously been linked to lower striatal dopamine D2 receptor expression. Replicating previous results,

**independent cohorts and in the entire sample (data of both cohorts combined).** Line charts at the left show mean values of correct responses (±s.e.m.) in A1 carriers (red) and A2 homozygotes (blue) in the first and the second half of trials for all four conditions. Bar plots at the right show the differences between mean (±s.e.m.) values of correct (red) and A2 homozygotes (blue) for each condition. This score represents the four-fold interaction of action by valence by time by genotype. Compared to the A2 homozygotes carriers of the A1 allele showed a diminished learning to withhold an action to receive a reward. *Post-hoc* comparisons via *t*-test: ∗*p <* 0*.*05.

participants were, irrespective of genotype, more successful in learning active choices in rewarded conditions and passive choices in punished conditions, with response inhibition to obtain a reward (*no-go to win*) being the condition most difficult to learn. The DRD2 TaqIA polymorphism exerted a modulatory influence on learning performance in the *no-go to win* condition with A1 carriers showing lower learning rates throughout the experiment.

It has to be emphasized that, despite the fact that in the present study learning curves of the two cohorts differed to some extent and initial performance of A1 carriers was not identical, we did yet observe a replicable attenuation of learning rates in A1 carriers that was specific to the *no-go to win* condition, and, importantly, the effect was even more pronounced when combining both datasets (using cohort as a covariate of no interest; see **Figure 2**).



*Means* ± *standard deviations are shown. A1*+*; carriers of the A1 allele. A1*−*; A2 homozygotes.*

It is important to note that there are two potential mechanisms by which valence can disrupt the choice of appropriate actions in the current task. The first mechanism is implemented at the time of the choice and can be seen as "Pavlovian" mechanism by which the anticipation of reward or punishment promotes action or inhibition, respectively (Dayan et al., 2006; Huys et al., 2011; Guitart-Masip et al., 2012b). The second mechanism is implemented at the time of outcome and is related to the role of DA within the striatum. According to a prevalent view in reinforcement learning and decision making, DA neurons signal reward prediction errors (Montague et al., 1996; Schultz et al., 1997; Bayer and Glimcher, 2005), in the form of phasic bursts for positive prediction errors and dips below baseline firing rate for negative prediction errors (Bayer et al., 2007), resulting in corresponding peaks and dips of dopamine availability in target structures, most prominently the striatum (McClure et al., 2003; O'Doherty et al., 2003, 2004; Pessiglione et al., 2006). In the striatum, increases of DA in response to an unexpected reward reinforce the direct pathway via activation of D1 receptors and thereby facilitate the future generation of go choices under similar circumstances, while dips in DA levels in response to an unexpected punishment reinforce the indirect pathway via reduced activation of D2 receptors and thus facilitate the subsequent generation of no-go choices in comparable situations (Frank et al., 2004, 2007; Wickens et al., 2007; Hikida et al., 2010; see **Figure 3**).

The effects related to the TaqIA polymorphism observed in the present study apparently reflect changes in the learning process, thus likely pointing to the function of DA in the ability to flexibly learn go or no-go choices based on the outcomes produced by previous actions. Our results are in apparent contrast to the effects previously reported in the same task after administration of levodopa. In that study, boosting DA levels resulted in a decoupling between action and valence that did not reflect any changes in the rate of learning (Guitart-Masip et al., 2013). Instead, the effects observed in that study boosted the asymptote reached by the participants that received levodopa. Using computational modeling, that effect was best characterized as a decreased influence of a Pavlovian control mechanism over the instrumental control mechanisms attempting to learn the task (Guitart-Masip et al., 2013). Similarly, in older adults, structural MRI measures of substantia nigra/ventral tegmental area (SN/VTA) integrity have also been linked to improved learning and a lower action bias (Chowdhury et al., 2013). One proposed explanation for the reduced coupling between action and valence in conditions associated with increased DA availability has been a likely increase of dopaminergic activity in the prefrontal cortex where DA influences the balance between different control mechanisms (Hitchcott et al., 2007). The implication of a prefrontal mechanism decreasing the Pavlovian influences on behavior and supporting performance of the *no-go to win* condition in this task has been shown in fMRI (Guitart-Masip et al., 2012b) and EEG experiments (Cavanagh et al., 2013). It should be noted, though, that, in the present study, we did not observe any behavioral differences as a function of the COMT Val108/158 Met polymorphism, which has previously been linked to prefrontal dopamine availability (Meyer-Lindenberg et al., 2005).

Receptor binding studies *in vitro* and *in vivo* have shown that A1 carriers show lower striatal D2 receptor expression (Noble et al., 1991; Thompson et al., 1997; Pohjalainen et al., 1998; Jonsson et al., 1999; Ritchie and Noble, 2003). On the other hand, A1 carriers also exhibit increased striatal DA synthesis, possibly as a result of reduced autoinhibitory signaling from presynaptic D2-type autoreceptors (Laakso et al., 2005). Previous behavioral and neuroimaging studies have in fact yielded results that would be best explained by parallel reduction of striatal postsynaptic D2 receptors and increased presynaptic dopaminergic activity in A1 carriers, with the latter also resulting in increased DA availability both in the striatum and in extrastriatal regions (Kirsch et al., 2006; Stelzel et al., 2010; Richter et al., 2013). According to those observations, A1 carriers would be assumed to show a less pronounced decrease of dopaminergic signaling after negative prediction errors in the indirect pathway and a shift to a more action-oriented behavioral pattern mediated by the direct pathway (**Figure 3**). Such a pattern bears some resemblance to the concept of behavioral impulsivity (Tomie et al., 1998; Flagel et al., 2010, 2011), and it is noteworthy in this context that the A1 allele has been linked to risk for impulsivity-related psychiatric disorders, most prominently alcohol dependence (Noble et al., 1991; Comings et al., 1996; Noble, 2003; Eisenberg et al., 2007; Wang et al., 2013). However, this does not explain, why A1 carriers exhibit a relatively specific performance disadvantage in the *no-go to win*, but not in the *no-go to avoid losing* condition. One possible reason would be that a punishment instead of a neutral feedback in the *no-go to avoid losing* condition might lead to a higher prediction error as compared to a neutral feedback instead of a reward in the *no-go to win* condition. Another reason might be that, for example, serotonin plays a specific role in punishmentrelated behavior (Daw et al., 2002; Boureau and Dayan, 2011; Cools et al., 2011; Guitart-Masip et al., 2012b, 2013; Den Ouden et al., 2013) and thus further modulates the performance in the *no-go to avoid losing* condition.

The investigation of modulators of stereotyped hard-wired behavioral responses is of interest to clinicians as it may help to develop novel treatment approaches for neurological or psychiatric disorders. The TaqIA polymorphism is one of the most extensively studied genetic variations in neuropsychiatric disorders with presumed dopaminergic dysfunction, and studies have pointed to a potential pleiotropic effect with A1 allele carriers showing an increased risk for addiction, but a lower risk for schizophrenia (e.g., Comings et al., 1996; Noble, 2003; Dubertret et al., 2004; Wang et al., 2013; Zhang et al., 2014). Moreover,

prediction errors in the form of phasic bursts for positive prediction errors and dips below baseline firing rate for negative prediction errors. Increases of DA in response to an unexpected reward reinforce the direct pathway via activation of D1 receptors and thereby facilitate the future generation of go choices under similar circumstances, while dips indirect pathway via reduced activation of D2 receptors and thus facilitate the subsequent generation of no-go choices in comparable situations. A1 carriers have less D2 receptors and thus would be assumed to have less limitation of dopaminergic signaling after negative prediction errors in the indirect pathway and a shift to a more action-oriented behavioral pattern mediated by the direct pathway.

studies in healthy humans have suggested a role of the TaqIA A1 variant in approach-related personality traits (Noble et al., 1998; Reuter et al., 2006; Lee et al., 2007; Smillie et al., 2010) and on motivated interference processing (Richter et al., 2013). The relation between the single nucleotide polymorphism (SNP) and instrumental learning has also been investigated. Previous studies have shown an impairment of the carriers of the A1 allele in no-go learning to avoid behaviors that yield negative outcomes (Klein et al., 2007; Frank and Hutchison, 2009; Jocham et al., 2009). However, those studies have only used conditions in which participants had to approach a reward or avoid a punishment. Since the interaction between action and valence has a pivotal influence on instrumental learning (Guitart-Masip et al., 2012b), such studies could not provide information on possible action by valence interactions, and the use of the valenced go/no-go-learning task with orthogonalized action and valence enables a more precise investigation of the contribution of the dopaminergic system in behavioral adaptation.

The TaqIA polymorphism, initially identified to be located on the DRD2 gene on human chromosome 11q22–23 (Grandy et al., 1989), is located 10kb downstream of the DRD2 termination codon on 11q23.1, within coding region of the adjacent ankyrin repeat and kinase domain containing 1 (ANKK1) gene (Dubertret et al., 2004; Neville et al., 2004). Because the DRD2 and ANKK1 genes are closely linked (Neville et al., 2004; Ponce et al., 2009), it has been proposed that genetic variations in linkage disequilibrium (LD) with the SNP might explain the observed relationship between the TaqIA and alterations of human dopaminergic neurotransmission. The SNP is indeed in LD with several polymorphisms on the DRD2 gene (Duan et al., 2003; Ritchie and Noble, 2003; Fossella et al., 2006) and one of them is the C957T polymorphism (rs6277) for which also modulations on instrumental learning have been observed (Frank et al., 2007, 2009; Frank and Hutchison, 2009). However, its influence on dopaminergic neurotransmission is not clear since *in vivo* and *in vitro* data are in conflict (Duan et al., 2003; Hirvonen et al., 2004; see also erratum by Hirvonen et al., 2004, 2009a,b) and no association was found between C957T and DA synthesis capacity *in vivo* (Laakso et al., 2005) and C957T and D2 receptor mRNA expression in *post mortem* brain tissue (Zhang et al., 2007). When controlling for a potential influence of this SNP in our analysis, the effect of TaqIA genotype was still significant. We cannot rule out, though, that another variant in the DRD2 gene—or perhaps in the ANKK1 gene—linked to TaqIA might be responsible for the observed genotype-related differences in learning rate.

In order to control for genetic influences of another genetic variant known to affect prefrontal DA levels and thereby cortical D1 receptor stimulation (Gogos et al., 1998; Matsumoto et al., 2003; Meyer-Lindenberg et al., 2005; Slifstein et al., 2008) we selected our participants to have comparable distributions of the COMT Val108/158 Met genotype. Importantly, the allelic distribution of COMT Val108/158 Met alleles did not differ significantly between TaqIA A1 carriers and A2 homozygotes.

It must nevertheless be kept in mind that genetic variations within the dopaminergic system do not exert their effects in isolation. Frank et al. (2007), for example, observed multiple roles for DA in reinforcement learning when investigating effects of the COMT Val108/158 Met, the DARPP-32, and the DRD2 C957T polymorphism on reward-based probabilistic learning. Even though we controlled for these polymorphisms in our experiment, we cannot completely rule out gene-gene interactions. Our moderately large sample sizes allowed us to examine effects of single genetic variants on behavioral outcomes, but the systematic analysis of gene-gene interactions would require substantially larger cohorts. In addition to the likely polygenic contribution of variants in the dopaminergic system to action by valence interaction, also other neuromodulatory transmitters must be considered in future studies.

# **CONCLUSION**

Our findings provide further evidence for a potential genetic basis of individual differences in probabilistic learning and, more specifically, suggest that genetically mediated differences in dopaminergic neuromodulation not only affect learning *per se*, but also can specifically affect behavioral phenomena like a Pavlovian action bias when a reward is expected. With respect to future research directed at individual differences in learning, our findings should thereby caution researchers to take into account the non-orthogonal nature of action by valence interactions.

#### **ACKNOWLEDGMENTS**

The authors would like to thank Iris Mann for help with testing and Maria Michelmann for help with genotyping. This project was supported by the Deutsche Forschungsgemeinschaft (SFB 779, TP A07 and A08) and the Leibniz Graduate School (PhD stipend to Adriana Barman, Master stipend to Marieke Klein). The authors have no conflicts of interest or involvement, financial or otherwise, to report.

## **REFERENCES**


reward anticipation correlate with reward-related ventral striatal dopamine release. *J. Neurosci.* 28, 14311–14319. doi: 10.1523/JNEUROSCI.2058-08.2008


and alcohol dependence. *Hum. Genet.* 132, 347–358. doi: 10.1007/s00439-012- 1251-6


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 25 June 2014; accepted: 18 July 2014; published online: 06 August 2014. Citation: Richter A, Guitart-Masip M, Barman A, Libeau C, Behnisch G, Czerney S, Schanze D, Assmann A, Klein M, Düzel E, Zenker M, Seidenbecher CI and Schott BH (2014) Valenced action/inhibition learning in humans is modulated by a genetic variant linked to dopamine D2 receptor expression. Front. Syst. Neurosci. 8:140. doi: 10.3389/fnsys.2014.00140*

*This article was submitted to the journal Frontiers in Systems Neuroscience.*

*Copyright © 2014 Richter, Guitart-Masip, Barman, Libeau, Behnisch, Czerney, Schanze, Assmann, Klein, Düzel, Zenker, Seidenbecher and Schott. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Corrigendum: Valenced action/inhibition learning in humans is modulated by a genetic variant linked to dopamine D2 receptor expression

Anni Richter <sup>1</sup> \*, Marc Guitart-Masip2, 3, Adriana Barman<sup>1</sup> , Catherine Libeau<sup>1</sup> , Gusalija Behnisch<sup>1</sup> , Sophia Czerney <sup>1</sup> , Denny Schanze<sup>4</sup> , Anne Assmann<sup>1</sup> , Marieke Klein1 †, Emrah Düzel 5, 6, 7, 8, Martin Zenker <sup>4</sup> , Constanze Seidenbecher 1, 8 and Björn H. Schott 1, 8, 9, 10

<sup>1</sup> Department of Neurochemistry and Molecular Biology and Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Magdeburg, Germany, <sup>2</sup> Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, London, UK, <sup>3</sup> Ageing Research Centre, Karolinska Institute, Stockholm, Sweden, <sup>4</sup> Institute of Human Genetics, Otto von Guericke University, Magdeburg, Germany, <sup>5</sup> Institute of Cognitive Neurology and Dementia Research, Otto von Guericke University, Magdeburg, Germany, <sup>6</sup> Institute of Cognitive Neuroscience, University College London, London, UK, <sup>7</sup> German Center for Neurodegenerative Diseases, Magdeburg, Germany, <sup>8</sup> Center for Behavioral Brain Sciences, Otto von Guericke University of Magdeburg, Magdeburg, Germany, <sup>9</sup> Department of Psychiatry, Charité University Hospital, Berlin, Germany, <sup>10</sup> Department of Neurology, Otto von Guericke University, Magdeburg, Germany

#### Edited by:

Daniela Laricchiuta, IRCCS Santa Lucia Foundation, Italy

> Reviewed by: Carlo Lai, Sapienza University, Italy

# \*Correspondence:

Anni Richter, anni.richter@lin-magdeburg.de

#### Present Address:

Marieke Klein, Department of Human Genetics, Donders Institute for Brain, Cognition and Behaviour, Radboud University Medical Centre, Nijmegen, Netherlands

†

Received: 20 January 2015 Accepted: 19 February 2015 Published: 11 March 2015

#### Citation:

Richter A, Guitart-Masip M, Barman A, Libeau C, Behnisch G, Czerney S, Schanze D, Assmann A, Klein M, Düzel E, Zenker M, Seidenbecher C and Schott BH (2015) Corrigendum: Valenced action/inhibition learning in humans is modulated by a genetic variant linked to dopamine D2 receptor expression. Front. Syst. Neurosci. 9:36. doi: 10.3389/fnsys.2015.00036 Keywords: dopamine D2 receptor, TaqIA, reward learning, motivated learning, action bias

## **A Corrigendum on**

## **Valenced action/inhibition learning in humans is modulated by a genetic variant linked to dopamine D2 receptor expression**

by Richter, A., Guitart-Masip, M., Barman, A., Libeau, C., Behnisch, G., Czerney, S., et al. (2014). Front. Syst. Neurosci. 8:140. doi: 10.3389/fnsys.2014.00140

We observed some errors that occurred during the genotyping of DARPP-32 rs907094. Naming of CC and TT homozygotes was swapped, and, furthermore, six genotypes were wrongly identified (three people changed from CT to CC, two people changed from CT to TT, and one person changed from TT to CT). All statistics that included DARPP-32 rs907094 genotype were recomputed. We have corrected the text in the corresponding text passages of the manuscript accordingly (last paragraph of the Results Section and **Table 2**). Importantly, these corrections did not affect our main findings, the effects attributable to the DRD2 TaqIA polymorphism.

Find below the last paragraph of the Results Section and **Table 2** with the corrected statistics including DARPP-32 rs907094 genotype.

### **Corrected version of the last paragraph of the Results Section**

Because the TaqIA polymorphism is located downstream of the DRD2 gene, the observed genotype effects might putatively result from linkage disequilibrium with other DRD2 polymorphisms, including the C957T. We indeed observed an imbalanced distribution of the C957T polymorphism (rs6277) among TaqIA A1 carriers vs. A2 homozygotes numerically in the first cohort (χ <sup>2</sup> = 4.04, p = 0.132) and significantly in the second cohort (χ <sup>2</sup> = 25.49, p < 0.001). Moreover, the DARPP-32 polymorphism (rs907094) was unequally distributed in the second cohort only (χ <sup>2</sup> = 7.62, p = 0.022). In order to rule out confounding effects, we included the polymorphisms as covariates in an additional ANCOVA. The same was done for COMT Val108/158Met (rs4680), because the cohorts

## **Corrected version of Table 2.**


Gender distribution, age (means ± standard deviations), number of smokers and nonsmokers. Allelic distributions for following polymorphisms: COMT Val108/158Met (mm, met homozygotes; vm, val/met heterozygotes; mm, met homozygotes), DAT1-VNTR (9+: carriers of the 9-repeat allele 9/9 and 9/10; 9−: 10-repeat homozygous subjects 10/10), C957T (CC/CT/TT carriers), and DARPP-32 (CC/CT/TT carriers). A1+, carriers of the A1 allele; A1−, A2 homozygotes.

were stratified with respect to that polymorphism. Importantly, the fourfold action by valence by time by genotype interaction for the TaqIA polymorphism remained significant [cohort 1: F(1, 82) = 4.67, p = 0.034, cohort 2: F(1, 90) = 4.65, p = 0.034], while there was no effect for C957T (cohort 1: p = 0.484, cohort 2: p = 0.832), DARPP-32 (cohort 1: p = 0.610, cohort 2: p = 0.235), or COMT Val108/158Met polymorphism (cohort 1: p = 0.149, cohort 2: p = 0.842).

# Acknowledgments

The authors would like to thank Iris Mann for help with testing and Maria Michelmann for help with genotyping. This project was supported by the Deutsche Forschungsgemeinschaft (SFB 779, TP A07 and A08) and the Leibniz Graduate School (PhD stipend to AB, Master stipend to MK).

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Richter, Guitart-Masip, Barman, Libeau, Behnisch, Czerney, Schanze, Assmann, Klein, Düzel, Zenker, Seidenbecher and Schott. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A new measure for the revised reinforcement sensitivity theory: psychometric criteria and genetic validation

Martin Reuter 1, 2, <sup>3</sup> \*, Andrew J. Cooper <sup>4</sup> , Luke D. Smillie<sup>5</sup> , Sebastian Markett <sup>2</sup> and Christian Montag<sup>6</sup>

*<sup>1</sup> Department of Psychology, University of Bonn, Bonn, Germany, <sup>2</sup> Laboratory of Neurogenetics, University of Bonn, Bonn, Germany, <sup>3</sup> Center of Economics and Neuroscience, University of Bonn, Bonn, Germany, <sup>4</sup> Department of Psychology, Goldsmiths, University of London, London, UK, <sup>5</sup> Melbounre School of Psychological Sciences, The University of Melbourne, Melbourne, VIC, Australia, <sup>6</sup> Department of Psychology, University of Ulm, Ulm, Germany*

Jeffrey Gray's Reinforcement Sensitivity Theory (RST) represents one of the most influential biologically-based personality theories describing individual differences in approach and avoidance tendencies. The most prominent self-report inventory to measure individual differences in approach and avoidance behavior to date is the BIS/BAS scale by Carver and White (1994). As Gray and McNaughton (2000) revised the RST after its initial formulation in the 1970/80s, and given the Carver and White measure is based on the initial conceptualization of RST, there is a growing need for self-report inventories measuring individual differences in the revised behavioral inhibition system (BIS), behavioral activation system (BAS) and the fight, flight, freezing system (FFFS). Therefore, in this paper we present a new questionnaire measuring individual differences in the revised constructs of the BIS, BAS and FFFS in *N* = 1814 participants (German sample). An English translated version of the new measure is also presented and tested in *N* = 299 English language participants. A large number of German participants (*N* = 1090) also filled in the BIS/BAS scales by Carver and White (1994) and the correlations between these measures are presented. Finally, this same subgroup of participants provided buccal swaps for the investigation of the arginine vasopressin receptor 1a (AVPR1a) gene. Here, a functional genetic polymorphism (rs11174811) on the AVPR1a gene was shown to be associated with individual differences in both the revised BIS and classic BIS dimensions.

Keywords: reinforcement-sensitivity-theory, anxiety, fear, revised RST questionnaire, AVPR1a, rs11174811

# Introduction

The Reinforcement Sensitivity Theory (RST) of personality has in recent years become one of the most prominent biologically oriented theories in personality psychology (Corr, 2008; Smillie et al., 2011). At the core of the classic form of this theory are the behavioral activation and behavioral inhibition systems (BAS and BIS, respectively). These systems regulate approach toward appetitive stimuli and avoidance/withdrawal of aversive stimuli. Individual differences in the functioning of the BIS and BAS are thought to provide the biological foundation for complex personality traits

#### *Edited by:*

*Daniela Laricchiuta, IRCCS, Santa Lucia Foundation, Italy*

#### *Reviewed by:*

*Joseph Kambeitz, University of Munich, Germany Fabio Ferlazzo, Sapienza University of Rome, Italy*

#### *\*Correspondence:*

*Martin Reuter, Department of Psychology, University of Bonn, Kaiser-Karl-Ring 9, 53111 Bonn, Germany martin.reuter@uni-bonn-diff.de*

> *Received: 05 August 2014 Accepted: 23 February 2015 Published: 16 March 2015*

#### *Citation:*

*Reuter M, Cooper AJ, Smillie LD, Markett S and Montag C (2015) A new measure for the revised reinforcement sensitivity theory: psychometric criteria and genetic validation. Front. Syst. Neurosci. 9:38. doi: 10.3389/fnsys.2015.00038* (see also Montag et al., 2013). Gray proposed that the BAS is anchored in mesolimbic dopaminergic pathways (e.g., Pickering and Gray, 2001), thereby sharing similar ideas with Panksepp's SEEKing system (Panksepp and Moskal, 2008) and Depue's Behavioral Faciliation System (e.g., Depue and Collins, 1999). Here, mesolimbic dopamine function is thought to underpin energized approach behavior toward appetitive stimuli (Schultz, 2007). Individuals with stronger dopaminergic firing in these brain regions might be characterized as full of energy, having a tendency toward outgoing explorative behavior, and being more motivated to pursue rewards (e.g., Leyton et al., 2002). In contrast, individuals with a more reactive BIS might be characterized as more anxious, avoidant, and more motivated to avoid threat or punishment<sup>1</sup> . According to the original conceptualization of RST, the BIS is hypothesized to be anchored around a core network comprising the septo-hippocampal-system (e.g., Gray, 1982).

A major revision to RST has resulted in a somewhat updated understanding of the systems described above (Gray and McNaughton, 2000; McNaughton and Corr, 2004), with particularly notable implications for the role of the BIS and what has now been termed the Fight Flight Freezing system (FFFS, reflecting Fear<sup>2</sup> ). The first major change from the "classic" to the "revised" RST is the removal of the distinction between conditioned and unconditioned stimuli (see McNaughton and Corr, 2004). In the classic version of the RST, the BAS and BIS were thought to be activated only by conditioned rewarding and punishing stimuli, respectively. In the revised RST, the BAS is proposed to be responsive to all rewarding and appetitive stimuli, while the FFFS is proposed to be responsive to all punishing and threatening stimuli. Conversely, the BIS is now thought to be activated by instances of goal conflict, such as when a threatening stimulus must be approached, or when mixed signals of reward and punishment are present. In rodent models, such conflict is well represented by a rodent being placed in an experimental setting prepared with cat odor. While the visual information clearly indicates that no cat is in close proximity, the olfactory senses of the rodent suggest otherwise. Such experiments have been conducted in a setting called the visible burrow system by Blanchard and Blanchard (1989) and Blanchard et al. (1993, 2001), among others. Activation of the BIS is still equated with the experience of anxiety, although this is now attributed to goal conflict, and is grounded in the "concurrent activity in the amygdala and septo-hippocampal system" (Gray and McNaughton, 2000, pp. 122–123). Observable behavior accompanying BIS activation is thought to include careful and slow approach behavior toward the potentially dangerous stimuli, and risk assessment behaviors (e.g., visual scanning of the environment). This careful approach behavior in a potentially dangerous situation is important, because it can generate new information to help solve the conflict (is a cat near, or not?) resulting either in activation of the BAS (hence, exploration behavior e.g., to seek for food), or activation of the FFFS, triggering withdrawal behavior such as fight, flight or freezing.

As outlined above, both the hippocampus and the amygdala have been outlined as playing an important role for the BIS (Gray and McNaughton, 2000; pp. 122–123). This idea has already received support from studies in the human neuroscience literature. For example, a study by Barrós-Loscertales et al. (2006) observed a positive correlation between gray matter volumes of the hippocampus and amygdala, and scores on a questionnaire measuring BIS reactivity. Supporting these findings, Cherbuin et al. (2008) were also able to observe a positive correlation between BIS scores and hippocampus volume. Of note, these studies administered self-report inventories which were originally developed to measure the classic BIS and BAS dimensions. Moreover, a review by our own group showed that similar personality constructs, such as Neuroticism or Harm Avoidance<sup>3</sup> , have also been linked to the hippocampus, but in the opposite direction—that is, a negative association between negative emotionality and gray matter volume in structures of the temporal lobe is also plausible (Montag et al., 2013).

One of the most pressing issues in personality psychology when dealing with the revised RST is the measurement of individual differences in the BIS (reflecting anxiety) and the FFFS (reflecting fear) in terms of the changes made to the theory. There already exists a first questionnaire, the Reinforcement Sensitivity Questionnaire (RSQ), measuring the revised RST, but the items were only published in a Serbian book chapter and not in English language. However, the authors have published an article on validation data of their RSQ where the principles of the questionnaire construction are described (Smederevac et al., 2014). In the RSQ the BAS is conceptualized with a focus on behaviors indicating sensitivity to signals of reward rather than on those indicating sensitivity to reward. The BIS is defined as conflict between worries (arising from the scanning of internal resources) and the outcome/feedback of real situations. The FFFS is represented by three distinct scales: Fight items include aggressive reactions to the emotion of fear caused by present threats, Freeze items express the inability to articulate necessary verbal responses to threat and Flight is defined as reaction to real danger which can be avoided. It is important to mention that the questionnaire construction and data collection of the RSQ and the Reuter and Montag rRST-Q happened parallel in time so that we had no chance to profit from the ideas and results of Smederevac et al. Theoretical overlap and difference between the RSQ and our rRST-Q are described in the Discussion section.

On this basis, the first aim of the present study is to provide a new measurement tool for the revised RST, with a particular focus on disentangling measurement of the BIS and FFFS constructs. In this initial investigation, we employ molecular genetic methods to investigate the validity of our new measure.

Of particular importance to the present study is that the emotion of anxiety is influenced by a large number of neurotransmitters, including gamma amino butyric acid (GABA; e.g., Nemeroff, 2002), and classic monoamines such as dopamine (e.g.,

<sup>1</sup>The important distinction between the emotions of fear and anxiety came with revised RST and will be discussed in the light of withdrawal/avoidance behavior further on.

<sup>2</sup>The FFFS was already included in the original version of the RST as the so called FFS and was activated by unconditioned unpleasant stimuli.

<sup>3</sup>BIS correlates with Harm Avoidance and Neuroticism at about 0.55–0.59 (Montag et al., 2013).

Montag et al., 2012) and serotonin (Lesch et al., 1996). In addition to these classic "anxiety molecules," recent years have seen a rise in studies investigating neuropeptides such as oxytocin and arginine vasopressin (AVP) to help better understand negative emotionality and personality (for an overview see Montag and Reuter, 2014). The nonapeptide oxytocin has become a major research focus as it seems to play an important role in social cognition and has been associated with trust behavior (Kosfeld et al., 2005). Kirsch et al. (2005) demonstrated that the nasal administration of oxytocin reduces amygdala activity while processing pictures depicting unpleasant content. Similar effects have been observed by Domes et al. (2007), who reported a down-regulation of the amygdala while processing emotional faces after administration of oxytocin. As noted above, Kosfeld et al. (2005) found that meeting a trusted person could trigger oxytocin secretion, which results in the dampening of alarm signals usually elicited by the amygdala when encountering a stranger. Therefore, oxytocin could be of particular relevance for understanding the emotion of (social) anxiety, elicited by uncertainty when meeting and engaging with strangers.

There has been even less social neuroscience research on the role of the nonapeptide vasopressin in the context of social anxiety. The initial studies in this area with humans point toward an equally important role for vasopressin in social cognition (Zink and Meyer-Lindenberg, 2012). Of particular importance is the study by Zink et al. (2010), reporting that the subgenual anterior cingulate cortex was more strongly activated under the influence of vasopressin, compared to placebo, when humans participated in a classic face matching paradigm using fearful and angry faces. During fear processing strong subcortical signals can be observed, putatively due to a lack of inhibition by the prefrontal cortex (Mobbs et al., 2007). Arginine-Vasopressin (AVP) could counteract these fear (or anxiety) effects by strengthening the PFC activity as a top-down fear/anxiety regulator. Further evidence for a role of AVP in anxiety/fear comes from animal research. Among others, Appenrodt et al. (1998) observed that the administration of AVP in septal regions attenuates anxietyrelated behavior in the form of longer time spent in the open arms of the elevated plus maze test in rats. The major target of AVP for cell signaling is the arginine vasopressin receptor 1 a (AVPR1a). Consequently, knocking out the gene coding for AVPR1a could be associated with a reduction in anxiety, as measured by the elevated plus maze test mentioned above (Egashira et al., 2007). In order to translate these interesting findings to humans, molecular genetic association studies have already investigated genetic variants on the AVPR1A and AVPR1B<sup>4</sup> genes in relation to individual differences in a vast range of human behaviors, including anxiety related personality traits (Meyer-Lindenberg et al., 2009; Kazantseva et al., 2014), altruistic behavior (Avinun et al., 2011), musical aptitude (Ukkola et al., 2009), pair bonding (Walum et al., 2008), and autism (Yirmiya et al., 2006; Yamasue, 2013).

In the present study, we hypothesized that genetic variation on the AVPR1a gene would be related to individual differences in measures of BIS (both the classic and revised form), but should not be associated with our measure of FFFS. We focused on the single nucleotide polymorphism (SNP) rs11174811 on the AVPR1a gene (located on chromosome 12q), because not only has it been associated with phenotypes related to anxiety/negative emotionality [e.g., stress reactivity, drug addiction, blood pressure, partnership satisfaction, and aggressive behavior (Maher et al., 2011; Nossent et al., 2011; Levran et al., 2014; Malik et al., 2014)], but also the functionality of this gene variant has been demonstrated by means of mRNA expression in postmortem brain tissue. Expression levels in samples homozygous for the major C-allele (genotype CC) were significantly lower than in samples with at least one minor A-allele (genotypes AA or CA; Maher et al., 2011).

In sum, the main aims of the present research are as follows: First, we report on the development of a new questionnaire measuring individual differences in the revised constructs of Gray and McNaughton's BAS, BIS and FFFS dimensions. Second, this new questionnaire, called Reuter and Montag's rRST-Q, is crossvalidated against the widely used Carver and White BIS/BAS scales, in order to examine the convergent and divergent validity of the new questionnaire. We would expect only moderate correlations between the same constructs measured across the two self-report-inventories given the differences between the classic and revised models of RST, as outlined above. We would also expect only low to moderate correlations between the revised FFFS measure in the rRST-Q and the BIS scale from the Carver and White scale, on the same basis. The third aim of the study was to examine whether a genetic variant is associated with individual differences in the BIS. As AVP has been understudied in the context of anxiety (although the first evidence points toward such an association, as outlined above) so far, we tested for a link between rs11174811 and the BIS, as measured by both the Reuter and Montag and the Carver and White scale. Given the small number of studies dealing with the functional polymorphism on the AVPR1a gene in the context of negative emotionality, we have not provided a directional hypothesis for this potential effect.

# Methods

# Participants

The results of this study will be presented across three sections. The German version of the rRST-Q reported on in the first section of the results was completed by N = 1814 participants (n = 704 males and n = 1110 females, mean-age: 24.86, SD = 7.28). The English translated version of the rRST-Q, also reported on in this first section, was filled in by N = 299 participants (n = 79 males and n = 220 females, mean-age: 24.12, SD = 8.49). The participants were predominantly university students in both the German and English samples. In the last two sections of the results, N = 1090 participants (n = 325 males and n = 765 females; mean-age: 25.27, SD = 8.09), from the German sample described above, filled in Reuter and Montag rRST-Q as well as Carver and White's (1994) BIS/BAS scales, and also provided buccal swaps for genotyping a genetic variation of the AVPR1a gene. The study was approved by the psychology ethics committee of the University of Bonn, Germany.

<sup>4</sup>These two genes code for the vasopressin 1a or 1b receptors. Of note, the gene coding for the 1b receptor has not been the major focus of research until now.

# Measures

Two questionnaires were administered to measure individual differences in RST-relevant personality constructs. We administered a new questionnaire called Reuter and Montag's rRST-Q to measure individual differences in the revised BAS, BIS, and FFFS constructs. Additionally, most of the German participants also completed the most widely used RST self-report measure, the Carver and White BIS/BAS scale, developed using the original RST model.

# *Reuter and Montag's rRST-Q*

Reuter and Montag's rRST-Q consists of 31 items, with a four point Likert scale ranging from "strongly disagree" to "strongly agree." The BAS is measured by eight items, the BIS by 11 items and the FFFS by 12 items. The original item pool for the rRST-Q consisted of 34 items; three items were excluded during the development process to improve both the factor structure and the internal consistencies of the scale. The German version of the scale was translated into English by a bilingual German-English speaker; the translated items were then checked by a native English speaker and some minor modifications were made to several of the items. This version was then back-translated to German by a different bilingual German-English speaker, and the resultant back-translated German items were checked against the original German items for consistency. **Tables 1**, **2** present all items from Reuter and Montag's rRST-Q in German and English.

The following theoretical considerations form the basis for the construction of Reuter and Montag's rRST-Q:

# Revised BAS

Higher BAS activity should be associated with energetic arousal and approach behavior toward appetitive stimuli, consistent with the BAS and similar systems being described as a "Go get it!" system (Panksepp, 1998). The BAS dimension in this scale has item content measuring approach and goal-directed behavior;

TABLE 1 | German version of *Reuter and Montag's rRST-Q*; Likert scaling: ① trifft für mich gar nicht zu, ② trifft für mich eher nicht zu, ③ trifft für mich eher zu, ④ trifft für mich genau zu.


*R, reversed item.*

*Note: The total FFFS scale score is comprised of high Flight/Freezing and low Fight scores. Therefore, high scores on Fight reflect low fear. So when calculating a total for the FFFS scale, the items for Fight 15R,16R,,17, 22 need to be reversed, resulting in 15, 16, 17R, and 22R.*


#### TABLE 2 | English version of *Reuter and Montag's rRST-Q;* Likert scaling: ① strongly disagree, ② disagree, ③ agree, ④ strongly agree.

*R, reversed item.*

*Note: The total FFFS scale score is comprised of high Flight/Freezing and low Fight scores. Therefore, high scores on Fight reflect low fear. So when calculating a total for the FFFS scale, the items for Fight 15R, 16R, 17, 22 need to be reversed, resulting in 15, 16, 17R, and 22R.*

those who score high on this BAS scale could be described as bold, adventurous and may show stronger energy and drive when approaching appetitive stimuli.

# Revised BIS

Higher BIS activity should reflect responses to goal conflict and situations of uncertainty, including hesitation, risk assessment or wary behavior. As proposed by Gray and McNaughton (2000), three kinds of conflict are possible in principle (i.e., approach– approach, approach–avoidance, and avoidance–avoidance). Individuals with a more reactive BIS will tend to have difficulty making decisions when two equally attractive or unattractive options are presented and one option needs to be chosen (e.g., in situations where conflict is apparent).

# Revised FFFS

In revised RST, the FFFS is associated with three kind of defensive or avoidant responses, namely Fight, Flight, and Freezing. Accordingly, in the rRST-Q, high overall trait FFFS is characterized by low fight, high flight and high freezing behavior. This may appear at odds with the notion of defensive attack (e.g., fight behavior) as a classic fear response, as observed in nearly all mammalian organisms. However, in the revised RST fight behavior is only observable if the distance between predator and prey is close to zero, leaving no option for flight or freezing. The probability of such situations occurring for human beings is extremely low (Corr et al., 2013). Furthermore, particularly fearful individuals are perhaps least likely to find themselves in a situation in which there is zero distance between them and a source of threat. As a consequence, and in line with the notion that activity of the FFFS is associated with withdrawal behavior in broad terms, we would characterize a high trait FFFS individual as high in flight and freezing behavior, but a low scorer on fight behavior. A person who is not willing to fight when being attacked might typically withdraw more quickly from unpleasant situations, compared to a person who is more willing to fight when threatened. Clearly when filling in a questionnaire such as this, asking a person to reflect on his or her behavior can only represent an indirect approach to understanding subcortical brain activity in the brain systems of the revised RST. In addition, operationalizing the FFFS as described here putatively leads to positive inter-correlations between all three FFFS subscales.

## *Carver and White BIS/BAS* Scale

The Carver and White BIS/BAS scale consists of 24 items. The BIS scale consists of seven items and the BAS scales comprise thirteen items. The BAS scale can be split in to three subscales: BAS drive (four items), BAS fun seeking (four items) and BAS reward responsiveness (five items). Four filler items are presented to participants, but not analyzed. The German translation of the Carver and White BIS/BAS scale by Strobel et al. (2001) was administered to the German participants in this study. The internal consistencies for the scales derived from the present data set (and contrasted with the data presented by Strobel et al., 2001) are presented in the Results section (see **Table 5**).

### Genetic Analyses

DNA was extracted from buccal cells. Automated purification of genomic DNA was conducted by means of the MagNA Pure <sup>R</sup> LC system using a commercial extraction kit (MagNA Pure LC DNA isolation kit; Roche Diagnostics, Mannheim, Germany). Genotyping of the AVPR1a SNP rs11174811 was performed by means of MALDI-TOF (Matrix-Assisted Laser Desorption/Ionization— Time of Flight) mass spectrometry using The Sequenom MassARRAY <sup>R</sup> system (Agena Bioscience).

# Results

The Results section of the study is split into three sections. The first section presents descriptive data as well as psychometric data (reliabilities) for both a German and an English version of Reuter and Montag's rRST-Q. In addition, confirmatory factor analyses (CFAs) are presented that test the revised RST model (e.g., its factor structure). The second section of the results reports correlations between the German version of the rRST-Q and a German version of Carver and White's BIS/BAS questionnaire, the latter being the most widely used questionnaire in RST research to date. The third section of the results reports a genetic validation of the new RST questionnaire, with a specific focus on the potential relation between BIS sensitivity and the AVPR1a gene.

# Section 1: Psychometric Analysis of *Reuter and Montag's rRST-Q*

In **Table 3**, means and standard deviations for the German version of Reuter and Montag's rRST-Q are provided, including descriptive statistics for the male and female participants separately.

The internal consistencies in terms of Cronbach's Alpha for BIS, BAS and FFFS were good in the German as well as in the English version. The rather low reliabilities for the FFFS subscales are due to the small number of items per scale (see **Table 4**). However, this is similar for the BAS subscales of the classic BIS/BAS questionnaire by Carver and White (see **Table 5**).

TABLE 4 | Internal consistencies (Cronbach's Alpha) for the scales of *Reuter and Montag's rRST-Q* in the German and English sample (data of the present study).


TABLE 5 | Internal consistencies (Cronbach's Alpha) for the *Carver and White BIS/BAS* scales in the German sample of the present study and the initial translation paper by Strobel et al. (2001).


TABLE 3 | Means and standard deviations for the full German sample and males and females separately for the different scales of *Reuter and Montag's rRST-Q* (data of the present study).


In order to test if the factor structure of the Revised Reinforcement Sensitivity Theory Questionnaire (rRST-Q) is in accordance with our theoretical assumptions, we ran CFAs using the LISREL software package (LISREL 8.80 by Jöreskog and Sorböm (1996); Science Software International, Inc). Given the ordinal nature of the questionnaire data (a 4-point Likert scale), the CFAs were based on polychoric covariance matrices and asymptotic covariance matrices. Parameter estimates were calculated using the Robust Diagonally Weighted Least Squares (DWLS) method. As indicated by the fit indices, the data showed good fit to our theoretical model in the German (Chi<sup>2</sup> = 4061.72, df = 431, p < 0.0001; RMSEA = 0.069; CFI = 0.92; see **Figure 1**), and in the English sample (Chi<sup>2</sup> = 871.27, df = 431, p < 0.0001: RMSEA = 0.060; CFI = 0.93).

# Section 2: Associations between *Reuter and Montag's rRST-Q* and the *Carver and White BIS/BAS* Scale

In **Table 6**, the inter-correlations between the Reuter and Montag rRST-Q dimensions and Carver and White's BIS/BAS scales are provided. Of note, the shared variance between the classic BAS from the Carver and White scale and the revised BAS from the new inventory is about 25%. Similarly, the shared variance between the classic BIS scale and its revised form was also around 25%. In **Table 7** we include additional information on the correlations between the subscales of the FFFS, and the revised BIS and BAS scales; **Table 8** provides correlations between the FFFS subscales and Carver and White's BIS/BAS.

# Section 3: Analysis of the Genetic Variation of the AVPR1a Gene in Relation to the Behavioral Inhibition System

In this third section of the results, we explored the relation of the AVPR1a gene and its functional polymorphism rs11174811 with both the classic and revised BIS scales. From the total sample described above in Section 1 of the results, a subgroup of n = 1090 participants provided buccal swaps for genotyping rs11174811. The genotype distribution was as follows: CC = 840, CA = 230, AA = 20 (Hardy Weinberg Equilibrium: Chi<sup>2</sup> = 0.84, df = 1, n.s.). A MANCOVA with the Carver and White BIS/BAS dimensions (and the BAS subscales) and with Reuter and Montag's scales revealed a significant effect of rs11174811 on both the classic and the revised BIS dimensions [F(2, 1087) = 7.93, p < 0.001 vs. F(2, 1087) = 5.03, p = 0.007, respectively]. As both of the BIS dimensions correlated with age (BIS: r = −0.14, p < 0.001 vs. rBIS: r = −0.19, p < 0.001), and with gender, with females having significantly higher scores [BIS: F(1, 1088) = 143.30, p < 0.001 vs. rBIS: F(1, 1088) = 69.15, p < 0.001], we undertook additional analyses, including gender as a second independent variable and age as a covariate. No gender by gene interaction effects could be observed on the BIS scales. The inclusion of age as a covariate did not change the significant influence of rs11174811 on the BIS. A post-hoc test revealed that the contrast for the genotypes CC vs. CA was significant. The group consisting of AA carriers was excluded from further interpretation at this point, because of the small group showing no clear trend in either the AA or AC direction (n = 20; see also **Figures 2**, **3**). As shown in **Tables 9**, **10**, no significant effect of rs11174811 could be detected on our measure of FFFS, nor on any of the other RST dimensions across both personality inventories.

# Discussion

This study had three key aims. First, we sought to develop a new self-report measure for the revised RST in order to better distinguish between aspects of personality concerned with fear and anxiety. The most widely used self-report measure in RST research, the Carver and White BIS/BAS scales, was developed under the classic model of RST, and the BIS scale in that measure arguably conflates processes related to the BIS and FFFS in the item content. Given the putative separation of the FFFS and the BIS in the revised RST in terms of behavioral functioning and their neuropsychopharmacological bases, self-report measures that seek to separate the FFFS and BIS are desirable. On that basis, and in line with revised RST, the new inventory attempts to disentangle the emotions of fear and anxiety by including separate scales for the revised BIS (reflecting anxiety) and for the FFFS (reflecting the emotion of fear). Of note, we designed the BIS scale to measure hesitation and cautious behavior in conflict situations, such as deciding between two (even potentially positive) options (e.g., "If I have the choice between two appealing options, I have difficulty deciding on one."). As well as difficulties in behavioral choice, cognitions related to tolerance of uncertainty are also reflected in the revised BIS in our questionnaire (e.g., "I find it hard to bear uncertainty."). The scale for the FFFS, measuring individual differences in fear tendencies, comprises the most important classes of behavioral fear responses, namely Fight, Flight, and Freezing. Finally, the BAS scale is designed to measure individual differences in reward-seeking, drive and energy (e.g., "I'm a spontaneous person." or "Most of the time I have a thirst for action.").

Despite some similarities in the conceptualization of the revised RST between the RSQ by Smederevac et al. (2014) and Reuter and Montag's rRST-Q there are also apparent differences. With respect to the BIS, the rRST-Q concentrates on conflicts without focusing on irrational interpretations of stimuli as the RSQ does. The conceptualization of the BAS is broader in the rRST-Q than in the RSQ: besides sensitivity to signals of reward, drive, energy and risk taking are also included.

The correlations between the dimensions within Reuter and Montag's rRST-Q show that the BAS is negatively associated with both the BIS and FFFS. As activation of the BAS is clearly associated with approach behavior or "wanting," this is not surprising, as BIS activation reflects orienting and risk assessment behavior (e.g., careful approach behavior, which can switch to activation of the FFFS in the presence of more overt and physically closer threats—ergo, avoidance behavior). In line with this, both the BIS and the FFFS are positively correlated and can be positioned on the side of negative emotionality. Importantly, from a psychometric point of view, our new inventory shows good internal consistencies across the scales and good model fit when using CFA to model the latent variables of the questionnaire. It should be noted that the internal consistencies of the Flight and Freezing subscales



TABLE 7 | Correlations between the FFFS dimension and its subscales and the rBIS/rBAS (German sample; *N* = 1090).


TABLE 8 | Correlations between *Reuter and Montag's* FFFS subscales and *Carver and White's BIS/BAS* scales (German sample; *N* = 1090).


are potentially a little lower than ideal, however they are each comprised of only several items, and so this may be expected.

The second aim of the study was to cross validate Reuter and Montag's rRST-Q with the Carver and White BIS/BAS scale. The results of this cross validation show that both the classic BAS and revised BAS, and also the classic BIS and revised BIS scale, correlate to about .50—hence 25% of the variance of these constructs overlap. This obviously also makes clear that a large portion of the variance does not overlap (75%), and so as a consequence the Reuter and Montag's rRST-Q are clearly measuring something related to yet distinct from the Carver and White dimensions. Future studies including both Carver and White's BIS/BAS scale and Reuter and Montag's rRST-Q are needed, particularly studies examining processes related to fear and anxiety in the context of revised RST, but also those studies examining BAS-related processes and functions. Establishing whether the Reuter and Montag's rRST-Q has incremental and/or divergent validity in relation to existing RST-related measures is clearly an important next step.

The final aim of this study was to examine individual differences of the BIS in relation to a genetic variation on the AVPR1a gene. In line with the previous literature, we showed that the gene coding for vasopressin 1a receptor is involved in human anxiety. Carriers of the CC variant of rs11174811 showed significantly elevated anxiety scores, measured in terms of Gray's Behavioral

FIGURE 2 | Association of genetic variation of the AVPR1a gene and anxiety measured with the revised BIS scale of *Reuter and Montag rRST-Q (Means and SEMs are depicted;* <sup>∗</sup> *the contrast is significant at p = 0.002 level)*.

Inhibition System. As already described above, expression levels in homozygous C-allele carriers (genotype CC) have been reported to be significantly lower compared to carriers of at least one minor A-allele (genotypes AA or CA; Maher et al., 2011). As a consequence, a putatively lower number of vasopressin 1a receptors are associated with elevated anxiety levels, because the anxiety lowering effects of vasopressin (Appenrodt et al., 1998) cannot unfold completely due to lower binding possibilities. But: This interpretation would be against the findings from genetic animal research showing that knocking out the AVPR1a gene is associated with lower anxiety (Egashira et al., 2007).



TABLE 10 | Mean and standard deviations of *Carver and White's BIS/BAS* scale depending on AVPR1a's rs11174811.


Interestingly, rs11174811 showed a significant effect on both the BIS measured with the Carver and White scale, as well as on the revised BIS measured with Reuter and Montag's rRST-Q. Given the correlation of 0.45 between the classic BIS and the revised BIS shown above, the genetic variation of the AVPR1a gene clearly targets the shared variance of both constructs. How can this be explained? When comparing Carver and White's BIS and Reuter and Montag's BIS scale it is apparent that Carver and White's BIS is a little more multifaceted compared to our revised BIS scale. More specifically, Carver and White included a wide range of BIS items in their questionnaire, ranging from explicitly feeling anxious (e.g., item 2 of their scale), to restlessness when being confronted with an unpleasant event (item 16 of their scale). In contrast, the revised BIS scale of our newly designed questionnaire includes no item explicitly referring to feeling anxious. Instead, Reuter and Montag's revised BIS scale describes being unable to bear uncertainty or often being indecisive, which targets one major issue in the revised RST. From our point of view, the overlap between the scales (and the genetic effect targeting the shared variance) could possibly be explained by the aspect of restlessness when being confronted with an unpleasant event (in the Carver and White questionnaire), which is close to our concept of being indecisive or overly careful when confronted with uncertainty.

Importantly, the genetic effect of rs11174811 was only significant in the context of BIS sensitivity; no significant effect was observed on the FFFS scale, measuring individual differences in fear and avoidance tendencies, nor any other RST scales on either of the questionnaires administered. The genetic variant investigated in this study seems to target anxiety, but not fear, in terms of the conceptualization of Reuter and Montag's rRST-Q. This finding supports the divergent validity of the BIS and FFFS dimensions in the rRST-Q, a key aim in the development of this questionnaire, and is potentially of wider importance for the revised model of RST, in terms of identifying neurobiological markers that reliably distinguish between trait measures of these constructs. Clearly, the genetic finding in this study represents a beginning point in this process, but an important beginning point nonetheless.

It should be noted that there are existing attempts to develop self-report measures in the context of revised RST (e.g., the "Jackson-5"; Jackson, 2009), and also attempts to modify the psychometric structure of measures designed under the classic RST in line with revised RST. For example, Heym et al. (2008) suggested the original Carver and White BIS scale could be decomposed into separate BIS and FFFS dimensions, based on an evaluation of the item wording and results of confirmatory factor analysis. Despite this, the research so far using these new or modified measures has tended to focus on validation using other psychometric self-report measures, laboratorybased behavioral tasks or "real-world" behaviors, and there has been limited research on the neurobiological markers associated with these new scales, particularly in terms of separating the BIS and FFFS. Thus, the genetic data reported in this study represents a relatively novel and important step in this endeavor.

# Conclusion

Reuter and Montag's rRST-Q is a new self-report measure, developed in line with theoretical assumptions derived from Gray and McNaughton's RST (2000). The psychometric properties of the scale, including its factorial structure and internal consistencies were supported in both a German and English language version of the measure. Correlations between Reuter and Montag's rRST-Q and an existing RST measure, the Carver and White BIS/BAS scales, showed that the new scale dimensions correlated in the expected direction with the Carver and White dimensions, but the correlations were not large enough to suggest high redundancy in the new dimensions. A first validation study using a molecular genetic approach found a significant association between a functional polymorphism on the AVPR1a gene (rs11174811) and the BIS. The genetic association was shown with respect to the BIS dimension in both the Carver and White BIS/BAS questionnaire and in the rRST-Q. Further, the genetic association was not shown for the FFFS dimension in the rRST-Q, supporting the divergent validity of the BIS and FFFS dimensions in this scale, and highlighting a potentially useful genetic marker that could be used to evaluate existing or new measures developed under the revised RST. This study should clearly be seen as a first step in the validation of this new revised RST measure. In

# References


particular, no validation of the revised BAS scale was attempted in this study, and this may be a focus for future work on this scale. More broadly, future studies will be needed to search for further genetic, endocrinological and brain imaging validation of this new inventory. In addition, this new tool will also need to be further evaluated in relation to other self-report measures and using theoretically relevant behavioral and experimental laboratory tasks.

# Acknowledgments

The position of CM is funded by a Heisenberg grant awarded to him by the German Research Foundation (DFG-MO 2363/3-1).


amygdala activation and personality traits in healthy humans. Mol. Psychiatry 14, 968–975. doi: 10.1038/mp.2008.54


of Individual Differences, eds T. Chamorro-Premuzic, S. von Stumm, and A. Furnham (Oxford: Blackwell Publishing Ltd), 101–131.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Reuter, Cooper, Smillie, Markett and Montag. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# In favor of behavior: on the importance of experimental paradigms in testing predictions from Gray's revised reinforcement sensitivity theory

# *Sebastian Markett 1,2\*, Christian Montag3 and Martin Reuter 1,2*

*<sup>1</sup> Department of Psychology, University of Bonn, Bonn, Germany*

*<sup>2</sup> Center for Economics and Neuroscience, University of Bonn, Bonn, Germany*

*<sup>3</sup> Department of Psychology, University of Ulm, Ulm, Germany*

*\*Correspondence: sebastian.markett@uni-bonn-diff.de*

#### *Edited by:*

*Daniela Laricchiuta, Istituto di Ricovero e Cura a Carattere Scientifico Santa Lucia Foundation, Italy*

*Reviewed by:*

*Alan Pickering, Goldsmiths, University of London, UK Adam Perkins, King's College London, UK*

**Keywords: anxiety, fear, approach/avoidance, affective neuroscience, reinforcement sensitivity theory (RST)**

Approach to desirable events or stimuli of reward and avoidance of undesirable events or stimuli of non-reward or punishment are powerful forces to drive behavior. The importance of approach and avoidance motivation has been outlined in many psychological theories and traditions, including psychodynamics, behaviorism and cognitivism (Elliot and Covington, 2001). Of all theoretical accounts, the *Reinforcement Sensitivity Theory* (RST) by Jeffrey Gray is probably the most elaborate perspective on approach and avoidance (Gray, 1971; Gray and McNaughton, 2000). Key contributions by Gray within this framework are the identification of neural circuits underlying approach and avoidance behavior, the description of these circuits as neuropsychological systems with circumscribed functions in terms of a *conceptual nervous system*, and the development of a theory of personality based on individual differences in the reactivity of these systems.

The plethora of empirical findings in response to the initial formulation of RST has led to a major revision of the theory (Gray and McNaughton, 2000). In the present commentary we focus on one key aspect of the revised RST, namely the processing of (non-)ambiguous dangerous stimuli, which plays a crucial role in disentangling the emotions of fear and anxiety. The revised RST distinguishes between the behavioral inhibition system (BIS) mediating anxiety and the fight flight freezing system (FFFS) reflecting fear. Whereas anxiety represents the emotion elicited when approaching potential threat (for example in foraging) fear represents a "get me out of here" emotion that operates during active avoidance of non-ambiguous threat and governs flight behavior or alternatively freezing or fight if flight is not an option (for example in the immediate vicinity of a predator). Both emotions are conceptualized as distinct entities depending on defensive direction with careful approach behavior in the context of potentially dangerous stimuli to clarify the stimuli's nature vs. avoidance of a clear threat. The BIS and FFFS are implemented in distinct but parallel neural streams (McNaugthon and Corr, 2006): Both neural streams are hierarchically organized along a rostral-cortical (e.g., cingulate, prefrontal cortex) to caudal-subcortical (e.g., periaqueductal gray, hypothalamus, amygdala) axis. Within each structure, different nuclei or subdivisions activate either fear or anxiety. Defensive distance, which is the perceived intensity of threat (Blanchard and Blanchard, 1990), determines which level of the hierarchy with its associated behavioral output becomes active in a given situation. Rostral regions along the hierarchy are thought to react to more distal threat while the caudal regions are activated by proximal threat (especially the periaqueductal gray). The functioning of the BIS (i.e., the emotion of anxiety), however, is not restricted to the conflict that emerges in the tension between approach to, vs. avoidance of potential threat but generalizes to all forms of conflict that results from incompatible goals such as avoidance/avoidance and approach/approach conflict.

The refinements to RST have been based on experimental work in rodents but the theory claims validity for human affective processing and personality as well. In animal research the emotion of anxiety has been investigated with a so called "Visible Burrow System" (e.g., Blanchard and Blanchard, 1989), where a rat is placed in a setting prepared with cat odor in the absence of an actual cat (Blanchard et al., 2001). To a rat, a cat is a clear threat but since its precise location cannot be inferred from its odor alone, flight is not likely to lead to safety. The different information processed by the visual and olfactory senses results in a conflict that triggers the behavioral inhibition system. By the activation of the BIS the rat stops its present exploration behavior (searching for food or a mate) and orients itself toward the potential danger. The rat resolves the conflict by obtaining further information through careful approach behavior. In consequence, either the behavioral approach system (BAS) is activated to return to the exploration of the environment or the FFFS to cope with the immediate danger (the latter is not the case in the Visible Burrow system paradigm).

The aforementioned cross species translation from rodents to humans requires empirical validation. For the design of validation studies, it is important to keep in mind that RST is composed of two main components: A state component that describes neural systems including their reactivity and their associated behavioral and emotional output and a trait component describing stable behavioral dispositions arising from individual differences in the neural systems' sensitivity (Corr and McNaughton, 2006). Initially, RST set out in an Eysenckian tradition as a biologically oriented theory on human individual differences at the trait level (Corr and Perkins, 2006) but the refinement of the hypothesized neurobehavioral systems (the BIS, the FFFS, and the BAS) in the theory's revision have been derived on the basis of ethopharmacological experiments and lesion studies in rodents. Importantly, the animals used in these studies were unselected rats, i.e., no different strains of rats bred for individual differences in fear- or anxiety proneness were used. Thus, the initial formulation of the revised RST has been based on the analysis of states and changes in states after experimental manipulation rather than traits. This highlights the fact that RST is not only a theory on personality but a more general theory on emotion, motivation and learning (Smillie et al., 2006).

So far, the majority of translational work has focused on human individual differences on the trait level, either by assessing predicted relationships between trait anxiety and trait fear (e.g., Perkins et al., 2007; Smederevac et al., 2014) or by testing the influence of individual differences in relevant traits on behavioral states (for meta-analysis see Leue and Beauducel, 2008). While these literatures were successful in testing important predictions within the scope of RST, they come with the downside that the use of self-report measures alone leaves a large body of RST untouched. Particularly, they do not allow for inference on the neural and neurochemical implementation of the hypothesized systems if no biological data is added to these studies. Furthermore, any attempt to link personality traits to individual differences in behavior or the reactivity of neural systems requires structurally valid psychometric tools. Special caution must be taken upon selecting such personality questionnaires in order to avoid circularity: As a structurally valid personality questionnaire is ideally constructed on the basis of the theory under scrutiny, any favorable result can be interpreted as evidence for the measurement tool or for the theory's prediction.

Here we argue for the importance of well-designed experimental paradigms that are crucial in the endeavor of translating non-human animal findings to our species. We hold that it is particularly important to derive paradigms that focus on main effects of systematically varied experimental context and pharmacological manipulation on recordable behavior and brain activity. The study of transient states rather than stable traits is an important step toward establishing general causal systems of human behavior that can be subjected to the study of individual differences in a second step. Of course, the problem of circularity mentioned above is not restricted to self-report measures and can also apply to behavioral tasks. To avoid or to reduce this problem, behavioral tasks need to be designed to resemble the animal tasks on which the refinements of RST are based. If such a translated task responds to different classes of drugs in humans as predicted from animal data, it can be confidentially used to test the individual differences part of the theory as well.

Important questions in the context of revised RST are: Can fear and anxiety be dissociated in humans? Are fear and anxiety elicited by active avoidance of and approach to threat, respectively? Can fear and anxiety be disentangled by biological markers like for example gene polymorphisms? And, are fear and anxiety modulated by panicolytic and anxiolytic drugs similar to pharmacological effects in rodents? Perkins and colleagues have addressed some of these questions with an experimental task that has been derived from a typical behavioral protocol in rodents (Perkins et al., 2009). In the joystick-operated runway task (JORT), participants determine an onscreen dot's speed in a runway by controlling a joystick. The dot is chased by another onscreen dot and if caught, a highly unpleasant burst of white noise is emitted to the participant via headphones. Thus, participants are motivated to escape the chasing dot and the amount of force participants exert on the joystick can be interpreted as a proxy for active avoidance or in other words fear. In a second version of the task, the participant's dot is not only chased by one dot, there is also another dot in the runway running in front of the participant's dot. The unpleasant burst of noise is not only emitted when the participant's dot gets caught but also if it is run into the dot upfront. Thus, in order to escape the chasing dot, participants have to approach a potential threat, which is experienced as conflict or, in other words, anxiety. The emotion of anxiety is quantified by the amount of back- and forth oscillations with the joystick handle while avoiding both threatening dots. Key findings from pharmacological experiments with the JORT are that anxiolytic but not anti-panic drugs affect anxiety in terms of approach-withdrawal oscillations (Perkins et al., 2009). A similar relationship between anti-panic drugs and active avoidance, however, was not observed. Active avoidance in the JORT was however affected by a risk gene variant for panic disorder as predicted by RST (Perkins et al., 2011). The study of genetic variation is also a feasible means to probe neurotransmitter systems and has the advantage of being free from side effects. In contrast to RST predictions, however, a more recent study confirmed a dose dependent effect of an anxiolytic drug on active avoidance, an effect that depended on baseline individual differences on a fear scale (Perkins et al., 2013). Taken together, the JORT was successful in confirming three out of five predictions. It failed, however, to provide evidence for the core theme of the revised RST, a double dissociation of fear and anxiety, either on the pharmacological or on the molecular genetic level. This emphasizes the requirement to validate the state part of the theory separately from the trait part. Replication of these results, however, is needed to justify refinement of human RST, especially since other behavioral experiments have confirmed that fear and anxiety can be separated in human facial expressions (Perkins et al., 2012).

A general problem with translational fear research is that levels of experimentally induced fear might vary considerably across species and only a rather small amount of fear may be triggered in humans for ethical reasons. To parallel findings from rodent studies, it might be wise to use innate (but harmless) fear stimuli such as spiders, which are likely to elicit phobic reactions in humans. Mobbs et al. (2010) varied the proximity of a huge tarantula to the feet of healthy human participants while they were undergoing neuroimaging to test the effect of threat proximity on the responsiveness of different levels of the neural fear hierarchy. This paradigm activates the FFFS because a dangerous stimulus is presented that is unavoidable. As predicted, distant threats were more likely to activate the rostral part of the fear axis (such as prefrontal cortices) while proximal threats activated the caudal parts of the hierarchy such as the amygdala or the periaqueductal gray. Similar findings were obtained by the same group (Mobbs et al., 2007, 2009) using a pacmanstyle computer game where the participant was chased through a labyrinth and was subjected to a painful cutaneous electric shock when caught. While these experimental protocols were successful in testing the neural fear hierarchy and its dependence upon defensive distance, they do not allow any conclusion on the anxiety hierarchy or on the dissociation of fear and anxiety. The combination of ethopharmacology with neuroimaging or etho-genetic imaging studies (e.g., Montag et al., 2008) will help to shed light on these questions.

Behavioral experiments as described above will overcome issues of social desirability in answering self-reports and provide ecologically valid data on human behavior—if operationalized in the correct way. An automated fear response elicited by an approaching spider, and the threat of loud bursts or electric shocks will be more instructive for neuropsychological theories than a questionnaire score that results from the reflection on past behavior, behavioral dispositions, and emotional states.

# **REFERENCES**


G. Andrews (Dunedin: Otago University Press), 124–133.


investigation of the revised reinforcement sensitivity theory of personality. *Emotion* 7, 252–261. doi: 10.1037/1528-3542.7.2.252


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 14 July 2014; accepted: 10 September 2014; published online: 30 September 2014.*

*Citation: Markett S, Montag C and Reuter M (2014) In favor of behavior: on the importance of experimental paradigms in testing predictions from Gray's revised reinforcement sensitivity theory. Front. Syst. Neurosci. 8:184. doi: 10.3389/fnsys.2014.00184*

*This article was submitted to the journal Frontiers in Systems Neuroscience.*

*Copyright © 2014 Markett, Montag and Reuter. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

MINI REVIEW ARTICLE published: 19 September 2014 doi: 10.3389/fnsys.2014.00175

# Cognitive reappraisal and expressive suppression strategies role in the emotion regulation: an overview on their modulatory effects and neural correlates

# **Debora Cutuli 1,2\***

<sup>1</sup> Department of Psychology, University "Sapienza" of Rome, Rome, Italy

<sup>2</sup> Laboratory of Experimental and Behavioral Neurophysiology, Santa Lucia Foundation, Rome, Italy

#### **Edited by:**

Daniela Laricchiuta, Istituto di Ricovero e Cura a Carattere Scientifico Santa Lucia Foundation, Italy

#### **Reviewed by:**

Giuseppe Curcio, Università degli Studi dell'Aquila, Italy Maria Beatrice Toro, SCINT Scuola di Specializzazione in Psicoterapia Cognitivo Interpersonale, Italy

#### **\*Correspondence:**

Debora Cutuli, Department of Psychology, University "Sapienza" of Rome, Via dei Marsi 78, 00185 Rome, Italy e-mail: debora.cutuli@uniroma1.it

Individuals regulate their emotions in a wide variety of ways. In the present review it has been addressed the issue of whether some forms of emotion regulation are healthier than others by focusing on two commonly used emotion regulation strategies: cognitive reappraisal (changing the way one thinks about potentially emotion-eliciting events) and expressive suppression (changing the way one behaviorally responds to emotion-eliciting events). In the first section, experimental findings showing that cognitive reappraisal has a healthier profile of short-term affective, cognitive, and social consequences than expressive suppression are briefly reported. In the second section, individual-difference findings are reviewed showing that using cognitive reappraisal to regulate emotions is associated with healthier patterns of affect, social functioning, and well-being than is using expressive suppression. Finally, brain structural basis and functional activation linked to the habitual usage of cognitive reappraisal and expressive suppression are discussed in detail.

**Keywords: emotion regulation, cognitive reappraisal, expressive suppression, brain volume, brain activation**

# **INTRODUCTION**

The number of studies on emotion regulation has dramatically increased in the past two decades. These studies strengthened our knowledge on how the effectiveness of emotion regulation is crucial for different aspects of healthy affective and social adaptation (Gross, 2001; John and Gross, 2004). Further, dysregulation of emotions typically characterizes mood and anxiety disorders (Gross and Thompson, 2007).

Two major emotion regulation strategies that have been particularly studied are cognitive reappraisal and expressive suppression (Gross and John, 1998). In particular, *cognitive reappraisal* is defined as the attempt to reinterpret an emotion-eliciting situation in a way that alters its meaning and changes its emotional impact (Lazarus and Alfert, 1964; Gross and John, 2003). *Expressive suppression* is defined as the attempt to hide, inhibit or reduce ongoing emotion-expressive behavior (Gross and Levenson, 1993; Gross and John, 2003).

Based on an analysis of how emotions unfold over time, it has been argued that cognitive reappraisal and expressive suppression have their primary impact at different points of the emotiongenerative process (**Figure 1;** Gross, 2001; Gross and John, 2003). Specifically, cognitive reappraisal is an antecedent-focused strategy that acts before the complete activation of emotion response tendencies has taken place. It thus might be expected to modify the entire temporal course of the emotional response before emotion responses have been completely generated. Expressive suppression is a response-focused strategy that intervenes once an emotion is already under way and after the behavioral responses have already been fully generated. It thus might be expected to require repeated efforts to manage emotional responses as they continually arise, challenging the individual's resources.

The usage of cognitive reappraisal allows to implement and produce interpersonal behavior that is appropriately focused on social interaction and is perceived by the others as emotionally engaging and responsive. At odds, expressive suppression comes relatively late in the emotion-generative process and principally modifies the behavioral aspect of the emotional responses, without reducing the subjective and physiological experience of negative emotion, which is not directly targeted by suppression and may thus continue to linger and accumulate unresolved. As expressive suppression comes late in the emotion-generative process, it requires the individual to effortfully manage emotional responses as they constantly occur. These repeated efforts deplete cognitive resources to the detriment of social performances and create a sense of discrepancy between inner experience and outer expression in the individual (Higgins, 1987). The final effect of this sense of inauthenticity can lead to negative feelings about the self, making more difficult the establishment of emotionally close relationships and rather contributing to avoidant, diverted and anxious relational behaviors (Sheldon et al., 1997; John and Gross, 2004).

In the following sections, experimental findings on cognitive reappraisal and expressive suppression will be briefly analyzed. Then, individual-difference findings on the dispositional usage

of these two strategies will be taken into account. Finally, brain structural basis and functional activation linked to the habitual usage of cognitive reappraisal and expressive suppression will be discussed in detail.

# **EXPERIMENTAL STUDIES**

In experimental studies, participants are exposed to emotioneliciting situations and randomly assigned to use cognitive reappraisal or expressive suppression strategies or to act naturally (control condition). Experimental studies use powerful research designs: in fact, by manipulating emotion-regulatory processes directly, they can demonstrate the immediate causal effects of particular strategies on dependent variables of interest, such as affective, cognitive and social consequences.

Overall experimental studies have demonstrated that cognitive reappraisal has a positive impact in the affective domain by decreasing negative emotion experience and negative emotion behavioral expression without any increase in physiological activation. At odds, suppression has a negative impact decreasing positive emotion experience and leaving unaltered the subjective negative emotion experience and exacerbating physiological activation (Gross and Levenson, 1993, 1997; Gross, 2002; Mauss et al., 2005; Hayes et al., 2010; Brans et al., 2013).

Cognitively, reappraisal results in unaltered or enhanced behavioral memory performance, while expressive suppression impairs memory performances (Richards and Gross, 1999, 2000; Dillon et al., 2007; Sheppes and Meiran, 2007, 2008; Hayes et al., 2010). Memory advantage for cognitive reappraisal may be subserved by the levels-of-processing effect (Dillon et al., 2007), which is characterized by deeper cognitive analysis of stimuli (Craik and Lockhart, 1972).

In experimental studies on the effects of emotion regulation strategies in social contexts, one member of each dyad is generally asked either to suppress, reappraise or to interact naturally with their conversation partner. When interacting with a person who was using suppression, subjects experienced more stress (i.e., greater increases in blood pressure) than when interacting with a person using reappraisal (Butler et al., 2003; Richards et al., 2003). Thus, while reappraisal has not detrimental effects, the cognitive costs of expressive suppression may concurr to compromise social functioning, as the suppressor fails to take up information needed to respond appropriately to the others and appears not tuned with the flow of the interaction.

# **INDIVIDUAL DIFFERENCE STUDIES**

Since experimental studies cannot account for the long-term, cumulative consequences of using particular regulatory strategies for the individual's emotional life, relationships and wellbeing, a complementary, correlational research approach was used. To this aim Gross and John (2003) developed a self-report questionnaire, the Emotion Regulation Questionnaire (ERQ), to assess individual differences in the usage of habitual, dispositional cognitive reappraisal and expressive suppression. Studies using ERQ have shown that the habitual use of these strategies varies systematically between individuals and is stable in time (Gross and John, 2003). Furthermore, cognitive reappraisal and expressive suppression resulted scarcely related to intelligence, social desirability and personality traits, but highly related to the constructs of inauthenticity, coping with stress and mood management (John and Gross, 2004).

Affectively, the use of cognitive reappraisal in everyday life is related to greater experience and expression of positive emotions and lesser experience and expression of negative emotions. By contrast, individuals frequently using expressive suppression experience and express less positive emotions, without differences in the negative ones (Gross and John, 2003; Abler et al., 2010; Larsen et al., 2012). However, expressive suppression may increase negative affect through its strict link with inauthenticity, specifically leading to feel bad about the self and even to depressive symptoms (John and Gross, 2004).

Cognitively, reappraisal has not effects on mnesic performances, while suppression is negatively related to memory, in particular for socially relevant information (Richards and Gross, 2000; Egloff et al., 2006; Hayes et al., 2010; Moore and Zoellner, 2012). In the domains of interpersonal functioning and well-being, cognitive reappraisal was interestingly associated with better psychological health. In fact, individuals who habitually use reappraisal showed lower symptoms of depression, were more satisfied and optimistic, and had higher self-esteem, environmental mastery levels, personal growth, self-acceptance, coping skills, sense of autonomy as well as better interpersonal relationships (Garnefski et al., 2001; John and Gross, 2004). At odds, suppressors feel to have less social support, worse coping abilities, lower life satisfaction, self-esteem, optimistic attitude about the future, higher avoidance and lack of close social relationships and support, all factors increasing the risk for depressive symptoms (Sheldon et al., 1997; John and Gross, 2004). Anyway, interesting recent studies demonstrated that culture has to be a moderator variable of emotion regulation, being the relation between expressive suppression and negative indicators of mental health stronger in the Western culture than in the Eastern one (Soto et al., 2011; Hu et al., 2014).

# **NEURAL CORRELATES OF COGNITIVE REAPPRAISAL AND EXPRESSIVE SUPPRESSION**

As the habitual use of emotion regulation strategies shows stable individual differences, it could be possible that these strategies, either as a *consequence* (i.e., pre-existing individual volume differences lead to differences in emotion regulation) or *precondition* (i.e., brain region volumes are affected by the usage of emotion regulation strategies) are associated with individual differences in brain volumes and functional activation. Several studies have investigated the underlying neurobiological substrates of cognitive reappraisal and expressive suppression usage.

Following an overview of the studies on brain structural and functional variations associated to the use of cognitive reappraisal and expressive suppression is presented.

## **BRAIN STRUCTURAL STUDIES**

In a magnetic resonance imaging (MRI) study, Welborn et al. (2009) investigated the relation between sex differences in orbitofrontal cortex (OFC) subregions and affective individual differences in healthy adults. As previously reported (Gross and John, 2003), women reported using suppression less frequently than did men. Volume differences based on participants' gender were also identified with men showing larger left planum temporal and women showing larger ventromedial prefrontal cortex (vmPFC), right lateral OFC, cerebellum and basal ganglia. Strikingly, vmPFC (but not OFC) volume was positively related to individual differences in cognitive reappraisal and negatively related to expressive suppression usage. Further, vmPFC volume fully mediated sex differences in emotion suppression and partly in cognitive reappraisal.

In another region of interest (ROI)-based neuroimaging study, Giuliani et al. (2011a) found a positive correlation between cognitive reappraisal and the volume of the dorsal anterior cingulate cortex (dACC), but not the ventral ACC, in healthy female subjects. No relations between dACC volume and expressive suppression, negative affect or age were found. Given that expressive suppression is an emotion regulation strategy that requires interoceptive and emotional awareness, the role of anterior insula in this process was further investigated (Giuliani et al., 2011b). It was demonstrated that anterior insula volume positively correlates to expressive suppression, but not with cognitive reappraisal and negative affect. These findings are consistent with the idea that trait patterns of emotion processing are related to brain structure and indicate that individual differences in cognitive reappraisal are related to different dACC volumes, while individual differences in expressive suppression are related to different anterior insula volumes.

Using an exploratory whole brain approach, Kühn et al. (2011) examined the structural correlates of the habitual use of expressive suppression of emotions. They found a positive correlation of right dorsomedial prefrontal cortex (dmPFC) volume with expressive suppression, but no association of any other brain area with cognitive reappraisal. As expected on the basis of the important role that dmPFC plays in self-control and voluntary inhibition of action (Brass and Haggard, 2007; Brody et al., 2007; Campbell-Meiklejohn et al., 2008; Kühn et al., 2009), the response-focused emotion regulation strategy of expressive suppression is associated with increased gray matter volume in the dmPFC. Even if it is not possible to rule out that the increased dmPFC volume in subjects with expressive suppression strategies is an *a priori* condition rather than a consequence of behavior, it could be speculated that expressive suppression is under internal control as consequence of the internalization of societal norms, customs and manners that govern the adequate or undesirable emotional expressions.

Recently, using a voxel-based morphometry (VBM) in a large sample of young individuals it was analyzed the association of gray matter volumes of the a priori ROIs, including amygdala, insula, dACC/paracingulate cortex, medial and lateral PFC, with cognitive reappraisal and expressive suppression usage as well as neuroticism (Hermann et al., 2013a). Interestingly, a positive association of cognitive reappraisal and neuroticism with amygdala volume was observed. Furthermore, expressive suppression resulted positively associated with dACC/paracingulate cortex and medial PFC gray matter volume. These findings underline the role of the amygdala in individual differences in cognitive reappraisal usage as well as neuroticism that was not found in previous studies. Additionally, the association of expressive suppression usage with larger volumes of the dACC/paracingulate cortex and medial PFC underpins the role of these regions in regulating emotion-expressive behavior. It is evident that Hermann et al. (2013a) did not replicate previous results regarding greater dACC (Giuliani et al., 2011a) and vmPFC (Welborn et al., 2009) volume in frequent using cognitive reappraisers, and larger insula (Giuliani et al., 2011b) and smaller vmPFC (Welborn et al., 2009) volume in individuals frequently using expressive suppression. In contrast, the positive correlation of expressive suppression with dACC/paracingulate cortex and with vmPFC gray matter volume is in line with the involvement of dmPFC in the network linked to the inhibition of actions (Kühn et al., 2009).

Although somewhat conflicting, overall brain structural studies demonstrate that distinct brain structural variations of gray matter volume in the amygdala, insula, dACC, vmPFC and dmPFC might underlie individual differences in cognitive reappraisal and expressive suppression usage. However, a replication of these results is still missing because most of the abovementioned studies focused on different brain regions. Additionally, methodological factors (e.g., VBM *vs.* ROI approach) as well as sample characteristics (e.g., gender and age of participants) prevent a reasonable comparison of the results.

## **BRAIN FUNCTIONAL STUDIES**

The neural basis of emotion regulation processes have been further investigated by several functional neuroimaging studies by manipulating emotion regulation strategies (Ochsner and Gross, 2005). Generally, negative affective pictures are used and participants are trained to reduce the emotional impact of the pictures by using cognitive reappraisal. It is well known that not all individuals experiencing adverse experiences develop anxiety disorders, as result of individual differences in the regulation of negative emotions. Anyway, a more frequent use of habitual (dispositional) cognitive reappraisal in daily life has been shown to be more adaptive. Interestingly, the down-regulation of negative emotions through cognitive reappraisal is indicated by increased activation of medial and lateral PFC along with a diminished activation of emotional arousal-related brain structures as amygdala and insula (Ochsner and Gross, 2005; Ochsner et al., 2012).

Furthermore, dispositional reappraisal has been associated with reduced insula, hippocampus and amygdala as well with stronger dACC and dorsolateral PFC activation in response to aversive emotional stimuli (i.e., pictures or faces; Drabant et al., 2009; Carlson and Mujica-Parodi, 2010; Hayes et al., 2010; Vanderhasselt et al., 2013; Hermann et al., 2014).

Recently, the correlation of habitual cognitive reappraisal usage with stronger down-regulation of amygdala activation during instructed emotion regulation was reported also in a group of patients with remitted depression and healthy controls by using functional MRI (fMRI; Kanske et al., 2012). Hermann et al. (2013b) found that dental phobic individuals with higher dispositional cognitive reappraisal scores showed a reduced activation of the right dmPFC and increased activation of the right vmPFC and the lateral OFC over the course of symptom provocation. Cognitive reappraisal was a predictor of habituation during exposure to phobic stimuli rather than symptom severity. Given that extinction learning as well as cognitive reappraisal are crucial components of exposure-based cognitive-behavioral therapy (CBT) of phobias, the findings by Hermann et al. (2013b) point out for the special importance of considering individual differences in general cognitive reappraisal abilities of phobic patients prior to exposure sessions and to improve these abilities if necessary in order to strengthen the (long-term) outcome of CBT.

Up-to-date few studies examined the neural correlates of expressive suppression in response to emotional stimuli (Ohira et al., 2006; Goldin et al., 2008; Hayes et al., 2010; Vanderhasselt et al., 2013). Ohira et al. (2006) demonstrated a reduced amygdala activation during suppression of emotions. In a further PET study, Goldin et al. (2008) demonstrated increased PFC, insula and amygdala activation during the suppression of disgust facial reactions in response to disgust-eliciting film clips. Individual differences in expressive suppression usage have been further associated with higher amygdala activation when inhibiting responses to sad *vs.* happy facial expressions (Vanderhasselt et al., 2013). Suppressing facial expressions in response to negative picture engaged bilateral insular cortex, supramarginal gyrus and middle frontal gyrus (Hayes et al., 2010).

In parallel with gray matter volume studies, taken together these studies on the functional activation during cognitive reappraisal and expressive suppression confirm that differential activation of the amygdala, insula, dACC, PFC and OFC might underlie individual differences in the use of different emotional strategies.

## **DISCUSSION**

Altogether experimental and individual difference studies underpin the crucial role of cognitive reappraisal and expressive suppression in adaptive as well as dysfunctional emotional processing and regulation. Furthermore, brain structural and functional studies depict a resulting brain network constituted by target regions for several emotional regulation processes. Namely, the amygdala has a crucial role in emotion regulation as it processes sensory information from the thalamus and somatosensory cortex and has bidirectional projections with hippocampus (emotional memories) and hypothalamus (physiological activation). The regulation of emotional processes is modulated by an rich net of interconnections among amygdala, insula (enteroception, sense of self) and the cortico-subcortical circuits of the OFC (saliency evaluation of emotional state, selection of adequate behaviors) and ACC (emotional state interpretation, motivated behavior). Also PFC (executive functions, cognitive elaboration) indirectly participates in the emotional regulation through its connections with OFC.

Not by chance association between amygdala gray matter volume and anxiety-related traits/states have been reported in numerous studies in healthy subjects (Barrós-Loscertales et al., 2006; Tottenham et al., 2010; van der Plas et al., 2010; Gerritsen et al., 2012) as well as altered activation and volume in the amygdala are common findings in mood and anxiety disorders (Etkin and Wager, 2007; Drevets et al., 2008; Irle et al., 2010; Atmaca, 2011; Kempton et al., 2011; Sacher et al., 2012). Furthermore, reduced activation of the vmPFC along with amygdala hyperactivation and a dysfunctional recruitment of ACC and dmPFC has been observed in patients with specific phobia and post-traumatic stress disorder (Schienle et al., 2007; Hermann et al., 2009; Milad et al., 2009), most likely indicating reduced cognitive control of emotional reactions. Interestingly, phobic individuals more frequently using cognitive reappraisal have an increased vmPFC activation during extinction learning and recall (Hermann et al., 2013b), probably related to a stronger extinction learning as following a successful CBT (Schienle et al., 2007).

The top-down emotional control network via cognitive reappraisal engages also OFC (Ochsner and Gross, 2005; Hermann et al., 2013b). By contrast, habitual bottom-up use of expressive suppression rely more heavily on the anterior insula (Giuliani et al., 2011a) and dACC/paracingulate cortex and medial PFC volume (Hermann et al., 2013a) as well as on increased insula, PFC and amygdala activation (Ohira et al., 2006; Goldin et al., 2008; Hayes et al., 2010; Vanderhasselt et al., 2013). In this neural correlates pattern the role of the insula emerges, not only in primarily supporting interoception and monitoring emotional awareness and outward emotional expression, but also as a relay point between the bottom-up signals from brain regions involved in emotional responding and inward emotional state, like the amygdala, and bottom-up signals from other regions involved in cognitive regulation and regulation goals, like the PFC (Nunn et al., 2008).

# **CONCLUSIONS**

As conclusive considerations, further studies are required to outline more in depth the relations among structural and functional data, trait and state emotion regulation and their interactions. In fact, given the strict relationship between expressive suppression, depression and stress-related symptoms (Moore et al., 2008), the question of whether this strategy is a vulnerability or causal factor remains still open. Otherwise, to evaluate its long term effects on anxiety, depression or other pathologies innovative clinical interventions could be designed training clients to cognitive reappraisal or even *positive reappraisal*, a recent trying to incorporate meditation mindfulness into cognitive therapy (Garland et al., 2009; Hanley and Garland, 2014).

Finally, another direction for the future studies is to carry out longitudinal researches that, allowing repeated observations of the effects of using particular emotion regulation strategies, would help to understand the causal order of effects of the habitual use of cognitive reappraisal or expressive suppression.

## **REFERENCES**


**Conflict of Interest Statement**: The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 15 July 2014; accepted: 01 September 2014; published online: 19 September 2014*.

*Citation: Cutuli D (2014) Cognitive reappraisal and expressive suppression strategies role in the emotion regulation: an overview on their modulatory effects and neural correlates. Front. Syst. Neurosci. 8:175. doi: 10.3389/fnsys.2014.00175*

*This article was submitted to the journal Frontiers in Systems Neuroscience*.

*Copyright © 2014 Cutuli. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

# Individual differences in approach-avoidance aptitude: some clues from research on Parkinson's disease

#### Alberto Costa<sup>1</sup> \* and Carlo Caltagirone1,2

<sup>1</sup> Department of Clinical and Behavioural Neurology, IRCCS Santa Lucia Foundation, Rome, Italy, <sup>2</sup> Dipartimento di Medicina dei Sistemi, Rome University Tor Vergata, Rome, Italy

Approach and avoidance are two basic behavioral aptitudes of humans whose correct balance is critical for successful adaptation to the environment. As the expression of approach and avoidance tendencies may differ significantly between healthy individuals, different psychobiological factors have been posited to account for such variability. In this regard, two main issues are still open that refers to (i) the role played by dopamine neurotransmission; and (ii) the possible influence of cognitive characteristics, particularly executive functioning. The aim of the present paper was to highlight the contribution of research on Parkinson's disease (PD) to our understanding of the above issues. In particular, we here reviewed PD literature to clarify whether neurobiological and neuropsychological modifications due to PD are associated to changes in approach-avoidance related personality features. Available data indicate that PD patients may show and approach-avoidance imbalance as documented by lower novelty-seeking and higher harm-avoidance behaviors, possibly suggesting a relationship with neurobiological and neurocognitive PD-related changes. However, the literature that directly investigated this issue is still sparse and much more work is needed to better clarify it.

Keywords: approach-avoidance, dopamine systems, Parkinson's disease, personality, motivation disorders, cognitive functioning, executive abilities

# Introduction

Actively seeking contact with rewarding stimuli and avoiding unpleasant conditions in the environment are critical in the functional adaptation of humans to their life context. Indeed, people learn early that in some conditions they have to maintain an approach attitude to pursue a desired goal and in other conditions they have to inhibit the tendency to move toward an object or a person to avoid negative outcomes. This implementation balance in approach-avoidance operations should allow the formation of behavioral modules at the level of the disposition to act, which represent the pre-conditions for obtaining correct knowledge of the world and one's own limits, successful access to resources and, at the same time, provide for one's own safety.

Within a psychobiological framework it has been posited that the activity of two main motivation systems modulates the approach-avoidance aptitude of an individual: the behavioral activation system and the behavioral inhibition system (Reinforcement Sensitivity Theory;

#### Edited by:

Maria V. Sanchez-Vives, Institució Catalana de Recerca i Estudis Avançats - Institut d'Investigacions Biomèdiques August Pi i Sunyer, Spain

#### Reviewed by:

Antonio Cerasa, Institute of Bioimaging and Molecular Physiology-CNR, Italy Astrid Müller, Hannover Medical School, Germany

#### \*Correspondence:

Alberto Costa, Department of Clinical and Behavioural Neurology, IRCCS Santa Lucia Foundation, Via Ardeatina 306, 00179 Rome, Italy Tel: +390651501190 Fax: +390651501388 a.costa@hsantalucia.it

> Received: 15 July 2014 Accepted: 05 March 2015 Published: 24 March 2015

#### Citation:

Costa A and Caltagirone C (2015) Individual differences in approach-avoidance aptitude: some clues from research on Parkinson's disease. Front. Syst. Neurosci. 9:43. doi: 10.3389/fnsys.2015.00043 Gray, 1970; Pickering and Gray, 2001). The first system was considered to mediate behavior related to gratifying conditions or potential positive outcomes of a situation and to be specifically sensitive to rewarding or non-punishing stimuli and to promote active searching for potentially rewarding conditions. By contrast, the behavioral inhibition system was considered particularly sensitive to punishment and non-rewarding stimuli. It modulated behavior by inhibiting appetitive responses and increasing arousal in order to improve attention to salient and relevant stimuli, e.g., potentially harmful stimuli, in the environment. The predominant activity of one of the two above systems was considered to lead to greater or even exclusive expression of behavioral moduli related to approach or, alternatively, to avoidance aptitudes, thus determining an individual's stable dispositional response mode to external stimuli.

Recent findings deriving from both animal (mammals) models and human studies suggest that the activity of the above-mentioned motivational systems and, thus, the degree to which approach and avoidance behaviors can be expressed in an individual, depends on the variable effects of both biological and psychosocial factors. In particular, results of studies with mammals show that the approach-avoidance aptitude may be modulated by the central activity of the neuropeptides oxytocin (OT) and arginine vasopressin (AVP) in target brain regions (Young, 2002); interaction with the dopamine reward system was also suggested (Skuse and Gallagher, 2009). Human data document that in healthy subjects personality traits and some cognitive processes may be related to the likelihood of adopting approach or avoidance behavior (Rettew et al., 2006; Spielberg et al., 2011). Finally, in persons suffering from psychopathological disorders, the approach-avoidance related motivational systems may show differential sensitivity to environmental stimulation (Muris et al., 2001; Hirano et al., 2002; Mitchell and Nelson-Gray, 2006). In view of the above observations, interest has recently been centered on individual differences in approach-avoidance behavior and on the possible role played by the interaction between the different psychobiological factors in moderating its expression.

# Aims of the Review

The purpose of this paper was to highlight findings deriving from research in individuals with PD that might contribute to clarifying the factors related to individual differences in approach and avoidance aptitude. In particular, we here reviewed PD literature to clarify whether neurobiological and neuropsychological modifications due to PD are associated to changes in approach-avoidance related personality features. We focused on this issue for three main reasons: First, PD clinical manifestations are primarily a consequence of dopamine dysfunction in neural networks whose activity is considered important for sustaining the activity of behavioral motivation systems (Young, 2002; Calabresi et al., 2006; Laricchiuta et al., 2014). Second, some data suggest that PD patients develop personality characteristics and psychopathological disorders associated with avoidance behavior (Meyer et al., 1999; Muris et al., 2001). Third, PD patients frequently present neuropsychological disorders involving cognitive functions that are critical for sustaining goal-directed behavior (Halliday et al., 2014). These three points will be discussed by focusing mainly on the results of studies that suggested a potential relationship between the modifications occurring during the course of PD and the expression of various aspects of approach and avoidance behavior.

# Methods

The studies were searched using electronic database Medline and PsychoInfo in a period including the first months of 2014. In both databases the same following keywords were used: Approachavoidance; Dopamine systems; Parkinson's disease; Personality; Motivation Disorders; Cognitive functioning; Executive abilities. The studies included in the review should investigate, in PD patients, personality traits that could be related to approachavoidance aptitude and their relationship with neurobiological and neuropsychological variables. A list of the studies that were considered with a description of main characteristics and results is reported in **Table 1**.

# Neurobiological Mechanisms of Approach-Avoidance Behavior and PD: Evidence of an Overlap

# Neurobiologic Correlates of Approach-Avoidance Behavior: Evidence from Non-PD Studies

Enter et al. (2012) found that dopamine transporter (DAT1) polymorphisms were related to different approach-avoidance behaviors when healthy adults were assessed using a task that had stimuli with emotional social valence (i.e., human faces). In particular, these authors demonstrated that, compared with DAT1 10-repeat homozygote carriers DAT1 9-repeat carriers showed an increased effect of the presented stimuli (happy and angry faces) in approach-avoidance responses. This finding suggests that the motivational behavioral systems of these subjects are more sensitive. The DAT is involved in dopamine reuptake in the striatum and the DAT1 9 repeat carriers have been reported to have lower levels of DAT than individuals with 10-repeat alleles, which indicates that these subjects have higher dopamine concentrations in the striatum (Heinz et al., 2000). Furthermore, in a recent functional magnetic resonance imaging (fRMI) study in healthy young subjects, Simon et al. (2010) documented greater ventral striatal and mesial orbito-frontal cortex activation when individuals who showed a high expression of rewardseeking behavior actually received rewards. By contrast, they found less ventral striatal activation when subjects who were more prone to inhibit appetitive behavior received a reward (Simon et al., 2010). These findings provide evidence in line with previous data from animals models that dopamine neurotransmission in neural networks (including the striatal structures) is critically involved in the modulation of motivation behavior (For a review on animal models see, Hoebel et al., 2007).


TABLE 1 | The table summarizes the results of main studies investigating approach-avoidance related functioning in individuals with Parkinson's disease (PD).

HCs: Healthy control subjects; PET: Positron emission tomography; TPQ: Tridimensional Personality Questionnaire (Cloninger, 1987); TCI: Temperament and Character Inventory (Cloninger et al., 1993); KSP: Karolinka Scales of Personality Questionnaire (Schalling et al., 1987); BFAC: Big Five Adjective Checklist (Caprara et al., 2002); MMPI Minnesota Multiphasic Personality Inventory (Dahlstrom et al., 1972); Neo-FFI: Neo-Five Factor Inventory (Costa and McCrae, 1992).

In particular, based on findings suggesting that dopamine activity would promote appetitive behavior (e.g., moving toward external stimuli, reward seeking), whereas acetylcholine would mainly enhance behavioral inhibition and aversive responses (Mark et al., 1995; Avena et al., 2006), Hoebel et al. (2007) proposed that dopamine interacts with acetylcholine in the ventral striatum (i.e., in the nucleus accumbens) to maintain a functional balance between approach and avoidance tendencies.

# Neurobiological Modifications in PD

PD is a well-known neurological disease that is primarily characterized by dysregulation of the nigro-striatal, mesolimbic and the mesocortical dopaminergic brain systems. (Owen, 2004; Dickson et al., 2009). More specifically, degeneration of the dopamine cells in the midbrain leads to precocious and severe dopamine depletion in the striatum, which first involves the rostrodorsal extent of the head of the caudate nucleus and, later, the ventral tegmental neurons that project to more ventral parts of this structure and to prefrontal and limbic regions (Yeterian and Pandya, 1991; Agid et al., 1993; Costa et al., 2009). In fact, in addition to movement disorders PD patients often display cognitive-behavioral deficits (Robbins and Cools, 2014). Although the role of dopamine brain transmission in causing cognitive-behavioral disorders in PD has not yet been completely clarified, in the early phase of the disease cognitive deficits are considered due to an imbalance between phasic dopamine activity in the dorsal striatum and tonic dopamine activity in the prefrontal cortex, which leads to reduced efficiency of flexibility processes (i.e., updating and set-shifting) (Cools, 2006; Cools and D'Esposito, 2011). With disease progression and the parallel greater involvement of dopamine transmission in the ventral striatum and the dopamine projections to the other structures of the mesolimbic system, reduced ability to decode and use environmental stimulation (e.g., reinforcers) to adopt functional behavior, and altered emotional processing and declarative memory disorders are observed. The hypothesis was also advanced that the disrupted equilibrium between the activity of dopamine and acetylcholine, which occurs in the striatum, could account for some of the cognitive-behavioral manifestations of PD (Calabresi et al., 2006). As stated above, dopamine projections to striatal structures are primarily affected in PD, thus causing a decrease of dopamine activity and, likely, a parallel increase of cholinergic tone. According to Calabresi et al. (2006) the altered dopamine-acetylcholine equilibrium could affect synaptic mechanisms of long-term potentiation and depression and of synaptic depotentiation, in some way modifying frontalstriatal interconnections and causing learning and executive disorders.

The above mentioned evidence suggests that PD may precociously cause functional and structural changes in frontalstriatal regions whose activity is supposed to be responsible for the modulation of approach and avoidance responses in animals as well as in humans. Thus, PD is an interesting natural human model for investigating the psychobiological mechanisms involved in learning and sustaining these behavioral aptitudes.

# Do the Personality and Psychopathological Features of PD Indicate an Approach-Avoidance Imbalance?

In the previous section we suggested that the neuropathological processes of PD might affect the functioning of brain circuitries involved in the mediation of approach and avoidance tendencies. This leads to the key question of whether these two main aspects of the behavioral motivational systems are impaired in PD patients. Some clinical reports are in line with this idea. In fact, a large proportion of PD patients suffer from depressive disorders and apathy (Aarsland et al., 2011; Martínez-Horta et al., 2013), which have been shown to be associated with a significant decrease of appetitive and self-initiated behaviors also in PD (Costa et al., 2006; Martínez-Horta et al., 2013; Damholdt et al., 2014; Spielberg et al., 2014). Anxiety symptoms, which are associated with avoidance behavior, particularly in the context of social interactions (Wong and Moulds, 2011), are also frequently described in these patients (Sagna et al., 2014). An opposite behavioral pattern, characterized by excessive attraction to rewarding stimuli, is observed in some PD individuals who develop impulse control disorders especially in response to the administration of dopamine therapy (Callesen et al., 2013).

More direct evidence of an imbalance between the behavioral activation and inhibition systems comes from research on the personality functioning of PD patients. These data document that personality traits such as novelty seeking, which mainly refers to the propensity towards active exploration in response to novel stimuli and the avoidance of frustration (Cloninger, 1987), are expressed to a lesser extent in PD patients compared to controls without neurologic diseases (Menza et al., 1993; Bódi et al., 2009; for a review see Poletti and Bonuccelli, 2012). By contrast, the personality trait of harm avoidance, which is characterized by the inclination to adopt a passive avoidance behavior, was found to be much more present in PD patients than in controls (Jacobs et al., 2001; Tomer and Aharon-Peretz, 2004; McNamara et al., 2008; Koerts et al., 2013). Other studies reported evidence of reduced extraversion in PD patients (Damholdt et al., 2014), which probably indicates their low aptitude for approach social interactions (McCrae and Costa, 1997).

# Personality Modifications May Predate Clinical Manifestation of PD

It was also hypothesized that personality changes might occur in a pre-clinical phase of PD. This hypothesis was mainly grounded on the observation that the presence of a personality with low novelty-seeking functioning, rigidity and caution predates the onset of extrapyramidal symptoms (for a review see Menza, 2000). Nevertheless, few longitudinal studies have been conducted to investigate this hypothesis. In this regard, some interesting data were reported by Bower et al. (2010) and by Arabia et al. (2010) in a cohort study in which more than 6,800 persons were followed for four decades. The Minnesota Multiphasic Personality Inventory, a validated psychometric test that investigates psychological disorders (Dahlstrom et al., 1972), was used to assess personality. The authors did not find a clear relationship between the constructs of introversion and extroversion and the risk of PD. However, results showed that an anxious personality, as assessed by the psychoastenia scale, is associated with a higher risk of developing PD with a hazard ratio of 1.63. This finding is quite interesting for our discussion because there are reports that anxiety symptoms are associated with avoidance behavior in both human studies (Muris et al., 2001; Wong and Moulds, 2011) and animal models (Toth and Neumann, 2013). Evidence that the neurobiochemical alterations of PD occur years before clinical manifestation of the disease also support the idea of a correlation with these personality changes. Nevertheless, findings from studies that directly correlated the personality changes and neurobiological modifications of PD are still sparse.

# Relationship between Personality Changes and Neurobiological Modifications in PD Findings Indicating a Positive Association

Bódi et al. (2009) documented a significant relationship between dopamine stimulation and novelty seeking in drugnaïve PD patients. In particular, they investigated the effect of dopamine agonist administration on different personality traits (i.e, novelty seeking, harm avoidance, reward dependence and persistence) and on reward-learning using a feedbackbased task. Patients underwent two assessments in which the examiners were blind to personality measures, test results and medication conditions, the first without taking any medication and the second after a 12-week period of treatment with the D<sup>2</sup> and D<sup>3</sup> dopamine receptor agonists pramipexole and ropinirole. At the first assessment PD patients showed significantly lower novelty-seeking and reward-learning scores than healthy controls. The second assessment showed that dopamine intake significantly improved PD patients' novelty-seeking scores and reward-learning performance so that they could no longer be distinguished from healthy controls on these measures (Bódi et al., 2009).

Tomer and Aharon-Peretz (2004) reported more complex results. They showed that left-right side asymmetry of dopamine-related pathology may differentially affect personality functioning in PD patients. In fact, findings from this study suggest that in these patients dopamine loss in the left hemisphere is associated with reduced novelty- seeking behavior while higher harm avoidance is related to dopamine loss in the right hemisphere (Tomer and Aharon-Peretz, 2004). These findings are in line with those of a previous PET study in PD patients in which Menza et al. (1995) demonstrated that 6-[18F]fluorodopa uptake in the caudate nucleus (of the left hemisphere) correlated with novelty-seeking scores.

Above observations are congruent with previous evidence in people without neurological disorders of the relevant role of dopamine transmission in the striatum in modulating sensitivity to reward- and novelty-seeking behavior (Leyton et al., 2002), and would sustain the general hypothesys that novelty seeking is strongly related to dopamine transmission (Cloninger, 2000; but see Paris, 2005 for a critical review). Also interestingly, the above findings are congruent with the hypothesis of brain hemispheric asymmetry in mediating activation and inhibition behavioral systems, with the left hemisphere more associated with approach and the right hemisphere with avoidance (Spielberg et al., 2013).

## Studies Documenting a Non-Linear Association between Personality Features and PD Related Neurobiological Changes

Partially divergent results in respect to those above discussed were reported in other studies with PD patients. Indeed, Kaasinen et al. (2004) showed an inverse correlation between novelty-seeking scores and dopamine receptor availability in the insula, a brain region highly interconnected with the striatum and suggested to be involved in different cognitive and emotional processes in PD (Christopher et al., 2014). The finding by Kaasinen et al. (2004) indicates that the likelihood of PD patients expressing a higher novelty-seeking aptitude corresponds with lower dopamine activity in this structure. In another independent PET study Kaasinen et al. (2001) also demonstrated that in unmedicated PD patients novelty-seeking scores did not correlate with 6- [18F]fluorodopa uptake in any of the target brain regions. Instead, higher harm avoidance scores were positively correlated with 6-[18F]fluorodopa uptake in the caudate nucleus.

In summary, an imbalance between the activity of behavioral activation and inhibition systems, characterized by lower approach and higher avoidance tendencies, seems to be present in PD patients (see **Table 1** for a synthesis of the results of the main studies). This is supported by observations of their reduced novelty seeking, sensitivity to reward and self-initiated behavior. However, results on the potential role played by the neurobiological changes of PD and this hypothesized imbalance are inconclusive. One limit of most studies investigating this issue in PD patients is the use of self-rating psychometric tools that require selfjudgment of one's own characteristics. Further studies using more objective, performance-based paradigms, which specifically assess approach and avoidance behaviors, would be more informative. For instance, to better understand the role of dopamine neurotransmission in target brain regions on these processes, a functional magnetic resonance protocol could be used to investigate how dopamine administration/withdrawal modulates neural activity in PD patients while they perform approach-avoidance procedures. Future investigations should also take into account several potentially confounding individual clinical (e.g., disease duration, disease severity, side of onset, pattern of movement disorders) and cognitive (e.g., presence of dementia, mild cognitive impairment or attention disorders) characteristics of the disease.

# Are the Executive Disorders in PD Potentially Related to Approach-Avoidance Tendencies?

# Results Evidencing an Association between Approach-Avoidance Behavior and Executive System in Individuals without PD

The interaction between approach and avoidance motivational systems and executive functions appears to be critical for the maintenance of goal pursuit and, thus, for the implementation of adaptive behavior (for a review see Spielberg et al., 2008, 2013). This interaction has been observed in both behavioral and neuroimaging investigations. Gray (2001) showed that the induction of approach and withdrawal motivation differentially affected subjects' performance on verbal and spatial working memory tasks. In a subsequent study Spielberg et al. (2011) showed that trait motivation modulated dorsolateral prefrontal cortex activity while healthy subjects were performing a selective attention task (i.e., the Stroop color-word task). Specifically, in a fMRI protocol, these authors explored changes in neural activity as a function of subjects' scores on questionnaires investigating approach and avoidance temperament. Results showed a significant positive correlation between approach temperament scores and activation in the left superior and middle frontal gyri. A positive association was also found between avoidance scores and activation in the middle frontal gyrus (bilaterally) and the left superior frontal gyrus (Spielberg et al., 2011).

Similar results indicating an interaction at the level of the prefrontal cortex between motivation and executive abilities were previously reported by Pochon et al. (2002) and by Taylor et al. (2004). Indeed, the prefrontal cortex has been consistently found to be highly involved in the mediation of several executive abilities (e.g., planning, working, multitasking, prospective memory) and it is considered fundamental for the correct organization of information and goal-directed behavior (Miller and Cohen, 2001; Burgess et al., 2011; Yuan and Raz, 2014). A recently proposed integrated view accounts for the interaction between approach-avoidance traits, state motivation and executive skills in coherently pursuing internal goals (Spielberg et al., 2013).

On the basis of the above observations it can be hypothesized that the qualitative characteristics of executive functioning influence the expression of individual approach and avoidance tendencies. This hypothesis is corroborated by the clinical manifestation of brain damage involving the prefrontal cortices. These patients often present cognitive and behavioral signs---such as a decrease in goal-directed behavior, apathy and disinhibition---that could be related to an imbalance between activation and inhibition systems.

# Results from Studies with PD Patients

We previously mentioned that PD patients present cognitive disorders early in the disease course, which are primarily related to dopamine dysfunction in the frontal-striatal circuitries (Costa et al., 2014a), with convergent evidence documenting their reduced ability to perform tasks sensitive to executive functions (Dirnberger and Jahanshahi, 2013; Kudlicka et al., 2013; Robbins and Cools, 2014). In particular, set-shifting and updating efficiency appear to be reduced early, likely affecting their ability to successfully maintain a goal- directed behavior (Cools, 2006; Cools and D'Esposito, 2011). In fact, these patients have difficulty in performing planning and multitasking tests and in spontaneously retrieving the intention to perform planned actions (Owen, 2004; Kliegel et al., 2011; Costa et al., 2014b,c).

Based on above observations documenting both an association between approach and avoidance motivational systems and executive functions in non-PD individuals and, in PD patients, a decreased efficiency of the executive system, we could hypothesize, in the latters, the existence of a relationship between executive dysfunctioning and approach-avoidance imbalance. Some findings are in line with these hypothesis (in **Table 1** a synthesis of main results is reported).

McNamara et al. (2008) reported, in 44 PD patients without dementia, a significant inverse correlation between harm avoidance rates, as assessed by means of the Temperament and Character Inventory, and a measure of cognitive flexibility and strategic access to information (i.e, verbal fluency). In an other study, Volpato et al. (2009) administered to 25 PD patients without dementia the Big Five Adjective Checklist to examine different personality traits, and the Tower of London test and Alternating Fluency test to investigate planning and setshifting, respectively. The authors found a significant correlation between the personality factor of openness to experience and PD patients'score on alternating fluencies, thus indicating a significant relationship between this personality factor and cognitive flexibility processes.

In a more recent study with 43 PD patients without dementia, Koerts et al. (2013) investigated the relationship between different components of executive functions (cognitive flexibility, inhibition, working memory, planning and verbal fluency) and various aspects of personality functioning (novelty seeking, harm avoidance, persistence and reward dependence). Neuropsychological tests and questionnaires (the Temperament and Character Inventory) were administered to assess cognitive and personality features, respectively. The authors found that PD patients' scores on cognitive flexibility measures significantly predicted reward dependence rates (Koerts et al., 2013).

In summary, taken together the above findings do not allow us to make firm conclusions about the nature of the relationship between executive functioning and approachavoidance imbalance in PD. Indeed, the reported associations between some personality factors (i.e, reward dependance and openness to experience) and cognitive flexibility is undoubtedly interesting as neural activity in the frontal-striatal circuitries has been demonstrated to be critical for both reward-related behavior and cognitive flexibility (O'Doherty, 2004; Bódi et al., 2009; Macdonald and Monchi, 2011). However, a better method for investigating the relationship between executive functioning and approach-avoidance aptitude in PD patients would be to test the effect of cognitive training aimed at potentiating executive abilities on approach-avoidance related behavior. The observation of executive improvement combined with a change in approach-avoidance behavior would be clear evidence of a causal relationship. In this regard it should be noted that some cognitive training has been proposed which significantly enhances some components of executive functioning, e.g., cognitive flexibility, which, as we discussed here, may be associated with approach-avoidance tendencies also in persons with PD (Calleo et al., 2012; Costa et al., 2014b).

# Conclusions

Currently, the main issues in the study of individual differences in the expression of approach and avoidance behaviors are (i) the role played by dopamine neurotransmission; and (ii) the possible influence of cognitive characteristics, particularly executive functioning. Regarding the first point, there is some evidence that dopamine and its interaction with other neuromodulators at the level of the mesocorticolimbic networks affects these processes by modulating the learning of individual response modalities (Skuse and Gallagher, 2009). However, this evidence is mainly derived from results of studies that used animal models; in fact, human data are inconclusive. Regarding the second point, also in this case the results of some studies suggest the existence of a potentially bidirectional relationship between cognitive (executive) functioning and approach-avoidance behavior. Extant findings are, however, still sparse and not univocal.

Here we highlight the potentially relevant contribution of research on PD in clarifying above issues. In fact, PD is characterized early by dopamine loss in brain circuitries including the frontal-striatal, mesolimbic and mesocortical pathways, that are implicated in sustaining the functioning of motivational behavioral systems and executive processes.

Some evidence seem to support the idea that PD may be associated to an alteration of approach-avoidance behaviors, wherein it documents that PD patients show reduced novelty seeking and higher harm avoidance expression. However, the literature that directly investigated the relationship between PD patients' neurobiological and neuropsychological modifications and approach-avoidance related personality functioning gave inconclusive results. Further research that overcomes the limits of previous studies (e.g., low sample size, clinical heterogeneity, heterogeneity of dopamine treatments, use of self-report questionnaires) are needed to further explore these important topics in PD.

# Acknowledgments

Contribution of the author AC: conception of the work, drafting the work, final approval of the version to be published, agreement to be accountable for all aspects of the work. Contribution of the author CC: conception of the work, revising the draft,

# References


final approval of the version to be published, agreement to be accountable for all aspects of the work.

investigation. Eur. J. Neurol. 13, 972--980. doi: 10.1111/j.1468-1331.2006. 01406.x


Neuropsychologia 49, 2166--2177. doi: 10.1016/j.neuropsychologia.2011. 01.024


in Advances in Individual Differences Research, ed A. Angleitner (Lengerich: PABST Science Publishers), 113--149.


Yuan, P., and Raz, N. (2014). Prefrontal cortex and executive functions in healthy adults: a meta-analysis of structural neuroimaging studies. Neurosci. Biobehav. Rev. 42, 180--192. doi: 10.1016/j.neubiorev.2014.02.005

**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Costa and Caltagirone. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

**REVIEW ARTICLE** published: 14 October 2014 doi: 10.3389/fnsys.2014.00194

# *James F. M. Cornwell 1, Becca Franks <sup>2</sup> and E. Tory Higgins 1\**

<sup>1</sup> Department of Psychology, Columbia University, New York, NY, USA

<sup>2</sup> Animal Welfare Program, University of British Columbia, Vancouver, BC, Canada

#### *Edited by:*

Daniela Laricchiuta, Istituto di Ricovero e Cura a Carattere Scientifico Santa Lucia Foundation, Italy

#### *Reviewed by:*

Raúl G. Paredes, National University of Mexico, Mexico Jesus De La Fuente, University of Almería, Spain

#### *\*Correspondence:*

E. Tory Higgins, Department of Psychology, Columbia University, 406 Schermerhorn Hall, 1190 Amsterdam Avenue MC 5501, New York, NY 10027, USA e-mail: tory@psych.columbia.edu

The hedonic principle—the desire to approach pleasure and avoid pain—is frequently presumed to be the fundamental principle upon which motivation is built. In the past few decades, researchers have enriched our understanding of how approaching pleasure and avoiding pain differ from each other. However, more recent empirical and theoretical work delineating the principles of motivation in humans and non-human animals has shown that not only can approach/avoidance motivations themselves be further distinguished into promotion approach/avoidance and prevention approach/avoidance, but that approaching pleasure and avoiding pain requires the functioning of additional distinct motivations—the motivation to establish what is real (truth) and the motivation to manage what happens (control). Considering these additional motivations in the context of moral psychology and animal welfare science suggests that these less-examined motives may themselves be fundamental to a comprehensive understanding of motivation, with major implications for the study of the "what," "how," and "why" of human and non-human approach and avoidance behavior.

**Keywords: motivation, approach, avoidance, promotion, prevention, truth, control, value**

# **INTRODUCTION**

The hedonic principle has existed for at least as long as we have had the capacity to write down our thoughts about ourselves, being recorded, for example, in the teachings of the ancient Greek philosopher Epicurus. In modern times, the principle reached its fullest expression as a foundation for human psychology and ethics in Bentham's (1789/2007) influential *An Introduction to the Principles of Morals and Legislation:* "Nature has placed mankind under the governance of two sovereign masters, *pain*, and *pleasure*. It is for them alone to point out what we ought to do, as well as to determine what we shall do" (Bentham, 1789/2007). The principle that humans and other animals approach desired end-states and avoid undesired endstates has served as the foundation for important theories across disciplines from political theory (Neumann and Morgenstern, 1944/2007) to behavioral economics (Simon, 1955). In many ways, our common economic life is built around this basic idea (Smith, 1776/2003), and it is one of the primary assumptions of many theories of animal behavior, both human and non-human (Thorndike, 1911, 1935). The fact that human beings and other animals approach pleasure and avoid pain has been treated as *the* fundamental principle from which all other examinations of motivation must flow (Watson, 1913; Freud, 1920/1950; Skinner, 1938).

The ascendance of approach and avoidance in the psychology literature co-occurred with the rise of behaviorism (Thorndike, 1911; Watson, 1913; Skinner, 1938, 1953). Behaviorist theorists, as a rule, argued that one cannot scientifically reason beyond those actions which can be directly observed and the contingencies that surround those actions—namely reinforcements and punishments (Thorndike, 1911). Any speculation as to the internal workings of those motivations, whether cognitive or otherwise, was eschewed as unscientific (Watson, 1913). The hard scientific work of a number of psychologists led to a cognitive revolution in the middle of the 20th century overthrowing behaviorist assumptions (e.g., White, 1959; Bandura, 1977), yet the premise regarding approach and avoidance as *the* fundamental distinction in motivation has remained (for a review, see Higgins, 1997).

Beginning in the late 20th century, many scientific discoveries were made further distinguishing between approaching positive end-states vs. avoiding negative end-states (Carver and Sheier, 1998; Carver, 2004), and extended this distinction into many additional areas of research. For example, Goal Orientation Theory has made significant contributions to our understanding of achievement motivation (Dweck, 1986; Pintrich, 2000; Elliot, 2005). Although future explorations of truth and control motives may discover interesting relations to achievement motivation, our model of motivation is theoretically orthogonal to these lines of research. For this reason, their possible interrelationships will not be discussed in detail here. For the purposes of this review, we focus on the contributions that have culminated in prominent theoretical advances in other fields dealing with motivation, including moral psychology (e.g., Janoff-Bulman et al., 2009) and animal welfare science (e.g., Fraser and Duncan, 1998). In this paper, we argue that, in part due to its resounding success as a theoretical framework for behavior, the approach/avoidance distinction is now at risk of becoming a "one size fits all" principle. Specifically, multiple kinds of motivations are now treated as if they entailed a simple approach/avoidance hedonic distinction

when, in fact, not only are they not simply hedonic, they are also distinct from one another in important ways. We propose that these distinctions matter to future research in many fields of study and that attending to them may yield important breakthroughs.

In this paper, though we argue that this treatment of approach/avoidance as a "one size fits all" principle of motivation is difficult to sustain in light of recent research, we fully recognize the importance of this distinction for motivation science. We thus begin the paper by outlining the ways in which the approach/avoidance distinction has made significant contributions to two areas of study: non-human animal behavior and moral psychology. We then review recent work in these fields demonstrating that other motivational distinctions need to be taken into account as well, and argue that some of these motives may in fact be fundamental to fully understanding the approach/avoidance distinction itself—looking within the "one size fits all" approach/avoidance principle to delineate the "what," "how," and "why" of these orientations and behaviors.

# **ADVANCES IN APPROACH AND AVOIDANCE MOTIVATION RESEARCH**

The hedonic principle has provided psychologists with considerable predictive power since the foundational days of psychology in the 19th century. This includes research on non-human animals driven by an understanding that some (though not all, see Higgins and Pittman, 2008) of the major components of human nature are shared with non-human animals through the branching process of evolution (Watson, 1913; Skinner, 1938, 1953)—and the search for essential components of human nature around which we can organize an ethics to guide our common life (Mill, 1863/2007; for a recent review, see Kitcher, 2011). Interestingly, though assumptions about approach and avoidance have underpinned the study of animal behavior and morality for over 100 years, some particular advances in each field have only been achieved by distinguishing between approach and avoidance in recent decades.

#### **APPROACH/AVOIDANCE IN NON-HUMAN ANIMALS**

For many, the appeal of the study of non-human animals lies in the ability of this line of inquiry to reveal fundamental truths about nature. Evolutionary theory in particular has shown us that human beings constitute only one species of animal, and that we are, at our most basic, mammals subject to many similar kinds of motivations as other animals. Approach and avoidance motivations have played a substantial part in understanding non-human animal behavior. For example, approach and avoidance orientations have been tied to underlying individual differences between animals. Certain animals, more than others, are willing to take risks in order to achieve their goals, while other animals, more than others, are consistently more risk-averse in their behavior (Wilson et al., 1994). Animals that fall into the former category have been classified as "bold," whereas those that fall into the latter category have been classified as "shy." These individual differences map onto a greater reliance on approach and avoidance orientations, respectively.

The bold/shy continuum has received an enormous amount of attention in recent years. Researchers have found individual differences on this continuum with attendant behavioral implications in squid (Sinn et al., 2008), fish (Toms et al., 2010), and lizards (López et al., 2005), just to name a few. Their underlying neurological differences have been studied (Reddon and Hurd, 2009), the impact of environmental context in their variation has been researched (Wilson et al., 1993), and the evolutionary origins of these individual differences have been explored (Wilson et al., 1994). Differences along this continuum have been linked to fundamental differences in stress responses (Oswald et al., 2012) and learning (Sneddon, 2003). It is clear that examining the difference in approach and avoidance inclinations, here understood as having either a bold or shy personality, has proven a to be a productive theoretical foundation for scientific exploration of non-human animals. However, as we will describe in more detail later, there are animal behaviors that cannot be understood from just a "one size fits all" approach/avoidance perspective. A more complete picture of non-human animal motivation requires the consideration of additional motivational distinctions.

## **APPROACH/AVOIDANCE IN MORAL PSYCHOLOGY**

Approach and avoidance motivations are fundamental in another way as well. They can help us to understand one of the most basic principles around which human societies organize themselves: ethics or morality. The principles of approach and avoidance have been integral to the study of ethics since the empiricists of the Scottish Enlightenment designated morality as an instrumental means for bringing about general happiness. For example, Hume (1751/1998), following Hutchinson (1725/2005), argued for the importance of positive and negative sentiments in morality, particularly in moral motivation. Mill (1863/2007) went so far as to argue that pleasure and pain provided aframework bothfor understanding and predicting human behavior and also for building a system of ethical dos and don'ts: things to approach and things to avoid.

Over the years, however, the predictive power of this hedonic approach has been called into question, perhaps most prominently byKahneman and Tversky (1979), whose work revealed important differences between what happens psychologically when people approach positive outcomes vs. avoid negative outcomes. In Prospect Theory, they argued that the motive to avoid pain "looms larger" than the motive to attain pleasure, and, importantly, people are relatively risk-seeking when avoiding negative outcomes (in the domain of losses) but are relatively risk-averse when approaching positive outcomes (in the domain of gains). In a manner similar to Prospect Theory's revisions of the nature of approach vs. avoidance in decision making, Janoff-Bulman et al. (2009) have shown that the two are different with respect to ethical systems. One system, the *proscriptive* system, motivates behavioral inhibition—*avoiding* moral wrongs. The other system, the *prescriptive* system, motivates behavioral activation—*approaching* moral rights. Their research has shown that approach and avoidance are not merely inverted images of one another, but that each has unique goals and tendencies and characteristics associated with it (including differences in political ideology; see Janoff-Bulman et al., 2008). Researchers using this paradigm have theorized important differences between the two systems that could have importance for research on ethics generally (Janoff-Bulman and Carnes, 2013).

These theories are important because they can aid in further understanding the nature of decision making and behavior in human society. However, there are additional motivational distinctions that need to be considered in order to have a more complete picture of ethics, and, as we will see, to understand animal behavior as well. In the next section, we distinguish between two distinct motivational systems around which approach and avoidance motivations are organized, and show how they have made important contributions to the areas of animal welfare research and moral psychology.

# **PROMOTION AND PREVENTION VALUE MOTIVES: THE "WHY" OF APPROACH AND AVOIDANCE**

Over the past two decades, research on approach and avoidance has been qualified by research on regulatory focus theory (Higgins, 1997, 1998). According to regulatory focus theory, the valued goals of approaching desired end-states and avoiding undesired end-states are organized into two independent and distinct motivational systems (Molden et al., 2008; Higgins, 2014). The *promotion* system approaches end-states related to nurturance, advancement, and growth while avoiding deprivation or stagnation—the motivation is to advance from the status quo "0" to a better state "+1." Success or pleasure in promotion is attaining a "+1" (gain) and failure or pain is not attaining a "+1" (non-gain). The *prevention* system approaches end-states related to security and safety while avoiding danger or threat—the motivation is to maintain a satisfactory status quo "0" against a worse state "−1." Success or pleasure in prevention is maintaining "0" (a non-loss) and failure or pain is not maintaining "0" (a loss). A visual representation of these systems with respect to the approach and avoidance systems is available in **Figure 1**.

We should note a few other relevant distinctions within regulatory focus theory (for a fuller account, see Scholer and Higgins, 2008). Both when approaching desired end-states and avoiding undesired end-states, humans and non-human animals can pursue their goal with either an eager strategy of seeking opportunities to make progress or a vigilant strategy of being careful and avoiding mistakes. Whether approaching desired end-states or avoiding undesired end-states, the preferred strategy (i.e., what fits) for those with a promotion focus is an eager strategy whereas the


preferred strategy for those with a prevention focus is a vigilant strategy (Higgins, 2000). But, notably, the non-preferred strategy (i.e., a non-fit) will also be used, sometimes because it is dictated by the situation (e.g., instructions about what strategy to use, like a team leader telling other team members to "be careful"). Finally, it should also be noted that either taking action (behavioral approach) or inhibiting action (behavioral avoidance) can occur in the service of strategic eagerness or in the service of strategic vigilance. For example, as we will see below, an animal concerned about safety (prevention) might be careful (vigilant strategy) to approach something new in the cage (behavioral approach) in order to make sure that it is safe, or approach a noxious object in the cage in order to bury it (behavioral approach).

The literature has seen a proliferation of research based on this model showing that the promotion and prevention distinction has significant explanatory power independent of the hedonic approach and avoidance motivations (Higgins et al., 1997, 2003; Förster et al.,1998,2001; Malaviya and Brendl,2014). This research has covered a variety of domains from persuasion (Cesario et al., 2004), to negotiation (Appelt et al., 2009), to consumer choice (Higgins, 2002), to interpersonal relationships (Molden et al., 2009). Theoretically, it has inspired more comprehensive models of the approach and avoidance motives, understanding them as being further subdivided into promotion goals (including "ideal" hopes and aspirations) on the one hand and prevention goals (including "ought" duties and obligations) on the other (Crowe and Higgins, 1997; Higgins, 1997). Recently, there is even evidence in neuroscience that the two kinds of distinctions—promotion vs. prevention in contrast to approach vs. avoidance—are associated with independent patterns of brain activation (Eddington et al., 2007; Strauman and Wilson, 2010). In this section, we will be focusing on new research applied to the fields of animal welfare science and moral psychology to highlight the fundamental character of the promotion vs. prevention distinction.

## **PROMOTION/PREVENTION IN NON-HUMAN ANIMALS**

The promotion vs. prevention distinction is well-established in humans, but recent research supports their existence in non-human animals as well. For example, one set of studies examined individual differences in behaviors reflecting promotion and prevention motivations among a group of zoo-housed cotton-top tamarins (Franks et al., 2013b). Through extensive observation, researchers identified individuals who consistently prioritized gains over safety (time spent eating in the open) or safety over gains (time spent hiding) in order to classify them as more promotion-focused or prevention-focused, respectively. Importantly, an approach/avoidance model of behavior (using the bold/shy conceptualization mentioned above) could similarly classify the above individuals as being more approachoriented ("bold") or avoidance-oriented ("shy"), respectively. Thus, it is unclear whether the monkeys appearing to prioritize safety are *avoiding* danger—which would be more in line with an avoidance-motivated model of behavior—or *approaching* security—which would be more in line with a preventionmotivated model of behavior. Similarly, it is unclear whether the monkeys appearing to prioritize gains are approaching *indiscriminately*—which would be more in line with an approach-motivated model of behavior—or are motivated by gains *specifically*—which would be more in line with a promotion-motivated model of behavior.

In order to test which model—approach/avoidance vs. promotion/prevention—made a better account of the animals' behavior, researchers placed two different kinds of novel enrichment items into the monkey's housing: one a "gain" enrichment and the other a "non-gain" enrichment (Franks et al., 2013b). If the primary difference separating the individuals described above was an approach/avoidance difference, then those classified as "approach"-oriented should approach *all* the novel objects more quickly than those classified as "avoidance"-oriented since coming into the open to examine a novel stimulus carries risk and should be unconditionally more aversive to the "avoidance"-oriented individuals than the "approach"-oriented individuals. However, if instead the two types of monkeys were actually different according to the promotion/prevention distinction, then this dynamic should only be *conditionally* true in the case of the "gain" object. In the case of the "non-gain" object, a regulatory focus model of behavior suggests that a promotion-focused individual should be uninterested because it has no gain potential, while the prevention-focused individual should be interested in cautiously examining the object in order to establish its status as a nonthreat. The results confirmed the regulatory focus distinction, with prevention-focused monkey *approaching* a novel "nongain" object *faster* than promotion-focused monkey (Franks et al., 2013b).

Research with rats has also shown that there are individual differences with respect to preferences for promotion vs. prevention goals. In one study, these goals were operationalized as time spent near a location containing food reward (promotion) vs. time spent near a location that turned the overhead light off, which, because they are nocturnal, creates security for rats (prevention; Franks et al., 2012). Though the promotion or prevention behavior of individual rats was stable in this test over several weeks, once again, from these behaviors alone it is unclear whether the observed differences are due to an approach/avoidance distinction or a promotion/prevention distinction. Were the darkness-preferring rats *avoiding* light as aversive or *approaching* darkness to maintain security? Were the treat-preferring rats approaching indiscriminately or approaching gains specifically?

To test these alternative hypotheses, the rats were observed in a different apparatus into which a noxious stimulus was introduced (Franks et al., 2012). If the rats preferring darkness were driven primarily by an avoidance orientation, then we would expect them to move as far away from the noxious stimulus as possible. If, however, the rats preferring darkness were driven by a prevention orientation, then we would expect them to *approach* the noxious stimulus in order to bury it, which is a rat's natural defensive behavior and means by which to restore safety (Pinel and Treit, 1978). The results favored the latter hypothesis: time with the noxious stimulus to bury it (vigilant behavioral approach in the service of prevention) was predicted by time spent maintaining darkness (prevention) and not by time spent with food reward (promotion). This animal behavior research is in line with recent

work in humans examining how a prevention focus can actually motivate riskier behavior (i.e., more approach-oriented behavior) when under conditions of loss or threat (Scholer et al., 2010).

Finally, another set of studies with a separate group of rats further distinguished promotion and prevention motivation from simple approach/avoidance motivation (Franks et al., 2014). Rats were again placed in an environment in which they could focus on maximizing gains (obtaining treats) and on maintaining safety (keeping the room dark) and again stable individual differences in promotion and prevention motivation were observed. In this experiment, however, researchers were able to collect a measure of chronic stress (or poor welfare) and found it to be inversely related to *both* promotion and prevention. This finding parallels empirical work in humans (Grant and Higgins, 2003) and corresponds to regulatory focus theory, which predicts positive emotions and well-being resulting from being effective at both promotion and prevention goals (Higgins, 1997). This finding would be somewhat puzzling from a simple approach/avoidance model of motivation and welfare, however, because if avoidance was driving the prevention animals' behavior, they should be more fearful than other animals and thus have lower welfare.

In sum, the behavior in non-human animals suggests a distinction between promotion goals and prevention goals *within* the approach and avoidance systems. Indeed, given its presence among non-human animals, even those as evolutionarily distant from humans as rats, this regulatory focus distinction—the "why" of approach and avoidance—may turn out to be a fundamental characteristic within the approach/avoidance distinction. This distinction is not limited to non-human animal welfare, however. As we discuss next, the role of promotion and prevention in moral psychology has been theoretically and empirically demonstrated as well.

## **PROMOTION/PREVENTION IN MORAL PSYCHOLOGY**

Over the past several years, there has been a push to examine the approach and avoidance distinction within moral psychology. However, until very recently, the independent role of promotion goals and prevention goals in the area of ethics has been largely unexplored. As with the non-human animal research outlined above, goals to approach moral behavior can be understood as stemming from either ideals or oughts, and efforts taken to avoid immoral behavior can similarly be understood as avoiding discrepancies with these distinct kinds of goals.

The first major exploration of promotion and prevention motivations in morality was carried out by Camacho et al. (2003). In a series of studies, these researchers found that different strategic framing of moral errors ("sins") created more intense feelings of regret depending on whether a person had a strong promotion or prevention focus. For example, those whose sins involved an error of *commission* (not being vigilant enough to avoid doing something bad) experienced more regret when they had a stronger prevention focus, whereas those whose sins involved an error of *omission* (not being eager enough to do something good) experienced more regret when they had a stronger promotion focus.

Like the animal behavior studies above, it would be possible to make similar predictions as those above relying only on the approach/avoidance distinction, instead stating that those who experience their errors of commission as more wrong are more motivated by avoidance inclinations and those who experience their errors of omission as more wrong are more motivated by approach inclinations. Are those with stronger promotion goals simply more concerned about approaching moral rights? Similarly, are those with stronger prevention goals simply more concerned about avoiding moral wrongs?

Other results from the Camacho et al. (2003) research provide evidence that the regulatory focus distinction matters for ethical responses beyond simple approach/avoidance. If the effects were due entirely to concern with approaching moral rights or avoiding moral wrongs, then the specific content of those rights and wrongs would not matter. However, they did matter significantly. Consistent with the regulatory focus perspective, those with a strong prevention focus experienced failures of social responsibility (a prevention security concern) as more wrong than those with a weak prevention focus (and those with strong promotion focus), whereas those with a strong promotion focus experienced failures to support others (a promotion nurturance concern) as more wrong than those with a weak promotion focus (and those with a strong prevention focus). Importantly, this effect was independent of whether the wrong was framed as an act of commission or omission.

In another study by Camacho et al. (2003), promotion and prevention had important effects with respect to positive moral evaluations as well. In this case, the difference between approach/avoidance on the one hand and regulatory focus on the other was made even more apparent. This study related participants' judgments of the perceived appropriateness of a conflict resolution *strategy* that was taken by his or her parent or guardian ("We ask that you think back to a time when you had a conflict with your parent or guardian and he/she resolved the conflict by...."). The strategic fit with promotion or prevention was *independent* of whether the strategy involved an attempt by the parent or guardian to increase future good behaviors (approach) or to decrease future bad behaviors (avoidance). The strategy could either be eager/positive (e.g., encouragement to succeed, providing opportunity), eager/negative (e.g., taking away a privilege, acting disappointed), vigilant/positive (e.g., safeguarding against undesired behaviors, alerting to potential dangers), and vigilant/negative (e.g., raising his/her voice, providing criticism).

If it were the case that the omission/commission difference associated with promotion/prevention was actually an effect of approach/avoidance, then those with a stronger promotion focus should see the positive approach resolutions as more appropriate than the negative avoidance resolutions, and those with a stronger prevention focus should see the negative avoidance resolutions as more appropriate than the positive approach resolutions. This pattern was not found. Instead, the researchers found that those with a stronger promotion focus preferred the eager resolutions over the vigilant resolutions regardless of whether the resolutions involved positive approach or negative avoidance. Similarly, those with a stronger prevention focus preferred the vigilant resolution over the eager resolution, regardless of whether the resolution involved positive approach or negative avoidance (Camacho et al., 2003).

Not only does this empirical evidence support the importance of the distinction between promotion and prevention independent of approach/avoidance in the area of ethics, more recent research indicates that recognizing this difference can help us to better understand some of the more well-known conundrums within moral psychology. For example, research on the promotion and prevention systems has shown that those processing in a promotion focus tend to make judgments and decisions based on feelings, while those processing in a prevention system tend to make judgments and decisions based on reasons (Pham and Avnet, 2004; Avnet and Higgins, 2006). The divide between feelings and reasons is analogous to the well-known division within moral psychology between socialintuitionists on the one hand who understand moral judgments and decisions as primarily arising from intuitions or affect (Haidt, 2001), and cognitive-developmental theorists on the other who approach morality as a fundamentally cognitive enterprise (Kohlberg, 1969). An intriguing possibility is that they may both be correct: processing moral goals in a promotion manner may rely on intuitions/affect, while processing moral goals in a prevention manner may rely on cognition/reasons.

So far the evidence has supported this hypothesis. Consider, for example, the infamous "incest" scenario put forward by social-intuitionists as a prototypical scenario evoking a negative intuitive response but a more muted deliberative response (Haidt et al., 1993). In this scenario, a brother and sister sleep together once, but not before taking precautions to avoid any negative consequences of the act (i.e., using two forms of contraception, agreeing to keep it a secret). Recent research has found that those who process this scenario in a promotion focus see the incest as more wrong than those who process it in a prevention focus (Cornwell and Higgins, unpublished manuscript). Importantly, in another study, Cornwell and Higgins (unpublished manuscript) examined the promotion/prevention distinction independent of the approach/avoidance distinction. In this study, participants were experimentally divided into four groups: promotion/approach, promotion/avoidance, prevention/approach, and prevention/avoidance. This study replicated the "stronger moral condemnation for promotion than prevention" effect for the incest scenario while also demonstrating the effect for another scenario (the equally infamous "dog-eating family" scenario, see Haidt et al., 1993) but, importantly, showed no effect of the approach vs. avoidance distinction (Cornwell and Higgins, unpublished manuscript). This research provides another example of how understanding distinct types of avoidance motives—the promotion version and the prevention version—can contribute to a better understanding of how moral judgments can have very different motivational underpinnings.

Another area in which the distinction between the promotion and prevention systems has proved relevant in ethics is in the examination of multiple moral actions across time. Much research has been devoted to understanding how different ways of framing moral goals can influence people to behave in either consistently good ways or to switch from good to bad (e.g., engaging in bad behavior because prior good behavior has "licensed" you to do

so; Sachdeva et al., 2009). For example, in one study, when participants purchased "green" environmentally friendly products, compared to those who purchased conventional products, they later behaved less altruistically and were more likely to cheat and steal on subsequent tasks (Mazar and Zhong, 2010). These latter immoral behaviors (i.e., cheating and stealing) were understood as "licensed" by prior moral behaviors (i.e., the purchase of a "green" product). However, in spite of research showing this effect, there are conditions in which behaving morally can actually lead to moral consistency rather than moral licensing (Conway and Peetz, 2012), and thus many questions surrounding these phenomena remain (Merritt et al., 2010). The promotion/prevention distinction provides some insight into the motivational underpinnings of these effects.

As noted above, research has shown that when people process decisions within the prevention system, they tend to use vigilant means to pursue stability and security, which leads them to favor the *status quo* options over alternatives (Chernev, 2004). One example of the prevention focus leading to a tendency toward preserving the status quo is the research finding that the endowment effect is stronger for individuals with a prevention focus than a promotion focus (Liberman et al., 1999). Another example is the propensity of prevention-focused individuals (but not promotion-focused individuals) to repeat in the present ways of doing things that they experienced in the past. Studies have shown that prevention-focused managers manage others with the style that they received when they were managed in the past, and that they preserve this status quo even when they disliked receiving this style of management (Zhang et al., 2011). The extension to moral psychology would predict that, compared to those who are more promotion-focused and those with a weak prevention focus, those with a strong prevention focus should be more likely to behave in a consistent manner, regardless of whether their original behavior was moral or immoral.

This prediction was supported in recent research. Participants with a strong prevention focus (whether measured chronically or experimentally induced) were more likely to repeat past good behaviors or repeat past bad behaviors compared to those who were either low in chronic prevention or induced into a promotion focus (Zhang et al., 2014). Importantly for the purposes of this paper, these effects occurred whether or not the initial moral behavior involved doing something bad, that is, an *immoral* act that ought to be *avoided*, such as cheating, or involved failing to do something good, that is, a *moral* act that ought to be *approached*, such as pledging money to a charity. This research thus provides a further demonstration of the importance of the promotion/prevention distinction within approach and within avoidance in the domain of ethics.

In sum, we see that the promotion/prevention distinction provides us with a more comprehensive view of the "why" of approach and avoidance. That is, when considering approach and avoidance as "why" we are motivated to do things, it is necessary to go beyond this simple distinction to recognize that there are two fundamentally different value systems within approach and within avoidance. Approaching desired end-states and avoiding undesired end-states in a promotion focus is fundamentally

different from approaching desired end-states and avoiding undesired end-states in a prevention focus. Therefore, to understand how the approach value system works it is necessary to know how it works in promotion *and* how it works in prevention. Similarly, to understand how the avoidance value system works it is necessary to know how it works in promotion *and* how it works in prevention.

But what of the other two questions we seek to answer: the "what" and the "how" of approach/avoidance motivation? In the next section, we provide an overview of the importance of these two questions for understanding approach and avoidance behaviors and provide evidence that they are as important to understand as value (promotion and prevention) and cannot be understood in terms of the approach/avoidance distinction alone.

# **TRUTH AND CONTROL MOTIVES: THE "WHAT" AND "HOW" OF APPROACH AND AVOIDANCE**

While the distinction between prevention vs. promotion goals is critical for advancing our understanding of how the approach and avoidance systems work, it is also important to examine what approach/avoidance presupposes. The study of humans and other animals tends to be motivated by the question of why people and animals behave the way they do, and approach and avoidance motives (further distinguished by promotion and prevention goals) address this by providing an answer (value from having desired results). But asking "why" of an action presupposes two additional things: it assumes that the action is bringing something about, and it assumes that the action is motivated by some understanding that the animal has of itself and its relation to its environment. The "bringing something about" (or managing to have an effect) constitutes the "how" of the behavior—*control motivation*. The understanding (or establishing what's real) constitutes the "what" of behavior *truth motivation*. Both control and truth need to be successful in order for the "why" to be successful. That is, control and truth need to *work together* with value for effective goal pursuit (Higgins, 2012). As such, it is not enough to consider approach/avoidance value alone, even when the promotion/prevention distinction is included in this consideration. In order to demonstrate the fundamental nature and independence of truth and control in goal pursuit, we will once again examine them in the context of the same two domains motivation in non-human animals and moral motivation in humans.

#### **TRUTH AND CONTROL IN NON-HUMAN ANIMALS**

Non-human animals do indeed approach valued end-states and avoid aversive end-states, but in order to effectively do so, they need to *learn* the contingencies of those end-states and actively *adapt* to their environment so that those ends might be achieved. That is, animals need to understand the contingencies and characteristics of their environments (truth) and take an active part in managing that environment (control) in order to bring about valued outcomes like security (prevention) and growth (promotion).

First, the "what" question is aimed at discovering the relevant ways in which the world works. Answering this question requires exploration and learning, both of which have been observed in

non-human animals, even when no valued outcome is present or attainable from them (Franks and Higgins, 2012). One of the earliest researchers to document observations of this tendency in non-human animals was Harlow (1950) who presented a group of rhesus macaques with a complex mechanical puzzle. The monkeys engaged with the puzzle in order to understand how it worked, in spite of the fact that they received no reward for doing so—in fact, adding a food reward to the task tended to disrupt learning rather than facilitate it (Harlow et al., 1950). This study suggested that the "what" question is inherently motivating independent of "why" valued outcomes. More recently, animal welfare scientists have shown that animals are motivated to explore and learn aspects of the "what" question and truth motivation in general. For example, rats will give up known reward and incur risk in order to explore novel environments (Franks et al., 2013a) and goats will interact with a learning device in order to obtain sips of water, even when water is available through less arduous means (Langbein et al., 2009).

Furthermore, we can see how this "what" question is essential to the example involving individual differences in cottontop tamarins' promotion and prevention focus described earlier (Franks et al., 2013b). As mentioned above, different animals were motivated to approach different stimuli in their environment with different response latencies depending on whether they were more motivated to approach security or approach gains. However, the example also involved the ability to *discriminate* between "gain" and "non-gain" stimuli, presupposing the motivation to learn that distinction. The monkeys were able to classify the stimuli as belonging to a particular selfregulatory category in order for them to respond to it in accordance with their regulatory focus orientation. Put another way, their ability to assess the "what" of the stimuli enabled their action in accordance with their preferred "why" (i.e., valued outcomes).

Second, animals also need to answer the "how" question—how to control or manage the situation in order to actually achieve the valued promotion or prevention, approach or avoidance goal. Animal welfare science has been instrumental in showing that the control or management activities themselves, such as the act of building a nest or pushing a lever to obtain food, are motivating independent of the actual result achieved. For example,Carder and Berkowitz (1970) describe the case of rats who could effortlessly attain food from a free food dish in front of them, but instead push the food dish out of the way in order to press a lever to make a pellet of the samefoodfall into thefood tray (for a general review of "contrafreeloading," see Osborne, 1977). In other words, beyond the desire to simply have good outcomes and the absence of bad outcomes, animals are also motivated to take an active part in bringing about these valuable results, even when doing so unnecessarily expends energy or involves risks (Franks and Higgins, 2012).

This control motivation is also apparent in the example involving the promotion-focused and prevention-focused rats cited in the previous section, particularly in the case of the preventionfocused rats (Franks et al., 2012). As mentioned above, some of the rats in the study were prevention-focused in the sense that they focused their energy on safety through maintaining darkness. When an aversive stimulus was placed within their environment, however, these same prevention rats were also the most approachoriented. In contrast to the interpretation based on the theoretical construct of approach/avoidance, in which animals are motivated to minimize pain and maximize pleasure, some of the rats in this experiment are actually *intensifying* an avoidable aversive experience (Franks et al., 2012). One way of making sense of this seeming inconsistency is to posit a distinct motivation to act upon the environment independent of immediate pleasurable or aversive experiences. Thus the rats in the experiment not only established the "what" of the stimulus (i.e., classify it as a threat), they were also motivated to act upon the environment, to take control of the situation, despite immediate costs. Put another way, their willingness to "pay" for managing their environment suggests the worth they placed on control, on the "how" of approach/avoidance.

The desire to achieve this "how" or control motivation has been observed in a number of instances involving non-human animals (for a more extensive review, see Franks and Higgins, 2012). For example, monkeys have been found to forgo the opportunity to receive their favorite treat in order to be able to choose from a variety of foods, many of which they dislike (Addessi et al., 2010). This trade-off suggests that having greater choice, and thus greater opportunity to manage the environment, is more motivating to the monkeys than a favorite outcome. Similarly, animal welfare scientists have long recognized that animals often prefer to engage in the activities that lead to desirable goals over simply receiving the goal without having the opportunity to manage the means by which it is achieved (Fraser and Nicol, 2011).

Interestingly, the motivation to actively manage one's environment has recently been posited as an important individual difference among chimpanzees. Analyses revealed that those animals who engage in the most task-switching—those animals who make the most active changes in managing the environment—are also the same individuals who engage in the most reconciliatory behavior following conflicts, a behavior that involves a great deal of monitoring and control (Webb et al., 2014). Thus the "how" motivation (control) is not only an integral and basic part of approach/avoidance goal pursuit, it is an independent motivation deserving of study in its own right.

These examples show that approaching desired end-states and avoiding undesired end-states presuppose that the individual *understands* its environment (truth) and is motivated to engage in the actions that *bring about* the goal (control). To provide evidence of their distinct importance, we have highlighted cases where these truth and control motivations can be observed independent of value motivations, but we appreciate that these cases are atypical. More often, truth and control motivations integrate with approach and avoidance motivations to create an effective whole—a topic we will discuss in more detail below. But, notably, in these typical cases it is still not approach and avoidance goals working alone. Rather, they are working together with, and depend on, the truth and control motivations. The presence of these distinct motivations in non-human animals points to their fundamental importance.

#### **TRUTH AND CONTROL IN MORAL PSYCHOLOGY**

In the recent modern era, valued outcomes have been argued to be the most fundamental feature of human motivation, and therefore what systems of ethics should concern themselves with most (e.g., Bentham, 1789/2007; Harris, 2010). However, this emphasis on outcomes appears to be an historical and cultural aberration. For example, Aristotle (2009) writes that happiness is constituted by a life of virtue in his *Nicomachean Ethics*, where contemplation (truth) is of paramount importance. Jesus of Nazareth admonishes us to put the "Kingdom of God" and God's "righteousness" prior to the acquisition of worldly possessions in the Gospel of Matthew (Thomas, 1997). The Buddha remarks that the essential qualities for a noble individual to attain are ethics and wisdom, explicitly setting aside things like high birth and wealth (Walshe, 1995). These thinkers and religious leaders tend to argue that ethics is bound up with motives *other than value*.

The Buddha's exhortation is particularly instructive in terms of which capacities he treats as ultimately noble: wisdom and ethics. "Wisdom" in this and related cases among ancient philosophers and religious thinkers is generally understood as a virtue like prudence, "good sense," or the ability to see and know how to respond to situations *as they really are* (i.e., truth; e.g., Aristotle, 2009). "Ethics," as it was understood by the Buddha and other these ancient thinkers, involves *control over the self* in order that the right actions are carried out effectively in the face of temptations like selfishness, fear, or self-indulgence. The resulting virtues make up the means by which an individual establishes what is real (truth) and manages what happens (control); frequently separated into "intellectual" and "moral" virtues, respectively (e.g., Aquinas, 1981/1274).

Thus, many systems of ethics see the settling on the *correct* courses of action and the *controlled* training of one's desires to be in line with those courses as *more fundamental* to becoming a good person than designing ethical systems to maximize valued outcomes. Questions of "what" and "how" matter critically for questions of morality. But this theoretical foundation does not exist only within philosophy and religion. By relating research on motivation to research in moral psychology, we can find empirical evidence for the conclusion that these two motives—truth and control—are fundamental to ethics.

One of the earliest questions psychologists asked about morality is how children come to understand what is right and what is wrong (Piaget, 1932/2008). It is not enough, for example, for pain to be *aversive*; children need to learn that pain is *bad or wrong*, and that causing it (to others and to the self) without good reason is *immoral*. In other words, children need to learn that actions and consequences can be right or wrong, good or bad, independent of mere subjective experience. Moral beliefs have an "objectivity" above subjective judgments like taste (Goodwin and Darley, 2008). How is this objectivity achieved?

One psychological mechanism that may shed light on the question of how moral understanding moves from subjective to objective is shared reality. According to the theory of shared reality (Hardin and Higgins, 1996; Echterhoff et al., 2009), human beings achieve a sense of objectivity from their subjective states when they perceive them as verified or shared by a trusted other. With respect to morality, human children observe the reactions of their parents toward particular behaviors, and toward statements about ethical truths, and then emulate within themselves (i.e., share) what they infer to be the perceived inner states (e.g., feelings, beliefs, goals) that underlie those reactions. In a sense, this formulation is an extension of observational learning (Bandura, 1977), but with shared reality it is the inner states that are imitated rather than just the observable behaviors. This is much like the "metamotivational" self-regulatory factor in achievement motivation, where individuals not only adopt the cognitions appropriate for achievement, but the goals (in our parlance, the "right" value motives) as well (Boekaerts, 1997).

Consider the following research with undergraduate participants for an example of how this process plays out regarding moral beliefs. In a recent study based on Asch's (1956) classic experiment on the effects of social influence on perception, participants adjusted their moral judgments to be in accord with those around them. In the study, participants were presented with ten moral and amoral behaviors (e.g., murder; telling a friend she doesn't look fat even though it's a lie) and asked to declare them either "morally acceptable" or "morally unacceptable."Together with the participants were four other"participants" who were actually confederates. These confederates were given directions to provide the "right" judgment on some trials and the "wrong" judgment on the "critical" trials that tested whether the participants would be influenced by the confederates to give the "wrong" judgment (i.e., moral judgments opposite to those determined to be the nearly unanimous—at least 97% agreement—private judgments in a pilot study; see Jago et al., 2014). On the "critical" trials in the main study, participants were significantly influenced to agree with the "wrong" judgment that was made by the confederates compared to responses in non-critical trials.

Importantly, these effects appear to be related to the epistemic characteristics of the behaviors in question. For example, one of the behaviors was"murder," and participants rated the judgment of that behavior as being relatively more "obvious" as being "morally unacceptable" than some of the other scenarios. A good example of a less obvious case of being "morally unacceptable" was one in which a friend tells another that she does not look fat in a pair of jeans even though she does. This scenario is morally ambiguous because, on the one hand, participants could argue that people should value honesty above anything else, and tell the truth even when it hurts a friend's feelings, but, on the other hand, people are often expected to set the truth aside in order to preserve the feelings of their friends. Thus, it is not surprising that the murder scenario was rated as more morally "obvious" than the lying scenario.

Interestingly for our purposes, if participants are, in fact, motivated to come to the *correct* judgment of the behaviors (truth), they should be more influenced by the unanimous majority opinion when those behaviors are less "obvious" because of greater uncertainty about what is the truth. Consistent with this prediction, the likelihood of providing judgments that agreed with the unanimous majority on "critical" trials significantly increased as the rated "obviousness" of the moral behaviors decreased. That is, it was when behaviors were more morally ambiguous or unclear that participants were more willing to adopt the views of the unanimous majority. They did not simply"go along to get along"or they would have been influenced on all of the "critical" trials equally regardless of how "obvious" the case was. Finally, there was also evidence that when the participants subsequently made private judgments they

not only maintained their group-influenced judgments, but also provided rational justifications for their judgments, which is consistent with their continuing in private to believe that they adopted as the truth. These findings support the conclusion that there is an independent motivation to arrive at the moral truth and shared reality is one way to achieve this.

Although establishing what is real is essential in goal pursuit, it is not enough by itself. Individuals are also motivated to act to take control—in accordance with their moral beliefs and, in doing so, determine the "how" of approaching moral rights and avoiding moral wrongs. The classic traditions cited earlier, as well as many contemporary moral psychologists, see moral motivation as providing a push or a pull to go beyond basic self-interest, beyond immediate pleasure or pain (Mansbridge, 1990). Through this lens of morality, the motivation to fulfill self-interest needs to be controlled in order to accomplish the actualization of moral behavior. For example, to be courageous in a threatening situation, an individual must control the self-interested desire to avoid pain. To be generous, an individual may need to control the selfinterested desire to approach a personal pleasure and instead give her money away. In fact, research has shown that when one does not perceive oneself as responsible for (i.e., having control over) one's own behavior, immoral behavior becomes more likely (Vohs and Schooler, 2008).

The non-human example of the rats responding to a noxious stimulus above contains a parallel—enduring an aversive stimulus in the short term in order to approach security. Human beings accomplish this motivational constraint on a grander scale through a sense of becoming (Higgins, 2005). Human beings are capable of seeing their singular actions or inactions, their approaches and avoidances, as instantiations of a larger long-term project of either moral maintenance or moral growth. In this way, human beings are able to actively act or inhibit their behavior in order to close the gap or maintain the concordance between their actual selves and their ideal- or ought-selves (Higgins, 2005). It is in this way that "what" you are approaching in morality or "why" you are approaching it are only part of the question—"how" you approach it matters critically as well. Morality needs to be pursued across time—sometimes a long time—and matching that controlled achievement to particular goals is a key to successful"ethical becoming." A common maxim for this everyday phenomenon is, "Life is a journey, not a destination."

Evidence for control motivation is evident across psychology, from self-determination theory (Deci, 1980; Deci and Ryan, 1985) to self-efficacy theory (Bandura, 1982, 1997). With respect to moral psychology, let's consider an example of the importance of "how" you do something in a study on charitable giving. Participants were provided with a prompt to go about their decision making in a particular way: either in a way that sees charitable giving as being related to fulfilling an ought *duty*, a goal that must be vigilantly maintained across time; or as being related to an ideal aspiration of *virtue*, a goal that must be eagerly attained across time. For each of these ways of giving to charity, when the way "fit" participants' particular regulatory focus—the ought duty way fitting with prevention; the ideal aspiration way fitting with promotion—there were significantly higher levels of giving among participants (Cornwell et al., unpublished manuscript).

This increase occurred because when the manner of actually going about the decision-making process (the "how") fit with the particular goals of the individual (the "why"), which made the prospect of giving charity more motivating, in accordance with regulatory fit theory (Higgins, 2000).

Emotional experiences also provide feedback on how well one is managing one's relation to promotion and prevention goals across multiple situations (Higgins, 2001)—not just "how do I achieve my goal?" but "how am I doing?" This experience also provides a basis for control. For example, if someone fails to maintain a basic moral standard regarding a duty or responsibility, he or she may feel "guilty," which can increase motivation to try harder to maintain those standards. Similarly, if someone succeeds at fulfilling an ideal moral standard regarding an aspiration, he or she may feel "virtuous,"which can increase motivation to continue to strive for more excellence in the future. The former emotion, being a negative agitation-related emotion, is a prevention focus failure emotion. The latter emotion, being a positive cheerfulnessrelated emotion, is a promotion focus success emotion (Higgins, 2001). Each of them provides specific feedback that affects future control motivation.

Importantly, the emotions are specifically relevant to the regulatory focus in question, and track the emotional experiences that provide for the most control. Feelings of failure are a "fit" for the prevention focus because they strengthen the vigilance that sustains prevention, leading to greater engagement. In contrast, feelings of success are a "fit" for the promotion focus because they strengthen the eagerness that sustains promotion, leading to greater engagement (Higgins, 2006). If moral emotions are relevant to the "how" of approach/avoidance, i.e., are relevant to control motivation, then we should see that moral and immoral behavior results in those emotions that fit a person's self-regulatory goals (promotion or prevention) because fit strengthens the engagement that contributes to more effective control. If emotions are not related to control, but instead simply represent positive and negative feedback for good and bad behavior—as a purely hedonic perspective might predict—then the type of emotional feedback should be unrelated to regulatory focus predominance.

These alternative perspectives were tested in another study on charitable giving conducted by Cornwell et al. (unpublished manuscript). Participants were again given the opportunity to donate some of their participant earnings to charity. After their decision, they were asked to report on their internal emotional experiences. Those who were predominantly promotion-focused (vs. prevention-focused) reported differences in how virtuous they felt (low virtue if they didn't give; high virtue if they did). In contrast, those who were more predominantly prevention-focused (vs. promotion-focused) reported differences in how guilty they felt (high guilt if they didn't give; low guilt if they did). The positive or negative emotional feedback (experienced differently depending on which type of regulatory focus goal the participant was pursuing) is related to the ongoing ethical project to motivate future action. By experiencing emotional control feedback that matches the "why" of the moral goals, individuals are more engaged and, thus, more able to engage effectively in control over themselves for the sake of their moral standards.

An interesting aspect of these studies of charitable giving is that they not only highlight the importance of the "how" for ethical motivation, but also suggest that the "how" and the "why" can *work together* (i.e., fit) to achieve the most ethical behavior, over and above the simple additive effect of each motivational element in isolation (Higgins, 2006). According to our model, if any of the fundamental aspects of motivation is lacking—the "what," "how," or "why"—the effectiveness of an individual's activity, whether it be an approach or avoidance activity, will be considerably diminished.

# **EFFECTIVENESS OF MOTIVE ORGANIZATION**

Each of these kinds of motivations (value, truth, and control) can be regarded as conceptually independent of one another providing additional motivational grounding for approach and avoidance motivations. Importantly, however, they also need to work together in order that approach and avoidance behaviors are pursued effectively (how the different motivations interact is illustrated in **Figure 2**). An implication of this conceptualization—alluded to above—is that this interactive process underlying approach and avoidance behavior and motivation can occur more or less effectively in a particular individual.

To understand the principles behind this effectiveness of motive organization, it may be useful here to lay out how it develops in humans. To reveal the central importance of organization, examining the independent development of truth, control, and value motivation in isolation is insufficient. Instead, a holistic, integrative perspective is required to reveal how truth, control, and value follow a developmental progression that results in their working together effectively. In the next section, we examine how this development occurs in human psychology to further highlight how motives of truth, control, and value are central to the understanding of approach and avoidance (see also Higgins, 2012).

(control), and ended up with the desired results (value). Goals are then reevaluated in light of this feedback, and the process begins again. When all of the motives fit with one another, the activity produced will "feel right."

## **DEVELOPMENT OF MOTIVES**

Many developmental psychologists who have worked to integrate Piagetian developmental models with new research in cognitive psychology continue to view the transition from infancy into childhood as involving different stages of development. A review of these stages reveals that infants and young children tend to emphasize truth and control effectiveness at least as much as valued outcomes. Moreover, examining these stages in light of similar developmental work by Erikson (1980) and Freud (1927/2011) further argue against a simple division of motivation into approach and avoidance.

According to Erikson (1980), human beings proceed through different phases of "conflict" during which they achieve particular capacities with which to engage with the world in an effective way. The earliest phase, occurring between birth and 2 years of age, is the conflict between trust and mistrust (the oral-sensory period), in which children attain the ability to distinguish between reality (i.e., truth) and fantasy with the help of their caregivers. This stage is primarily related to the desire to understand how the world works, and is achieved through understanding how other human beings in their environment understand the world by sharing in their sense of reality (Erikson, 1980). Children have some basic outcome needs at this stage (e.g., warmth and nourishment), but these needs are not the same as desires for *particular* outcome goals, which develop much later. In many ways, the fulfillment of these basic needs via caregivers serves not just to achieve the outcome, but also to achieve relational motives, which are in many ways connected to epistemic motives, particularly with respect to understanding reality (Hardin and Higgins, 1996). This overall view is consistent with Piagetian and neo-Piagetian perspectives in which the first developmental stage is the sensorimotor stage, when the child learns about its truth and control relationship with the world (Piaget, 1983).

The next phase of development for Erikson (1980) occurs around two to 4 years of age, and is understood as the conflict between autonomy vs. shame and doubt (the muscular-anal period). This stage is the period in which children learn how to control their own behavior and manipulate themselves relative to the world (control). It is also the stage in which children are able to mentally represent objects in their minds not immediately present to their senses (Erikson, 1980). These changes offer children the capacity to reflect on how their actions influence the world and make it change based upon their actions. They also begin to think about how the world might be or might become, not simply how it is. Thus children are motivated in this stage to effectively develop the ability to control aspects of the world within their range of influence.

The third phase of development according to Erikson (1980) is represented by the conflict between initiative and guilt (locomotor-genital), which occurs around the ages of four and five. It is during this stage that children begin to integrate their desires and needs with their understanding of the world and their abilities to act upon it. In other words, during this stage, children begin to have mental representations of goals associated with significant others' viewpoints on them (Erikson, 1980; Higgins, 1989, 1991). It is also worth noting that during this latter portion of the pre-operational stage, Piaget (1983) noted the development of

what he called intuitive thought—the phase during which children begin to ask "why?" During this phase of development (roughly equivalent to Freud's Oedipal stage), children also become capable of inferring the inner states of others, including what others want the child to become (ideals and oughts). This progress is an essential development in children's control capacity because children can now take into account others feelings about them, beliefs about them, and goals for them when making choices about what to do or not do (Higgins, 1989, 1991).

It is following this final stage that the effectiveness in the three motivational domains can finally be integrated into a full-fledged system of goal pursuit. At this age, children are finally able to achieve a structural integration of the "what," "how," and "why" of behavior. Importantly, this classic developmental work indicates that the motive for truth and control are developmentally foundational for a full-fledged approach/avoidance goal system. It also suggests that those individuals who are most effective at integrating these value, truth, and control motivations to form an integrated whole would experience the most effective approach and avoidance. We now review some recent research in moral psychology supporting this view and note the theoretical possibilities of extending this research to animal welfare science.

#### **ORGANIZATION OF MOTIVES IN MORAL PSYCHOLOGY**

As argued above, the "what," "how," and "why" questions of approach and avoidance are of fundamental importance in considering questions of right and wrong, good and bad. The preceding developmental account suggests that the most moral human beings would be those that have effectively organized these motivations to ethical ends—those for whom the questions of "what,""how," and "why" all flow *together* into ethical activity. The opinions and reasoning of philosophical and religious thinkers converge on this matter. For them, human beings are not simply the sum total of their behaviors, but have a point of unity about which a judgment can be made: a character or soul—in this case, the animating principle of life (in Greek, *psyche*, in Latin, *anima*), rather than a Cartesian "ghost in the machine." Those in ancient and classical traditions tend to divide the human soul into three parts: the affective, the volitional, and the rational, typically denoted as the affections, will, and reason, which, in turn, correspond to the motivational constructs of value, control, and truth, respectively. As early as Plato (1992), the best human soul would be one in which the three different aspects worked together for the good of the person (Crombie, 1962). For these thinkers, this working together of the soul constitutes the "good life" and has two major implications for moral psychology.

First, it suggests that that those in whom the three forms of motivation work together would also be the most likely to engage in behavior generally deemed to be ethical. Preliminary evidence supports this hypothesis. We have developed a scale to measure the degree of relational integrity among the three motivations, called the "Effectiveness of Motive Organization" scale (EMO; Cornwell et al., unpublished manuscript). This scale correlates not only with higher effectiveness in each of the three kinds of motivation separately (higher measured truth, control, and value effectiveness; Franks, 2012), but also with lower levels of variance among the independent motives. In other words, the people who

score highest on the EMO scale also have the most integrated (i.e., equally high) levels of truth, control, and value effectiveness, suggesting greater mutual support and the absence of a dominant or deficient motivation.

Importantly for the research on ethics, the EMO scale was correlated with Benevolence values over and above other values as measured by the Schwartz Value Inventory (over and above other values such asAchievement or Stimulation; Schwartz,1992), which are theoretically associated with self-transcendence and altruism. Moreover, scores on the EMO scale significantly predicted the likelihood that an individual will engage in charitable giving in the 4 weeks after measurement, the frequency of self-reported altruistic behaviors, and the likelihood of helping in an experimentally created ambiguous situation (Cornwell et al., unpublished manuscript).

The second implication stemming from the link between the "good life" and an effective motive organization is that the most effective means by which to achieve this integration of motivations should occur at the person level rather than the behavioral level that is, at the level of moral character rather than moral behavior. An implication of the earlier point regarding the development and integration of the three forms of motivation is that *all* of the motivations are implicated in every behavior an individual engages in or inhibits. Thus, a person's moral character (i.e., how likely he or she is to engage in moral behavior or inhibit immoral behavior) may be directly related to the *integrity* of his or her motives. This level of analysis most fruitfully occurs at the level of the individual as a whole, rather than the particulars of any given behavior. Notably, the word "integrity" itself refers both to having united (integrated) characteristics *and* a strong moral character.

In the moral psychology literature, there is growing evidence that the construct of moral character is of paramount importance, even though early attempts by trait theorists to measure it empirically were largely regarded as failures (Hartshorne et al., 1930). Researchers have theorized that incorporating judgments of character into our theories of moral judgment would greatly improve their predictive capacity and perhaps help us to understand otherwise puzzling judgments and behavior (Pizarro and Tannenbaum, 2011). Empirical work on the subject has also shown that when making judgments, individuals often judge whether a behavior is *the sort of thing a good person would do* rather than simply judging it according to its negative consequences or conformity to universal rules or norms (Inbar et al., 2011). Furthermore, recent work has shown that judgments of moral character actually predominate over other important dimensions of social judgment (Goodwin et al., 2014). Research has also demonstrated the importance of virtues and character strengths in understanding behavior and success (Peterson and Seligman, 2004). Finally, research has shown that encouraging moral behavior among young children is most effective when their character is commended for performing a particular altruistic behavior as opposed to rewarding them (Grusec and Redler, 1980).

These last results are of particular interest given the earlier developmental account. They suggest that it is only after the development of each of the independent motivations (truth, control, and value) and their integration into a relational whole can a

truly "moral" human being come about. In light of this formulation, it is interesting to note that Freud (1927/2011) himself argued that the developmental stage immediately following the locomotor-genital stage (after which each of the three motives is present) is that in which conscience develops. The empirical research cited above on the advantage of praising moral character (vs. rewarding moral behavior) shows that praise during an earlier stage of development is ineffective. The study found that among 5-year-olds, subsequent behavior did not differ as a function of whether their character was praised for their altruistic behavior or they were rewarded for it. However, among 8-yearolds, praising the character of children *did* increase subsequent altruism, whereas reward did not. This difference in the efficacy of character praise vs. reward is consistent with the view that moral character presupposes the ability to organize these motivations in an effective way—children did not respond to praise until age 8, which is after the proposed developmental account above is complete (Grusec and Redler, 1980).

Thus we see how the three kinds of fundamental motivations and their effective organization are critically important for investigating the domain of ethics. Given the centrality of this domain to the lives of so many people, its fundamental nature is apparent. Yet it remains to be seen whether the organization of these different motives is important for non-human animals as well. In the final section, we discuss some of the research suggesting that this question should be answered in the affirmative.

#### **ORGANIZATION OF MOTIVES IN NON-HUMAN ANIMALS**

Though there is a strong philosophical foundation for linking the three forms of motivations and their effective organization to the discipline of ethics, the relation to non-human animals is not as obvious. Non-human animals may have certain characteristics that cause them to behave in ways that we might understand as a kind of precursor to the comprehensive ethical systems found in humans (e.g., Flack and De Waal, 2000). Nevertheless, they may not have certain fundamentally human capacities of consciousness nor take into account the inner states of others the way that humans do (Higgins, 2005), and thus they would not organize themselves according to morality in the same manner. However, by adopting the idea of the "soul" as an integrative animating principle rather than a "rational" ghost in the machine, the same principle could be applied to non-human animals. Indeed, many pre-Enlightenment thinkers, though acknowledging the different capacities of human beings relative to other animal species, nevertheless attribute souls to non-human animals (e.g., Aristotle, 1986). Thus, if the three motivations have measurable outcomes in humans in the domain of ethics, there may be analogous outcomes for an effective motive organization in non-human animals.

The area in which this concept may be of particular interest is in the field of animal welfare science. In addition to moral values and behaviors, the EMO scale is also highly correlated to various measures of well-being (Cornwell et al., unpublished manuscript), such as life satisfaction (Diener et al., 1985) and the perception of one's life as meaningful (Baumeister et al., 2013). These two relations closely mirror "pleasure" and "meaning"in the pleasure-meaning-engagement triad of human happiness outlined by Peterson et al. (2005; see also Haidt, 2006), and there is other evidence that the EMO scale is related to engagement as well. Thus, the "good life" may be "good" in two senses: "good" as being morally good *and* "good" as being well. Since the latter form of "good"is something that human and non-human animals share, we predict that those animals with the most effective motives organization are the ones that have the best welfare. Indeed, this hypothesis was already implicated in our theoretical exploration of each of these motivations in the context of non-human animals.

While no research to date has directly tackled the question of how motive organization relates to welfare, there are several lines of evidence pointing to the utility of this framework (Franks and Higgins, 2012). For this reason, we believe it could be productive to pursue it as a model for future welfare research. For example, the motivation for food—a valuable outcome—is certainly a hallmark of good welfare: loss of appetite is a strong indicator of illness and poor welfare. Nevertheless, an unchecked motivation for food can also be a sign of poor welfare (D'Eath et al., 2009). An individual who is so preoccupied with food (unrestrained value motivation) that it loses interest in changes to its surroundings (diminished truth motivation) or loses the motivation to engage in species-typical activities (diminished control motivation) has a poor organization of motives and is likely to suffer from poor welfare. Similarly, individual animals subjected to learned helplessness experiment conditions learn that they have no control over the outcomes in their life (Maier and Seligman, 1976), which reflects a disorganization of motives that coincides with poor welfare. Thus, we see preliminary evidence that, as in the domain of moral psychology, the relative effectiveness of the three fundamental forms of motivation could potentially be of critical importance to research among non-human animals as well. As we believe that developing these ideas and testing them across species is an important line of inquiry, we hope to see more research in animal welfare science examining the utility of this framework.

### **FINAL COMMENTS**

Throughout this paper we have argued that the study of approach and avoidance motivation may benefit from the incorporation of additional perspectives on what other motivations it needs to work with (truth and control motivation) and how, as a value motivation, it can be further differentiated (promotion approach/avoidance vs. prevention approach/avoidance). We have discussed three independent motivations that are presupposed by approach/avoidance: namely truth (the "what" of approach/avoidance), control (the "how" of approach/avoidance), and value (the"why"of approach/avoidance), and we have emphasized the importance of considering how they *work together*. We have provided evidence for our perspective by examining these motivational constructs in human moral psychology and non-human animal welfare science. In so doing, we have noted phenomena with which a purely hedonic approach cannot grapple. We concluded by noting that the *organization* of these three kinds of motives may be of central importance to the larger story of approach/avoidance.

For both moral psychology and animal welfare science, attention to the integrity of motives would involve regarding the individual as a whole at the appropriate level of analysis to strive for a complete understanding of how integrity relates to different kinds of effectiveness (moral character for humans and wellbeing for all animals). This framework is a reconceptualization of motivation that goes beyond the hedonic principle to extend our ability to address the full complexity of human and non-human behavior.

The hedonic principle that animals approach pleasure and avoid pain has provided scientists with substantial explanatory and predictive power. In recent years, however, some of its limitations as a "one size fits all" distinction have become apparent. In this paper, we have discussed the ways that new developments in motivation science have contributed to the growth of the fields of moral psychology and animal welfare science, and the contributions extend beyond these two alone. Understanding the nature of motivation is essential for understanding humans and other animals, and it is critical that this understanding be equipped to answer the fundamental questions of "what," "how," and "why" when it comes to approach and avoidance.

## **REFERENCES**


Freud, S. (1950). *Beyond the Pleasure Principle.* New York, NY: Liverlight. (Original work published in 1920.)


Osborne, S. R. (1977). The free food (contrafreeloading) phenomenon: a review and analysis. *Anim. Learn. Behav.* 5, 221–235. doi: 10.3758/BF03209232


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 20 June 2014; accepted: 22 September 2014; published online: 14 October 2014.*

*Citation: Cornwell JFM, Franks B and Higgins ET (2014) Truth, control, and value motivations: the "what," "how," and "why" of approach and avoidance. Front. Syst. Neurosci. 8:194. doi: 10.3389/fnsys.2014.00194*

*This article was submitted to the journal Frontiers in Systems Neuroscience.*

*Copyright © 2014 Cornwell, Franks and Higgins. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited andthatthe original publication inthis journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Individual differences in response to positive and negative stimuli: endocannabinoid-based insight on approach and avoidance behaviors

# **Daniela Laricchiuta1,2\* and Laura Petrosini 1,3**

1 IRCCS Fondazione Santa Lucia, Rome, Italy

<sup>2</sup> Department of Dynamic and Clinical Psychology, Faculty of Medicine and Psychology, University "Sapienza" of Rome, Rome, Italy

<sup>3</sup> Department of Psychology, Faculty of Medicine and Psychology, University "Sapienza" of Rome, Rome, Italy

#### **Edited by:**

Mikhail Lebedev, Duke University, USA

#### **Reviewed by:**

Takefumi Kikusui, Azabu University, Japan Jony Sheynin, University of Michigan, USA Clare Marie Mathes, Baldwin Wallace University, USA Mikhail Kalinichev, H. Lundbeck A/S, Denmark

#### **\*Correspondence:**

Daniela Laricchiuta, IRCCS Fondazione Santa Lucia, via del Fosso di Fiorano 64, 00143 Rome, Italy e-mail: daniela.laricchiuta@ uniroma1.it

Approach and avoidance behaviors—the primary responses to the environmental stimuli of danger, novelty and reward—are associated with the brain structures that mediate cognitive functionality, reward sensitivity and emotional expression. Individual differences in approach and avoidance behaviors are modulated by the functioning of amygdaloidhypothalamic-striatal and striatal-cerebellar networks implicated in action and reaction to salient stimuli. The nodes of these networks are strongly interconnected and by acting on them the endocannabinoid and dopaminergic systems increase the intensity of appetitive or defensive motivation. This review analyzes the approach and avoidance behaviors in humans and rodents, addresses neurobiological and neurochemical aspects of these behaviors, and proposes a possible synaptic plasticity mechanism, related to endocannabinoid-dependent long-term potentiation (LTP) and depression that allows responding to salient positive and negative stimuli.

**Keywords: personality traits, endocannabinoid system, dopaminergic system, reward system, fear system, neuroimaging**

## **INTRODUCTION**

Many different labels have been proposed over the years to cover the definition of approach and avoidance. An Approach-Withdrawal distinction was introduced by Schneirla (1965) that argued that in all organisms the motivation is grounded in overt behavioral actions toward or away from stimuli. Subsequently, Davidson (1992) re-utilizing such a distinction presumed that action tendencies are grounded in differently lateralized cortical activation. In their analysis of emotion, Lang et al. (1997) used an Appetite-Aversion distinction to characterize two brain systems that underlie emotions: Appetite connotes consummatory and approach-oriented tendency, whereas Aversion connotes defensive and avoidance-oriented tendency. On the other hand, Lewin (1935), Miller (1944), and McClelland et al. (1953) conceptualized an Approach-Avoidance distinction in terms of valence-based processes, rather than over behavior. More recently, Elliot and Church (1997), Elliot and Thrash (2002), Elliot (2006), and Elliot (2008) addressed the issue, proffering the Approach-Avoidance distinction that expands the previous Approach-Withdrawal distinction in terms of energization of the behavior by (motivation), or direction of the action toward (behavior), positive stimuli in the case of the approach, and in parallel, energization of the behavior by, or direction of the action away from, negative stimuli in the case of the avoidance. Thus, positive or negative valence of the stimulus is considered the core of Approach-Avoidance distinction. The approach and avoidance behaviors appear to be the primary reactions to novel, rewarding, and dangerous stimuli on which all successive responses are based in order to gain successful adaptation. The approach system is considered a motivational system that activates reward-seeking behavior associated with impulsivity/exploration, whereas the avoidance system is considered an attentional system that promotes appetitive response inhibition or active overt withdrawal (McNaughton and Gray, 2000; Pickering and Gray, 2001; Carver and Miller, 2006).

The approach and avoidance behaviors are biologically based and constitutionally ingrained, since all organisms, following a phylogenetic gradient, are "preprogrammed" to approach or avoid particular classes of stimuli (Elliot, 1999, 2005, 2008; Elliot et al., 2006). The phylogenetically early mechanisms engender low-level responses to concrete stimuli, and complex mechanisms mediate sophisticated responses to a broader range of stimuli (Elliot et al., 2006). Approach and avoidance behaviors have been described not only across but also within phyla. Within the same species, some individuals have a greater tendency to approach or avoid a stimulus, also in relation to the age and context. For example, both in humans and animals, very young individuals are more sensitive than adults to the experiences linked to approach and avoidance, as early socialization or desensitization (Rothbart and Bates, 1998; Jones and Gosling, 2008; Sullivan et al., 2008). The adolescents exhibit emotional lability, impulsivity and proclivity to seek rewards and novel sensations (Fairbanks, 2001; Spear, 2002; Adriani and Laviola, 2004; Hefner and Holmes, 2007; Good and Radcliffe, 2011), even if sometimes these tendencies are maintained in adulthood (Roberts et al., 2001; Henderson and Wachs, 2007; Krishnan et al., 2007). However, increased sensitivity to reward is reversed in adolescents who are characterized in early childhood as having a behaviorally inhibited temperament (Helfinstein et al., 2011).

Excessive approach or avoidance behavior can lead to psychopathological disorders, as attention-deficit/hyperactivity disorders, depression and substance abuse on one hand, or anxiety and post-traumatic stress disorders on the other hand (Meyer et al., 1999; Muris et al., 2001; Kasch et al., 2002; Mitchell and Nelson-Gray, 2006). Thus, individual differences in approach and avoidance may represent predictors of vulnerability (or resilience) to neuropsychiatric diseases. Many of these conditions show sex differences in age of onset, risk, prevalence and symptomatology (Lynch et al., 2002; Costello et al., 2003; Rutter et al., 2003; Zahn-Waxler et al., 2008). In adolescence and adulthood, testosterone might increase susceptibility for some neuropsychiatric conditions by tipping the balance between approach and avoidance. For example, testosterone decreases avoidance by attenuating unconscious fear-responses (Hermans et al., 2006, 2007) and reducing sensitivity to punishment (van Honk et al., 2004), as well as it increases approach by enhancing sensation- and reward-seeking behaviors (van Honk et al., 2004; Coates and Herbert, 2008) and motivation to act (Campbell et al., 2010; Bos et al., 2012). The females exhibit a prolonged avoidance duration in a computerbased approach-avoidance task (Sheynin et al., 2014a,b). However, females may have a higher propensity for cocaine-induced approach-avoidance conflict (Back et al., 2005; Zakharova et al., 2009). In particular, the behavioral effects of drug rewarding stimuli vary across the reproductive cycle with specific "at risk" phases in respect to reward seeking. For example, women report higher drug-induced pleasure during the follicular phase than during the luteal phase (Evans et al., 2002), and female rats display greater reward-seeking behavior during estrus compared to other cycle phases (Feltenstein and See, 2007; Kerstetter et al., 2008, 2013).

# **CONCEPTUAL SPACE OF APPROACH AND AVOIDANCE BEHAVIORS**

Motivation is based on an intricate array of active approach and avoidance mechanisms. Functionally, approach and avoidance motivation are viewed as instigators of valenced propensities. They influence immediate affective, cognitive, and behavioral inclinations in response to real or imagined stimuli and orient individuals consistently across domains and situations. In humans, although some actions may derive directly and invariably from these proclivities, the ultimate behavior may be self-regulated and subjected to strategic planning, so that individuals can override their initial inclinations and redirect behavior (e.g., putting an approach behavior into action to override a basic avoidance tendency). The separate systems for approaching incentives and avoiding threats show individual differences and are sustained by disparities in brain structure and function. Personality traits are linked to neurobiological measures, such as neurotransmitter metabolites (Cloninger, 1986, 1987; Limson et al., 1991; Cloninger et al., 1993; Kim et al., 2002), markers that are associated with *in vivo* neuroimaging (Sugiura et al., 2000; Canli et al., 2001; Youn et al., 2002; Kumari et al., 2004), and morphometry (cortical thickness and volumes) in specific brain regions (Yamasue et al., 2008; Gardini et al., 2009; DeYoung et al., 2010; Picerni et al., 2013; Laricchiuta et al., 2014b,c,d). Approach and avoidance are related to and distinct from the central constructs of personality related in turn to the *trait adjective*, *affective disposition*, and *motivational system* constructs (Gable et al., 2003; Quilty and Oakman, 2004).

*Trait adjective* includes extraversion and neuroticism. Extraversion is the tendency to be sociable, active, optimistic, and to have high sensitivity to positive stimuli. Conversely, neuroticism is the tendency to be worrisome, prone, emotionally unstable, insecure, and to have high sensitivity to negative stimuli (Eysenck, 1981; Costa and McCrae, 1992). The specific sensitivity to positive or negative stimuli affects perceiving, attending, thinking, encoding, and recalling such stimuli. Eysenck (1981) proposed that extraversion is linked to a general cortical "arousability" and that neuroticism correlates with a low threshold for activation in the limbic system. In accordance, Eisenberger et al. (2005) suggested that neuroticism is the result of a neural system that detects a mismatch between actual and expected situations—a function that is carried out by the dorsal anterior cingulate cortex. DeYoung et al. (2010) reported that neuroticism covaries positively with the volume of the cingulate gyrus and negatively with the volume of the dorsomedial prefrontal cortex and posterior hippocampus regions that are associated with threat, punishment, and negative affect. Recent results have shown that cerebellar white matter (WM) and gray matter (GM) volumes negatively covary with neurotic personality traits (Schutter et al., 2012). In parallel, extraversion covaries positively with the volume of the medial orbitofrontal cortex, which mediates the processing of rewardrelated information (DeYoung et al., 2010). Further, a positive association between patterns of synchronous neuronal activity and extraversion has been described in the cerebellum (Wei et al., 2011).

*Affective disposition* includes positive and negative emotionality, i.e., the tendency to experience positive or negative emotion and engage life in a positive or negative manner, respectively (Tellegen, 1985; Digman, 1990). Whereas positive emotionality is related to approach motivation and is elicited by appetitive stimuli (hedonic stimuli, reward cues, safety signals), negative emotionality is associated with avoidance motivation and is elicited by aversive stimuli (negative stimuli, threat cues, punishment signals). Individuals with high positive emotionality exhibit high energy, optimism, and openness toward others and the future. They tend to focus on the pleasant characteristics of themselves and others. Individuals with high negative emotionality exhibit high levels of distress, anxiety, irritability, fear, pessimism about the future, and dissatisfaction. They call attention to their own unpleasant characteristics and those of others. Electroencephalographic recordings revealed that positive and negative emotionality is associated with left and right prefrontal cortex activation, respectively (Wheeler et al., 1993). The link between the extraversion/neuroticism and the positive/negative emotionality is often discussed with regard to emotional reactivity. Extraverts and neurotics respond to stimuli with more intense emotions than introverts and non-neurotics. High levels of approach behavior in extraverts often lead to affective benefits. Unlike negative emotionality, which promotes withdrawal behavior, positive emotionality spurs exploratory behavior. The broaden-and-build theory of positive affect by Fredrickson (2001, 2004) suggests that once a positive emotionality is experienced, one seeks to expand and continue the experience that encourages the subject to approach novel situations, ideas, and individuals that are related to the object of interest. The author hypothesizes the development of an *upward spiral* in which positive emotions and the broadened thinking they engender influence one another reciprocally, leading to appreciable increases in emotional well-being over time. Positive emotions may trigger these upward spirals by building resilience and influencing the ways that people cope with adversity. Complementarily, the author hypothesizes a *downward spiral* in which negative emotionality and the narrowed pessimistic thinking it engenders influence one another reciprocally, leading to ever-worsening mood, till depression.

*Motivational system* includes behavioral activation system (BAS) and behavioral inhibition system (BIS). The reinforcement sensitivity theory proposes that the BAS produces positive affect and facilitates approach behaviors in response to conditioned appetitive stimuli, whereas the BIS generates negative affect and facilitates avoidance behaviors in response to conditioned aversive stimuli, especially in novel situations (Gray, 1987; Gray and McNaughton, 2000; McNaughton and Corr, 2004, 2014). Recently, Simon et al. (2010) examined the relation between individual differences in reward sensitivity and neural processing during expectation and reception of a reward, by using functional magnetic resonance imaging (MRI) during a monetary incentive delay task. Subjects with a high BAS exhibited greater activation of the ventral striatum during receipt of the reward, and greater activation of the medial orbitofrontal cortex during receipt and omission of the reward, demonstrating that approaching or avoiding reward-related situations have a distinct relationship with neural processing of the reward. Further, even amygdala responses appear to be positively associated with BAS (Beaver et al., 2008). Resting-state functional MRI demonstrated that BIS correlates negatively with the cerebellum and positively with the frontal gyrus (Kunisato et al., 2011). Increased fetal testosterone (FT) predicted increased BAS by biasing caudate, putamen, and nucleus accumbens to be more responsive to positively compared with negatively valenced information (Lombardo et al., 2012). In contrast, FT was not predictive of BIS, suggesting that testosterone in humans may act as a fetal programing mechanism on the reward system and influence behavioral approach tendencies later in life.

Interestingly, human approach-avoidance behavior has been assessed mainly by self-report questionnaires (e.g., Eysenck, 1981; Costa and McCrae, 1992; Cloninger et al., 1993; Taylor and Sullman, 2009), which query the respondent about the type and frequency of behaviors, and assign a score on each answer. Recently, in a human study on approach and avoidance tendencies the individual differences have been assessed on the Sensitivity to Punishment and Sensitivity to Rewards Questionnaire split into four subscales: Punishment that measures avoidance tendencies related to BIS; Impulsivity/Fun-Seeking, Drive, and Reward Responsivity that measure approach tendencies related in turn to BAS (Lombardo et al., 2012). Furthermore, to more directly evaluate avoidance behaviors, in humans several studies have used mild electric shocks (Lovibond et al., 2008, 2013; Delgado et al., 2009), or aversive visual or auditory stimuli (Dymond et al., 2011) as the aversive events that could be avoided. To evaluate approach behaviors, most human studies have employed monetary incentive tasks allowing the analysis of responses occurring during both expectation and receipt of reward or during the omission of reward (Schlund and Cataldo, 2010; Simon et al., 2010). A number of other studies have used the presentation of primary reinforcers, as somatosensory, olfactory or more often pleasant taste stimuli (O'Doherty et al., 2000, 2002). Another line of human studies has considered computer-based tasks (Molet et al., 2006; Schlund et al., 2010; Sheynin et al., 2014a,b), some of which take the form of a videogame, in the idea that even though no negative (e.g., electric shock) or positive (e.g., pleasant taste or money incentive) stimulus is delivered, people are nonetheless motivated to avoid aversive events and to approach rewarding events within the game. In the same vein, recently in a human study on approach-avoidance conflict a computer game was used in which the collection of monetary tokens provided the approach motivation, while the possibility that a virtual predator might wake up and remove all tokens provided a potential threat, and thus the avoidance motivation (Bach et al., 2014).

# **APPROACH- AND AVOIDANCE-RELATED PERSONALITY TRAITS AND BRAIN STRUCTURAL VARIATIONS**

Within theories of personality, another model directly related to approach and avoidance is that related to the primary basic personality temperament and character traits by Cloninger (Cloninger, 1987; Cloninger et al., 1993). In his temperament and character inventory (TCI), he described four temperamental traits: Novelty Seeking (NS), Harm Avoidance (HA), Reward Dependence (RD), and Persistence (P). Novelty seeking is an approach-related personality trait and refers to the tendency to act. High NS scores reflect a greater tendency toward exploratory activity in response to novelty, impulsive decisionmaking, extravagant approaches to reward cues, and rapid loss of temper. The advantages of high NS are excitability, curiosity, enthusiasm, and quick engagement with anything that is new and unfamiliar. Conversely, its disadvantages are indifference, lack of reflection and intolerance to monotony, anger, inconsistency in relationships, and quick disengagement whenever a wish is frustrated. Harm avoidance is an avoidance-related personality trait and is the tendency to inhibit behaviors, acting with caution and apprehension. High HA scores indicate proclivity to respond intensively to aversive stimuli or signals of punishment or nonrewards, and they lead to pessimistic worry in anticipation of problems, fear of uncertainty, shyness with strangers, and rapid fatigability. The adaptive advantages of high HA are cautiousness and careful planning when a hazard is likely. Its disadvantages arise when a hazard is unlikely but still anticipated which leads to maladaptive inhibition and anxiety. Reward dependence is the inclination to maintain ongoing behaviors that have been associated with reinforcement and to express persistence, social attachment, and dependance on approval by others. High RD scores reflect to be tenderhearted, sensitive, dedicated, dependent, and sociable. The adaptive advantage of high RD is sensitivity to social cues, which facilitates affectionate social relations and genuine care for others. Its disadvantages are related to suggestibility and loss of objectivity, which are frequently encountered with people who are excessively socially dependent. Persistence refers to the ability to maintain arousal and motivation internally in the absence of an immediate external reward. High P scores indicate hard-working, perseverance, ambitiousness, and perception of frustration as a personal challenge. The adaptive advantage of a high P is the use of behavioral strategies when a reward is intermittent but the contingencies remain stable. Its disadvantages are related to perfectionist perseverance when contingencies change rapidly.

Within the factors that contribute to individual differences, gender influences HA (females have higher HA scores than males), and age influences NS (young subjects have higher NS scores than elders) (Cloninger et al., 1993; Fresán et al., 2011; Westlye et al., 2011). Although individuals with depression (Ono et al., 2002), bipolar mania (Loftus et al., 2008), schizophrenia (Fresán et al., 2007), substance use disorders (Conway et al., 2003), pathological gambling (Martinotti et al., 2006), and anxiety disorders (Kashdan and Hofmann, 2008) have NS or HA scores higher than healthy subjects, NS and HA are clearly nondysfunctional behaviors and contribute to adaptive functioning. Further, NS and HA provide mechanisms to expand the range of stimuli and possibilities, protect one from potentially aversive contexts, supply the appropriate feedback for sculpting the brain and develop interest in specific domains. Structural neuroimaging studies on the regional specificity of brain-temperament relationships have demonstrated that the strength of fiber tracts from the hippocampus and amygdala to the striatum predicts the individual differences in NS (Cohen et al., 2009). Further, NS correlates positively with the volume of the frontal and posterior cingulate cortex; HA is negatively associated with the volume of the orbitofrontal, occipital, and parietal areas; RD correlates negatively with the volume of the caudate nucleus and frontal gyrus; P has a positive association with the volume of the precuneus, paracentral lobule, and parahippocampal gyrus (Gardini et al., 2009). Negative relationships between HA and anxiety-related traits and volumes of the entire brain (Knutson et al., 2001) and orbitofrontal (DeYoung et al., 2010) and left anterior prefrontal (Yamasue et al., 2008) cortices have been also reported. In parallel, increased HA is linked to decreased micro-structural integrity in widely distributed fiber tracts that include the corticolimbic pathways (Westlye et al., 2011). Furthermore, subjects with low NS and high HA scores have a relatively low striatal dopaminergic receptor density (Montag et al., 2010).

Assuming that the variability in an approach-related personality trait, such as NS, and an avoidance-related personality trait, such as HA, is normally distributed, in a large cohort of healthy subjects of both sexes and a wide age range (18–67 years), we tested the hypothesis that macro- and micro-structural variations in specific brain areas correlated with scores on the TCI temperamental scales (Picerni et al., 2013; Laricchiuta et al., 2014c,d). Region of interest (ROI)-based and voxel-based morphometry (VBM) analyses were used to assess macro-structural organization, and diffusion tensor imaging (DTI) scan protocol was used to evaluate micro-structural organization (Picerni et al., 2013; Laricchiuta et al., 2014b,c,d). Diffusion tensor imaging measures the diffusion of water molecules through tissues, detects microstructural variations in the brain, and provides physiological information that is not available using conventional MRI (Le Bihan, 2007; Basser and Pierpaoli, 2011). The DTI indices that we used were Mean Diffusivity (MD) for GM and Fractional Anisotropy (FA) for WM, which reflect with great accuracy in space and time the subtle changes in cell structure which accompany various physiological and pathological states. In particular, low values in MD or high values in FA indicate high integrity and efficiency, and advanced organization of brain micro-structure. Variations in water diffusion parameters are linked to variations in cognitive functions (Piras et al., 2010, 2011) and personality dimensions (Westlye et al., 2011; Bjørnebekk et al., 2012, 2013).

We found that increased volumes of the bilateral caudate and pallidum were associated with higher NS scores (**Figure 1A**), and increased MD measures in the bilateral putamen correlated with higher HA scores (Laricchiuta et al., 2014c). Further, greater cerebellar volumes were linked to higher NS scores, and reduced cerebellar volumes were associated with higher HA scores (Laricchiuta et al., 2014d; **Figure 1B**). These associations were observed in the cerebellar WM and cortex of both hemispheres. A greater-than-average volume might reflect greater-than-average power to perform specific functions. Human and animal evidence favors the larger-is-more-powerful position: training on particular tasks or experiencing complex environment increases the volume of functionally related brain structures (Boyke et al., 2008; Pangelinan et al., 2011; Di Paola et al., 2013). Thus, it is reasonable to assume that volume tends to covary positively with function. We also noted positive associations between the volumes of vermian lobules VIIb, VIII, and Crus 2 and NS scores (**Figure 2**; Picerni et al., 2013). The relationship between NS scores and cerebellar structures was also observed at the micro-structural level, as evidenced by the DTI data. The triad including increased volume, decreased MD, increased FA indicates that the macroand micro-structural features of the posterior vermis support approach behaviors.

These novel data that implicate a cerebellar substrate for approach- and avoidance-related personality traits extend the relationship between brain areas and personality to a structure that, until now, was believed to be involved primarily in motor and cognitive functions (Oliveri et al., 2007; Torriero et al., 2007; De Bartolo et al., 2009; Foti et al., 2010; Cutuli et al., 2011; Hampe et al., 2013), much less in emotional processes (Schmahmann and Sherman, 1998; Schmahmann et al., 2007; Timmann and Daum, 2007) and even less in personality individual differences (O'Gorman et al., 2006). Anatomo-clinical analyses indicate that

**FIGURE 2 | Positive association between cerebellar gray matter volumes and NS scores**. Coordinates are in Montreal Neurological

positively associated with Novelty Seeking (NS) scores. **(B)** The volumes of

the cerebellum is a critical neuromodulator of intellect and mood and that the posterior vermis, the so-called limbic cerebellum, chiefly regulates emotion and affect (Schmahmann, 2004; Stoodley and Schmahmann, 2010; Stoodley et al., 2012). Impaired executive and spatial functions, language deficits, and personality changes have been described in subjects with lesions of the posterior lobe and vermis (cerebellar cognitive-affective syndrome) (Schmahmann and Sherman, 1998). MRI studies have shown structural and functional abnormalities in the cerebellum in patients with personality, anxiety, or depression disorders (Pillay et al., 1997; De Bellis and Kuchibhatla, 2006; Fitzgerald et al., 2008; Baldaçara et al., 2011a,b). This evidence implicates the cerebellum in affective processing which affects personality characteristics. Moreover, the psychopathological profiles of patients who are affected by cerebellar diseases describe them as impulsive, obsessive, hyperactive, disinhibited, and developing ruminative and stereotypical behaviors—features that affect their personality style (Schmahmann et al., 2007). Even data in healthy subjects indicate limited capacity for emotional regulation after repetitive inhibitory transcranial magnetic stimulation over the cerebellum (Schutter and van Honk, 2009). The direct reciprocal connections between the cerebellum and basal ganglia (**Figure 3**, dashed black line) (Hoshi et al., 2005; Bostan and Strick, 2010; Bostan et al., 2010) constitute the neuroanatomical basis for the cerebellar influence on reward-related behaviors and motivationrelated information processing—functions that, until now, have been attributed only to the basal ganglia (Wise, 2004; Delgado, 2007; Palmiter, 2008). It is likely that the cerebellum accelerates the "force" with which the reward is experienced (Schmahmann et al., 2007). Cerebellar activity signals when the sensory input differs from memory-driven expectations, provides a sensory prediction error, guides exploratory drive in novel environments, allows a flexible switching among multiple tasks or alternatives, and renders functions faster and more adaptive (Restuccia et al., 2007). The cerebellum performs these functions by refining the rate, rhythm, and force of the behavior and adjusting it for given situations. Essentially, the cerebellum receives information from the cortex and basal ganglia and sends a "corrected" signal back. In particular, based on cerebellar detection of error/novelty, Ito (2008) proposed that in the motor and cognitive domains the cerebellum develops both forward and inverse models. In the forward model, the cerebellum is informed by the cortex and basal ganglia with regard to information load, plans, and intentions about the upcoming behavior and on the characteristics of the

left and right volumes. Linear fits (solid black lines) are reported.

Institute (MNI) space. In figure left is left.

turn projects to the hypothalamus, and directly or indirectly (via orbitofrontal cortex) to the dorsal striatum. These connections are involved in avoidance responses (solid red line). The outputs from the amygdala also reach the ventral striatum and the orbitofrontal cortex, and these connections are involved in approach responses (solid green line). The dorsal striatum receives

environment in which the behavior is manifested. Thus, the cerebellum develops a progressive, short-cut, anticipatory model (Wymbs and Grafton, 2009; Seidler, 2010; van Schouwenburg et al., 2010). As the behavior and cognition are repeated and the anticipatory predicted feedback is received, the cerebellum becomes increasingly accurate in its predictive capacities and allows behavior to become faster, more precise, and independent of cortical control. With successful repetitions, behavior that is governed consciously by the cerebellar forward model becomes increasingly automated and the cerebellar "inverse" model is developed. This permits rapid and skilled behavior to occur at an unconscious level. The cerebellum is constantly constructing multipairs of models that constitute a complex modular architecture for adaptively regulating motor, cognitive, and emotional material. In triggering the new mental activity, the cerebellum could warn the prefrontal cortex about the absence of internal models that match the novel information, maintain the newly generated internal models, and incorporate them into routine schemes of thought. To successfully manage novelty, the cerebellum and neocortical/subcortical areas must neurons. Direct pathway projects to the internal globus pallidus and substantia nigra, whereas indirect pathway projects to the substantia nigra by way of the external globus pallidus and subthalamic nucleus. Also the bidirectional striatal-cerebellar network (dashed black line) is involved in the emotional and motivational processes linked to approach and avoidance.

be co-activated. Timing, prediction, and learning properties of the cerebellum, once integrated in the circuits that are formed with the neocortex, basal ganglia, and limbic system (**Figure 3**), could affect the control of complex novelty-related functions (D'Angelo and Casali, 2013). Thus, this widespread two-way communication sustains basal ganglia and cerebellar involvement in motor functions and cognitive and behavioral processing. Cortico-basal-cerebellar communication may influence and sustain even processes that are linked to individual differences in approach and avoidance behaviors (**Figure 3**, dashed black line). The basal ganglia and cerebellum have complementary roles in facilitating motivation that sustains and reinforces personality features. The positive correlation between basal ganglia and cerebellar volumes and NS scores and the negative association between basal ganglia and cerebellar volumes and HA scores are consistent with the varying levels of engagement that subjects with various personality traits require to their subcortical circuitries. In fact, subjects who search for unfamiliar situations, make the unknown known, explore new environments, display increased tendency toward risk-taking, sensation-seeking, and immediate reward-seeking, lack inhibition, as novelty seekers do, need very rapid detection of unfamiliar events, flexible switching among tasks, alternatives, and contexts, and fast adaptation to change. All these functions heavily engage basal ganglia and cerebellum.

# **APPROACH AND AVOIDANCE BEHAVIORS IN ANIMALS**

It is still very difficult to study the brain mechanisms of human subjective experience like emotion or motivation. Although the neuroimaging techniques are rapidly advancing, they reveal little about the precise working of neurons and trafficking of molecules in the brain activity related to approach and avoidance. Further, neuroimaging studies are correlative and cannot deliver answers about the nature and cause of the associations between structure and function. The techniques required to detail the mechanisms of brain functions usually cannot be used with humans for ethical and practical reasons, but animal research allows for use of these techniques, much as invasive they can be. In the following sections we address the experimental research on approach and avoidance behaviors, facing neurobiological, neurochemical and synaptic aspects.

## **TOOLS FOR STUDYING APPROACH AND AVOIDANCE BEHAVIORS**

In a wide range of animal species individual differences in approach and avoidance behaviors have been observed, based on direction of the action toward positive (e.g., rewarding) stimuli or away from negative (e.g., dangerous) stimuli, on neophilic or neophobic responses, or on exploratory or withdrawal behaviors (Greenberg, 2003). In an attempt to model in rodents the human individual differences in approach and avoidance behaviors, many behavioral testing paradigms have been employed because almost all behavioral tests encompass approach or avoidance facets. In fact, although most tests are devoted to test spatial, discriminative, mnesic, attentive functions as well as emotional components, in many behavioral tests it is possible to emphasize the component of approach and avoidance. Overall, the tests integrate the approachavoidance conflict designed to promote or inhibit an ongoing behavior characteristic for the animal, such as forcing or vise versa contrasting the tendency of mice to engage in exploratory activity, reward- or novelty-seeking behaviors, and social interaction. Notably, the explorative drive represents the prerequisite to recognize and seek for rewarding or novel stimuli and includes many components, such as suppression of the discomfort caused by unfamiliar spaces, exit from known starting areas, acquisition or use of efficient foraging strategies, and snapshots of the target view and representation-forming procedures.

Among the various tests, the mostly used are the Light-Dark Exploration Test, Social Interaction Test, Novelty-Induced hypophagia test, Approach-Avoidance conflict paradigm, Approach/Avoidance (A/A) Y-maze, and Open Field (OF) test (Bailey and Crawley, 2009).

As for the *Light-Dark Exploration Test*, the chamber is formed by a cage divided into two unequal compartments by a dark partition with a small aperture located in the bottom center. The smaller compartment is painted black and covered by a hinged lid. The larger compartment is uncovered with transparent sides and is brightly lit by fluorescent room lighting. Thus, the animal is exposed to environment with protected (dark compartment) and unprotected (light compartment) areas. The inherent conflict between exploratory drive and risk avoidance is thought to inhibit exploration. Most mice naturally demonstrate a preference for the dark protected compartment. The key measure for assessing approach-avoidance behavior is a willingness to explore the lighted unprotected area. Such proclivity is reflected in the number of transitions between compartments, and in the time spent in each compartment. An increase in exploratory activity is interpreted as a release of exploratory inhibition and novelty-seeking behavior. In fact, mice exhibiting higher levels of anxiogenic/avoiding-like behavior will make fewer transitions between the brightly illuminated, open area and the dark, enclosed compartment. Further, the time spent in risk assessment is another measure of anxiety/avoidance-related behavior. Risk assessment includes a stretch-attend posture in which the head and forepaws extend into the lighted area but the remainder of the body stays in the dark compartment (Bailey and Crawley, 2009).

As for the *Social Interaction Test*, unfamiliar animals are allowed to directly or indirectly interact in an arena. Time spent in interacting is recorded. Anxiolytic/approaching-like behavior is inferred if social interaction time increases and general motor activity remains unaffected. Conversely, decreased time spent in engaging social behavior indicates anxiogenic/avoiding-like behavior. The times engaged in aggressive (attack, aggressive unrest), avoiding (vigilant posture, escape and defense activity), approaching (following, social sniffing, over-under climbing) behaviors as well as in motor activities (rearing, walking) are scored (File and Seth, 2003).

*Novelty-Induced hypophagia test* is based on the typical behavior of the rodents that consume very limited quantities of any new even if highly palatable food and only after considerable investigation. This response is unconditioned, requires no training, and can be elicited in food-deprived or satiated animals by substituting a highly palatable food source for standard food. As the test sessions go on, the latency to the first taste decreases and the total amount of consumed food increases (Dulawa and Hen, 2005).

*Approach-Avoidance conflict paradigm* consists of a rectangular box subdivided into two compartments. One distinctive visual cue is associated with each compartment: one compartment has white walls and black floor, whereas the other one has black walls and white floor. For three consecutive days, the animal is placed in only one compartment that becomes familiar. In the following days, the animal placed in the familiar compartment is allowed to freely explore the whole apparatus (both familiar and novel compartments). The time spent in each compartment and frequency of crossings between compartments are indices of approach and avoidance behaviors (Adriani et al., 1998; Zoratto et al., 2013).

*A/A Y-maze* has a starting arm from which two arms stemmed, arranged at an angle of 90◦ to each other (**Figure 4A**). One of the two arms has black and opaque floor and walls and no light inside, while the other one has white floor and walls and is lighted. At the end of each arm of choice there is a food tray. The depth of the tray prevents mice from seeing the reward at a distance

but allows for an easy reward (eating) and the appreciation of reward scent, not reducing the olfactory cues. Since the appetites for palatable foods have to be learned (Wise, 2006; Lafenêtre et al., 2009), a week before behavioral testing the animals have to be exposed to a novel palatable food (Fonzies, KP Snack Foods, Munchen, Germany) in their home cages for three consecutive days (Bassareo et al., 2002). At the beginning of behavioral testing, mice are subjected to 1-day habituation phase in which all Y-Maze arms are opened to encourage maze exploration. During habituation phase, no food is present in the apparatus. To increase the motivation to search for the reward, 12 h before exposure to the experimental set-up, the animals are slightly food deprived by limiting the food access to 12 h/day. Such a regimen has to result in no significant body weight loss. Testing phase consists of two 10-trial sessions with 1 min-inter-trial interval. In the Session 1 (S1), the mouse is placed in the starting arm and may

choose to enter one of the two arms, both containing the same standard food reward. During the Session 2 (S2; starting 24 h after S1), the white arm is rewarded with the highly palatable food, while the black arm is rewarded with the standard food pellet. Thus, the A/A Y-maze task requires an animal to choose between two conflicting drives: reaching a new reward (highly palatable food) in an aversive (white and lighted) environment or reaching a familiar food (standard pellets) in a not aversive (black and opaque) environment. The considered parameters were: white choices, the frequency of entry into the white arm in S1 and S2; A/A conflict index, the difference in the number of white choices between S1 and S2; entry latencies exhibited in white and black arms, separately or regardless arm color or reward in each trial of both S1 and S2.

*Open field* apparatus consists of a wide circular arena delimited by a wall (**Figure 4B**). In S1, a mouse is allowed to explore the empty OF and its baseline level of activity is measured. In S2, the object is put in the arena center. Notably, the approach to the object requires the subject to overcome its innate fear toward open spaces and indicates thus that the animal is reacting to the mismatch between the initial (empty arena) and new (presence of the object) situations. Novelty preference is considered an inverse index of anxiety whereby an anxious mouse tends to avoid the potential dangers associated with a novel and unknown environment. The considered parameters were: total and peripheral distances traveled in the arena; central crossings; freezing duration; number of defecation boluses; latency and time of contact with the object.

In these tasks there is a clear conflict between positive and negative poles that simultaneously evoke approach and avoidance behaviors. Typically, when the positive and negative poles have similar strengths, the subject remains suspended or, at best, gravitates toward the slightly heavier pole of the conflicting situation. Many other tests are devoted to selectively assess behaviors of approach (as drug intake, response to positive conditioned stimulus, brain self-stimulation) or avoidance (as conditioned taste aversion, operant behavior to avoid an electric shock by a leverpress, aversive brain stimulation).

## **NEUROBIOLOGICAL ASPECTS**

Approach and avoidance behaviors are posited to emerge from mechanisms operative in the spinal cord (Berntson et al., 2003; Schutter et al., 2011), brain stem (Berridge and Peciña, 1995; Nelson and Panksepp, 1998; Challis et al., 2013) and cortex (Nasser and McNally, 2012). Namely, approach and avoidance behaviors are associated with the corticolimbic circuitry that comprises the prefrontal cortex, amygdala, and striatum and that controls cognitive functions, attention, reward sensitivity, and emotional expression (**Figure 3**; Cain and LeDoux, 2008; LeDoux, 2012; Bravo-Rivera et al., 2014). The intensity of appetitive or defensive motivation-related behaviors are modulated by the levels of neurotransmitters (dopamine, acetylcholine), neuropeptides (corticotrophin-releasing hormone, oxytocin, orexin), and neuromodulators (endocannabinoids) (Robbins and Everitt, 1996; Berridge, 2000; Gerra et al., 2000; Linfoot et al., 2009; Groppe et al., 2013; Mogi et al., 2014). Understanding neurochemical systems is crucial in addressing approach and avoidance topic (Tops et al., 2010). The avoidance situations (satiation, conditioned taste aversion, aversive brain stimulation) have the acetylcholine release in common, while the approach situations (eating, sugar bingeing, drug intake, positive conditioned stimulus, brain self-stimulation) have the dopamine release in common (Hoebel et al., 2007). However, it has to be considered that dopamine is an important factor also in responding to positive punishment provoked by the exposure to an aversive stimulus, and is involved in the motor aspects of both approach and avoidance behaviors. In the nucleus accumbens it has been demonstrated that dopamine and acetylcholine exert opposing roles in the control of GABAergic output in relation to approach and avoidance, and acetylcholine counteracts any excessive approach behavior mediated by the dopamine (Helm et al., 2003; Kelley et al., 2005; Hoebel et al., 2007). Interestingly, adult offspring of

dams treated with corticosterone and a tryptophan-deficient diet showed increased avoidance behavior in the approach-avoidance conflict paradigm and anhedonia toward highly palatable reward in an operant progressive ratio test (Zoratto et al., 2013). These behaviors were associated with reduced dopamine and serotonin levels in the prefrontal cortex and reduced striatal and increased hypothalamic Brain Derived Neurotrophic Factor (BDNF) levels. Also neuropeptides are retained to be critical in approach and avoidance behaviors and have been much studied in animal research over the last several years. It has been demonstrated that in odor-recipient rats the odor cues from healthy conspecifics induced approach behavior, while the odor cues from sick conspecifics produced avoidance response (Arakawa et al., 2008, 2009, 2010a, 2011). In the odor-recipient rats, c-Fos mRNA expression was induced in olfactory bulb, amygdala, bed nucleus of stria terminalis, and hypothalamic paraventricular nucleus (Arakawa et al., 2010b). Interestingly, in the amygdala, the expression of oxytocin receptor mRNA was increased when the rats were exposed to healthy conspecific odor, while induction of arginine vasopressin receptor mRNA was found when exposed to sick conspecific odor. Into the amygdala the infusion of an antagonist of oxytocin receptor blocked approach behavior to "healthy" odor, while the infusion of antagonists of arginine vasopressin receptor inhibited avoidance response to "sick" odor. Thus, the approach and avoidance behaviors appear to involve similar brain regions but with different mechanisms (Ikemoto and Panksepp, 1999; Cain and LeDoux, 2008; Nasser and McNally, 2012). Recent findings indicate that also the orexins, hypothalamic neuropeptides that regulate feeding and sleeping behaviors, modulate avoidance behaviors. Rats treated with an antagonist of orexin-1 receptor approached a typically negative stimulus (cat odor) more than vehicle-treated rats (Staples and Cornish, 2014). Notably, exposure to cat odor induced Fos expression in the hypothalamus, suggesting that hypothalamic system is functionally involved with antipredator defensive behaviors (Blanchard et al., 2005). In accordance, microinjections of orexins in the paraventricular thalamic nucleus that innervates the amygdala decreased approach behavior to novelty in rats, indicating a negative emotional state (Li et al., 2010).

A very significant neuromodulatory system on approach and avoidance behaviors in humans (McDonald et al., 2003; Van Laere et al., 2009) as well as rodents (Pattij et al., 2007; Lafenêtre et al., 2009) is the endocannabinoid system (ECS) that deserves a detailed description.

As we recently demonstrated, spontaneous forms of approach and avoidance behaviors rely on ECS modulation in corticolimbic and striatal areas (Laricchiuta et al., 2012b, 2014a,d).

## **NEUROCHEMICAL ASPECTS: ENDOCANNABINOID AND DOPAMINERGIC SYSTEMS**

After their synthesis from arachidonic acid, endocannabinoids, such as anandamide (AEA) and 2-arachidonoylglycerol (2-AG), modulate synaptic transmission by stimulating cannabinoid type-1 (CB1) receptors (Freund et al., 2003; Piomelli, 2003; Marsicano and Lutz, 2006; Matias and Di Marzo, 2007; Kano et al., 2009). These receptors are primarily expressed in the corticolimbic, striatal and cerebellar pathways (Herkenham et al., 1990; Katona et al., 1999; Marsicano and Lutz, 1999; Palmiter, 2008; Koob and Volkow, 2010). Cannabinoid type-1 receptors presynaptically inhibit glutamatergic and GABAergic neurotransmission (Pagotto et al., 2006; Matias and Di Marzo, 2007; Kano et al., 2009) and this inhibitory control of excitatory and inhibitory neuronal subtypes determines the bimodal effects of endocannabinoids (Bellocchio et al., 2010). Thus, the ECS is engaged in myriad of physiological functions. During neural development, the ECS mediates neuronal proliferation, migration, and axonal growth (Berghuis et al., 2007; Harkany et al., 2008; Mulder et al., 2008; Trezza et al., 2008). Throughout life, the ECS influences synaptic transmission, neuroprotection, and neuroinflammation (Fowler and Jacobsson, 2002; Cota et al., 2003; Maldonado et al., 2006; Marsicano and Lutz, 2006; Kano et al., 2009; Lutz, 2009; Fowler et al., 2010). Further, the ECS governs emotional processes, anxiety, stress coping and extinction of aversive memories (Witkin et al., 2005; Lutz, 2007, 2009; Patel and Hillard, 2008; Laricchiuta et al., 2013). The involvement of the ECS in fear extinction is supported by the different responses of the human subjects genotyped for two polymorphisms of CB<sup>1</sup> receptors in a fear-potentiated eyeblink startle reflex paradigm (Heitland et al., 2012). In adults with traumarelated psychopathologies, increased CB<sup>1</sup> receptor availability in the amygdala is associated with increased attentional bias to threat and increased severity of the symptomatology linked to threat (re-experiencing, avoidance, and hyper-arousal), but not the symptomatology linked to loss (emotional numbing, depression, generalized anxiety) (Pietrzak et al., 2014). Also a common polymorphism that affects the enzymatic degradation of endocannabinoids by fatty acid amide hydrolase (FAAH) is linked to reactivity of the amygdala in relation to threat during a face allocation task involving fearful and angry faces, and to reactivity of the striatum in relation to reward in a gambling task with positive and negative feedback (Hariri, 2009). Further, the individuals with the FAAH polymorphism exhibit quick habituation of amygdala reactivity to threat (Gunduz-Cinar et al., 2013). Thus, the effects of the FAAH polymorphism demonstrate the engagement of ECS in the defensive and appetitive motivational systems (Conzelmann et al., 2012). Moreover, genetic deletion or inhibition of FAAH has context-dependent anxiolytic effects, as demonstrated in mice tested on Elevated Plus-Maze and Light-Dark Exploration Test (Naidu et al., 2007; Moreira et al., 2008).

In mice, experimental manipulations with strong rewarding and reinforcing properties, such as cocaine-induced conditioned place preference, spontaneous running wheel activity, and sucrose consumption, are associated with hypersensitivity of striatal GABAergic synapses to CB<sup>1</sup> receptor stimulation (Centonze et al., 2007a,b; De Chiara et al., 2010). Conversely, social defeat chronic stress down-regulates CB1-controlled GABAergic striatal neurotransmission in mice (Rossi et al., 2008). Notably, the reinforcing effects of the primary rewards (food or drug) or the environmental stimuli associated with them enhance the dopaminergic release in corticolimbic and basal ganglia areas (**Figure 3**, yellow solid line) (Bassareo et al., 2002; Lupica and Riegel, 2005; Alcaro and Panksepp, 2011). Endocannabinoid system and dopaminergic system dynamically interact in controlling neuronal, endocrine, and metabolic responses to reward (Di Marzo et al., 2004; Fernández-Ruiz et al., 2010). In rats, the ECS inhibition on mesolimbic dopaminergic neurons influences the processes of attribution of salience to the reward represented by cocaine and heroin (De Vries et al., 2001; Fattore et al., 2003). The ECS has been implicated in several dopamine-related disorders, such as schizophrenia (Robson et al., 2014), Parkinson's disease (Maccarrone et al., 2003), and drug addiction (Maldonado and Rodríguez de Fonseca, 2002; Rivera et al., 2013; Nader et al., 2014). In these conditions, ECS involvement likely reflects the activity of midbrain dopaminergic neurons and their target structures (Berke and Hyman, 2000; Everitt and Wolf, 2002; Castelli et al., 2011).

To analyze individual differences in spontaneous approach and avoidance behaviors, we tested adolescent (about post-natal day 32nd) C57BL/6JOlaHsd inbred mice in the A/A Y-maze (Laricchiuta et al., 2012b, 2014a,d). In the large sample of mice (more than seven hundred) tested in the A/A Y-maze task, we assigned the individuals into three phenotypes—avoiding (∼6% of individuals that spontaneously reacted with withdrawing responses to the conflicting stimuli), balancing (∼25% of individuals that reacted with balanced responses to the conflicting stimuli), and approaching (∼7% of individuals that reacted with advancing responses to the conflicting stimuli, Laricchiuta et al., 2012b, 2014d; **Figure 4A**). All mice had similar explorativity levels in the initial trials of the task, but only approaching animals maintained high reactivity as trials went by. To eliminate the "food" and "palatability" dimensions and maintain the conflicting drives given by a new object placed in an anxiogenic central location of a wide arena, OF task has been used. In the OF, only the approaching animals were highly explorative and attracted by the new object (**Figure 4B**; Laricchiuta et al., 2012b). The close relation between approach behavior and explorativity has been proposed also in human studies that report that impulsivity and extraversion (Martin and Potts, 2004; Cohen et al., 2005), and risk aversion and low motivation (Tobler et al., 2007) are related to each other.

Because the A/A Y-maze and OF tasks integrate approachavoidance conflict, the inevitable anxiogenic component that is linked to the conflict had to be considered. No differences in anxiety-related parameters of both tasks (defecation boluses, freezing times and central crossings) were found in the three phenotypes. Also, in the Elevated Plus-Maze, a well-validated anxiety test, all animals had similar anxiety levels.

To analyze the neuronal correlates of the approach and avoidance behaviors displayed by the three sub-populations of animals, we analyzed the CB1-mediated neurotransmission in medium spiny neurons (MSNs) of the dorsomedial striatum that is crucially involved in motivated and goal-directed behaviors (Palmiter, 2008; Koob and Volkow, 2010; Laricchiuta et al., 2012b). Presynaptic control of CB<sup>1</sup> receptors on GABAergic transmission in the dorsostriatal MSNs was nearly absent in the avoiding animals but rose increased in the approaching animals. Specifically, application of a CB<sup>1</sup> receptor agonist (HU210) to striatal slices provoked peak reductions of GABAA-mediated inhibitory postsynaptic currents of approximately 40%, 20%, and 0% in approaching, balancing, and avoiding animals, respectively. By enhancing the AEA endogenous tone with URB597, a drug that inhibits FAAH, the avoiding animals exhibited increased approach behavior and explorative drive. These behavioral responses were paralleled by the rescue of CB<sup>1</sup> receptor sensitivity to HU210. On blocking CB<sup>1</sup> receptors with AM251, a CB<sup>1</sup> inverse agonist, the approaching animals reduced their contact times with object and explorative behavior in the OF task, behaviors accompanied by complete inhibition of CB<sup>1</sup> receptor activity. Thus, the behavioral features of the avoiding and approaching animals treated with ECS agonists and antagonists tended to fade. In a nut shell, the treatment rendered them less inhibited and less "advanced", respectively. These findings were confirmed by counterbalancing the pharmacological manipulations in avoiding and approaching animals. Avoiding animals that had a reduced CB<sup>1</sup> control on GABAergic MSNs when further inhibited by AM251 treatment did not display any behavioral as well as electrophysiological modification in comparison to avoiding animals treated with vehicle. In parallel, approaching animals that had an enhanced CB<sup>1</sup> control on GABAergic MSNs when further potentiated by URB597 treatment did not display any behavioral as well as electrophysiological modification in comparison to approaching animals treated with vehicle.

Balancing animals treated with URB597 developed a robust approach behavior toward palatable food in the A/A Y-maze and the new object in the OF task (Laricchiuta et al., 2014a). In these animals, the administration of AM251 alone or in combination with URB597 attenuated the approach behavior toward palatable food in the A/A Y-maze and the new object in the OF test, and suppressed the effects of HU210 on dorsostriatal GABAergic MSNs. These findings demonstrate that the effect of URB597 on approach behavior is mediated by CB<sup>1</sup> receptors. Notably, in balancing animals, haloperidol (dopaminergic D<sup>2</sup> receptor antagonist) blocked their approach behavior toward palatable food in the A/A Y-maze and the new object in the OF task, like AM251 did, and suppressed the effects of HU210 on dorsostriatal GABAergic MSNs (Laricchiuta et al., 2014a). These findings are consistent with the observation that D<sup>2</sup> stimulation activates the dorsostriatal ECS, which in turn influences the GABAergic MSNs (Centonze et al., 2004, 2007a,b), and with the disparities in impulsivity that are associated with differences in monoamines in the striatum and nucleus accumbens in inbred rodents (Moreno et al., 2010).

In balancing animals, the co-administration of URB597 and haloperidol counteracted the effects of haloperidol on approach behavior in the A/A Y-maze but not in the OF task. Further, ECS potentiation combined with D<sup>2</sup> receptor blockade arose only when the reward was represented by palatable food (Laricchiuta et al., 2014a). Such a facilitatory effect on food reinforcement was due to the higher salience of palatable food, based on the hedonic properties of its palatability, compared with the lower salience of the object, regardless of its novelty. On the electrophysiological level, CB<sup>1</sup> receptor sensitivity to HU210 was rescued when URB597 and haloperidol were co-administered. These findings are consistent with the increased preference for palatable substances (evaluated by sucrose drinking) and sweet taste (evaluated by behavioral and electrophysiological responses to sweet mixtures) that is induced by the administration of exogenous cannabinoids or endocannabinoids (Higgs et al., 2003; Jarrett et al., 2005; Yoshida et al., 2010). In parallel, in rodents the AM251 treatment decreased the palatable food intake (Di Marzo and Matias, 2005; Pagotto et al., 2006). Further, mice injected with the selective CB<sup>1</sup> antagonist Rimonabant repeatedly exposed to novel palatable food or a novel object, exhibited decreased reactivity to palatable food intake, but not to novel object (Lafenêtre et al., 2009). Cannabinoid type-1 antagonists decreased and CB<sup>1</sup> agonists increased dopamine release induced by rewarding stimuli (Fadda et al., 2006; Solinas et al., 2006). Thus, by regulating the dopaminergic processes the striatal ECS increased the hedonic aspects of food-seeking, evaluated by an operant reinstatement procedure in rats (Duarte et al., 2004). Further, exogenous cannabinoids increased the hedonic reactions to highly palatable food (sucrose) but did not affect the reactions to aversive (quinine and saturated NaCl solutions) tastes. Consistent with the ability of cannabinoids to increase sucrose palatability, under cannabinoid pretreatment the sucrose induced a release of dopamine in the nucleus accumbens (De Luca et al., 2012).

As previously reported, enhanced or reduced CB1-mediated control on dorsostriatal GABAergic MSNs was associated with spontaneous approach/exploratory or avoidance behaviors, respectively (Laricchiuta et al., 2012b). A possible explanation for this observation could have been that approaching, balancing, and avoiding animals had varying densities of CB<sup>1</sup> receptors and disparate activities of FAAH in the brain regions that govern the approach and avoidance behaviors. To test this hypothesis, we measured the density of CB<sup>1</sup> receptors (by using [3H]CP55,940 binding autoradiography) and FAAH activity in many brain regions in the three subpopulations of mice (Laricchiuta et al., 2012a). Because significant changes in receptor density do not necessarily translate into gross alterations in receptor functionality or the presence of receptor reserve, we also examined CB<sup>1</sup> receptor functionality (by using [35S]GTPγS binding autoradiography). Notably, only approaching animals had higher CB<sup>1</sup> receptor functionality in the amygdaloid nuclei and hypothalamic dorsomedial nucleus. Interestingly, when compared with balancing animals, both approaching and avoiding animals, which attribute increased motivational salience to stimuli, had greater CB<sup>1</sup> receptor densities in the amygdaloid nuclei and hypothalamic ventromedial nucleus. An intriguing parallel on the relation between opposite temperamental traits and similar receptor availability is provided by a PET study that reported the lower availability of striatal dopamine D2/<sup>3</sup> receptors in healthy subjects with both high or low sensation-seeking, in comparison to subjects with moderate sensation-seeking (Gjedde et al., 2010).

Thus, the subcortical circuit that involves the amygdala and hypothalamus appears to drive individual differences in response to motivational cues, regardless of the opposite direction of the behavioral output. Amygdala mediates the processing of significant stimuli in conditioned fear learning (Pape and Pare, 2010), emotional memory (McGaugh, 2004; LaBar and Cabeza, 2006; LeDoux, 2012), assessment of novel (Schwartz et al., 2003; Weierich et al., 2010), ambiguous (Davis and Whalen, 2001), and threatening (LeDoux, 2000; Cain and LeDoux, 2008; Pape and Pare, 2010) stimuli. Further, in the amygdala, CB<sup>1</sup> receptors presynaptically inhibit GABAergic neurotransmission (Freund et al., 2003). In theory, in avoiding and approaching animals the decreased inhibitory neurotransmission due to increased CB<sup>1</sup> expression could influence the amygdaloid output that converges on other limbic regions, such as the hypothalamus that in turn mediates the reactive component (autonomic and somatic responses) of action. Hypothalamic ventromedial nucleus that regulates ingestive behavior and energy homeostasis exhibits the highest level of CB<sup>1</sup> and cannabinoid receptor gene expression (Herkenham et al., 1990; Marsicano and Lutz, 1999; Jamshidi and Taylor, 2001; Pagotto et al., 2006). The increased CB<sup>1</sup> density in the hypothalamic ventromedial nucleus in avoiding and approaching animals (and the greater CB<sup>1</sup> functionality in the hypothalamic dorsomedial nucleus in approaching animals) could influence their autonomic and somatic responses and affect their phenotypes.

Overall, our data demonstrate that in response to conflicting stimuli, mice exhibit variance of spontaneous behaviors, ranging from avoiding to approaching (Laricchiuta et al., 2012b, 2014a,d). The increased hedonic response and explorative behavior of the approaching animals are linked to greater CB1 mediated control on dorsostriatal inhibitory neurotransmission. Conversely, the inhibitory response to reward of the avoiding animals correlates with decreased CB1-mediated control on dorsostriatal inhibitory neurotransmission. The robust differences among behavioral phenotypes in striatal CB1-mediated currents are not a direct consequence of striatal CB<sup>1</sup> receptor expression levels, but they reflect more subtle changes in ECS signaling (Laricchiuta et al., 2012a). In this context, significant evidence indicates that striatal neurotransmission is important for generating anticipatory/preparatory responses in the presence of a conditioned stimulus paired with a positive or negative unconditioned stimulus (Berridge and Robinson, 1998; Ikemoto and Panksepp, 1999; Cardinal et al., 2002).

It has been proposed that the subjects that attribute higher salience to reward-related cues may be vulnerable to addiction (Flagel et al., 2009; Robinson and Flagel, 2009; Saunders and Robinson, 2010), and the subjects that show higher NS behavior may be vulnerable to depressive-like symptoms (Duclot and Kabbaj, 2013). Conversely, the subjects that attribute higher value to aversive cues may be vulnerable to anxiety and post-traumatic stress disorders (Bush et al., 2007; Yehuda and LeDoux, 2007). By using a Pavlovian conditioned approach procedure, Morrow et al. (2011) classified the rats based on whether they learned to approach and interact with a cue that predicted food reward (sign-tracker animals) or conversely learned to go to the location of the food delivery (goal-tracker animals). Sign-trackers were more fearful of discrete cues that predicted foot-shock, while goal-trackers exhibited greater contextual fear even in the absence of discrete cues, suggesting that a subset of individuals attributes high salience to predictive cues regardless of emotional valence. Because motivational systems have evolved primarily to support drives and to direct actions, their outputs facilitate information processing, motor recruitment, action readiness, and affective and attentional engagement.

# **A POSSIBLE SYNAPTIC SCENARIO OF APPROACH AND AVOIDANCE BEHAVIORS**

As underlined by McNaughton and Corr (2014), the approach and avoidance behaviors have to be anchored to the long-term global sensitivities of the underpinning neural systems. Considering the huge bulk of experimental and human findings (see Elliot, 2008 for an overview), we propose a possible synaptic scenario of approach and avoidance behaviors.

**Figure 3** schematizes the main brain structures retained to mediate approach and avoidance behaviors. Information from the sensory systems reaches the thalamus that in turn projects to neocortex and amygdala, first to its lateral and then to its central nucleus (**Figure 3**, solid black line) (Pape and Pare, 2010). Outputs from the lateral to central and basal nuclei are critical in the increased processing of salient stimuli, whether they are pleasant or aversive (Cain and LeDoux, 2008). The amygdala in turn projects to the hypothalamus (Miguelez et al., 2001). Notably, in the amygdaloid and hypothalamic nuclei the avoiding and approaching animals display an increased density of CB<sup>1</sup> receptors (Laricchiuta et al., 2012a). Furthermore, from the amygdala direct or indirect (via orbitofrontal cortex) outputs reach the dorsal striatum and these connections appear to be involved in avoidance responses (**Figure 3**, solid red line) (Lang and Bradley, 2008). The outputs from the basolateral and central amygdaloid nuclei reach the ventral striatum and the orbitofrontal cortex, and these connections appear to be likely contributors to the execution of approach behavior (**Figure 3**, solid green line) (Lang and Bradley, 2008). Since both amygdaloid-hypothalamic-striatal and striatal-cerebellar networks are involved in the emotional and motivational processes linked to putting into action behaviors toward or away from emotionally salient stimuli, the striatum that inherently serves as a gating mechanism represents a crucial crossroad in the neuroanatomical geography of approach and avoidance behaviors (McNab and Klingberg, 2008; Koziol et al., 2010). The goal-directed and hedonic nature of the striatal contribution to action is supported by pioneering studies on "compulsory approaching syndrome", in which animals with striatal lesions compulsively followed and contacted humans, other animals, or even stationary objects (Villablanca et al., 1976), and on reinforcing and rewarding effects of striatal micro-stimulations in animals (Plotnik et al., 1972; Phillips et al., 1976, 1979) and humans (Lilly, 1960; Heath, 1963). The dopaminergic nature (Kilpatrick et al., 2000) of the reinforcing and rewarding effects has been conclusively confirmed by recent innovative optogenetic studies (Tsai et al., 2009; Bass et al., 2010; Adamantidis et al., 2011; Witten et al., 2011). Striatal neurons appear to not respond to movement *per se* but rather to features of the movement that supports reinforcement, such as the anticipation or expected reward value (Kawagoe et al., 1998; Schultz et al., 2000, 2003). However, striatal neurons and dopaminergic release play a role not only in reward processing but also in aversive processing (Ferreira et al., 2003, 2008; Pezze and Feldon, 2004; Matsumoto and Hikosaka, 2009; Bromberg-Martin et al., 2010; Cohen et al., 2012). Roitman et al. (2005) showed that distinct

populations of striatal neurons respond to rewarding (sucrose) or aversive (quinine) taste. Besides the amygdaloid projections, the striatum receives also glutamatergic inputs from neocortical and thalamic areas (**Figure 3**, solid blue line) and dopaminergic inputs from the substantia nigra (**Figure 3**, solid yellow line). These inputs establish synapses with striatal GABAergic MSNs and cholinergic interneurons (Calabresi et al., 2014). The MSNs are distinct in "direct" and "indirect" pathway projection neurons (DeLong, 1990; Graybiel et al., 1994). Direct pathway MSNs project to the internal globus pallidus and substantia nigra pars reticulata (SNr; **Figure 3**, dashed green line), whereas indirect pathway MSNs project to the SNr by way of the external globus pallidus and subthalamic nucleus (**Figure 3**, dashed red line). The activation of the direct or indirect pathways facilitates or inhibits the motor output, respectively (Durieux et al., 2009). In this framework, Kravitz and Kreitzer (2012) propose that positive reinforcement (caused by the presence of a positive stimulus) may be associated with plasticity that enhances synaptic efficacy (long-term potentiation, LTP) onto direct pathway neurons, whereas positive punishment (caused by the presence of a negative stimulus) may be associated with LTP of indirect pathway neurons. Conversely, negative reinforcement (caused by the absence of a negative stimulus) may be associated with plasticity that depresses synaptic efficacy (long-term depression, LTD) onto indirect pathway neurons, whereas negative punishment (caused by the absence of a positive stimulus) may be associated with LTD of direct pathway neurons (**Figure 5A**). By applying this interesting schema to the approach-avoidance (A/A Y-Maze) task we used (Laricchiuta et al., 2012b, 2014a,d), the reinforcements and punishments can be labeled as depicted in **Figure 5B**. Notably, the substrate for the cross-talk between direct and indirect pathways is represented by ECS that induces the LTD of the dorso-striatal MSNs and of their afferent and efferent connections (Lovinger, 2010). However, an opposite synaptic consequence results when the activation of ECS is kept persistent. In fact, in the dorso-striatal MSNs the long-lasting activation of the ECS impairs both LTD and the reversal of LTP (Nazzaro et al., 2012), mechanisms of synaptic plasticity involved in the habit formation (as drug-related habits or compulsive behaviors) and in reinforcement- or reward-related behaviors (Gerdeman et al., 2003; Gerdeman and Lovinger, 2003; Kravitz et al., 2012; Nazzaro et al., 2012). Interestingly, in our approaching or avoiding mice the striatal ECS is potentiated or downregulated, respectively (Laricchiuta et al., 2012b). It is reasonable to hypothesize, although it has been not yet demonstrated, that such ECS modulations may influence the mechanisms of synaptic plasticity, by reducing the LTP reversal in the approaching animals, and by increasing the LTD in the avoiding animals (**Figure 5C**). The next step of this chained modeling is linked to the rewarding or aversive nature of the direct and indirect pathways. Specifically, are the neurons activated by rewarding stimuli belonging to the direct pathway and the neurons activated by aversive stimuli belonging to indirect pathway? Optogenetic activation of direct or indirect pathway neurons heightens or impairs the strength of cocaine-induced conditioned place preference, respectively (Lobo et al., 2010). Consistently, the activation of direct or indirect pathway neurons heightens or impairs amphetamine sensitization (Ferguson et al., 2011). Furthermore, impaired dopamine-mediated transmission of direct pathway neurons reduces cocaine-locomotor sensitization and impairs conditioned place preference for a food reward, and conversely the impaired transmission of indirect pathway neurons evokes aversive learning deficits (Hikida et al., 2010). Moreover, the stimulation of direct or indirect pathway evokes the rapid learning to contact or to avoid a trigger, respectively (Kravitz et al., 2012), exerting then an opposite control over not just movement, as classically indicated (DeLong, 1990; Graybiel et al., 1994), but also on approach and avoidance behaviors. Thus, in response to the previous questions, it appears that the direct pathway activation is rewarding and indirect pathway activation is aversive. Once more it is possible to hypothesize that by modulating the synaptic plasticity of direct and indirect pathways neurons, the ECS might shift the behavior toward the most significant component of any conflicting context (in the case of approaching behavior: positive reinforcement against negative punishment; in the case of avoiding behavior: negative reinforcement against positive punishment), determining thus the ultimate behavioral outcome (**Figure 5D**). The further final step of the chained modeling can be performed by integrating the schema by Kravitz and Kreitzer (2012), the findings by Nazzaro et al. (2012) and our own results (Laricchiuta et al., 2012b). We suggest that by decreasing the reversal of LTP the potentiation of ECS on direct pathway might contribute to the approach behavior, prompting

the animal toward the positive reinforcement (palatable food). Conversely, by increasing LTD the de-potentiation of ECS on indirect pathway might contribute to the avoidance behavior, prompting the animal toward the negative reinforcement (dark environment) (**Figure 5E**).

# **CONCLUSIONS**

Approach and avoidance behaviors are the foundation of emotional and motivational experience. These behaviors are modulated by the functioning of the network encompassing the subcortical structures implicated in the action (amygdala, dorsal striatum, cerebellum) and re-action (amygdala, hypothalamus) to salient stimuli. The nodes of this network are strongly interconnected and the final behavioral output probably depends upon the weight of the various nodes. By acting on them the endocannabinoid and dopaminergic systems increase the intensity of appetitive or defensive motivation (Häring et al., 2011; Fiorillo, 2013; Ohno-Shosaku and Kano, 2014; Piomelli, 2014). Large individual differences in endocannabinoid and dopaminergic transmission at the striatal, limbic and cortical level have been described in animals (Verheij and Cools, 2008; Yamamoto et al., 2013; Coria et al., 2014; Flagel et al., 2014) and humans (Moresco et al., 2002; Van Laere et al., 2009), as if the primitive model of response to salient stimuli is maintained as a "phylogenetic footprinting" that allows survival and adaptation.

# **AUTHOR'S CONTRIBUTIONS**

Daniela Laricchiuta and Laura Petrosini wrote the paper and revisited it critically for important intellectual content, approving the final version of the paper and agreeing to be accountable for all aspects of the work.

# **REFERENCES**


a review of single administration studies. *Front. Neuroendocrinol.* 33, 17–35. doi: 10.1016/j.yfrne.2011.01.002


Lewin, K. (1935). *A Dynamic Theory of Personality.* New York: McGraw-Hill.


propensity to attribute motivational salience to an aversive cue. *Behav. Brain Res.* 220, 238–243. doi: 10.1016/j.bbr.2011.02.013


increases dopamine levels in the nucleus accumbens shell in rats. *J. Neurochem.* 98, 408–419. doi: 10.1111/j.1471-4159.2006.03880.x


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 01 July 2014; accepted: 28 November 2014; published online: 22 December 2014*.

*Citation: Laricchiuta D and Petrosini L (2014) Individual differences in response to positive and negative stimuli: endocannabinoid-based insight on approach and avoidance behaviors. Front. Syst. Neurosci. 8:238. doi: 10.3389/fnsys.2014.00238 This article was submitted to the journal Frontiers in Systems Neuroscience*.

*Copyright © 2014 Laricchiuta and Petrosini. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

# ADVANTAGES OF PUBLISHING IN FRONTIERS

FAST PUBLICATION Average 90 days from submission to publication

COLLABORATIVE PEER-REVIEW

Designed to be rigorous – yet also collaborative, fair and constructive

RESEARCH NETWORK Our network increases readership for your article

# OPEN ACCESS

Articles are free to read, for greatest visibility

## TRANSPARENT

Editors and reviewers acknowledged by name on published articles

GLOBAL SPREAD Six million monthly page views worldwide

## COPYRIGHT TO AUTHORS

No limit to article distribution and re-use

IMPACT METRICS Advanced metrics track your article's impact

SUPPORT By our Swiss-based editorial team

EPFL Innovation Park · Building I · 1015 Lausanne · Switzerland T +41 21 510 17 00 · info@frontiersin.org · frontiersin.org