# **DOPAMINERGIC FOUNDATIONS OF PERSONALITY AND INDIVIDUAL DIFFERENCES**

**Topic Editors Luke D. Smillie and Jan Wacker**

#### *FRONTIERS COPYRIGHT STATEMENT*

© Copyright 2007-2015 Frontiers Media SA. All rights reserved.

All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.

The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.

Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.

Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.

As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.

All copyright, and all rights therein, are protected by national and international copyright laws.

The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.

Cover image provided by Ibbl sarl, Lausanne CH

**ISSN** 1664-8714 **ISBN** 978-2-88919-432-2 **DOI** 10.3389/978-2-88919-432-2

# *ABOUT FRONTIERS*

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

# *FRONTIERS JOURNAL SERIES*

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing.

All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

# *DEDICATION TO QUALITY*

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view.

By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

# *WHAT ARE FRONTIERS RESEARCH TOPICS?*

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area!

Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# **DOPAMINERGIC FOUNDATIONS OF PERSONALITY AND INDIVIDUAL DIFFERENCES**

Topic Editors: **Luke D. Smillie,** The University of Melbourne, Australia **Jan Wacker,** University of Hamburg, Germany

# Table of Contents


Andrew J. Cooper, Éilish Duke, Alan D. Pickering and Luke D. Smillie

*51 Extraversion and Anterior Vs. Posterior DMN Activity During Self-Referential Thoughts*

Gennady G. Knyazev

*61 Extraversion Differentiates Between Model-Based and Model-Free Strategies in a Reinforcement Learning Task*

Anya Skatova, Patricia A. Chan and Nathaniel D. Daw

*71 Perceived Stress Predicts Altered Reward and Loss Feedback Processing in Medial Prefrontal Cortex*

Michael T. Treadway, Joshua W. Buckholtz and David H. Zald

*81 The Role of the Dorsoanterior Striatum in Implicit Motivation: The Case of the Need for Power*

Oliver C. Schultheiss and Anja Schiepe-Tiska


Masahiro Kawasaki and Yoko Yamaguchi

*119 The Role of D4 Receptor Gene Exon III Polymorphisms in Shaping Human Altruism and Prosocial Behavior*

Yushi Jiang, Soo Hong Chew and Richard P. Ebstein

*126 The Influence of Dopaminergic Gene Variants on Decision Making in the Ultimatum Game*

Martin Reuter, Andrea Felten, Sabrina Penz, Anna Mainzer, Sebastian Markett and Christian Montag

*134 Dopaminergic Foundations of Schizotypy as Measured by the German Version of the Oxford-Liverpool Inventory of Feelings and Experiences (O-LIFE)—a Suitable Endophenotype of Schizophrenia*

Phillip Grant, Yvonne Kuepper, Eva A. Mueller, Catrin Wielpuetz, Oliver Mason and Juergen Hennig


Colin G. De Young

# Dopaminergic foundations of personality and individual differences

#### *Luke D. Smillie1 \* and Jan Wacker <sup>2</sup>*

*<sup>1</sup> Melbourne School of Psychological Sciences, The University of Melbourne, Melbourne, VIC, Australia*

*<sup>2</sup> Department of Psychology, University of Hamburg, Hamburg, Germany*

*\*Correspondence: lsmillie@unimelb.edu.au*

#### *Edited and reviewed by:*

*Hauke R. Heekeren, Freie Universität Berlin, Germany*

**Keywords: dopamine, personality, extraversion, schizotypy, reward**

For several years, theory and research in Personality Neuroscience has linked dopamine function with various aspects of personality and individual differences. This literature builds on research in basic neuroscience concerning the role of dopamine in behavior and experience, with the aim of understanding the ways in which this neurotransmitter system influences *regularities* in behavior and experience. We organized this special issue on "Dopaminergic Foundations of Personality and Individual Differences" with the goal of illuminating the diversity of roles that dopamine plays in personality and individual differences. To introduce this topic, we provide a brief sketch of the current understanding of the functions of the dopamine system. In doing so, we place the diverse contributions to this research topic in the context of this rich, evolving literature.

# **WHAT ROLE DOES DOPAMINE PLAY IN BEHAVIOR AND EXPERIENCE?**

The dopamine system can be divided into several anatomically defined branches or pathways. The nigrostriatal pathway (projecting from the substantia nigra to the striatum) is involved in motor control, and has long been of interest in the context of Parkinson's Disease and its therapeutic management via dopamine replacement (see Cenci, 2007). It was initially thought that motor control was the primary or even sole function of dopamine (e.g., Koob, 1982). However, this perspective has given way to a reward-processing interpretation of dopamine, focussed primarily on the mesolimbic pathway (projecting from the ventral tegmental area to limbic and forebrain areas including the striatum) (Robbins and Everitt, 1996; Wise, 2004; Schultz, 2007). One early theory helped integrate these diverging perspectives by proposing that the ventral striatum, a target of both nigrostriatal and mesolimbic dopamine, was responsible for converting motivation (i.e., to approach desire goal states) into action (Mogenson et al., 1980).

The reward-processing functions of dopamine have been discussed in terms of motivation by reward, enjoyment of reward, and learning from reward—or "wanting," "liking" and "learning" (Berridge et al., 2009). Initially it was theorized that dopamine mediated reward "liking"—the hedonic impact of rewarding stimuli (Wise, 1982), and that these pleasure responses sustained reward-directed behavior. This theory enjoyed widespread influence for some time, and explains why dopamine was popularly dubbed "the pleasure chemical," but has now been abandoned (Wise, 2004). One critique came from the addiction literature, which showed that dopamine-mediated escalation of drug dependence is accompanied by *decreased* pleasurable responses to those drugs (Robinson and Berridge, 2003). This favors the theory that dopamine mediates motivational "wanting" of reward by conferring stimuli with "incentive salience"—the process through which stimuli become motivationally attractive (Robinson and Berridge, 2003; Berridge et al., 2009). Dopamine is also thought to be responsible for reward learning, with phasic dopamine activity providing the "teacher" signal hypothesized in reinforcement learning models (Schultz et al., 1997; Schultz, 2007). Although reward wanting theories appear compatible with reward learning theories, they have not yet been integrated into a cohesive theoretical framework (see Alcaro et al., 2007).

Dopamine also has a major role in cognitive function and dysfunction. The mesocortical dopamine pathway (projecting from the ventral tegmental area to the dorsolateral prefrontal cortex and the anterior cingulate cortex) is implicated in higher cognitive functions such as working memory and decision-making (Robbins et al., 1996; Arnsten, 1998; Floresco and Magyar, 2006). Although these appear strikingly different to the motivational functions of the mesolimbic dopamine system, mental representations and operations seem likely to facilitate motivated action. That is, the mesocorticolimbic dopamine pathways may jointly coordinate the *"anticipation of reward and activation of representations in the PFC needed to achieve it"* (Miller and Cohen, 2001, p. 182). The higher cognitive functions of dopamine have implications for creative behavior, which is typically operationalized using tests of cognitive flexibility and divergent thinking. Ashby et al. (1999) suggest that this may explain the apparent impact of induced positive affect on creativity; positive affect is often preceded by reward delivery, which will often stimulate dopamine release. Finally, an enduring theory has posited a central role for dopamine in the cognitive disturbances seen in schizophrenia (e.g., Gray et al., 1991). A later iteration of this theory has related *mesocortical* dopamine to cognitive deficits (e.g., executive dysfunction) and negative symptoms (e.g., anhedonia), and *mesolimbic* dopamine to positive symptoms (e.g., hallucinations and delusions) (Lindenmayer et al., 2013).

This brief overview is only intended to orient the reader, illustrate the breadth of processes to which dopamine has been linked, and thereby foreshadow the diversity of topics addressed in this special issue. For more in-depth perspectives on dopamine function the interested reader is encouraged to consult the references cited here.

# **WHAT ROLE DOES DOPAMINE PLAY IN PERSONALITY AND INDIVIDUAL DIFFERENCES?**

#### **EXTRAVERSION AND REWARD-PROCESSING**

Perhaps the earliest and most influential perspective on the role of dopamine in personality was Gray's (1973) suggestion that dispositional variation in the reward-processing functions of the dopamine system would likely manifest as a major, tobe-identified personality dimension. This dimension was later identified as extraversion (Depue and Collins, 1999), an enduring proposal that is currently the dominant neurobiological perspective on this trait (see Smillie, 2013), and has motivated over one-third of the contributing articles to this special issue.

Our first two articles demonstrate that the effects of dopaminergic pharmacological agents are entirely dependent on extraverted personality: *Depue and Fu* observe contextual facilitation of incentive motivation processes in extraverted individuals for whom a contextual ensemble was paired with a dopamine agonist. These findings appear to link extraversion with the dopamine-driven processes that associate contexts with reward. *Chavanon and colleagues* demonstrate dose-dependent effects of a dopamine antagonist on an EEG-recorded neural activity localized to prefrontal regions innervated by the mesocorticolimbic dopamine pathway)—and that these effects are diametrically opposed for extraverted and introverted individuals. These findings are potentially explained in terms of extraversion-related individual differences in pre- and post-synaptic responsivity to the dopamine antagonist.

Our next two articles relate extraversion to EEG-derived indices of reward system activity: *Cooper and colleagues* replicate their recent finding that extraversion is associated with a neural index of the dopaminergic teacher signal specified in models of reinforcement learning, and show that this generalizes to a conceptually related trait concerning reward anticipation. *Knyazev* shows that, in more extraverted individuals, there is a relation between self-referential thoughts and alpha power in the posterior hub of the Default Mode Network—a resting state network that has been implicated in self-centered cognition, and which appears to have a basis in dopaminergic neurotransmission.

A further two articles focussing on extraversion and rewardprocessing employ computational models of reward learning: *Pickering and Pesola* identify a number of specific parameters within biologically-plausible models of reward learning that potentially represent the neural substrates of traits such as extraversion (e.g., those that modulate the strength of the neuroplastic effects of phasic dopamine cell firing). *Skatova and colleagues* model extraversion-related differences on a reinforcement learning task in terms of two distinct forms of learning that are difficult to dissociate in typical studies of this kind. After discarding participants who were not engaged with the learning task, they found that extraversion is related to error-driven learning processes, distinct from other learning processes.

#### **OTHER REWARD-RELATED INDIVIDUAL DIFFERENCES**

Reward-related processes have also been implicated in several individual differences constructs beyond extraversion: *Treadway and colleagues* focus on chronic perceptions of stress, which they find is associated with reduced processing of reward and punishment in the medial prefrontal cortex. *Schultheiss and Schiepe-Tiska* focus on the implicit motive "need for power" (i.e., the tendency to experience power over others as rewarding), which they theorize may have a basis in dopamine-driven learning processes centered on the striatum. *Richter and colleagues* report that the degree to which both monetary rewards and punishments modulate reaction time and BOLD measures of interference processing (i.e., objective indicators of differential reinforcement sensitivity) covaries with the DRD2/ANKK1 TaqIA polymorphism of the dopamine D2 receptor gene. Relatedly, findings by *Kawasaki and Yamaguchi* suggest that the degree to which visual working memory capacity increases for preferred versus non-preferred colors may constitute another useful indicator of reward sensitivity that may be linked to brain dopamine in future work.

The rewarding impact of prosocial actions and outcomes (e.g., Harbaugh et al., 2007) suggests that prosocial behavior may also be linked with dopaminergic reward-processing. In line with this, *Jiang and colleagues* conclude from their qualitative review of the literature that a specific dopaminergic gene variant, the D4 receptor gene exon III (DRD4) polymorphism, influences prosocial behavior depending on environmental influences. *Reuter and colleagues* underscore this conclusion by showing that the extent to which individuals behave fairly in the ultimatum game depends on genetic variants of not only the DRD4 but also the D2 receptor gene (DRD2/ANKK1 TaqIA).

#### **COGNITIVE PROCESSES**

The role that dopamine appears to play in symptoms of schizophrenia has clear implications for individual differences reflecting psychosis-proneness and schizotypy. In the first of two papers on this topic, *Grant and colleagues* report that scores on a German translation of the "Oxford-Liverpool Inventory of Feelings and Experiences" (O-LIFE; Mason and Claridge, 2006) are related to a number of dopamine-related genetic polymorphisms. Conversely, *Ettinger and colleagues* examine the association between two measures of psychosis-proneness and neural activity (fMRI) during procedural learning. Their observed associations with BOLD response in several dopamine-relevant regions, including the striatum, are consistent with dopamine's role in both procedural learning and psychosis-proneness.

As noted in our brief introduction, the role of dopamine in cognitive function also appears to extend to creative problem solving. *Chermahini and Hommel* report a replication of their prior work showing a curvilinear association between spontaneous eye-blink rate (a putative non-invasive marker of central dopamine tonus) and creativity (divergent thinking), with optimal performance at average eye-blink rates (presumably reflecting average levels of central dopamine).

#### **THEORETICAL INTEGRATION**

How can we make coherent sense of the variety of individual differences phenomena in which dopamine appears to be involved? In the final article in our special issue, *DeYoung* proposes that the over-arching function of the dopamine system is to promote exploration, which he divides into cognitive exploration (driven by salience-coding dopamine neurons, and linked with trait domains such as openness/intellect) and behavioral exploration (driven by value-coding dopamine neurons, and linked with trait domains such as extraversion). His model provides an elegant framework for integrating the various contributions to this special issue, as well as the broader literature concerning the dopaminergic foundations of personality and individual differences.

#### **SUMMARY AND OUTLOOK**

The sixteen articles in this special issue are a testament to the significant advances that have been made in personality neuroscience and related fields in recent years. Some of these articles have yielded novel findings, while others have served the important task of replicating and consolidating existing research. Overall, they should leave most readers convinced that dopamine function does play a role in personality and other individual differences. Equally, they demonstrate that there is no simple oneto-one correspondence between the neurotransmitter dopamine and any single personality trait. This has often been noted (e.g., Zuckerman, 2005) but is perhaps tempting to ignore. In recognition of this complexity, a challenge for future research is to develop and evaluate more integrative perspectives concerning the multiple neurobiological bases of so-called dopaminergic traits, and the multiple ways in which dopamine influences regular patterns of behavior and experience.

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 03 September 2014; accepted: 09 October 2014; published online: 30 October 2014.*

*Citation: Smillie LD and Wacker J (2014) Dopaminergic foundations of personality and individual differences. Front. Hum. Neurosci. 8:874. doi: 10.3389/fnhum. 2014.00874*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2014 Smillie and Wacker. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# On the nature of extraversion: variation in conditioned contextual activation of dopamine-facilitated affective, cognitive, and motor processes

# *Richard A. Depue\* and Yu Fu*

*Human Development, Cornell University, Ithaca, NY, USA*

#### *Edited by:*

*Jan Wacker, Philipps-Universität Marburg, Germany*

#### *Reviewed by:*

*Charles S. Carver, University of Miami, USA Luke D. Smillie, The University of Melbourne, Australia Juergen Hennig, Justus-Liebig-University Giessen, Germany*

#### *\*Correspondence:*

*Richard A. Depue, Human Development, Laboratory of Neurobiology of Personality, 243 MVR Hall, Cornell University, Ithaca, NY 14853, USA e-mail: rad5@cornell.edu*

Research supports an association between extraversion and dopamine (DA) functioning. DA facilitates incentive motivation and the conditioning and incentive encoding of contexts that predict reward. Therefore, we assessed whether extraversion is related to the efficacy of acquiring conditioned contextual facilitation of three processes that are dependent on DA: motor velocity, positive affect, and visuospatial working memory. We exposed high and low extraverts to three days of association of drug reward (methylphenidate, MP) with a particular laboratory context (Paired group), a test day of conditioning, and three days of extinction in the same laboratory. A Placebo group and an Unpaired group (that had MP in a different laboratory context) served as controls. Conditioned contextual facilitation was assessed by (i) presenting video clips that varied in their pairing with drug and laboratory context and in inherent incentive value, and (ii) measuring increases from day 1 to Test day on the three processes above. Results showed acquisition of conditioned contextual facilitation across all measures to video clips that had been paired with drug and laboratory context in the Paired high extraverts, but no conditioning in the Paired low extraverts (nor in either of the control groups). Increases in the Paired high extraverts were correlated across the three measures. Also, conditioned facilitation was evident on the first day of extinction in Paired high extraverts, despite the absence of the unconditioned effects of MP. By the last day of extinction, responding returned to day 1 levels. The findings suggest that extraversion is associated with variation in the acquisition of contexts that predict reward. Over time, this variation may lead to differences in the breadth of networks of conditioned contexts. Thus, individual differences in extraversion may be *maintained* by activation of differentially encoded central representations of incentive contexts that predict reward.

**Keywords: dopamine, extraversion, conditioning, cognition, motor velocity, positive affect**

# **INTRODUCTION**

Extraversion represents a higher-order personality trait that has been identified in virtually all classificatory systems of the structure of personality, including Eysenck and Gray's models (Gray, 1994), the Five-Factor model (Costa and McCrae, 1992), Tellegen's Multidimensional Personality Questionnaire (MPQ) model (Tellegen and Waller, 2008), and Zuckerman's Alternative Five-Factor model (Zuckerman, 2002). The phenomenology of extraversion is described similarly in all of these models, and is characterized by adjectives that connote a state of positive affect and strong motivation of desire and wanting, as well as by feelings of being excited, enthusiastic, active, peppy, strong, confident, and optimistic (Watson and Tellegen, 1985; Berridge, 2004).

Jung (1921) insightfully placed this positive motivational state in a larger context in his description of extraversion. He suggested that extraversion is characterized by broad engagement with the environment which is supported by the positive affective state emphasized by others. Jung's notion suggests that there is a broad class of environmental stimulus that elicits positive affective engagement, and Gray (1994) extended that notion by arguing that the stimulus class is composed of rewards. Thus, extraversion may represent individual differences in the extent to which environmental rewards elicit positive affective engagement as a means of obtaining those rewards.

Due to conceptually similar phenomenological features, we drew an analogy between this positive affective state in humans and incentive motivation as described in the animal literature (Depue and Collins, 1999; Depue and Morrone-Strupinsky, 2005; Depue and Fu, 2012). Incentive represents a motivational system identified in all mammals, and is elicited by the broad stimulus class of unconditioned and conditioned incentive stimuli that induce forward locomotion and strong subjective feelings of reward. This analogy suggested that, if extraversion represents the manifestation of an incentive reward system, then the trait may be in part influenced, as this motivation is in animals, by the activity of the mesocorticolimbic dopamine (DA) projection system. This projection system originates mainly in the ventral tegmental area (VTA) of the midbrain, and sends afferents to several limbic regions, including the nucleus accumbens (NAc) in the ventral striatum and the amygdala, and to many cortical regions, including the orbital cortex (Depue and Collins, 1999; Depue and Morrone-Strupinsky, 2005; Fields et al., 2007).

In rats and monkeys, dose-dependent DA receptor activation in the VTA-NAc pathway mediates the acute rewarding effects of stimulants, and facilitates a broad array of incentive motivated behaviors, including locomotor activity to novelty and food; as well as exploratory, aggressive, affiliative, and sexual behavior (Depue and Collins, 1999; Berridge, 2007). In single-unit recording studies in monkeys, large populations of VTA DA neurons are activated preferentially by appetitive incentive stimuli (Schultz et al., 1995, 1997; Mirenowicz and Schultz, 1996; D'Ardenne et al., 2008; Schroeder et al., 2008), and DA cells, most numerously in the VTA, respond in proportion to the magnitude of both conditioned and unconditioned incentive stimuli (Fields et al., 2007; Schultz, 2007; Bromberg-Martin et al., 2010). Similarly, NAc cells increase firing to primary and conditioned signals of reward and novelty during intervals when reward is expected, and during engagement in rewarding social activity.

In humans, incentive motivation is associated with both positive *emotional* feelings such as elation and euphoria, and *motivational* feelings of desire, wanting, craving, potency, and self-efficacy (Depue and Collins, 1999). This is in contrast to positive feelings that accompany reward consummation, which is associated with feelings of gratification, quiescence, liking, and calm pleasure (Depue and Morrone-Strupinsky, 2005; Smillie et al., 2012). DA activity is related to the former, but not the latter, subjective emotions. Thus, neuroimaging studies have found that, during acute cocaine or amphetamine administration, the intensity of a participant's subjective euphoria increased in a dosedependent manner in proportion to DA-agonist binding to the DA uptake transporter (and hence DA levels) in the ventral striatum (Volkow et al., 1997). Moreover, DA-induced activity in the NAc was linked equally strongly (if not more strongly) to motivational feelings of desire, wanting, and craving, as to the emotional experience of euphoria (Breiter et al., 1997). And the degree of activation by positive or rewarding stimuli or agonist-induced DA release in healthy human ventral striatum and other regions of reward circuitry (e.g., amygdala, medial orbitofrontal cortex, and anterior cingulate cortex) assessed by fMRI and PET were correlated strongly with (i) feelings of euphoria, (ii) extraversion and similar traits of novelty seeking and affective impulsivity, (iii) DA-relevant gene polymorphisms, and (iv) pharmacological indicators of DA functioning (Depue et al., 1994; Depue, 1995; Berke and Hyman, 2000; Drevets, 2001; Canli et al., 2002; Kumari et al., 2004; Knutson and Cooper, 2005; Mobbs et al., 2005; Reuter and Hennig, 2005; Reuter et al., 2006; Deckersbach et al., 2006; D'Ardenne et al., 2008; Zald et al., 2008; Smillie et al., 2009; Bromberg-Martin et al., 2010; Buckholtz et al., 2010; Haber and Knutson, 2010; Baik et al., 2012). Hence, taken together, the animal and human evidence supports the notion that the VTA DA-NAc pathway is a primary neural circuit for incentive reward (Bromberg-Martin et al., 2010; Haber and Knutson, 2010; Sesack and Grace, 2010), and that extraversion is related to activity in that pathway (Wacker et al., 2006, 2012, 2013).

While VTA DA activation is critical for inducing incentive motivation in NAc, VTA DA neuron responses also play a role in facilitating the association between those stimuli that predict reward (i.e., conditioned stimuli) and motivated behavior that obtains reward (Schultz et al., 1997; Montague et al., 2004; Schultz, 2007). With regard to associative learning, mere DA neuron activation without exogenous reward produced a preference for the context paired with phasic DA firing. Concordantly, DA neuron firing was gradually time-locked to the presentation of a conditioned cue that predicted sucrose delivery, and phasic DA release correlated positively with conditioned approach behavior toward the cue (Stuber et al., 2008). This associative process includes the following steps. The optimal stimuli for activating VTA DA neurons are *unpredicted* unconditioned rewards (e.g., food, sweet liquid). Such biologically salient stimuli are evaluated for their emotional significance in the basolateral amygdala (BLA) and medial orbital frontal cortex (mOFC). If such stimuli have sufficient incentive salience, these and other corticolimbic areas then activate VTA DA neurons (Berke and Hyman, 2000; Myer-Lindenberg et al., 2005; Fields et al., 2007; Kauer and Malenka, 2007; Stuber et al., 2008; Zellner and Ranaldi, 2010), which release DA into the NAc as a means of increasing incentive motivation to obtain the reward. Subsequently, neutral cues in the current context that consistently predict reward are associated with reward (become CSs) in the BLA and mOFC (Elliott et al., 2003; Gottfried et al., 2003; Simmons et al., 2007; D'Ardenne et al., 2008), which in turn activate VTA DA neurons prior to the occurrence of primary reward (Zellner and Ranaldi, 2010). This process is shown in **Figure 1** during an experiment's progression: VTA DA neurons show increased activity in the presence of neutral stimuli that consistently predict reward, and a concurrent decrease in activity to the unconditioned reward, until DA responding has transferred completely to the conditioned incentive stimuli (Schultz et al., 1997; Galvan et al., 2005; Day et al., 2007; Schultz, 2007; Stuber et al., 2008). Thus, VTA DA discharge ratchets backward in time so as to respond to earlier and earlier predictors of reward. Therefore, DA activity is critical to the control of appetitive behavior by conditioned incentive stimuli—specifically, to link stimuli predicting reward, which activate VTA neurons, to the response-facilitation mechanism in the NAc (Schultz et al., 1997; Depue and Collins, 1999; Nestler, 2001; Depue and Morrone-Strupinsky, 2005; Berridge, 2007; Stuber et al., 2008; Zellner and Ranaldi, 2010).

The acquisition of a reward-predictive neural structure is enhanced when VTA DA activation results in release of DA in the NAc. DA release in the NAc plays a critical role in the formation of complex contextual ensembles that predict the occurrence of reward in a much more detailed manner than do single CS incentives (Depue and Morrone-Strupinsky, 2005; Depue and Fu, 2012). The array of stimuli that comprise the full context that precedes the occurrence of primary reward converge on the NAc (O'Donnell, 1999). These corticolimbic inputs originate from many perceptual processing pathways, but importantly also from those areas that compute the incentive salience of contextual stimuli, including the BLA, mOFC, and extended amygdala (e.g., bed nucleus of the stria terminalis) (Groenewegen et al., 1999a,b; O'Donnell, 1999; Berke and Hyman, 2000; Depue and Morrone-Strupinsky, 2005). The end product of this compression is a contextual ensemble that is encoded for incentive salience or value. That ensemble is further compressed in a cortico-cortical

loop, which terminates in the mOFC where the ensemble is associated with an expected outcome (i.e., probability and magnitude of reward; Alexander et al., 1990; O'Donnell, 1999; Amodio and Frith, 2006). It is not surprising then that it is the mOFC that provides the major source of activation of VTA DA neurons when predictive contexts of reward occur (Taber et al., 1995; Carr and Sesack, 2000; Zellner and Ranaldi, 2010). The magnitude of the encoded incentive salience of the mOFC contextual ensemble is thus translated into the magnitude of mOFC-VTA DA activation and, in turn, NAc DA-facilitated incentive motivation.

The acquisition of contextual ensembles is strongly dependent on DA in the NAc. Corticolimbic regions carrying contextual information innervate NAc neurons in close proximity to VTA DA projections to the NAc (O'Donnell, 1999; Depue and Morrone-Strupinsky, 2005; Sesack and Grace, 2010). It is here that DA facilitates the development of long-term potentiated connections of corticolimbic afferents to NAc neurons (Nestler, 2001; Goto and Grace, 2005; Kauer and Malenka, 2007; Shen et al., 2008; Stuber et al., 2008). Presumably, the more DA that is released in the NAc, (a) the greater the strengthening of the connection of contextual afferents on NAc neurons, and (b) the greater the number of afferents thus facilitated. *Hence, variation in DA input to the NAc will modulate the strength of the contextual ensemble, and hence the capacity of that ensemble to subsequently elicit incentive motivation, positive affect, and approach behavior (i.e., extraverted behavior)*.

The importance of this model is that individual differences in VTA DA-NAc reactivity to reward, as found in extraversion, could modify the associative conditioning of unconditioned rewards to neutral contextual cues, and thereby create differences in the strength and breadth of individuals' networks of reward-relevant contexts. Exactly this prediction has been confirmed in animal studies, where a significant correlation between DA functioning and contextual conditioning was demonstrated (Hooks et al., 1992; Cabib, 1993; Jodogne et al., 1994; Wassum et al., 2011). The implication of these findings is that variation in the strength and breadth of reward-predictive contextual networks could play a critical role in the *maintenance* of individual differences in extraverted behavior over time.

Expanding a small preliminary study on conditioning and extraversion, we more fully investigated these possibilities by studying the *acquisition* and *extinction* over seven consecutive days of conditioned contextual facilitation of DA-modulated motor, affective, and cognitive processes in a DA agonist (methylphenidate)-paired context in high and low subgroups of extraverts. We predicted and found that high extraverts who had context paired with methylphenidate showed significantly greater conditioned contextual facilitation across all three processes relative to low extraverts. Indeed, low extraverts showed little, if any, conditioning under these experimental conditions. Moreover, conditioning was verified not only on a conditioning Test day, but also by demonstrating (a) robust conditioned responses on the first day of extinction under placebo in the absence of unconditioned drug effects, and (b) the decay of conditioned responding over a three-day extinction period.

### **MATERIALS AND METHODS DESIGN**

A study design with three consecutive phases was used (**Figure 2**): (i) *Association* (days 1–3), in which MP or placebo (lactose) is associated with laboratory context for three days. MP and placebo were administered in identical capsules double-blind to drug and extraversion score. On the basis of preliminary studies, three Association days were used; even one day with low doses of DA agonist is adequate in rats to demonstrate acquisition of contextual association to incentive processes (Anagnostaras and Robinson, 1996; Robinson and Berridge, 2000); (ii) *Test* (day 4), in which degree of contextual facilitation of responding is assessed under MP conditions; and (iii) *Extinction* (days 5–7), three days of placebo, where the first extinction day (day 5) assessed the presence of conditioned context-facilitated responding in the absence of unconditioned drug effects, which provides direct evidence of a motivational effect of conditioned cues (Anagnostaras and Robinson, 1996; Everitt et al., 2001).

Three experimental conditions, each with high and low subgroups of extraverts (i.e., six groups total), paired MP exposure with laboratory context (*Paired*) or did not (*Unpaired* and *Placebo*). On each Association day, all three experimental conditions received MP or placebo in each of two contextually distinct laboratories (Lab *A,* followed by Lab *B*—in which participants read emotionally neutral magazines supplied by the experimenter, as they also did in Lab A when not involved in tasks). This procedure equated Paired and Unpaired conditions for MP exposure but within different laboratory contexts (see **Figure 2**) (Anagnostaras and Robinson, 1996). Following previous research (Anagnostaras and Robinson, 1996; Robinson and Berridge, 2000), the context of Labs A and B differed in physical dimensions, flooring, wall colors and decorations, lighting, furniture, and experimenters. Because psychostimulants, including MP, strongly *amplify* conditioned-cue activation of behavior via DA release in the NAc (Parkinson et al., 1999; Robinson and Berridge, 2000; Everitt et al., 2001), all conditions received MP on Test day. MP was administered on Test day, because expression of conditioned drug effects are contextdependent. Therefore, despite receiving MP, the control groups above should not express facilitation of responding as should the group that has acquired conditioned facilitation. This allowed an assessment on Test day of the extent to which contextual cues had acquired incentive properties in the Paired condition relative to unconditioned effects of MP in Unpaired and Placebo groups.

**PARTICIPANTS**

The MPQ (Tellegen and Waller, 2008) extraversion scale was used. It correlates with EPQ extraversion (0.62, *P <* 0*.*01), incorporates content of the extraversion scales measured by the NEO-PI (Costa and McCrae, 1992; Church, 1994), is influenced by strong genetic variation (Tellegen et al., 1988), and its positive affect or emotionality interpretation is supported by convergent-discriminant relations to the state dimension of positive affect (Zevon and Tellegen, 1982; Watson and Tellegen, 1985; Tellegen and Waller, 2008). MPQ extraversion scores were obtained from 92% (*N* = 2997) of Cornell freshmen, which has an MPQ profile equivalent to other university samples and to the general population within the age range of 19–24 years (Tellegen and Waller, 2008). High and low extraversion subgroups were randomly selected from the top and bottom deciles, respectively, of MPQ extraversion scores, and then were randomly assigned to the three experimental conditions. Selected participants were medically and psychiatrically normal and taking no medications, as verified blind to MPQ score by (i) medical interview and physical exam by a physician, and (ii) psychiatric interview using the latest version of the SCID (nonpatient version), DSM-IV criteria, and the Personality Disorders Examination (Loranger, 1994) for Axis II disorders. We excluded participants with (a) cardiovascular, immune, or endocrine disorders or who were taking medications for these or other conditions that might interact with MP; (b) Axis I and II disorders because such conditions may affect DA functioning in unpredictable ways; (c) substance abuse or dependence; and (d) a recent (within last two years) smoking history, since nicotine may interact with DA. We have found that frequency of smokers does not differ above or below the MPQ extraversion median. To detect illicit drug use, participants received a confidential drug screen the day prior to each study day. No illicit drug use was detected.

Of the 74 initially selected male participants, 70 (95%) participated. As is expected due to strict decile selection criteria, MPQ extraversion scores did not differ significantly between comparisons of all low subgroup combinations (all *P*'s *<*0.70) nor between comparisons of all high subgroup combinations (all *P*'s *<* 0.70) across experimental conditions (**Table 1**). The 70 participants were also selected on the basis of their falling within the middle six deciles on MPQ Negative Emotionality (Neuroticism) and Constraint (impulsivity scale). Therefore, high



and low extraversion participants were equivalent (not significantly different) on these other MPQ traits. Males (Caucasian; age: 19–21 years; weight: 62–88 kg) rather than females were used because DA efficacy markedly varies across the menstrual cycle (Depue et al., 1994). The number in each of the six experimental groups is: Paired High Extraversion: (PH = 15); Paired Low Extraversion: (PL = 15); Unpaired High Extraversion: (UPH = 10); Unpaired Low Extraversion: (UPL = 10); Placebo High Extraversion: (PBH = 10); Placebo Low Extraversion: (PBL = 10). Because the critical comparison in this study is between paired high vs. paired low extraversion, the *N* for those two groups is higher than for the other groups. Written informed consent was obtained from all participants in a protocol approved by Cornell University's institutional review board.

#### **METHYLPHENIDATE (MP)**

MP was used because (a) MP exerts its DA-agonist effects by increasing release of DA from presynaptic terminals, thereby activating an array of DA receptor subtypes; (b) MP binds with similar or greater magnitude to the same DA-uptake transporter as cocaine and amphetamine at presynaptic sites in cortex and striatum, especially the NAc; (c) regional distribution of MP binding in human brain is almost identical to cocaine; and (d) MP strongly induces NAc-facilitated incentive motivated behaviors, including (i) rewarding properties in conditioned place preference, (ii) self-administration in primates, and (iii) positive affect, energy, and euphoria in humans at doses of 0.5 mg/kg or less that correlates with its % DA-uptake binding in ventral striatum (Volkow et al., 1995, 1997, 1998, 2001).

MP was also used because of its specificity of action to DA at doses used here. In individual limbic and coritical brain regions, there are varying mixtures of D1, D2, D3, D4, and D5 receptors (Strange, 1993). The control of motor, emotion, and motivation processes by DA in these brain regions will, therefore, be dependent on DA interacting with various combinations of receptor isoforms. With respect to behavioral effects of D1 and D2 and D1/D2 mixed agonists and antagonists in interaction with MP, MP has its behavioral effects via both D1 and D2 receptors in a dose-dependent manner (Koek and Colpaert, 1993; Strange, 1993). Importantly, compounds not directly involving DA receptors, and compounds with antagonist properties at CNS receptors other than DA (including alpha 1 and 2 and beta noradrenergic, and 5HT 2 and 1A receptor antagonists), either did not interact with MP behavioral effects, or did so only at such high doses that extreme behavioral adverse effects occurred (Koek and Colpaert, 1993). Moreover, affinitiy for the 5HT transporter is not only much lower for MP than amphetamine and cocaine, but also affinity for this transporter is not associated with the reinforcing properties of MP (Ritz et al., 1987; Little et al., 1993). Thus, at the relatively low dose used in the current study, MP's major effects appear to be on both D1 and D2 (and perhaps other DA) receptor families. Since DA facilitation of incentive motivation, positive affect, and initiation of locomotion appears to involve at least both D1 and D2 receptors (Depue and Collins, 1999), MP is a better agonist to study extraversion processes than bromocriptine or bupropion (Vassout et al., 1993), which both have mainly D2 receptor effects. MP also appears to have a more specific DA transporter binding affinity, relative to noradrenergic and serotonergic affinities (Weiner, 1972), than amphetamine and to some extent cocaine.

Percent binding of MP to the DA-uptake transporter provides one means of judging the "saturation" effects of an MP dose, and is correlated significantly with induced positive affect in humans (Volkow et al., 1997). We used an oral MP dose of 0.6 mg/kg based on the fact that at this dose (a) % DA transporter binding is ∼80% or more (Volkow et al., 1998, 2001); (b) a sufficiently long, stable peak plateau (∼90 min) is associated with the positive affect effects of MP (Volkow et al., 1997, 1998), permitting sufficient time for our task administration (∼1 h) at peak MP concentrations; (c) no significant negative affect is observed; and (d) clearance is ∼10 h, indicating wash-out by the next day (Volkow et al., 2001). In addition, in humans, retest stability for the binding and time-course characteristics of MP (0.5 mg/kg) is very high (Volkow et al., 1995). Finally, in humans, MP has a very low adverse effect profile when orally administered acutely in low dose (0.5 mg/kg or less) (Aoyama, 1994; Wang et al., 1994; Volkow et al., 1995).

#### **EXPERIMENTAL STIMULI**

The extent to which MP-induced reward is associated with context in the Paired condition is reflected in facilitation of responding elicited by *general contextual features* of Lab A. General context-reward association, like conditioned place preference, is an implicit Pavlovian process that is acquired more readily and with greater resistance to extinction than is the pairing of explicit, discrete stimuli with reward (Holland, 1992; Graybiel, 1998). The number of conditioning sessions required for general context vs discrete stimuli in animals is ∼1:20 session ratio, respectively. To assess the success of associative conditioning of Lab A to MP, we used five 20-s video clips that differed in their (i) association with laboratory context, (ii) MP drug effects, and (iii) inherent incentive value. The five video clips were presented in Lab A via VCR in randomized order, each separated by a 1-min rest interval, on a 56-inch TV monitor located 12 feet in front of participants.

The content of *three* of the video clips, shown on Association day 1 and Test day 4, were initially incentive-neutral, but differed in their representation of the Lab A context and in their association with MP drug reward: (i) *Library*: a moving pan across the front of Cornell's main library, which has no association with Lab A or drug reward; (ii) *Labfront*: a moving video pan across the front of Lab A, which participants continually faced during the study because they were seated facing the front of the lab; and (iii) *Portrait*: a large poster of a female portrait in the front of Lab A. The latter two stimuli vary in two other ways: *Labfront* (i) represents an implicit general contextual stimulus, which is rapidly and strongly conditioned in animals, and (ii) such general contextual stimuli are likely processed in the dorsal visual stream (i.e., via peripheral vision). In contrast, *Portrait* (i) represents an explicit, discrete stimulus object that is conditioned more slowly in animals and (ii) such discrete stimuli are likely processed in the ventral visual stream (i.e., as object recognition). Differential facilitated responding on Test day 4 is a direct test of an acquired incentive salience for *Labfront* and *Portrait* compared to *Library*.

*Two* additional previously validated video clips (Morrone et al., 2000; Morrone-Strupinsky and Depue, 2004), also shown on Association day 1 and Test day 4, had no association with drug reward or the general context of Lab A (outside of the 5-minute exposure on day 1). The two clips, however, differed in inherent incentive value and appetitive approach motivation, to which extraverts respond vigorously, but not in calm pleasurable feelings, to which extraverts do not respond vigorously (Morrone et al., 2000; Morrone-Strupinsky and Depue, 2004; Smillie et al., 2012): (iv) *Rainforest (low incentive)*: neutral rainforest scenes, and (v) *Football (high incentive and approach motivation, rather than a calm pleasurable emotional state*: a triumphant football game sequence (scoring of a touchdown). The rationale for comparing these two clips is to assess whether the Lab A context had acquired facilitatory effects on unfamiliar stimuli that had not been paired with Lab A or with MP. The incentive response elicited by any stimulus is a joint function of both the conditioned incentive value of the context and of the inherent incentive value of the unfamiliar stimulus (Jodogne et al., 1994; Schultz et al., 1997; Robinson and Berridge, 2000). Stimuli with little inherent incentive value, like *Rainforest*, will not be facilitated substantially by a conditioned context. While the incentive response to the *Football* clip relative to the *Rainforest* clip is expected to naturally differ on day 1, whether that incentive response will evidence an enhancement on day 4 relative to day 1 depends on the success of the conditioning procedure in interaction with the natural incentive value of the unfamiliar stimulus. Therefore, if there is an enhanced incentive response to *Football* on day 4 relative to day 1, but no enhancement for *Rainforest*, then one may conclude that the enhanced response to *Football* on day 4 was dependent on contextual conditioning (Robinson and Berridge, 2000).

Preliminary research showed that *Library*, *Rainforest*, *Labfront*, and *Portrait* were initially rated on both the 10-point positive and negative affect scales used in this study (see below) as neutral in affective state [*N* = 50 college males; Positve Affect Means (SDs) = 1.1 (0.05), 1.01 (0.03), 1.08 (0.04), 2.03 (0.07), respectively, where a rating of 1 or 2 = neutral affect state]. *Football* was rated 4.1 (1.2), where 4 = mild positive affect state. Mean negative affect ratings were generally around 1, and did not exceed 2.2 (neutral affect state).

#### **MEASURES**

Three variables, measured only in Lab A, indexed conditioned context facilitation on motor, affective, and working memory processes. All three variables are strongly dependent on VTA DA projections to the NAc or dorsolateral prefrontal cortex (working memory variable). The three variables were assessed only on Association day 1 and on Test day 4 to avoid excessive task repetition, with affective and motor variables being measured (in that order) after each of the video clips. Working memory was measured only once on these two days, immediately after the video clip presentations. During the Extinction phase, only motor and affective responses to video clips were measured—on the first (day 5) and final (day 7) days of extinction. The cognitive task was not assessed in Extinction, because it is subject to repetition effects (Luciana et al., 1992).

#### *Motor velocity*

*Velocity* of motor behavior is (i) specifically related to incentive processes facilitated by DA predominantly in the NAc (Le Moal and Simon, 1991; Depue and Collins, 1999), (ii) activated by drug-associated conditioned cues (Hyman and Malenka, 2001), and (iii) correlates (*r* = 0*.*68, *P <* 0*.*01) with % DA-uptake binding in human NAc (Volkow et al., 1998). Therefore, velocity of finger tapping was measured as in Volkow et al. (1998). Finger tapping was performed on a laptop computer space bar for 6 s using the dominant hand with palm resting on the laptop base so that taps were performed solely by finger-wrist movement. To control for variation in reaction time (RT), which affects number of taps in the first second, only the last 5 s of tapping were analyzed. Preliminary studies using 20 s of tapping showed that differences between individuals are most marked in the initial 5-s period of tapping (after 1 s correction for RTs).

#### *Positive affect*

Positive affect, which reflects a state of positive incentive motivation (Zevon and Tellegen, 1982; Watson and Tellegen, 1985; Watson and Clark, 1997; Depue and Collins, 1999; Tellegen and Waller, 2008), was assessed by a rating scale similar to a previously validated scale described in detail elsewhere (Morrone et al., 2000; Morrone-Strupinsky and Depue, 2004). This and similar scales have excellent internal consistencies, retest reliabilities, and factor homogeneity (Watson and Tellegen, 1985; Watson et al., 1988; Krauss et al., 1992). They are also correlated with (i) % DA-uptake binding specifically in human ventral striatum (Volkow et al., 1997), (ii) DA-agonist challenge and responses to the video material used here (*r* = 0*.*57, *P <* 0*.*01) (Depue et al., 1994; Volkow et al., 1997; Morrone et al., 2000; Morrone-Strupinsky and Depue, 2004), and (iii) extraversion (*r* = 0*.*49, *P <* 0*.*01) (Morrone et al., 2000). Intraclass correlation between MP-induced peak affect ratings obtained 2–3 months apart is 0.58 (*P <* 0*.*05; *N* = 20, ranging from top to bottom decile on MPQ extraversion). Negative affect state was also rated at the same times as positive affect, but the former showed little (nonsignificant) variation from 1 to 2 (neutral mood state), and no significant activation by MP. Therefore, negative ratings are not discussed further.

The positive and negative affect rating scales are visual analog scales ranging from 1 (neutral affect state) to 10. Point 10 was anchored by adjectives found to be most highly correlated with positive and negative affect states (Watson and Tellegen, 1985). The positive adjective anchors were: *active, elated, enthusiastic, excited, peppy, strong* (where all adjectives were listed under point 10 on the scale). Participants were instructed to rate their emotional response on the scale to each clip.

The positive affect rating scale was displayed on a laptop monitor, and ratings were made directly on computer. For the affect and motor measures, the stimulus–response sequence was: (a) audiovisual prompt on the monitor, preparing the participant for the video clip, (b) video clip, (c) positive affect rating (∼3 s), (d) 6 s of tapping, the timing of which started with the first tap and ended with an audio stop-beep produced by the laptop, and (e) 1-min rest interval between video clips. Participants were trained *prior* to the study on the laptop, tapping procedure, and rating scales.

#### *Visuospatial working memory task*

This measure reflected conditioned incentive effects derived from the general laboratory context of Lab A. The task, validated and described previously (Luciana et al., 1992, 1998; Luciana and Collins, 1997), is dependent in primates and humans on VTA DA projections to dorsolateral prefrontal cortex, and is facilitated by MP (Oades and Halliday, 1987; Luciana et al., 1992, 1998; Luciana and Collins, 1997; Devilbiss and Berridge, 2008; McNab et al., 2009; Aart et al., 2011). Briefly, during each trial, participants observed a central fixation point (a black "+") on a computer monitor for 3 s. Next, a visual cue (a blackened circle against a white background) appeared in peripheral vision within a 360◦ Circumference for 200 ms (too brief to make a saccadic eye movement), after which the cue and fixation point disappeared and the screen blackened for delay intervals of 0.5 s, 4.0 s, or 8.0 s. After the delay, participants indicated the screen location of the cue with a light pen (FTG Data Systems, Inc.). Twenty-four trials (8 for each delay), with a 2-s inter-trial interval, were completed, with delay intervals randomly interspersed and cue locations randomized over trials. Visual cues were presented randomly at two different locations in each of four quadrants (8 trials) for each delay. Working memory accuracy was computer assessed by use of the hypotenuse of a triangle formed by the actual target location and the vertical and horizontal deviations from the actual target indicated by the participant by use of the light pen. RT was also recorded by computer.

As described previously (Luciana et al., 1992, 1998; Luciana and Collins, 1997), MP drug effects on attentional, arousal, perceptual, and sensorimotor processes involved in a targeted visual search (but not specifically in working memory tasks) were assessed on day 4 by use of (a) a non-mnemonic spatial location task of 16 stimulus trials with no response delay, where accuracy and latency to respond were computer recorded; and (b) a bi-letter cancellation task, where number of omission and commission errors (unmarked target letters and incorrectly marked non-target letters, respectively) were tabulated. Order of these tasks was: non-mnemonic spatial location, working memory task, bi-letter cancellation task. These tasks were given on day 1 and day 4 immediately after all the video clips had been viewed and responded to for affective and motor variables.

#### **PROCEDURE**

Participants were habituated to Labs A and B during two prestudy visits to the labs. Participants completed the 2½ h protocol sometime between noon and 6 p.m. for seven consecutive days. MP and placebo were administered with water in Lab A upon arrival, and tasks and measures occurred over a 1-h period beginning 1 h post-drug ingestion. Participants fasted from midnight prior to each study day, and were on a low monoamine diet for three days prior to and during the study.

#### **RESULTS**

As recommended by others (Anagnostaras and Robinson, 1996; Volkow et al., 1997, 1998; Robinson and Berridge, 2000), magnitude of conditioning was assessed as % change from Association day 1 to Test day 4 on the three dependent variables: motor velocity (finger tapping), positive affect ratings, and visuospatial working memory accuracy. Within the Placebo (PB) and Unpaired (UP) conditions, the high and low extrovert subgroups showed no significant difference on Association day 1 or in % change from day 1 to Test day 4 for any of the five video clips (alpha adjusted for number of analyses, *P <* 0*.*005). Thus, a 4 (subgroups: PBL, PBH, UPL, UPH) × 5 (video clips) ANOVA with repeated measures on the second factor revealed no significant main effects for subgroups [*F(*3*,* <sup>144</sup>*)* = 1*.*45, *P* = 0*.*36] or video clips [*F(*4*,* <sup>144</sup>*)* = 1*.*32, *P* = 0*.*39] on *motor velocity* on day 1. A 4 (subgroups) × 5 (video clips) ANOVA with repeated measures on the second factor revealed no significant main effects for subgroups [*F(*3*,* <sup>144</sup>*)* = 1*.*61, *P* = 0*.*48] or video clips [*F(*4*,* <sup>144</sup>*)* = 1*.*13, *P* = 0*.*59] on *positive affect* ratings on day 1. Finally, a 4 (subgroups) × 3 (working memory delay intervals) ANOVA with repeated measures on the second factor revealed no significant main effects for subgroups [*F(*3*,* <sup>72</sup>*)* = 1*.*39, *P* = 0*.*38] or delay intervals [*F(*2*,* <sup>72</sup>*)* = 1*.*47, *P* = 0*.*46] on *day 1* for *working memory*.

A 4 (subgroups) × 5 (video clips) ANOVA with repeated measures on the second factor revealed no significant main effects for subgroups [*F(*3*,* <sup>144</sup>*)* = 1*.*34, *P* = 0*.*42] or video clips [*F(*4*,* <sup>144</sup>*)* = 1*.*44, *P* = 0*.*51] on % change from Association day 1 to Test day 4 for *motor velocity*. In addition, a 4 (subgroups) × 5 (video clips) ANOVA with repeated measures on the second factor revealed no significant main effects for subgroups [*F(*3*,* <sup>144</sup>*)* = 1*.*21, *P* = 0*.*54] or video clips [*F(*4*,* <sup>144</sup>*)* = 1*.*68, *P* = 0*.*33] on % change from Association day 1 to Test day 4 for *positive affect* ratings. Finally, a 4 (subgroups) × 3 (working memory delay intervals) ANOVA with repeated measures on the second factor revealed no significant main effects for subgroups [*F(*3*,* <sup>72</sup>*)* = 1*.*42, *P* = 0*.*35] or delay intervals [*F(*2*,* <sup>72</sup>*)* = 1*.*39, *P* = 0*.*42] on % change from Association day 1 to Test day 4 for *working memory*.

Thus, none of the four extraversion subgroups comprising PB and UP experimental conditions showed evidence on motor velocity, positive affect, or working memory of conditioning (i.e., no significant % change from day 1 to day 4 on any measure), nor did they differ significantly from each other on day 1. Therefore, these low and high extraversion subgroups were combined, leaving the larger PB and UP groups (now each with an *N* of 20). The low and high subgroups in the paired condition represent the strong test of differential conditioning, so they were of course not combined.

#### **GROUP COMPARISONS OF MOTOR VELOCITY AND POSITIVE AFFECT RATINGS**

Alpha adjusted for the number of analyses for the following analyses is *P <* 0*.*008. A 4 (groups: PB, UP, PL, PH) × 5 (video clips) ANOVA with repeated measures on the second factor revealed no significant main effects for groups [*F(*3*,* <sup>272</sup>*)* = 1*.*48, *P* = 0*.*44] nor for video clips [*F(*4*,* <sup>272</sup>*)* = 1*.*51, *P* = 0*.*51] on day 1 for *motor velocity*. A 4 (groups: PB, UP, PL, PH) × 5 (video clips) ANOVA with repeated measures on the second factor revealed significant main effects for groups [*F(*3*,* <sup>272</sup>*)* = 19*.*26, *P <* 0*.*001; partial eta squared = 0.10] and for video clips [*F(*4*,* <sup>272</sup>*)* = 15*.*59, *P <* 0*.*001; partial eta squared = 0.11] on % change from Association day 1 to

Test day 4 for *motor velocity*. The Groups × Video Clips interaction was also significant [*F(*12*,* <sup>272</sup>*)* = 10*.*43, *P <* 0*.*001; partial eta squared = 0.14]. Tukey *post-hoc* comparisons revealed that PH significantly exceeded all of the other three groups in % change for motor velocity on *Labfront, Portrait*, and *Football* video clips (all *P*'s *<* 0.003), but not on *Library* and *Rainforest* (all P's *>* 0.30) (**Table 2**; **Figures 3A–E**). In addition, none of the other three groups (PB, UP, PL) differed significantly from each other for motor velocity on any of video clips for motor velocity (all *P*'s *>* 0.30). Indeed, PB, UP, and PL groups generally showed a decrease in % change in motor velocity.

A 4 (groups: PB, UP, PL, PH) × 5 (video clips) ANOVA with repeated measures on the second factor revealed no significant

**Table 2 | Means (SDs) of motor velocity for association and extinction phases.**


*The association phase represents data for the four groups for Association day 1, Test day 4, and percent (%) change from day 1 to day 4 as a function of stimulus scene. The extinction phase shows data for all groups on days 5 and 7, and % change for only the PH group on days 5 and 7 for the stimulus scenes on which conditioning was observed (Labfront, Portrait, Football). Data are rounded. PB, placebo; UP, unpaired; PL, paired low extraverts; PH, paired high extraverts.*

main effects for groups [*F(*3*,* <sup>272</sup>*)* = 1*.*433, *P* = 0*.*49] nor for video clips [*F(*4*,* <sup>272</sup>*)* = 1*.*46, *P* = 0*.*45] on day 1 for *positive affect* ratings. A 4 (groups: PB, UP, PL, PH) × 5 (video clips) ANOVA with repeated measures on the second factor revealed significant main effects for groups [*F(*3*,* <sup>272</sup>*)* = 21*.*37, *P <* 0*.*001; partial eta squared = 0.17] and for video clips [*F(*4*,* <sup>272</sup>*)* = 16*.*92, *P <* 0*.*001; partial eta squared = 0.15] on % change from Association day 1 to Test day 4 for *positive affect* ratings. The Groups × Video Clips interaction was also significant [*F(*12*,* <sup>272</sup>*)* = 10*.*28, *P <* 0*.*001; partial eta squared = 0.23]. Tukey *post-hoc* comparisons revealed that PH significantly exceeded all of the other three groups in % change for positive affect on *Labfront*, *Portrait*, and *Football* video clips (all *P*'s *<* 0.003), but not on *Library* and *Rainforest* (all *P*'s *>* 0.30) (**Table 3**; **Figures 4A–E**). In addition, none of the other three groups (PB, UP, PL) differed significantly from each other on any of video clips for positive affect (all P's *>* 0.30). Indeed, PB, UP, and PL groups generally showed a decrease in % change in positive affect.

Thus, only PH showed a significant increase in % change from Association day 1 to Test day 4 in both motor velocity and positive affect to the three video clips that were either paired with MP and Lab A context (*Labfront*, *Portrait*) or had high inherent incentive value (*Football*). PH did not evidence increases in % change for video clips that were not paired with MP or Lab A context (*Library*) or that had low inherent incentive value (*Rainforest*). The % change increase in motor velocity by PH was substantial, ranging from increases of 19–28%, being greatest for *Football*. The % change increase in positive affect ratings by PH was particularly substantial, ranging from increases of 105–126%, being greatest for *Portrait* [note that although the female Portrait may have been more rewarding to the male participants, this analysis was on the change from day 1 to day 4, and hence represents a conditioning effect only]. For PH, within-subject increases in % change in motor x affect variables correlated (Pearson productmoment) significantly for *Labfront* (*r* = 0*.*49, *P <* 0*.*05), Portrait (*r* = 0*.*52, *P <* 0*.*05), and *Football* (*r* = 0*.*50, *P <* 0*.*05), indicating a joint conditioned contextual facilitation across two different DA-modulated response systems within participants.

#### **GROUP COMPARISONS OF VISUOSPATIAL WORKING MEMORY**

Alpha was adjusted to number of analyses at *P <* 0*.*03. A 4 (groups: PB, UP, PL, PH) × 3 (delay intervals) ANOVA with repeated measures on the second factor revealed no significant main effects for groups [*F(*3*,* <sup>136</sup>*)* = 1*.*53, *P <* 0*.*39] nor for delay intervals [*F(*2*,* <sup>136</sup>*)* = 1*.*49, *P <* 0*.*34] on day 1 for *visuospatial working memory accuracy*. A 4 (groups: PB, UP, PL, PH) × 3 (delay intervals) ANOVA with repeated measures on the second factor revealed significant main effects for groups [*F(*3*,* <sup>136</sup>*)* = 18*.*45, *P <* 0*.*001; partial eta squared = 0.18] and for delay intervals [*F(*2*,* <sup>136</sup>*)* = 21*.*72, *P <* 0*.*001; partial eta squared = 0.23] on % change from Association day 1 to Test day 4 for *visuospatial working memory accuracy*. The Groups × Delay interaction was also significant [*F(*6*,* <sup>136</sup>*)* = 13*.*13, *P <* 0*.*001; partial eta squared = 0.31] (**Table 4**; **Figure 5**). Tukey *post-hoc* comparisons revealed that the four groups did not differ in % change from day 1 to day 4 in working memory accuracy for the delay interval of 0.5 s (all *P*'s*>*0.30). However, PH significantly exceeded all of the other

three groups in % change for working memory accuracy at delay intervals of 4.0 s and 8.0 s (all *P*'s*<*0.003). None of the other three groups (PB, UP, PL) differed significantly from each other at any of the delay intervals (all *P*'s *>*0.30). Indeed, PB, UP, and PL groups showed decreases in % change in working memory accuracy at all delay intervals. Finally, PH showed a significant increase in % change from delay interval 0.5 s to 4.0 s (*P <* 0*.*003), as well as a significant increase in % change from delay interval 4.0 s to 8.0 s (*P <* 0*.*003) (see **Table 4** and **Figure 5**). The % change increases for PH were substantial, ranging from +29% at delay 4.0 s to +47% at delay 8.0 s, which is in accord with the demands on DA functioning in dorsolateral prefrontal cortex during increasingly long working memory delay periods (Luciana et al., 1992, 1998; Luciana and Collins, 1997).

For PH participants, the % change increase at 8.0 s delay correlated significantly with the % change increase in motor

velocity (*r* = 0*.*49, *P <* 0*.*05) and positive affect (*r* = 0*.*57, *P <* 0*.*05) to the *Football* video clip, again indicating a joint conditioned contextual within-subject facilitation across three different DA-modulated response systems within participants. [Affective responses to the *Football* clip were used here to correlate with the other dependent variables, because it had the strongest affective induction of positive affect].

Finally, MP drug effects on attentional, arousal, perceptual, and sensorimotor processes involved in a targeted visual search (but not specifically in working memory) were assessed by use of a non-mnemonic spatial location task of 16 stimulus trials with no response delay (0.0 s) on day 4, where accuracy was computer recorded. Adjusted alpha was *P <* 0*.*007. There was no significant main effect for One-Way ANOVA's comparing accuracy [*F(*3*,* <sup>64</sup>*)* = 1*.*23, *P* = 0*.*45] or RT [*F(*3*,* <sup>64</sup>*)* = 1*.*51, *P* = 0*.*48] of the four groups at a delay of 0.0 s. In addition, a bi-letter



*The association phase represents data for the four groups for Association day 1, Test day 4, and percent (%) change from day 1 to day 4 as a function of stimulus scene. The extinction phase shows data for all groups on days 5 and 7, and % change for only the PH group on days 5 and 7 for the stimulus scenes on which conditioning was observed (Labfront, Portrait, Football). Data are rounded. PB, placebo; UP, unpaired; PL, paired low extraverts; PH, paired high extraverts.*

cancellation task was also used to assess MP drug effects on attentional, arousal, perceptual, and sensorimotor processes on day 4, where number of omission + commission errors (unmarked target letters + incorrectly marked non-target letters, respectively) were tabulated. There were no significant main effects for the four groups in a One-Way ANOVA in bi-letter accuracy scores [*F(*3*,* <sup>64</sup>*)* = 1*.*43, *P* = 0*.*42]. Taken together, these findings indicate that MP effects on attentional, arousal, perceptual, and sensorimotor processes do not account for group differences in the working memory results.

#### **MOTOR VELOCITY AND POSITIVE AFFECT IN THE EXTINCTION PHASE**

Extinction-phase data represent % change in motor velocity and positive affect from day 1 to each of days 4, 5, and 7 (% change in days 1 to 4 is used as the conditioning baseline for assessing extinction effects). Because only PH demonstrated significant conditioning (all other groups showed a level line across days 4–7; **Tables 2**, **3**), only the PH Extinction data are analyzed for the three video clips that evidenced conditioning: *Labfront*, *Portrait*, and *Football* (**Table 4**; **Figures 6A**,**B**). Alpha was adjusted for number of analyses at *P <* 0*.*13. A 3 (video clips) × 3 (days 4, 5, 7) ANOVA with repeated measures on both factors revealed a significant main effect for days [*F(*2*,* <sup>84</sup>*)* = 14*.*37, *P <* 0*.*001; partial eta squared = 0.15], but no significant main effect for video clips [*F(*2*,* <sup>84</sup>*)* = 1*.*92, *P* = 0*.*43], on % change in *motor velocity* (**Figure 6A**) from Association day 1 to day 4, 5, and 7. Tukey *post-hoc* tests showed that % change on Test day 4 vs. first extinction day 5 was not significant for any of the three video clips (all *P*'s *>* 0.30), indicating that conditioned contextual facilitation occurred on day 5 in the absence of unconditioned MP drug effects. Comparison of % change on day 5 vs. day 7 showed that day 5 significantly exceeded day 7 for all three video clips (all *P*'s *<* 0.003). As seen in **Figure 6A**, by day 7 motor responding was at or near the level of day 1 (indicated by the 0% change dashed line) on all three video clips.

A 3 (video clips) × 3 (days 4, 5, 7) ANOVA with repeated measures on both factors revealed a significant main effect for days [*F(*2*,* <sup>84</sup>*)* = 19*.*42, *P <* 0*.*001; partial eta squared = 0.28], but no significant main effect for video clips [*F(*2*,* <sup>84</sup>*)* = 1*.*62, *P* = 0*.*38], on % change in *positive affect* (**Figure 6B**) from Association day 1 to day 4, 5, and 7. Tukey *post-hoc* tests showed that % change on day 4 vs. day 5 was not significant for any of the three video clips (all *P*'s *>* 0.30), indicating that conditioned contextual facilitation occurred on day 5 in the absence of unconditioned MP drug effects. Comparison of % change on day 5 vs. day 7 showed that day 5 significantly exceeded day 7 for all three video clips (all *P*'s *<* 0.003). As seen in **Figure 6B**, by day 7 positive affect ratings were at or below the level of day 1 (indicated by the 0 % change dashed line) on all three video clips.

#### **DISCUSSION**

The current findings suggest that extraversion is positively related to brain processes that associate contexts with reward. The robustness of this conclusion is indicated by five findings:


of conditioning (Holland, 1992; Graybiel, 1998) [implicit, contextual stimuli (*Labfront*) vs. explicit, discrete stimuli (*Portrait*)], and that are likely processed along different brain pathways [i.e., ventral (*Portrait*) and dorsal (*Labfront*) visual streams]. Thus, broad conditioned contextual facilitation was observed across different domains (motor, affective, and cognitive) and for different types of stimuli (general context and a discrete object stimulus) for PH participants.


Thus, high extraverts that had context paired with MP in Lab A during the Association phase of the study (i.e., PH) manifested broad conditioned contextual facilitation across motor, affective,

Association phase. Zero % change indicates no change from day 1 to day 4. PB, placebo; UP, unpaired; PL, paired low extraverts; PH, paired high extraverts.

**Table 4 | Means (SDs) for % change in visuospatial working memory in the association phase.**


*PB, placebo; UP, unpaired; PL, paired low extraverts; PH, paired high extraverts.*

and cognitive processes, where the three processes correlated in magnitude of facilitation within participants, and which persisted into the first day of Extinction when no unconditioned effects of MP were present. These conditioned effects were not observed in high or low extraverts who had no exposure to MP in Lab A (i.e., PB and UP), or who had been exposed to MP but in a different lab context (i.e., UP in Lab B). Indeed, PB and UP groups generally showed a moderate loss of contextual facilitation on Test day 4 relative to Association day 1, apparently due to having found repeated presentation of the Lab A context to be absent of incentive value without MP exposure.

Most importantly, low extraverts exposed to MP in Lab A (i.e., PL) apparently experienced little or no rewarding effects from the MP dose used in this study, since they manifested no significant conditioned contextual facilitation on Test day 4 relative to Association day 1. This suggests that PH participants are more sensitive than PL participants to the MP-induced reward generated by the dose used here. This would support the notion that extraversion is characterized by individual differences in reactivity to reward or incentive stimuli, and that these differences have implications for contextual conditioning (Depue et al., 1994; Gray, 1994; Depue and Collins, 1999).

Several lines of evidence suggest that DA modulation contributes to the relation between extraversion and the magnitude of conditioned contextual facilitation of responding. First, DA functioning in the NAc in animals is strongly correlated with (a) the acquisition of reward-induced conditioned contextual responding (Hooks et al., 1992; Cabib, 1993; Jodogne et al., 1994; Wassum et al., 2011), (b) the magnitude of incentive attributed to context (Hooks et al., 1992; Cabib, 1993; Jodogne et al., 1994; Robinson and Berridge, 2000), and (c) the efficacy of drug-associated cues to markedly enhance DA release and gene expression in the NAc (Berke and Hyman, 2000; Everitt et al., 2001). Second, as reviewed above, MP is a potent DA agonist and inducer of feelings of reward in humans. It was the pairing of MP with context in our study that was critical to demonstrating contextual facilitation in PH participants in that equivalently high extraverts in conditions that did not pair MP with context (i.e., PB and UP participants) did not acquire such conditioned facilitation. Third, the presence of conditioned facilitation in PH participants on the first day of Extinction (where no unconditioned MP effects were present) is also consistent with cue-induced NAc DA activity (Ranaldi et al., 1999; Devilbiss and Berridge, 2008). Fourth, as discussed above, the dependence of facilitation of motor velocity, positive affect, and visuospatial working memory processes on VTA DA projections to the NAc and dorsolateral prefrontal

cortex, respectively, is well established in animals and humans (Luciana et al., 1992, 1998; Luciana and Collins, 1997; Depue and Collins, 1999; Devilbiss and Berridge, 2008; McNab et al., 2009; Aart et al., 2011). Fifth, the increasing efficacy of contextual facilitation of working memory with longer response delays found here, when demands on DA facilitation are increasing, is also consistent with a role for DA (Luciana et al., 1992, 1998; Luciana and Collins, 1997). And sixth, that only PH but not PL participants acquired a context-incentive reward association may reflect the positive relation between DA functioning and extraversion reviewed above.

VTA DA neural subgroups positioned more laterally in midbrain project to the NAc, where DA release enhances incentive facilitation of locomotor activity and positive affect (Depue and Collins, 1999; Olson et al., 2005; Fields et al., 2007). In contrast, more medially located VTA DA neural subgroups project to cortical regions, such as the dorsolateral prefrontal cortex, and facilitate working memory processes (Goldman-Rakic, 1987; Luciana et al., 1992, 1998; Fields et al., 2007). The fact that incentive motivational processes reflected by motor and affective variables, as well as cognitive processes indexed by visuospatial working memory, similarly evidenced conditioned contextual facilitation, and that these three variables correlated in % change with each other within participants, suggests that afferents from corticolimbic regions carrying contextual information to the VTA have broad excitatory effects across distinct VTA DA nuclear subgroups (Oades and Halliday, 1987; Taber et al., 1995; Luciana et al., 1998; Groenewegen et al., 1999b; Berke and Hyman, 2000; Carr and Sesack, 2000). Thus, contexts that have been associated with reward appear to facilitate not only incentive motivational processes that activate approach to reward (Berke and Hyman, 2000; Hyman and Malenka, 2001), but also cognitive processes that mediate behavioral strategies and outcome expectancies that guide goal-oriented decisions and behaviors (Everitt et al., 2001; Hyman and Malenka, 2001). This perspective suggests that extraversion involves both affective and cognitive components in engaging with rewarding goals (Gray and Braver, 2002; Depue and Fu, 2012).

The conditioned contextual effects found in PH are specific to the trait of extraversion. This is because we used selection criteria that limited our participants to the middle six deciles on the two major higher-order traits of neuroticism and constraint (impulsivity). While this selection method helps to assure specificity of results to extraversion, it also creates study participants that do not represent the full range of combinations of extraversion with other higher-order traits. Such combinations (e.g., high extraversion and low constraint) may modify conditioning effects (Depue and Fu, 2012). Future studies will need to assess the effects of interactions of traits on the conditioning process.

**FIGURE 6 | Extinction (placebo during days 5, 6, and 7) of conditioned contextual facilitation of motor velocity (A) and positive affect (B) to successfully conditioned video clips (***Labfront***,** *Portrait***, and** *Football***) in PH participants (who were the only participants to condition).** Degree of extinction of conditioned contextual facilitation is indexed as % change (change from day 1) in responding on Test day 4, day 5, and day 7. Responding on day 5 is a strong index of conditioning in that facilitated responding (degree of similarity to facilitated responding on Test day 4) occurs only to context, because the unconditioned effects of methylphenidate are absent. PH, paired high extraverts.

At a broader level, the current findings shed further light on the nature of extraversion. Two points are worth emphasizing about extraversion. First, as much research in genetics, pharmacology, psychology, and neuroscience now suggests, a major contributor to variation in extraverted behavior is individual differences in the functional properties of the VTA DA-NAc/cortical pathways. Second, variation in DA functioning is manifested by the eliciting effects of environmental incentive stimuli, which as our study suggests can be conditioned incentives as well. Therefore, as shown in **Figure 7**, the expression of extraverted behavior can be illustrated by a threshold model that represents a central nervous system weighting of the external and internal factors that contribute to initiation of behavior (Stricker and Zigmond, 1986; White, 1986; Depue and Collins, 1999). In the case of extraversion, the threshold would be weighted most strongly by the joint function of two main variables: (i) the magnitude of incentive stimuli, which ultimately is mainly a function of the magnitude of reward induced by an unconditioned or conditioned incentive stimulus, and (ii) level of DA postsynaptic receptor activation. The interaction of these two variables creates a trade-off function in **Figure 7**, where pairs of values (of incentive stimulus magnitude and DA activation) specify a diagonal representing the minimum threshold value for activation of incentive reward processes that manifest as extraverted behavior. Because the two input variables are interactive, independent variation in either one not only modifies the probability of behavior, but it also simultaneously modifies the value of the other variable that is required to reach a minimum threshold of reward and extraverted behavior.

A threshold model allows behavioral predictions that have implications for conceptualizing the nature of extraversion. A *trait* dimension of DA postsynaptic receptor activation is represented on the horizontal axis of **Figure 7**, where two individuals

**FIGURE 7 | A minimum threshold for facilitation of feelings of reward and extraverted behavior is illustrated as a trade-off function between incentive stimulus magnitude (left vertical axis) and dopamine (DA) postsynaptic receptor activation (horizontal axis).** Range of effective (facilitating) incentive stimuli is illustrated on the right vertical axis as a function of level of DA activation. Two hypothetical individuals with low and high trait DA postsynaptic receptor activation (demarcated on the horizontal axis as A and B, respectively) are shown to have narrow (A) and broad (B) ranges of effective incentive stimuli, respectively.

with divergent trait levels are demarcated: *A* (low trait level) and *B* (high trait level). These two divergent individuals may be used to illustrate the effects of trait differences in DA receptor activation on both acquisition and maintenance of extraverted behavior.

First, as **Figure 7** indicates, for any given incentive stimulus, the degree of DA response will on average be larger in individual *B* vs. *A*. Because the degree of DA activity is correlated with the magnitude of *positive affect* that is naturally elicited by incentive stimuli [e.g., increased enthusiasm, activity, desire, wanting, optimism], this positive emotional experience is also predicted to be more enhanced in *B* vs. *A*.

*Second*, trait differences in incentive activation may have marked effects on the *range* of effective (i.e., reward- and behavior-inducing) incentive stimuli. This is illustrated in **Figure 7**, where the right vertical axis represents the range of effective affiliative stimuli. Increasing trait levels of DA activation (horizontal axis) are associated with an increasing efficacy of weaker incentive stimuli and, thus, with an increasing range of effective incentive stimuli. In **Figure 7** individuals *A* and *B* have a narrow vs broad range, respectively. Significantly, the broader range for individual *B* suggests that on average *B* will experience more *frequent* elicitation of positive emotional experiences associated with reward.

Third, if individual B experiences more frequent and more enhanced reward to incentive sitmuli, animal research suggests that this experience is associated with the quantity of DA release in the NAc and with a graded increase in the frequency and duration of VTA DA neuronal activity (White, 1986; Nishino et al., 1987; Blackburn et al., 1989; Schultz et al., 1995). Thus, variation in DA activation by incentive stimuli may not only influence the level of experienced reward, but also may lead to variation in the strength of DA-facilitated associative processes that link neutral stimuli with reward (Phillips et al., 2003; Simmons and Neill, 2009; Wassum et al., 2011). *The outcome of these interactions may be the acquisition of a more elaborate associative network linking reward to incentive stimuli in*

#### **REFERENCES**


110, 1397–1414. doi: 10.1037/0735- 7044.110.6.1397


*individual B*. The findings of the current study support such a proposition.

Finally, the *maintenance* of individual differences in extraversion may relate to the very factors that promote variation in the acquisition of conditioned incentive stimuli. The latter would be expected to result in variation in the strength and breadth of the encoded memory *network* of conditioned positive incentives (i.e., a contextual ensemble) that represents the general context and specific features associated with subsequent reward. Such differences in reward-encoding of memory representations of salient contexts could have marked effects on the maintenance of extraverted behavior through the operation of cognitive processes of working memory integrated in prefrontal cortical regions. In prefrontal regions, symbolic central representations of the salient context associated with reward can be held on-line as a means of (a) "reliving" and predicting the expected reward from engagement with a salient context, and (b) guiding motivated approach to the goal (Goldman-Rakic, 1987; Waterhouse et al., 1996; Damasio, 1999; Rolls, 2000). Thus, individuals *A* and *B* may develop *differences in their capacity to facilitate over time subjective reward and extraverted behavior due to differentially encoded central representations of salient contexts and their expected outcome (most likely held in mOFC* (Depue and Collins, 1999). Put differently, individual differences in extraversion may be *maintained* by activation of differentially encoded central representations of incentive contexts that predict reward. The implications of the current study are that, in high extraverts, who are predicted to have a lower threshold of behavioral facilitation, this process will involve: (i) more *frequent* activation of incentive; (ii) by a *broader* network of conditioned contexts that; (iii) elicit more strongly encoded central representations of related rewarding events and their expected outcomes.

## **ACKNOWLEDGMENTS**

This work was supported by grant R01 MH 55347 (Richard A. Depue) from the National Institute of Mental Health.


rewarding, aversive, and alerting. *Neuron* 68, 815–834. doi: 10.1016/j.neuron.2010.11.022


cortex to the ventral tegmental area: target specificity in the synaptic associations with mesoaccumbens and mesocortical neurons. *J. Neurosci.* 20, 3864–3873.


model of affiliative bonding: implications for conceptualizing a human trait of affiliation. *Behav. Brain Sci.* 28, 313–395. doi: 10.1017/S0140525X05000063


*Nature of Emotion: Fundamental Questions,* eds P. Ekman and R. J. Davidson (New York, NY: Oxford University Press), 329–331.


imaging of reward prediction. *Curr. Opin. Neurol.* 18, 411–417. doi: 10.1097/01.wco.0000173463. 24758.f6


Personality predicts activity in reward and emotional regions associated with humor. *Proc. Natl. Acad. Sci. U.S.A.* 102, 16502–16506. doi: 10.1073/pnas.0408457102


reinforcement and locomotor activity by D-amphetamine. *J. Neurosci.* 19, 2401–2421.


behavioral events," in *Models of Information Processing in the Basal Ganglia,* eds J. Houk, J. J. Davis, and D. Beiser (Cambridge, MA: MIT Press), 216–229.


excitatory synaptic strength onto midbrain dopamine neurons. *Science* 321, 1690–1692. doi: 10.1126/science.1160873


(COMT) gene: on defining and measuring psychological phenotypes in neurogenetic research. *J. Pers. Soc. Psychol.* 103, 213–227. doi: 10.1037/a0026544


mechanisms in autism. *Psychol. Rev.* 103, 457–489.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 30 March 2013; paper pending published: 19 April 2013; accepted: 02 June 2013; published online: 13 June 2013.*

*Citation: Depue RA and Fu Y (2013) On the nature of extraversion: variation in conditioned contextual activation of dopamine-facilitated affective, cognitive, and motor processes. Front. Hum. Neurosci. 7:288. doi: 10.3389/fnhum. 2013.00288*

*Copyright © 2013 Depue and Fu. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

# Paradoxical dopaminergic drug effects in extraversion: dose- and time-dependent effects of sulpiride on EEG theta activity

# *Mira-Lynn Chavanon\*, Jan Wacker and Gerhard Stemmler*

*Department of Psychology, Philipps-Universität Marburg, Marburg, Germany*

#### *Edited by:*

*Luke D. Smillie, The University of Melbourne, Australia*

#### *Reviewed by:*

*Richard Depue, Cornell University, USA Gennady Knyazev, Academy of Medical Sciences, Siberian Branch, Russia William Revelle, Northwestern University, USA*

#### *\*Correspondence:*

*Mira-Lynn Chavanon, Department of Psychology, Philipps-Universität Marburg, Gutenbergstr. 18, 35037 Marburg, Germany. e-mail: chavanon@uni-marburg.de*

Dopaminergic drugs frequently produce paradoxical effects depending on baseline performance levels, genotype, or personality traits. The present study for the first time aimed to specify the mechanisms underlying such opposite effects using the following recently reported scenario as an example: depending on the personality trait agentic extraversion (agentic facet, aE; i.e., assertiveness, dominance, ambition, positive emotionality) the selective dopamine D2 receptor antagonist sulpiride (200 mg) had opposite effects on resting posterior vs. anterior theta activity in the electroencephalogram (EEG). In order to better describe these opposite pharmaco-EEG effects and to generate hypotheses regarding the underlying mechanisms, we measured the EEG intermittently over 5 h in 80 healthy male volunteers extremely high or low in aE who had received either placebo or one of three doses of sulpiride (50, 200, or 400 mg). The findings suggest a model postulating stronger pre- vs. postsynaptic subreceptor effects in high aE individuals compared to low aE individuals. Future studies may now systematically apply the model to other examples of paradoxical dopaminergic drug effects and examine the molecular basis of individual differences in pre- vs. postsynaptic dopamine D2 subreceptor sensitivities and densities.

**Keywords: electroencephalogram, theta activity, dopamine, sulpiride, agentic extraversion**

# **INTRODUCTION**

The effects of psychopharmacological manipulations of dopamine often show striking variability across individuals with the same drug (e.g., a dopamine agonist, a dopamine antagonist, caffeine) either increasing or decreasing measures of brain activity, cardiovascular activity, mood reaction and task performance depending on baseline values (Takeshita and Ogura, 1994; Bitsios et al., 2005), baseline performance (Mehta et al., 2004; Finke et al., 2010), dopamine synthesis capacity (Cools et al., 2009), working memory span (Kimberg et al., 1997, 2001; Mattay et al., 2000; Mehta et al., 2000; Gibbs and D'Esposito, 2005, 2006; Frank and O'Reilly, 2006; Wallace et al., 2011), dopaminergic genotypes (Mattay et al., 2003; Kirsch et al., 2006; Apud et al., 2007; Cohen et al., 2007; Roussos et al., 2009; Mueller et al., 2011; van Holstein et al., 2011; Rokem et al., 2012) and personality traits like psychoticism (Corr and Kumari, 2000), sensation seeking (Netter and Rammsayer, 1991; Hutchison et al., 1999), impulsivity (Corr and Kumari, 1997; Cools et al., 2007; Clatworthy et al., 2009; Zack and Poulos, 2009), and extraversion (Revelle et al., 1976; Rammsayer et al., 1993; Corr and Kumari, 1997; Rammsayer, 1998; Wacker et al., 2006; Wacker and Stemmler, 2006; White et al., 2006; Chavanon et al., 2007; Smillie and Gokcen, 2010). Understanding the precise mechanisms underlying such paradoxical effects would offer important insights into the dopaminergic foundations of various domains of personality. In the present study, we aim to explore these mechanisms using the strong moderating effect of extraversion on the consequences of the dopamine D2 receptor antagonist sulpiride on resting electroencephalogram (EEG) theta topography observed by Wacker et al. (2006) as an example.

The study by Wacker et al. (2006) aimed to test Depue and Collins' (1999) suggestion that individual differences in a dopamine-based incentive motivational system, the Behavioral Facilitation System (BFS), underlies the personality trait of extraversion—more specifically its agentic facet (aE) encompassing drive, achievement striving, assertiveness as well as positive affective motivational states (elation, desire—wanting, energy) and vigorous and persistent goal-directed behavior in a wide range of achievement-related and social contexts <sup>1</sup> . Neurobiologically the BFS, which closely resembles Gray's (1994)

<sup>1</sup>Although impulsivity is often viewed as a potential trait manifestation of dopamine (for recent research supporting this view see e.g., Dalley et al., 2007; Buckholtz et al., 2010) and as the trait resulting from individual differences in reward and incentive salience processing, empirically it seems to be more strongly related to serotonin (see reviews and data by Carver, 2005; Crockett et al., 2009; Robbins and Crockett, 2010). Moreover, it should be noted that the most common measures of impulsivity are heterogenous encompassing items assessing agentic extraversion and constraint (lower order traits like risk taking, novelty seeking, boldness, adventuresomeness, boredom susceptibility) or assessing motor and cognitive impulsivity. For this reason, measures of impulsivity are neither consistently highly interrelated, nor consistent in their correlation with extraversion (Depue and Collins, 1999). Thus, the Depue model only predicts significant empirical relations between impulsivity and dopamine for those measures of impulsivity more closely related to aE.

Behavioral Approach System, is tied to the mesocorticolimbic dopamine system (MDS; Depue and Collins, 1999) which plays an important role in reward processing (Knutson and Cooper, 2005; Berridge, 2007) and projects from the dopaminergic cells of the ventral tegmental area (VTA) to limbic and cortical areas, such as the nucleus accumbens, cingulate cortex, prefrontal and orbitofrontal cortex (Depue and Collins, 1999; Wise, 2004; Bjorklund and Dunnett, 2007). Individual differences in functional properties of the MDS are thought to create respective differences in the BFS and hence in incentive motivation, approach/goal-directed behavior and aE (Depue and Collins, 1999). Consequently, aE should be associated with individual differences in brain dopamine.

Broadly supporting the aE-dopamine hypothesis, neuroimaging studies have reported associations between extraversion and activation at rest or in response to positive or rewarding stimuli within regions such as ventral striatum (i.e., caudate, putamen, nucleus accumbens), amygdala, medial orbitofrontal cortex (OFC), and anterior cingulate cortex (ACC; Canli et al., 2002; Kumari et al., 2004; Mobbs et al., 2005; Deckersbach et al., 2006) that Depue and Collins (1999) identified as particularly important in the dopaminergic circuitry of reward and approach behavior. In addition, psychopharmacological studies linked extraversion to individual differences in the hormonal response to a challenge with a selective dopamine receptor agonist (Depue et al., 1994; Depue, 1995) and molecular genetic studies have repeatedly found associations between extraversion and variants of dopaminergic genes (e.g., Reuter and Hennig, 2005; Reuter et al., 2006; Smillie et al., 2010).

Rather than focusing exclusively on genetic contributions and instead of using either expensive neuroimaging technology or invasive measurements of blood hormone levels Wacker et al. (2006) opted for an easily obtainable non-invasive EEG index, for which they expected both an association with aE and sensitivity to MDS activity: posterior vs. anterior EEG theta activity. In the meantime, the correlation between aE and this measure was replicated in several studies (Knyazev, 2009, 2010; Wacker and Gatt, 2010; Köhler et al., 2011). Recent studies using the low-resolution electromagnetic tomography algorithm (LORETA) suggest that major sources of posterior vs. anterior EEG theta index are likely located in the ACC (Knyazev, 2010; Chavanon et al., 2011) and the OFC (Knyazev et al., 2012). Also, initial molecular genetic studies have related posterior vs. anterior EEG theta activity to the COMT polymorphism (Val/Val carriers displayed increased posterior vs. anterior EEG theta activity and higher E scores; Wacker and Gatt, 2010) and the dopamine D2 receptor (DRD2) polymorphisms SNP19 rs1076560 and -141C Ins/Del (Köhler et al., 2011). For this index of resting posterior vs. anterior, EEG theta activity Wacker et al. (2006) observed that instead of the usual positive correlation with aE a significant negative correlation was present after administration of sulpiride (200 mg). Thus, sulpiride had completely opposite effects in individuals high vs. low in extraversion.

Besides aE, neuropharmacological studies have revealed an inverted U-shaped relation between working memory functioning and dopaminergic activity (see Arnsten, 1998, for a review). Given that both working memory and extraversion are currently thought to at least partly rely on brain dopamine, it seems reasonable to assume that dopamine connects the two in a systematic way. This suggestion is also corroborated by the fact that the MDS, vital to the concept of aE, is also the main dopaminergic projection to the frontal cortex and thus central to the inverted U-shaped relation between working memory and frontal dopamine. Recent studies revealed that extraversion predicts both working memory performance (Lieberman and Rosenthal, 2001; Chavanon et al., 2007) and working memory-related prefrontal brain activity (Gray and Braver, 2002; Kumari et al., 2004; Gray et al., 2005). Intriguingly, Wacker et al. (2006) reported that the disordinal effects on EEG theta topography were paralleled by paradoxical effects on 2- and 3-back working memory performance: whereas under placebo high aE showed shorter reaction times than low aE, which matched prior observations by Lieberman and Rosenthal (2001), sulpiride reversed these reaction time differences by speeding up low aE and slowing down high aE.

Such opposing or paradoxical effects of dopaminergic drugs have commonly been accounted for by the inverted U-shape principle (often *post-hoc*): Two groups (e.g., high vs. low aE) differ in their baseline levels of dopamine and hence occupy different initial locations on an inverted U-shaped function linking dopamine levels and the dependent variable. Administration of a dopaminergic drug (e.g., a D2 agonist) shifts the groups to different arms of the inverted U-shaped function, producing opposite drug effects for the two groups (**Figure 1**). However, more direct tests of the inverted U-shaped model that use varying drug doses are extremely rare. At the present time there are no empirical data available that elucidate the mechanisms on which such an inverted U-shaped curve between dopamine and posterior vs. anterior theta is based in the context of aE. Without specifying which distinct processes or mechanisms contribute to an inverted U-shaped relationship, it is just a function of representation.

However, plausible alternative explanatory models for paradoxical effects can also be derived from the pharmacodynamic profiles of the dopaminergic drugs administered. For example, in the lower dosage range sulpiride enhances dopaminergic transmission and dopamine synthesis (Tagliamonte et al., 1975) as well as dopamine release by its antagonistic binding to the presynaptic D2/D3-autoreceptors (see review by Rankin et al., 2009), which explains its antidepressant impact, whereas at high doses postsynaptic blockade and reduced dopamine signaling predominate (Westerink and Devries, 1989; Serra et al., 1990; Kuroki et al., 1999). Hence, those two processes might contribute to the paradoxical effects observed by Wacker et al. (2006). A dose of 200 mg sulpiride—as used in the study by Wacker et al. (2006)—likely produces both pre- and postsynaptic effects, although presynaptic effects are thought to prevail (Mueller et al., 2011). Paradoxical dopaminergic effects in different individuals might arise from systematic differences in the time courses of pre- and postsynaptic drug effects. For example, due to different baseline levels of dopamine the responses to sulpiride might be shifted in time in high vs. low aE causing differing effects at a specific point of time (**Figure 2A**).

Finally, because sulpiride shows high affinity to D2 and D3 subreceptors (Strange, 2001), which are both highly expressed in midbrain structures and function at least partly as presynaptic autoreceptors (Rankin et al., 2009), and a lower affinity to D4 receptors, which are mostly expressed in prefrontal cortex, hippocampus, amygdala, and pituitary (Oak et al., 2000), sulpiride's pharmacological profile slightly expands with increasing doses as D4 receptors are additionally simulated<sup>2</sup> . Individual differences in any of these receptor densities or sensitivities might lead to paradoxical effects (**Figure 2B**), if they affect the balance of presynaptic (i.e., effects on DRD2 and DRD3 presynaptic autoreceptor subtypes) vs. postsynaptic (i.e., effects on DRD2, DRD3, and DRD4 postsynaptic receptor subtypes) drug effects, resulting in distinct patterns of response dominance.

Aiming to compare the models' (**Figure 2**) power to explain aE-driven paradoxical effects of sulpiride on posterior vs. anterior theta activity and 2-back working memory performance we measured the EEG intermittently over 5 h in individuals either extremely high or low in aE who had received either placebo or one of three doses of sulpiride (50, 200, 400 mg). We expected that responses of individuals high and low in aE would follow an inverted U-shaped function and/or that individuals high and low in aE differ systematically in the time course or dominance of sulpiride's pre- and postsynaptic effects.

# **MATERIALS AND METHODS PARTICIPANTS**

To select participants either extremely high or low in aE, we recruited a pool of *N* = 422 male, university or high school student volunteers, to fill in a German short scale of Tellegen's Multidimensional Personality Questionnaire designed to measure aE (see Wacker et al., 2006). In order to obtain greater homogeneity within aE groups the extreme group selection was based on the primary scales: participants scoring above the median in each of the three primary scales constituted the high aE extreme group, whereas participants with scores below or equal to the median in all three primary scales constituted the low aE extreme group. By virtue of this selection procedure the participants of the present study scored either above the top tercile (high aE) or below the bottom tercile (low aE) of the distribution of total aE scores. Preselected participants were further screened for their handedness (inclusion criteria: right-handed) and participants' health status was checked via interview: self-reports of chronic or acute diseases especially cardiovascular or gastrointestinal ailment or functional abnormalities of the liver or the kidney led to rejection from participation, as did habitual smoking of more than ten cigarettes per day, regular use of other drugs, and treatment with prescription drugs in the last 3 months. Furthermore, lifetime absence of psychiatric disorders was ascertained by a brief clinical interview based on DSM-IV criteria. *N* = 88 healthy male participants met the inclusion criteria and finally agreed to participate in the study. The study was approved by the Ethics Committee of the German Society for Psychology (Deutsche Gesellschaft fuer Psychologie) and performed in accordance with the Declaration of Helsinki. All volunteers gave written informed consent and

<sup>2</sup>In the past 20 years numerous studies aligned the dopamine receptor D4 gene and its polymorphisms to personality, althought there is only limited empirical evidence to assume a DRD4—aE or approach-related personality trait relation (for a recent meta-analysis see Munafo et al., 2008).

were paid for participation (80C, approximately \$120). Eight participants were excluded from statistical analysis, because they had less than 30 epochs of artifact-free EEG data due to excessive artifacts (eye and muscle movements; *n* = 5) or due to technical malfunction (*n* = 3) in more than two data recordings. Data are reported from 40 high-aE subjects (mean age = 22.70, *SD* = 2.53; range 19–30) and 40 low-aE subjects (mean age = 23.93, *SD* = 3.06; range 20–31). The participants of each aE extreme group were randomly assigned to either the placebo or one of the three D2 antagonist groups. A full description of the sample is given in **Table 1**.

#### **EXPERIMENTAL DESIGN**

The experimental design was a placebo-controlled, double-blind design defined by the between-subjects factors aE (high, low) and substance (placebo, D2 antagonist sulpiride dosages 50, 200, 400 mg) and the within-subjects factor time since substance (0.5, 1.5, 2.5, 3.5, and 4.5 h after substance intake).

All substances were delivered in capsules, which had the same appearance and were matched for weight to assure that the experimenter and the participant were blind to the pharmacological treatment. Sulpiride is a substituted benzamide derivative, shows high affinity within the nanomolar range to D2 and D3 receptors and a weaker affinity within the micromolar range to D4 receptors (Strange, 2001), and acts predominantly on the MDS (Mauri et al., 1996). Sulpiride appears to lack effects on norepinephrine, acetylcholine, serotonin, histamine, or gamma-aminobutyric acid receptors; it is rather slowly absorbed from the gastrointestinal tract, with peak serum levels occurring within 1–6 h after oral ingestion and elimination half-life is in the range of 3–10 h (Mauri et al., 1996). A major advantage of sulpiride is that adverse side effects are very rare (e.g., McClelland et al., 1990; Meyer-Lindenberg et al., 1997; Wacker et al., 2006). Regarding sulpiride's efficacy, the current literature suggests that low doses (50–150 mg) affect presynaptic autoreceptors producing its antidepressant efficacy, whereas higher doses (>800 mg) induce antipsychotic, postsynaptic D2 receptor effects (Serra et al., 1990). Based on this clinical profile it is assumed that 50 and 200 mg sulpiride could induce both pre- and postsynaptic D2 receptor effects but presynaptic effects predominate (see supportive data in Mueller et al., 2011). Based on the data of Mehta et al. (2008), 400 mg seems to induce stronger striatal occupancy compared to 200 mg sulpiride. In the same vein, a decreased striatal activation to reward seen with 400 mg sulpiride is in keeping with the hypothesis that inhibition of dopamine transmission (via postsynaptic effect) predominates 400 mg sulpiride (McCabe et al., 2011). In addition, the maximal prolactin response to 50 and 200 mg are time shifted (Sugnaux et al., 1983): the response to 50 mg sulpiride occurred 1 h later compared to 200 mg. Thus, the postsynaptic effects dominate later in time for low compared to high doses and consequently, presynaptic effects had to peak earlier in time for high compared to low doses.

#### **TWO-BACK WORKING MEMORY TASKS**

In the present study, we employed the same 2-back paradigm as in Wacker et al. (2006). Participants are presented with a series of stimuli and asked to judge for each item as quickly and accurately as possible whether it matches the stimulus that preceded it by two places in the sequence (2-back task). Participants responded to each letter with their dominant right hand. For each of the five 2 back tasks a pseudorandomized sequence (30% target trials; 70% non-target trials) of 48 practice and 168 evaluated trials was generated. Of the non-target trials, 15% were lure trials, which are 1-back and 3-back repeats included as foils.

As stimuli we used single white letters (Times New Roman, 60 pt) each appearing in the center of a 15-TFT display for


*Notes: Values given as means (standard errors). aE, Main effect agentic Extraversion, F*(*1*, *<sup>72</sup>*)*; aE* × *S: interaction of Trait aE and Substance, F*(*3*, *<sup>72</sup>*)*; EPQ-R, Eysenck Personality Questionnaire Revised (E, Extraversion; P, Psychoticism; N, Neuroticism); MAE, Marburg Agentic Extraversion Scale; MPQ NE, Multidimensional Personality Questionnaire–Negative Emotionality Scale; ZKPQ, Zuckerman–Kuhlman Personality Questionnaire (Act, activity; AH, aggression-hostility; ImpSS, impulsive sensation seeking; NA, neuroticism-anxiety; Soc, sociability); WMC, working memory capacity as measured by the automated span task; CFT, general fluid intelligence as measured by the short version of the Culture Fair Test (Scale 3). \*p* <sup>≤</sup> <sup>0</sup>.05*, \*\*\*p* <sup>≤</sup> <sup>0</sup>.001*, two-tailed.*

500 ms followed by a blank, black screen for another 1650 ms. Participants were expected to respond during this 2150 ms interval. The end of each trial was marked by a 350-ms auditory feedback, notifying whether the preceding reaction was correct and fast enough ("correct," "incorrect," "too slow"). "Too slow" referred to a correct reaction that was slower than a latency criterion, which was defined as the 90th percentile of the individual reaction time distribution for correct reactions during the practice trials of each 2-back task. For the computation of the individual percentiles, reaction times longer than three standard deviations above the individual mean were excluded. A new trial started right after the trial feedback (ITI = 0 ms; ISI = 2500 ms). For each of the five 2-back task presentations, the following performance measures were calculated: (a) the mean reaction time for correct reactions to targets, (b) the percentage of correct reactions to targets, and (c) the variability of reaction times for all correct reactions. For statistical analysis the reaction times and variability were square root transformed to normalize distributions. In order to control for unspecific attentional substance effects, we also introduced a 0-back task. The set-up was identical to the 2-back task, but participants were asked to indicate whether the present letter was a "q" or not. For lure trials we used 1-back and 2-back repeats as foils. It should be noted that Wacker et al. (2006) did not report significant effects for the 0-back task.

# **INTELLIGENCE TEST, WORKING MEMORY CAPACITY, AND PERSONALITY QUESTIONNAIRES**

The participants completed the short version of Cattell's Culture Fair Test Scale 3 (CFT; Cattell and Weiß, 1971) in order to control for fluid intelligence as a possible confound and the automated version of the operation span task (Unsworth et al., 2005) in order to control for working memory capacity, which has already been shown to produce paradoxical effects with respect to the D2-receptor agonist bromocriptine (Kimberg et al., 1997). In addition, participants completed the German versions of the Zuckerman-Kuhlman Personality Questionnaire (ZKPQ; Zuckerman, 2002), the Eysenck Personality Questionnaire-Revised (EPQ-R; Ruch, 1999), and the MPQ-Negative Emotionality Scale (Tellegen and Waller, 2008). EPQ-R measures Eysenck's personality traits of E, Neuroticism, and Psychoticism. The ZKPQ measures Zuckerman's "Alternative Big Five," Aggression-Hostility, Neuroticism-Anxiety, Sociability, Activity, and Impulsive Sensation Seeking.

### **SETTING AND APPARATUS**

The experiment was conducted in two adjacent rooms. The experimental room (4 × 3.4 m) had no windows, was air-conditioned (22◦C), sound-attenuated, and had a largely non-technical appearance. Participants sat comfortably in a reclined position. A 15-TFT monitor (Natcomp, Bad Homburg, Germany) and a response box (XQMS, Frankfurt, Germany) were placed in front of the participants. Electrodes were connected to a customized head box (Neuroscan, Sterling, VA), where EEG and electrooculogram (EOG) signals were preamplified with a gain of 30 (input impedance 10 M-). The adjacent room contained a 32-channel SynAmps 5083 amplifier (Neuroscan, Sterling, USA) and the technical equipment for experimental control and data acquisition. A Power Macintosh G4 performed data recording, data visualization, and data storage using LabView 5.0 software (National Instruments, Austin, USA). An IBM-compatible computer running Presentation 10.3 (Neurobehavioral Systems, Albany, USA) displayed stimuli and delivered prerecorded instructions.

# **PROCEDURE**

The experiment was conducted in two separate sessions. In Session 1 the experimenter conducted a standardized clinical interview in order to check for lifetime absence of DSM-Axis I psychiatric disorders. Then participants completed the automated span task, the CFT and the personality questionnaires. Finally, they were trained on a attention control task and 2-back working memory to reduce potential practice effects for pharmaco-session (Wesnes and Pincock, 2002), in which EEG was recorded.

During Session 2 (starting at 8 a.m.; on average 1.5 days after session 1; range 1–9 days) the experimenter first conducted a semi-standardized interview to check protocol compliance (i.e., fasting, sleep duration and abstinence from alcohol, cigarettes, and caffeine for the last 12 h), and then positioned electrodes and explained emotion self-reports. The experimenter reminded the participants to sit quietly to help prevent artifacts in the EEG recordings and participants were told to relax with their eyes opened for a 10-min rest period with five embedded 1-min recordings. At the end of the rest period, participants filled in several self-reports on their current mood (Wacker et al., 2006). Participants then received either placebo or sulpiride together with a light breakfast. Thirty minutes after breakfast and substance intake, the experimental session started. It consisted of five blocks, with each of these blocks following the same set-up: first a 4-min rest period at which the EEG data reported here were recorded, then a 0-back attentional control task, next a 2 back working memory task to obtain behavioral measures for dopamine dependent cognitive processes, a 4 min post-task waiting phase terminated by a performance feedback, and finally a 30 min recreation period. A post-experimental semi-standardized interview concluded the experiment about 5.5 h after medication.

### **DATA ACQUISITION, RECORDING, AND ANALYSIS**

Vertical and horizontal electrooculogram (EOG) was recorded from four electrodes. EEG was recorded from 29 Ag/AgCl sintered ring electrodes (impedances <5 kOhm for EEG, <1 kOhm for the ground electrode AFZ; <10 kOhm for EOG) positioned in accordance with the International 10–20 system (Jasper, 1958) using an elastic electrode cap (Easy Caps, Germany). All sites were online referenced to Cz. EEG and EOG signals were amplified with a 32-channel SynAmps 5083 amplifier (EEG: gain 500; EOG: gain 100; input impedance 10 MOhm), digitally filtered (bandpass 1–50 Hz for EEG; lowpass 1 kHz for EOG; 50 Hz Notch filter) and stored (sampling rate: 2 kHz). Then signals were down-sampled to 250 Hz and converted to physical units. Subsequent pre-processing was carried out using BrainVision Analyzer 2 (Brain Products, Munich, Germany) and EEGLAB (Delorme and Makeig, 2004). Low-pass filters were located at 30 Hz and high-pass filters at 1 Hz. After visual rejection of data portions containing non-stereotyped artifacts (e.g., large muscle artifacts, swallowing, cable movement, etc.), concatenated EEG data were submitted to extended infomax-independent component analysis. Independent components reflecting eye blinks, lateral eye movements, line noise, and heartbeat pulses were identified visually and discarded by back-projecting all but these components to the data space. Corrupted channels flagged as artifact-contaminated for more than 1/4 of the recording were estimated using spherical spline interpolation (Perrin et al., 1989). In 2.13% of the data recordings Fz or Pz needed interpolation. Overall 2.25 % of the recorded channels were interpolated. Data portions and recordings with more than two corrupted channels were discarded. Next, all data epochs of 2.048 s were once again semi-automatically screened for artifacts.

All artifact free epochs were referenced to average electrodes and submitted to a Fast Fourier Transform (50% Hammingwindowed, padded symmetrically with zeros up to 1000 data points). The resulting estimates of power density (μV2/Hz; resolution 0.25 Hz) were clustered into theta (4.00–7.75 Hz) and delta (1.00–3.75 Hz) frequency bands both of which were shown to be sensitive to aE-related baseline/resting differences in posterior vs. anterior power distribution (Chavanon et al., 2011) and sulpiride (Wacker et al., 2006). Since the pattern of results for delta frequency data was almost identical to the theta pattern, we decided to restrict the presentation of results to the latter. As studies by Knyazev and colleagues (Knyazev, 2009, 2010; Knyazev et al., 2012) reported aE-related differences in posterior vs. anterior activity for higher frequency bands, we inspected broad alpha (8–12.75 Hz) and beta (13–29.75 Hz) frequency bands. All effects of interest (i.e., Substance, Substance × Time, Substance × Trait aE, or Substance × Trait aE × Time) were non-significant for both higher frequency bands, all *p*s > 0.5.

Power values were normalized by logarithmic transformation before statistical testing (see e.g., Davidson et al., 2000). The posterior vs. anterior EEG index was computed separately for each band as ln-transformed power at Pz minus ln-transformed power at Fz. In order to obtain reliable data recordings, only those with more than 30 artifact-free epochs (approximately 1 min) were kept (1.25% missing data). On average, EEG-analyses were based on 71.07 artifact-free epochs (*SD* = 19.44, range = 31– 121) for post substance periods and 76.19 artifact-free epochs for the initial pre-drug baseline (*SD* = 22.18, range = 39–144). The number of artifact-free epochs was not associated with the Pz-Fz score, average correlations across data recordings *r*(79) = −0.05. To control for individual baseline differences, the main statistical analysis was performed on reactivity scores computed by subtracting the pre-drug baseline. Prior to this subtraction, we ran a 2 × 4 ANOVA with Trait aE (2; high, low) and Substance (4; Placebo, 50, 200, 400 mg sulpiride) as group factors. Regarding posterior vs. anterior activity in theta band, there was neither significant effect of Substance, *F*(3, <sup>72</sup>) = 0.74, *p* = 0.53, nor an interaction effect of Substance × Trait aE, *F*(3, <sup>72</sup>) = 0.56, *p* = 0.65, for the initial, pre-drug resting period. However, as reported in detail in Chavanon et al. (2011), there were strong baseline differences between high and low aE in posterior vs. anterior theta activity, *F*(1, <sup>72</sup>) = 40.90, *p* < 0.0001, *d* = 1.51. High aE subjects showed more posteriorly located theta activity, whereas

low aE depicted a more frontal pattern (Wacker et al., 2006, 2010). Posterior vs. anterior theta reactivity data was checked for normality prior to analysis using the Shapiro-Wilk test.

#### **STATISTICAL DATA ANALYSIS**

For all dependent variables a 2 × 4× 5 repeated measures ANOVA with Trait aE (2; high, low) and Substance (4; Placebo, 50, 200, 400 mg sulpiride) as group factors and Time (5; 0.5, 1.5, 2.5, 3.5, 4.5 h since substance) as repeated factor was fitted in SAS/STAT (SAS Institute Inc., 1997) PROC MIXED. The error variance covariance matrix was specified as completely general.

Significant ANOVA interactions were followed by a priorispecified contrasts tested with an α-level of 0.05, two-tailed. Contrasts for the posterior vs. anterior EEG theta reactivity scores were specified for three different interactions. These contrasts depict (a) the Substance × Time interaction regarding substance effects (placebo vs. sulpiride group) over time, (b) the Substance × Trait aE interaction focusing on (b1) substance effects (placebo vs. sulpiride group) within aE groups and (b2) dose-response relations within aE groups receiving sulpiride (a priori specified orthogonal contrasts for unequally spaced linear and quadratic dose-responses across 50, 200, and 400 mg), and finally (c) substance effects (placebo vs. sulpiride group) within aE groups over time (c1) to identify the first significant substance effect and (c2) to characterize time courses with orthogonal polynomial trends (linear, quadratic, cubic). Effect sizes (*r*contrast) for those latter repeated measures contrasts on temporal patterns (i.e., c2) were computed according to Furr and Rosenthal (2003); otherwise, Cohen's d was calculated.

## **RESULTS**

#### **PHARMACOLOGICAL SIDE-EFFECTS AND BLINDNESS TO SUBSTANCE GROUPS**

Participants did not report any adverse side effects. The ratings of nausea and dizziness averaged across experimental phases were very low (<1 on a 9-point scale with 0 = not at all applicable, 1 = not applicable) in all eight experimental groups. The percentage of participants, who guessed in a forced choice question in the post-experimental interview that they had received a pharmacologically active substance, did not differ between the substance groups [placebo: 40%, 400 mg sulpiride: 25%, 200 mg sulpiride: 30%, 50 mg sulpiride: 40%; *Chi*2(3) <sup>=</sup> <sup>1</sup>.51, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>.68]. When asked to evaluate the confidence in their guess, none of the participants reported to be 100% sure (*M* = 66%, *SD* = 19%). Thus, it can be concluded that the participants were blind to the experimental condition as intended.

#### **CHANGES IN POSTERIOR vs. ANTERIOR ACTIVITY**

A significant main effect for Trait aE indicated that low compared to high aE showed a shift toward more posterior vs. anterior EEG theta [*F*(1, <sup>72</sup>) = 52.85, *p* < 0.0001, *M* = 0.076 and *M* = −0.069, *SEM* = 0.014 for low and high aE]. Furthermore, we observed a significant main effect of Time [*F*(4, <sup>72</sup>) = 9.95, *p* < 0.0001], which was best described by a linear trend toward more anterior vs. posterior theta across time [*t*(72) = −5.65, *p* < 0.001; quadratic and cubic trends were non-significant, *p*s > 0.08]. The significant interaction effect of Trait aE × Time [*F*(12, <sup>72</sup>) = 6.73, *p* < 0.0001] could be traced back to diametrically opposed quadratic trends [*t*(72) = −5.58, *p* < 0.0001]. However, these effects were further qualified by significant higher order interaction effects (see below).

A Substance main effect [*F*(3, <sup>72</sup>) = 19.23, *p* < 0.0001] revealed a linear dose response with 50 mg sulpiride inducing a shift toward stronger posterior theta and 400 mg inducing a shift toward a stronger anterior theta [*t*(72) = 3.89, *p* < 0.0025, quadratic ns]. In addition, the expected interaction of Substance × Time was observed [*F*(12, <sup>72</sup>) = 6.73, *p* < 0.0001]. This interaction effect was due to stronger quadratic trends over time for sulpiride groups compared to placebo [*t*(72) ≥ |1.78|, *p* ≤ 0.01], with an opposite direction for 50 mg sulpiride compared to 200 [*t*(72) = 4.06, *p* = 0.0001] and 400 mg, [*t*(72) = 4.48, *p* < 0.0001].

Most importantly, the predicted interactions of Substance × Trait aE [*F*(3, <sup>72</sup>) = 18.81, *p* < 0.0001] and Substance × Trait aE × Time [*F*(12, <sup>72</sup>) = 3.74, *p* < 0.001], were also highly significant. These expected aE based modulations of sulpiride effects were subsequently probed by a priori contrasts.

#### *A priori contrasts*

*Substance* **×** *Trait aE.* The tests of the central a priori contrasts are documented in **Table 2**. The corresponding means (and *SEM*s) are shown in **Figure 3**. High aE participants, who had received 200 and 400 mg sulpiride, exhibited a significant shift toward more anterior vs. posterior theta distribution compared to placebo (*d* = 0.60 for 200 mg and *d* = 1.30 for 400 mg) and the opposite was true for low aE (*d* = −1.39 for 200 mg and *d* = −1.01 for 400 mg). Compared to placebo the lowest dose of 50 mg sulpiride resulted in changes toward more posterior theta that were highly significant in low aE (*d* = −1.67), but non-significant in high aE (*d* = −0.35). Dose-response analyses using orthogonal polynomials for unequally spaced factor levels revealed that linear dose-responses were stronger for high aE than for low aE [*t*(72) = 2.89, *p* = 0.005, *d* = 0.68]. For quadratic trends all contrasts were non-significant (*p*s > 0.25).

*Substance***×** *Trait aE* **×** *Time.* As expected neither placebo group (high or low aE) showed any significant trends across time (all *t*(72) values ≤ |0.95|, *p*s ≥ 0.35). **Figure 4** displays the differential time courses of reactivity scores observed for high and low aE

**Table 2 | Substance effects within and between high and low aE in posterior vs. anterior theta reactivity:** *t***-values of contrasts (effect size** *d***).**


*Notes: aE, agentic Extraversion; 50, 200, and 400 refer to the groups that received 50 mg, 200 mg, or 400 mg sulpiride. N* <sup>=</sup> <sup>80</sup>*. df* <sup>=</sup> <sup>72</sup>*, \*p* <sup>=</sup> <sup>0</sup>.05*, \*\*\*p* <sup>≤</sup> <sup>0</sup>.001*, two-tailed.*

within placebo and sulpiride groups. The associated a priori contrasts are provided in **Table 3**. All aE groups that received sulpiride exhibited significant substance effects. All those effects—except for high aE 50 mg sulpiride—remained significant for at least three consecutive recording times and, thus, lasted for at least 2 h (see **Table 3**).

Under 50 mg, high aE significantly differed from their placebo control group as early as 0.5 h after intake. Notably, for all high aE groups the first response to sulpiride was a shift toward posterior theta activity, although this shift was not significant for 200 and 400 mg sulpiride. Contrasts for high aE participants further revealed that compared to placebo a first statistically reliable response to 200 and 400 mg sulpiride occurred after 1.5 h. While for 200 mg the substance-induced shift toward anterior theta lasted for about 2 h (1.5–3.5 h after intake), it lasted 3 h for 400 mg (1.5–4.5 h after intake).

Substance effects occurred earlier in time for low than high aE: half an hour after substance intake there was a reliable shift toward posterior theta in all sulpiride groups. While for 400 mg this effect lasted approximately 2 h, substance effects of 200 mg and 50 mg were significant for 3 h.

Maximal posteriorization response to 50 and 200 mg sulpiride was delayed compared to 400 mg in low aE (2.5 vs. 1.5 h for 50/200 and 400 mg, respectively). It should be noted that the linear dose-response pattern for low aE changed: whereas from 0.5 to 1.5 h after intake 400 mg induced stronger effects than 50 mg, this was reversed at 2.5 and 3.5 h. For high aE maximal anteriorization responses to 400 mg and 200 mg occurred 2.5 h after intake. The linear dose-response pattern (400 mg > 200 mg > 50 mg) remained stable from 2.5 h on.

Characterizing time courses by polynomial trends revealed that high and low aE depicted opposing quadratic time components, and this was true for all sulpiride groups (see *t*-values in **Table 3**). High aE showed an increase in *anterior* vs.

**FIGURE 3 | Disordinal interactions of aE × Substance on posterior vs. anterior theta reactivity (***M* **and** *SEM* **values) contrasting high and low aE groups which received placebo (PLC) with each of the three different doses of sulpiride (SLP; 50, 200, or 400 mg).**

posterior theta followed by a decrease, whereas low aE exhibited an increase in *posterior* vs. anterior theta followed by a decrease. Within the 50 mg sulpiride groups, low aE additionally showed a cubic component that was mainly due to a sharp rise to posterior theta at 2.5 h and a significant reduction in posterior theta at 4.5 h (see **Figure 4**), whereas nonlinear trends for high aE were not significant. After 400 mg, both low and high aE groups depicted an additional linear trend over time.

#### **WORKING MEMORY PERFORMANCE: 2-BACK TASK**

Neither reaction time for correct target responses nor percentage of correct target responses in the five 2-back tasks showed any effect related to Substance or aE. In contrast to the percentage of correct target responses, for which no effects were observed, reaction times speeded up with each hourly task block [*F*(4, <sup>72</sup>) = 21.25, *p* ≤ 0.0001].

For reaction time variability we observed an main effect of Time [*F*(4, <sup>72</sup>) = 4.61, *p* ≤ 0.003], which was described by a cubic trend across time [*t*(72) = 3.28, *p* < 0.01, all other trends *p*s > 0.07]. Furthermore, a significant effect of Substance [*F*(3, <sup>72</sup>) = 3.28, *p* = 0.026] was revealed. Variability was lower under placebo compared to all doses of sulpiride [*t*(72) values > 2.23, *p*s < 0.03], while there were no significant differences between the three sulpiride dosages [*t*(72) values ≤ |0.49|, *p*s ≥ 0.62; average variability in ms (SD): placebo 80 (26), 50 mg sulpiride 103 (34), 200 mg sulpiride 97 (30), 400 mg sulpiride 101 (36)].

Controlling for attentional effects in the three performance measures as measured with the 0-back task (entered as repeated measures covariate, see Winer, 1971) did not change the pattern of results.

Grand means for the performance measures are given in **Table 4** for each hour. Note that even in the first task block despite a comparable percentage of correct target responses, both reaction time and reaction time variability were considerably lower than in the previous study by Wacker et al. (2006), possibly due to the practice session on a separate day introduced in the present work [reaction time (*SD*) : 444 ms (100) vs. 600 ms (149), reaction time variability (SD): 98 (31) ms vs. 157 ms (53), correct target responses (SD): 74% (13.3) vs. 72% (17) for present vs. Wacker et al., 2006, respectively].

#### **SPECIFICITY TO aE**

To check whether the effects of sulpiride on posterior vs. anterior EEG theta activity were modulated by other (correlated) personality traits (either EPQ-R-neuroticism, ZKPQ-aggression/hostility, ZKPQ-impulsive sensation seeking, MPQ-negative emotionality), age, weight, general fluid intelligence or working memory capacity, we calculated a series of ANCOVAs using the statistical model described above, but


 *high size Cohen's d for simple Substance and Substance* × *aE effects at each time point (between subjects). size*

#### *andlowaE)andSubstance*×*aEeffects(tetraddifferences;i.e.,differenceoftheeffectsofeachsulpiridedosevs.placeboinhighvs.lowaE)atrcontrastfor trends across time (linear, quadratic, and cubic pattern; repeated measures). \*p*≤0.05*, \*\*p*≤0.01*, \*\*\*p*≤0.001*, two-tailed.*

**Table 4 | Grand means (SD) of the performance measures in 2-back task for each hour since substance intake.**


now entering in turn each variable as an additional covariate, its two-way interaction with Substance, its two-way interaction with Time and its three-way interaction with Substance and Time. The results of these supplementary analyses revealed that the interactions Substance × Trait aE and Substance × Trait aE × Time remained significant [Trait aE × Substance: *F*(3, <sup>68</sup>) ≥ 10.78, *p* ≤ 0.0001; Trait aE × Substance × Time: *F*(12, <sup>68</sup>) ≥ 2.31, *p* ≤ 0.015], indicating that the effects reported above are indeed specific to aE.

# **DISCUSSION**

The present study focused on paradoxical dopaminergic effects and confirmed that the effect of sulpiride on posterior vs. anterior theta activity strongly depends on aE. Low aE showed more frontally distributed theta than high aE, and under 200 and 400 mg sulpiride this difference was reversed: high aE showed a shift toward anterior theta, but low aE, a shift toward posterior theta. Furthermore, we found marked aE-related response differences across time. Thus, the present findings support the basic idea that besides general responses to pharmacological agents and static models like an inverted U-function, time aspects of pharmacological effects contain valuable information regarding the biological basis of Extraversion. While EEG theta activity proved sensitive to the paradoxical effects of sulpiride on high and low aE, such effects could not be detected for 2-back working memory performance. Based on the present findings we will discuss in detail differential pre- and postsynaptic responses in high and low E as one possible explanatory mechanism after briefly refreshing the most important features of sulpiride's pharmacodynamic profile.

#### **THE PHARMACODYNAMICS OF SULPIRIDE DOSES**

Sulpiride shows high affinity within the nanomolar range to D2 and D3 receptors and a weaker affinity within the micromolar range to D4 receptors (Strange, 2001), and acts predominantly on the MDS (Mauri et al., 1996). Regarding sulpiride's biphasic action and clinical efficacy, the literature suggests that low doses (50–150 mg) affect presynaptic D2/D3-autoreceptors (see review by Rankin et al., 2009) producing its antidepressant efficacy, whereas higher doses (>800 mg) induce antipsychotic, postsynaptic D2 receptor effects (Westerink and Devries, 1989; Serra et al., 1990; Kuroki et al., 1999). Based on this clinical profile it is assumed that 50 and 200 mg sulpiride used here could induce both pre- and postsynaptic D2 receptor effects but presynaptic effects predominate (Mueller et al., 2011). Furthermore, a dose of

*a Effect* *b Effect* 

400 mg induced stronger striatal occupancy compared to 200 mg (Mehta et al., 2008) and produced a marked decrease in striatal activation to reward (McCabe et al., 2011). These data suggest that the inhibition of dopamine transmission (via postsynaptic effects) predominates the effects of 400 mg sulpiride. In a nutshell, there is a dose-dependent biphasic action that relates to the balance of pre- to postsynaptic effects (50 mg > 200 mg presynaptic vs. 400 mg postsynaptic predominance). In addition, pharmacokinetic data showed that within the tuberoinfundibular system the maximal prolactin response to 50 and 200 mg are time shifted (Sugnaux et al., 1983): the response to 50 mg sulpiride occurrs1 h later compared to 200 mg. Thus, postsynaptic effects rush in or dominate later in time for low compared to high doses. Hence one could expect that in the present study 400 mg reach the plasma levels for pre- and postsynaptic effects in the MDS earlier in time compared to 200 mg and the postsynaptic effects to 50 mg -if at all- will be observed last.

#### **EVALUATION OF INVERTED U-SHAPED MODEL (FIGURE 1)**

This model assumes that dopamine and posterior vs. anterior theta are linked by an inverted U-shaped function and that equal doses of sulpiride influence dopamine levels in the same, commensurate direction for high and low aE. This implies that all observed effects are necessarily presynaptic, increasing dopamine levels. Thus, the model would make the following prediction for the present data: low aE are typically located on the low dopamine left side of the inverted U and sulpiride shifts them up the ascending limb through presynaptic blockade. The same mechanism pushes high aE up to the top of the curve and beyond (descending limb). When focusing the time points where substance effects were most pronounced (0.5–3.5 h; see **Figure 5**), this predicition fits the empirical data, although the position of high aE for the 50 mg dose is ambiguous and the size of shifts differ between high and low aE for the 50 mg dose. In addition, this model focuses on the interaction effect of Trait aE and Substance, and hence, it cannot explain any of the effects qualified by time.

#### **EVALUATION OF ALTERNATIVE EXPLANATORY MODELS (FIGURE 2)**

Model A assumes that sulpiride produces pre- (50 mg) or preand postsynaptic (200, 400 mg) responses of similar magnitude in low and high aE groups that are, however, shifted in time. Applied to the present data model A would suggest that more posterior vs. anterior theta directly tracks dopamine levels (posterior shift = presynaptic blockade, dopamine increase; anterior shift = postsynaptic blockade, dopamine decrease). In high aE the presynaptic effects are visible under 50 mg at 0.5 h. For higher doses significant presynaptic effects should have appeared before 0.5 h. Postsynaptic effects for both higher doses started as early as 1.5 h. Conversely, in low aE only presynaptic effects were evident and enduring. In neither group postsynaptic net effects were found within 4.5 h. Although model A can principally account for the findings, it is necessary to assume that significant portions of the responses occurred within the first 30 min for high aE (missing parietalization) and after 4.5 h for low aE (missing anteriorization). These time points were not included in data sampling here. Given that serum levels of sulpiride have been reported to peak

**FIGURE 5 | Projection of the present Substance × aE interaction effect onto an inverted U-shaped relationship between posterior vs. anterior theta activity and dopamine level (Figure 1).** Postulating such an inverted U-shaped function implies that sulpiride (SLP) acts presynaptically. In order to facilitate interpretation we used mean values based on posterior vs. anterior theta activity across 0.5–3.5 h (i.e., time range of maximal substance effects). Thus, by increasing dopamine level through presynaptic sulpiride action, the typical posterior vs. anterior theta activity values of high and low aE observed under placebo (circles) are shifted to different locations on this function (squares). For high aE under 50 mg sulpiride, the position on the inverted U-shaped function could either be on the left or the right arm (indicated by asterisks). The arrows indicate aE-based differences in the magnitude of presynaptic action (*x*-axis) to 50 mg, which must be assumed—in either case—in order to accommodate the observed Substance × aE interaction pattern within the inverted U-shape model.

within a widely varying interval (1–6 h), a direct test of these assumptions in future studies seems warranted in order not to dismiss the model prematurely. However, the observed differences in response magnitude for high and low aE are not covered by the model exclusively assuming a aE-related time shift in responses.

Model B holds that in high aE postsynaptic effects should dominate at least for the two higher doses of sulpiride, whereas for low aE net presynaptic effects should be observable for all doses, but particularly for the 50 mg dose. In high aE 50 mg should generate at least a small presynaptic response. Once again assuming that posterior vs. anterior theta directly tracks dopamine levels the observed pattern closely matches these predictions: low aE only showed presynaptic effects peaking earlier for the higher dose than for the lower ones, whereas high aE primarily showed postsynaptic effects for the two higher doses, with 400 mg peaking later than the presynaptic effects observed for low aE. In high aE we obviously observed an initial presynaptic net effect after 0.5 h for 50 mg, but presynaptic effects for 200 and 400 mg were non-significant. An explanation for this pattern in high aE might be that even under 50 mg presynaptic effects are opposed (but never outweighed) by postsynaptic effects in the time interval around 1–4.5 h during which peak sulpiride plasma levels most likely occur. Alternatively, the lack of evidence for more enduring net presynaptic effects in high aE under 50 mg may be due to ceiling effects in our EEG measure (i.e., high aE may have already demonstrated a maximally posterior distribution of theta activity under placebo).

It should be noted that aE-related differences in the D2 like subreceptors DRD2, DRD3, and DRD4 might account for this pattern. For example, a simple aE-related difference in postsynaptic D4 receptors might lead to aE × Substance × Time interaction because sulpiride's pharmacological profile expands across time and additionally stimulates D4 receptors: if low aE have less DRD4 receptors than high aE they would -in contrast to high aE- show less DRD4 related postsynaptic effects that typically rush in when sulpiride reaches micromolar concentrations. Taken together, model A can accommodate some of the observations whereas model B can explain the complete pattern of findings although the precise contribution of D2-like receptors is not elucidated with the present research.

#### **INTEGRATION OF RECENT RESEARCH**

Investigating the posterior vs. anterior EEG theta activity with polymorphisms related to dopamine D2 receptor functioning, a recent study showed that SNP19 rs1076560, which is implicated in the regulation of two isoforms of the DRD2 receptor (Zhang et al., 2007), and -141C Ins/Del rs1799732, which has been associated with altered expression of the DRD2 in the striatum, were significantly associated with posterior vs. anterior EEG delta/theta activity (Köhler et al., 2011). Particularly, the SNP19 rs1076560 polymorphism might be relevant to the present data and their interpretation, because this polymorphism is associated with relative expression of the DRD2 long isoform (D2L), which is mainly postsynaptic and the DRD2 short isoform (D2S), which is mainly presynaptic and serves as an autoreceptor regulating dopamine synthesis and release (Usiello et al., 2000) in the frontal cortex. Furthermore, D2S receptors are the most abundant autoreceptor subtype in the midbrain (Khan et al., 1998) and provide potent inhibition of dopamine release. However, the SNP19 rs1076560 T allele shifts splicing from short to long receptors, decreasing the D2S/D2L ratio relative to the G allele and therefore the T allele is associated with putatively greater levels of midbrain dopamine. Köhler et al. (2011) reported that the T allele compared to the G allele was associated with less posterior vs. anterior EEG delta/theta activity and carriers had numerically lower scores in Extraversion. Combining those findings and the hypothetic principles of tonic and phasic dopaminergic activity (Grace, 1991) would lead to the following prediction: low aE might more frequently be carriers of the T-allele and consequently have higher tonic midbrain dopamine levels. High dopamine levels result in a lower (phasic) responsivity of postsynaptic receptors, leading sulpiride's presynaptic effects to prevail. For high aE the lower dopamine level results in higher responsivity of postsynaptic receptors, leading to sulpiride's postsynaptic effects to prevail. This is exactly what we found in the present study. Combining the present pharmacological design with the genetic approach used in Köhler et al. (2011) could provide a direct test of this model.

Regarding the functional significance of the posterior vs. anterior theta measure, there are some aspects we would like to point out. As anterior theta is generated in ACC (Ishii et al., 1999), we recently probed the ACC as a potential source of posterior vs. anterior theta and found that especially theta in the rostral portion (rACC) was strongly associated with low values in our EEG measure (Chavanon et al., 2011). In line with the present results, ACC is known to respond to dopaminergic challenges (Vollm et al., 2004). Interestingly, high levels of inhibitory rACC delta/theta activity (i.e., presumably low ACC activity) have been associated with both blunted nucleus accumbens reward responses and anhedonia, i.e., reward-insensitive behavior and blunted positive emotionality or, arguably, extremely low aE (Wacker et al., 2009). Furthermore, ACC activity predicts the psychopharmacological treatment response in depressive patients (Korb et al., 2009). Thus, low aE individuals may have demonstrated a sulpiride-induced "antidepressive" reaction in rACC mirrored by posterior vs. anterior theta. Pizzagalli (2011) recently argued that the rACC plays a key role in treatment outcomes due to its prominent position in the default network. He related the antidepressive rACC response to adaptive self-referential processing which parallels our suggestion that posterior vs. anterior theta (and low inhibitory rACC theta) is positively associated with optimistic future-oriented mentation about one's self and personally significant issues (Chavanon et al., 2011). However, data by Knyazev and colleagues (Knyazev, 2012, 2013; Knyazev et al., 2012) and unpublished data from our group supported the idea that aE is associated with higher theta activity in the default mode network.

Regarding the posterior component of the posterior vs. anterior theta index, Chavanon et al. (2011) reported that inferior parietal and insular cortex were negatively associated with aE. Those results converge with a recent study showing that the insula is inversely related to the willingness to work for reward (Treadway et al., 2012), which is a major facet of aE (i.e., persistent reward striving). Because the insula receives dopaminergic innervation (Gaspar et al., 1989) and expresses D1-like and as well as to a lesser extent D2-like receptors (Hurd et al., 2001), it can be speculated that—in addition to the rACC—the insula might have contributed to the results presented here. Other structures which might have contributed could be the inferior parietal cortex, precuneus and posterior cingulate which were a) functionally connected to the striatum under resting conditions (Di Martino et al., 2008) and b) recently linked to extraversion (Knyazev, 2013). However, it should be kept in mind that based on its neuroanatomy, the dopaminergic system exerts its influence more strongly on frontal brain structures than on posterior brain structures (Cools and D'Esposito, 2011). Thus, compared to Chavanon et al. (2011), the present data presumably rely more heavily on the anterior component of the theta index due to the pharmacological manipulation of the MDS.

### **LIMITATIONS**

The following caveats should be noted: (1) The present study was conducted with male participants and thus leaves generalizability to women open. (2) The assessment of sulpiride effects was limited to posterior vs. anterior theta activity and 2-back working memory performance. In contrast to Wacker et al. (2006) who reported diametrically opposite effects on multiple levels, the present paradoxical effects were restricted to the EEG measure. Unfortunately, we cannot explain the lack of effects on working memory performance. Because other biological indicators such as plasma dopamine levels were not assessed, a validation of the EEG measure with other dopamine biomarkers or a dopmamine-related cognitive phenotype is missing here. (3) Although sulpiride is a highly selective dopamine D2 receptor antagonist, we cannot rule out that the effects observed are due to interactions with other neurotransmitter systems rather than purely dopaminergic. (4) Furthermore, we cannot rule out that there were substance effects before our initial measurement at 30 min and after the final measurement at 4.5 h. Thus, a definitive conclusion concerning the time-course model requires a study with an even more extended recording interval. (5) The exact contributions of D2S, D2L, D3, and D4 subreceptors could not be disentangled in the present study. (6) Without including molecular genetic indicators of functional dopaminergic properties (e.g., polymorphisms related to densities of pre-to postsynaptic D2 receptors; Zhang et al., 2007), it remains a data-based, plausible hypothesis to assume differential pre- to postsynaptic differences in high vs. low aE. Further pharamcogenetic studies including different substances (e.g., selective D2/D3 agonists/antagnonists, D4 antagonists) may help to elucidate and refine the model proposed here.

#### **CONCLUSIONS**

Using resting posterior vs. anterior theta activity, we demonstrated that sulpiride's effects play out differently for individuals

#### **REFERENCES**


personality psychology, convergence with theory in other areas, and potential for integration. *Pers. Soc. Psychol. Rev.* 9, 312–333.


high and low in aE. Whereas the present findings cannot fully rule out that these differences are exclusively due to shifts in the time course of the drug responses, a more parsimonious model holds that low aE individuals are more sensitive to presynaptic, and high aE to postsynaptic sulpiride effects. These data not only add to the still limited evidence for a dopaminergic basis of aE, but also help to generate new hypotheses on the neurobiological mechanisms underlying the frequently observed paradoxical effects of dopaminergic drugs: pre- and postsynaptic reactivity depends on personality-correlated baseline dopamine levels. This factor contributes to the variability in the EEG-effects and possibly to the clinical efficacy of dopaminergic drugs. Future research may probe these suggestions and investigate the molecular basis of individual differences in pre- vs. postsynaptic dopamine D2 subreceptor densities and sensitivities.

# **ACKNOWLEDGMENTS**

This work was supported by Deutsche Forschungsgemeinschaft Grant Ste 405/9-3. The authors thank all participants and student research assistants involved in this study.


*of Psychophysiology*, eds J. T. Cacioppo, L. G. Tassinary, and G. G. Bernston (Cambridge: Cambridge University Press), 27–51.


and methylphenidate on visual attention capacity: a TVA-based study. *Psychopharmacology* 210, 317–329.


factors moderate subjective and psychophysiological responses to d-amphetamine in humans. *Exp. Clin. Psychopharmacol.* 7, 493–501.


component analysis. *Biol. Psychol.* 89, 515–524.


T. W. (2004). Impaired set-shifting and dissociable effects on tests of spatial working memory following the dopamine D2 receptor antagonist sulpiride in human volunteers. *Psychopharmacology (Berl.)* 176, 331–342.


*Neuropsychopharmacology* 36, 183–206.


automated version of the operation span task. *Behav. Res. Methods* 37, 498–505.


dopamine release – brain dialysis provides direct evidence for mediation by autoreceptors localized on nerve-terminals. *Neurosci. Lett.* 99, 197–202.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 13 December 2012; paper pending published: 29 January 2013; accepted: 16 March 2013; published online: 08 April 2013.*

*Citation: Chavanon M-L, Wacker J and Stemmler G (2013) Paradoxical dopaminergic drug effects in extraversion: dose- and time-dependent effects of sulpiride on EEG theta activity. Front. Hum. Neurosci. 7:117. doi: 10.3389/ fnhum.2013.00117*

*Copyright © 2013 Chavanon, Wacker and Stemmler. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

# Individual differences in reward prediction error: contrasting relations between feedback-related negativity and trait measures of reward sensitivity, impulsivity and extraversion

#### **Andrew J. Cooper <sup>1</sup>\*, Éilish Duke<sup>1</sup> , Alan D. Pickering<sup>1</sup> and Luke D. Smillie<sup>2</sup>**

<sup>1</sup> Department of Psychology, Goldsmiths, University of London, London, UK

<sup>2</sup> Melbourne School of Psychological Sciences, The University of Melbourne, Melbourne, VIC, Australia

#### **Edited by:**

Jan Wacker, Philipps-Universität Marburg, Germany

#### **Reviewed by:**

Johannes Hewig, Julius-Maximilians-University Würzburg, Germany Diane Santesso, University of Winnipeg, Canada

#### **\*Correspondence:**

Andrew J. Cooper, Department of Psychology, Goldsmiths, University of London, New Cross, Lewisham Way, London, SE14 6NW, UK e-mail: a.cooper@gold.ac.uk

Medial-frontal negativity occurring ∼200–300 ms post-stimulus in response to motivationally salient stimuli, usually referred to as feedback-related negativity (FRN), appears to be at least partly modulated by dopaminergic-based reward prediction error (RPE) signaling. Previous research (e.g., Smillie et al., 2011) has shown that higher scores on a putatively dopaminergic-based personality trait, extraversion, were associated with a more pronounced difference wave contrasting unpredicted non-reward and unpredicted reward trials on an associative learning task. In the current study, we sought to extend this research by comparing how trait measures of reward sensitivity, impulsivity and extraversion related to the FRN using the same associative learning task. A sample of healthy adults (N = 38) completed a battery of personality questionnaires, before completing the associative learning task while EEG was recorded. As expected, FRN was most negative following unpredicted non-reward. A difference wave contrasting unpredicted non-reward and unpredicted reward trials was calculated. Extraversion, but not measures of impulsivity, had a significant association with this difference wave. Further, the difference wave was significantly related to a measure of anticipatory pleasure, but not consummatory pleasure. These findings provide support for the existing evidence suggesting that variation in dopaminergic functioning in brain "reward" pathways may partially underpin associations between the FRN and trait measures of extraversion and anticipatory pleasure.

#### **Keywords: extraversion, pleasure, feedback-related negativity, reward, event related potential, behavioral approach system**

Monitoring and evaluating cues in the environment for their motivational significance and reward value represents a crucial aspect of decision-making and goal-directed behavior. Cues that provide feedback on whether outcomes of actions have been better or worse than expected are critical in the updating of behavior in response to environmental demands. In particular, reinforcement learning models often train a so-called actor circuit (which associates stimuli with responses) using a teaching (reinforcement) signal sent from a critic circuit, which compares actual with predicted outcomes. Such models typically stress the temporal prediction of reinforcement, relying on so-called temporal difference learning (see Sutton and Barto, 1998, for a review). Dopaminergic (DA) projections from the ventral tegmental area (VTA) to the nucleus accumbens and the anterior cingulate cortex (ACC) appear to play a key role in signaling the degree to which events are better or worse than expected. This has been termed reward prediction error (RPE) signaling (e.g., Schultz, 1998, 2007), and this DA-based RPE signaling function has been widely argued to be a central part of the biological underpinning of reinforcement learning mechanisms within actor-critic (e.g., Houk et al., 1995), temporal difference (e.g., Montague et al., 1996), and other (e.g., Brown et al., 1999) models.

The ACC appears to play an important integrative role as a recipient of RPE signals, using this information to assign reward value to cues, evaluate the effect of previous actions and select subsequent actions, amongst other functions (Gehring and Willoughby, 2002; Holroyd and Coles, 2002; Paus, 2001). The DA mediated RPE signal has been associated with a negative deflection in the event related potential (ERP) approximately 200–300 ms after the presentation of motivationally salient feedback, and is largest in magnitude over medial-frontal brain areas (Holroyd and Coles, 2002; Nieuwenhuis et al., 2004). Although originally identified with an earlier response to error commission (often termed error-related negativity), the negative deflection 200–300 ms post-stimulus has been observed in the absence of any overt choice or response, and so has been termed feedback-related negativity (FRN; Yeung et al., 2005). Studies using source localization analyses and functional imaging have suggested the ACC is the source of the FRN (Gehring and Willoughby, 2002; Ruchsow et al., 2002; Holroyd et al., 2004; Martin et al., 2009). Holroyd and colleagues have suggested that the FRN is modulated by phasic DA responses to unpredicted rewards and unpredicted non-rewards that serve as inputs to the ACC. In this way, the FRN has been linked with the generation of RPE signals that are transmitted via ascending dopaminergic pathways; phasic decreases in DA activity in response to unpredicted non-rewards result in a more negative FRN, while increases in phasic DA activity result in a less negative FRN (Holroyd and Coles, 2002; Nieuwenhuis et al., 2004; Holroyd and Krigolson, 2007).

On the bases outlined above, the difference in the magnitude of FRN should be larger when comparing unpredicted rewards and unpredicted non-rewards (i.e., the calculation of a difference wave), compared to the difference in FRN magnitude between predicted rewards and non-rewards. In other words, we can potentially characterize the difference in FRN magnitude for unpredicted reward and unpredicted non-reward trials as an index of RPE signaling. The existing empirical evidence has generally supported an inverse relation between the likelihood of outcome and the magnitude of an FRN difference wave between unpredicted reward and unpredicted non-reward trials (Walsh and Anderson, 2012). For example, Potts et al. (2006) examined ERPs after participants had completed a passive associative learning task that manipulated the likelihood of reward and nonreward. They showed that FRN was most negative for unpredicted non-reward trials and least negative for unpredicted reward trials in the expected time window (i.e., 200–300 ms post stimulus presentation). Notably, FRN was elicited in the absence of any requirement for a behavioral response to the task stimuli, suggesting that FRN can reflect feedback monitoring in a general sense and is not necessarily a response-locked deflection.

The field of personality neuroscience seeks to identify the neurobiological mechanisms, along with the key operational parameters of these mechanisms, that contribute to the long-term patterning of affect, behavior and cognition (DeYoung, 2010). There has long been recognition that individual differences in the sensitivity of brain systems involved in the processing of reward contribute to variation in higher order personality traits (Gray, 1973; Depue and Collins, 1999; Pickering and Gray, 1999). This has often been considered in the framework of what has been termed the Behavioral Approach System or Behavioral Activation System (BAS; Pickering and Smillie, 2008). There has been less consensus, however, on the personality trait/s that might best reflect BAS functioning. One candidate trait is extraversion (Depue and Collins, 1999; Smillie, 2013), a trait associated with positive affect, behavioral approach and agency (Wilt and Revelle, 2009). There has also been a focus on impulsivity and/or sensation-seeking, and other traits associated with anti-social behavior (Zuckerman, 1984; Pickering, 2004). This latter effort has been complicated by the recognition that factor analyses of impulsivity self-report scales show a multidimensional structure (e.g., Whiteside and Lynam, 2001; Cyders and Coskunpinar, 2011). Further still, researchers have developed novel scales to measure theory-driven conceptualizations of dispositional BAS functioning; these have also typically been multidimensional in their factorial structure, breaking down in to what might be broadly termed reward sensitivity and impulsivity traits. An example along these lines would be the Carver and White (1994) BIS/BAS scales; these scales have three BAS factors, reward responsiveness, drive and fun-seeking. More generally, factor analyses of multiple self-report "BAS-related" scales typically used in this research show a multi-factorial structure; these factors have often been labeled reward drive (or reward sensitivity) and rash impulsiveness (Dawe and Loxton, 2004; Cooper et al., 2008).

As there is now substantial empirical support for FRN as an index of DA RPE signaling, it serves as a useful tool for evaluating personality traits that have a putative basis in DA functioning. For instance, Smillie et al. (2011) used the same associative learning task reported by Potts et al. (2006; see earlier) to examine the association between FRN and individual differences on a trait measure of extraversion, assessed using the Eysenck Personality Questionnaire—Revised (EPQ-R; Eysenck and Eysenck, 1991). The study used an extreme groups design, including individuals exceeding 1 SD unit above or below the mean for extraversion. Each trial in the task involved the sequential presentation of two images, each of which was either a gold bar or a lemon. For 40% of trials there was a sequence of two gold bars followed by a monetary reward (predicted reward), while for another 40% of trials there was a sequence of two lemons followed by no reward (predicted non-reward). In addition, on 10% of trials a gold bar was followed by a lemon and no reward was delivered (unpredicted non-reward), while on the remaining 10% of trials a lemon was followed by a gold bar and a reward was delivered (unpredicted reward). Participants were not required to make any behavioral responses as part of the task, and simply observed the monetary outcomes. The results replicated findings by Potts et al. (2006), with FRN being significantly larger for unpredicted non-reward trials compared to unpredicted reward trials. Furthermore, Smillie et al. (2011) found that the difference in FRN for unpredicted non-reward and unpredicted reward trial types was larger for Extraverts, such that FRN was more negative following unpredicted non-reward and less negative following unpredicted reward. A subset of these participants had provided a DNA sample for a separate study, and so FRN was also examined in relation to the dopaminergicrelated gene polymorphism DRD2/ANKK1. The results showed that those carrying at least one copy of the A1 allele had a larger difference wave contrasting unpredicted non-reward and reward trials, although this difference failed to reach formal statistical significance.

Prior to Smillie et al. (2011), a number of studies had reported associations between error and feedback-related ERP components and personality traits related to reward and punishment sensitivity (e.g., Boksem et al., 2006, 2008; Balconi and Crivelli, 2010; Tops and Boksem, 2010). For example, Boksem et al. (2006) related scores on the Carver and White (1994) BIS/BAS scales with ERP responses to errors on the Eriksen Flanker Task. They found that BIS scores were positively correlated with error-related negativity 50–100 ms post-stimulus response, with BAS scores unrelated to this component. Conversely, a later error positivity deflection was significantly and positively related to BAS scores, largely driven by the fun-seeking subscale, but was not significantly related to BIS scores. Recently, several other studies have examined relations between FRN and BAS-related traits more specifically. Lange et al. (2012) examined the relations between scores on the BIS/BAS scales and FRN in response to a twochoice task that manipulated reward expectation. They found that FRN was significantly more negative following unpredicted non-reward in the extinction phase of the task for individuals higher on the BAS scale, as they predicted. Conversely, individuals higher on the BIS scale showed a significantly less negative FRN in relation to unpredicted non-reward. Notably, an aggregated BAS scale score was used in this analysis, so it is unclear how the different BAS facets related to the FRN in this case. It leaves open the possibility that different facets of the BAS, in this case reward responsiveness, drive and fun-seeking, may have diverging associations with FRN magnitude. More recently, Bress and Hajcak (2013) examined FRN responses to a gambling task, and associations with a self-report measure of reward responsiveness (RR; Van den Berg et al., 2010) and a signal detection behavioral task designed to assess bias towards reward. This self-report measure of RR shares some items with the reward responsiveness and drive scales from the BIS/BAS scales; an inspection of the item content in this new RR scale arguably suggests that it reflects behavioral approach and agency, rather than the enjoyment or consummation of reward. Bress and Hajcak (2013) found that self-reported RR using this new scale significantly correlated with the difference between FRN response to gains and losses, such that those higher on RR had a larger magnitude difference wave.

In sum, there is a small but growing body of research suggesting that individual differences in BAS-related personality traits are related to the magnitude of FRN. To date, however, these studies have tended to either examine individual BAS traits in isolation, or have used aggregated BAS scales, potentially obscuring important dissociations in the relations between different BAS traits and FRN magnitude.

Another potential issue with the previous research on FRN and BAS-related traits, and arguably an issue in the reward processing and personality literature more generally (see Smillie, 2013), is an under-appreciation of the distinction between different aspects of reward processing, and what these aspects might mean for individual differences in BAS-related personality traits. For example, in the addiction literature a distinction has been made between reward "wanting", referring to the motivated approach of and feelings of desire for reward, with a putative basis in mesolimbic dopaminergic functioning, and reward "liking", referring to feelings of enjoyment or satisfaction upon reward consummation, with a putative basis in forebrain opioid circuitry (Berridge et al., 2009). Efforts in other areas of clinical psychology have highlighted the potential value in dissociating the motivational and consummatory components of reward processing, particularly in relation to the reward processing deficiencies often seen in depression and schizophrenia (Treadway and Zald, 2011, 2013). Gard et al. (2006) have developed a selfreport questionnaire, the Temporal Experience of Pleasure Scale (TEPS), which sought to measure trait individual differences in anticipatory pleasure (TEPS-ANT) and consummatory pleasure (TEPS-CON). While there has been some evidence to suggest dissociations between the two scales in relation to reward processing deficits in schizophrenic patients (Gard et al., 2007), the evidence on this front is mixed (Strauss et al., 2011). More generally, further research is needed to validate the psychometric distinction between these constructs.

Our aim in this study was to extend the existing data by examining how a broad array of BAS-related personality measures relate to FRN magnitude, using the same associative learning task reported in Smillie et al. (2011). The personality inventories we included cover constructs that have currently or previously been thought to at least partly reflect variation in the BAS, and so might be considered candidates for relating to neural indices of dopaminergic functioning. These include measures of extraversion, impulsivity and reward sensitivity/anhedonia. We were particularly interested in examining the TEPS in this context, the subscales of which potentially capture the distinction between reward wanting and liking. Given the putative basis of the approach or anticipatory element of reward processing in dopaminergic functioning, we would predict that the TEPS-ANT scale would significantly vary with FRN, but we would expect to see no significant association between the TEPS-CON scale and FRN. Similarly, with the BAS scales from the Carver and White (1994) scales, we would predict that FRN would be significantly predicted by the drive subscale, but not by reward responsiveness (the item content of which appears to capture consummatory aspects of reward processing) or fun-seeking (which appears to measure impulsivity; Smillie et al., 2006). Our measure of extraversion in the current study was the same as that used in Smillie et al. (2011); the EPQ-R, and so we would expect that this measure would also relate significantly with FRN, but we would predict no association with the psychoticism scale from the EPQ-R (as this is where impulsivity lies within the Eysenckian "giant three" framework). Measures used in the clinical assessment of anhedonia included in the current study, the Snaith-Hamilton Pleasure Scale and the anhedonia subscale from the Beck Depression Inventory, both tend to have item content reflective of the consummatory aspects of reward processing, and so we would not expect to see a significant association with FRN for these measures.

# **METHODS**

# **PARTICIPANTS**

Thirty-eight right handed individuals aged between 19 and 42 years (*M* = 24.39, SD = 4.76) participated in this study in exchange for cash (£15); 20 of these participants were male (52.6%). Participants were largely recruited from among students at Goldsmiths, University of London, UK. No participants reported a personal history of psychiatric illness and all participants had normal or corrected-to-normal vision. Participants were recruited via leaflets and social networking sites. All participants provided written consent to take part in the study. The experimental procedure, including EEG set-up, was outlined prior to the start of the experiment and participants were given the opportunity to ask questions and were made aware that they could withdraw participation at any point during the study. These procedures were approved by the Goldsmiths Department of Psychology Ethics Committee.

# **MEASURES**

After completion of consent and information forms, and prior to the EEG recording and completion of the experimental task, participants completed a battery of personality measures comprised of the following:

# **Temporal experience of pleasure scale (TEPS)**

The TEPS (Gard et al., 2006) is an 18-item questionnaire designed to measure individual differences in anticipatory pleasure (TEPS-ANT; 10 items) and consummatory pleasure (TEPS-CON; 8 items). The TEPS-ANT subscale measures feelings of pleasure associated with anticipation and eagerness for upcoming events e.g., "*When something exciting is coming up in my life, I really look forward to it*". The TEPS-CON subscale measures feelings of pleasure associated with the consumption and savoring of current rewarding events e.g., "*A hot cup of coffee or tea on a cold morning is very satisfying to me*". Participants indicated their agreement with the 18 statements using a 6-point Likert-type scale ranging from *strongly disagree* to *strongly agree*. Individual item scores were summed for each subscale, such that high scores equate to stronger feelings of pleasure. In the current study, Cronbach's α was 0.72 for TEPS-ANT and 0.50 for TEPS-CON. The somewhat low reliability estimate for the TEPS-CON subscale is consistent with some previous studies that have used this questionnaire, albeit with a Chinese language version (Chan et al., 2012) and an English language version used with patients diagnosed with schizophrenia (Buck and Lysaker, 2013).

# **The BIS/BAS scales**

The Carver and White (1994) BIS/BAS Scales are a measure comprising a BIS scale (7 items) and three BAS scales: reward responsiveness (5 items), drive (4 items) and fun-seeking (4 items). Each item was answered using a four-point Likert scale, ranging from 1 ("*very false for me"*) to 4 ("*very true for me"*). Previous research has shown the scales have satisfactory internal reliability and construct validity (Carver and White, 1994; Gomez et al., 2005). Item scores for each subscale were summed, with higher scores equating to higher approach and inhibition. Cronbach's α-values in the current study for reward responsiveness, drive, fun-seeking and BIS were 0.69, 0.78, 0.60, and 0.76, respectively.

# **The eysenck personality questionnaire—revised (EPQ-R)**

The EPQ-R (Eysenck and Eysenck, 1991) is a widely used measure of personality that provides scores for extraversion (23 items), neuroticism (24 items), and psychoticism (32 items). The extraversion subscale includes items that reflect behavioral approach and agency, while the psychoticism subscale includes items that reflect impulsive and anti-social behavior. The neuroticism subscale includes items that reflect negative affective states and emotional instability. Respondents indicated their agreement with each statement using a dichotomous yes/no response format. Item scores for each subscale were summed, with higher scores equating to higher levels of the respective trait. The EPQ-R has been used extensively in past research, and has been shown to have good reliability and validity. In the current study, Cronbach's α-values for extraversion, neuroticism, and psychoticism were 0.74, 0.86, and 0.76, respectively.

# **The beck depression inventory (BDI)**

The Beck Depression Inventory–II (BDI-II; Beck et al., 1996b) is a widely-used self-report measure assessing the severity of depressive symptoms over the previous 2 weeks, with good reported reliability and validity (Beck et al., 1996a). Items 4 ("*satisfaction with things"*), 12 ("*interest in other people"*), 15 ("*effort in doing things"*), and 21 ("*interest in sex"*) can comprise an anhedonia subscale (e.g., Pizzagalli et al., 2005), and this subscale was also examined separately in the current study. Cronbach's α-values for the total BDI scale and the anhedonia subscale in the current study were 0.85 and 0.38, respectively. All participants in this sample had the same response to item 21 (i.e., no change in interest in sex), therefore this item was not included in the calculation of Cronbach's α for the BDI total and anhedonia subscales.

# **Snaith-Hamilton pleasure scale (SHPS)**

The Snaith-Hamilton Pleasure Scale (SHPS; Snaith et al., 1995) is a 14-item self-report measure of the pleasure felt when engaging in various everyday activities (e.g., "*I would enjoy a warm bath or refreshing shower"*). Respondents indicated the degree to which they agreed with each statement using a four-point scale ("*Strongly Disagree"*, "*Disagree"*, "*Agree"* and "*Strongly Agree"*). All statements are positively worded. To derive a total score, either of the "disagree" responses to an item is given 1 point, and either of the "agree" responses is given 0 points; thus, total scores can range from 0–14, with higher scores indicative of higher levels of anhedonia. Cronbach's α for the SHPS in the current study was 0.68.

# **TASK DESIGN AND PROCEDURE**

Following completion of the personality measures, participants were seated in a noise-shielded room in front of the computer screen showing the experimental task and the EEG recording procedure was initiated, as outlined below. Once the EEG equipment had been fitted, participants were given instructions for the task. The experimental task used in the current study was the same as that described in Smillie et al. (2011), which itself had been based on an earlier task used by Potts et al. (2006). The task was presented to participants as being similar to a "fruit machine" used in gambling venues in the UK (often called a "slot machine" or "poker machine" outside the UK). The task used a passive S1-S2 randomized-block design, with two within-subject factors representing the differences in trial-type: reward *vs*. non-reward, and predicted *vs*. unpredicted. S1 and S2 comprised images of either a gold bar or a lemon. Participants were instructed to simply observe the trials on the screen and attend to the outcome of each trial, and that they did not need to make any overt actions in response to the presentations.

Each trial sequence began with a fixation point (300 ms), followed by the presentation of S1 (500 ms), a second fixation point (300 ms), presentation of S2 (500 ms), and then feedback in the form of a numeric representation of the trial and cumulative earnings (600 ms). To help minimize blink artifacts, a "*blink now"* message appeared on the screen at the end of each trial as part of an irregular inter-trial interval (2000–3600 ms), and participants were encouraged at the beginning of the task to restrict blinking to this period if possible.

Participants completed 30 practice trials to ensure that they understood the task. They subsequently received a total of 480 experimental trials (8 blocks with 60 trials per block), which were separated by rest periods. On 40% of trials S1 and S2 were gold bars and participants received a reward (£0.50) (predicted reward; 192 trials). On another 40% of trials S1 and S2 were lemons, and participants received no reward (predicted nonreward; 192 trials). On 10% of trials S1 was a gold bar and S2 was a lemon, and participants received no reward (unpredicted non-reward; 48 trials). Conversely, on the remaining 10% of trials S1 was a lemon and S2 was a gold bar, and a reward (£0.50) was received (unpredicted reward; 48 trials). Cumulative "winnings" from each trial were reset between blocks, and participants were told that they would be paid their "winnings" from the highestpaying block (which was fixed at £15 for all participants). After completion of the task, the EEG equipment was removed and participants were debriefed on the aims of the study.

#### **EEG RECORDING AND ANALYSIS**

Continuous EEG data were acquired from 64 active Ag/AgCl electrode channels placed in accordance with the extended 10– 20 system using Easycap® elastic electrode caps. In order to detect eye movements [electrooculogram (EOG)], two electrodes were placed on the sub- and supra-orbit of the right eye to monitor vertical eye movements, and an additional two electrodes recorded the horizontal EOG from the external canthi of both left- and right eyes. The active electrode system did not require impedance measurements. Data were amplified using a BioSemiActiveTwo® amplifier. To help ensure that the recorded data was of a high standard, the experimenter continuously monitored the incoming EEG data, and participant attention and body movements were observed via a closed circuit video camera. All data were sampled at 512 Hz, and further filtered offline using a 0.1–100 Hz bandpass filter. An average reference was applied to the data. The data was segmented in to 500 ms epochs, beginning 100 ms before S2 onset and finishing 400 ms post S2-onset. Individual epochs were extracted for the onset of the different trial types (unpredicted reward, unpredicted non-reward, predicted reward, predicted non-reward), and these were time-locked to the S2 onset.

Artifacts were automatically detected according to a maximum/minimum voltage criterion (±70 µV on target frontal channels and EOG channels), and then kept or rejected after visual inspection. Following artifact rejection, there was a mean of 36.51 (SD = 9.86) and 36.70 (SD = 9.84) trials available for subsequent analysis for unpredicted reward and unpredicted nonreward trials, respectively. For the more common trial types, there was a mean of 145.84 (SD = 37.36) and 146.46 (SD = 37.95) trials available for subsequent analyses for the predicted reward and predicted non-reward trial types, respectively. There were no significant correlations between the number of trials after artifact rejection for each of the trial types and scores on any of the personality variables. The FRN was averaged across six medialfrontal sites (F1, F2, Fz, FC1, FC2, and FCz), and a grand average was calculated for each participant for each of the four conditions.

In line with the approach by Smillie et al. (2011), we exported the mean ERP amplitude during a time window of 200–300 ms post S2-onset for analysis. To provide alternative estimates, we

**Table 1 | Correlations between the trait self-report measures and the averaged difference between the ERP response to unpredicted reward and non-reward trials.**


Note: \* p < 0.05, \*\* p < 0.01. RPE = Reward prediction error index, EPQ P = Eysenck Personality Questionnaire—Revised psychoticism, EPQ E = Eysenck Personality Questionnaire—Revised extraversion, EPQ N = Eysenck Personality Questionnaire—Revised neuroticism, SHPS = Snaith-Hamilton Pleasure Scale, BAS-DR = Behavioral Approach System—drive, BAS-FS = Behavioral Approach System—fun-seeking, BAS-RR = Behavioral Approach System—reward responsiveness, BIS = Behavioral Inhibition System, BDI = Beck Depression Inventory, BDI-AN = Beck Depression Inventory anhedonia, TEPS-ANT = Temporaral Experience of Pleasure Scale—Anticipatory, TEPS-CON = Temporal Experience of Pleasure Scale—Consummatory. n = 25 for all correlations involving the EPQ-R; n = 38 for all other correlations. EPQ-R extraversion and RPE correlation is reported using one-tailed testing; all other correlations are reported as two-tailed tests.

also extracted data from (a) the six medial-frontal electrode sites mentioned above, but using a longer time window post S2-onset (e.g., 200–350 ms); (b) from the same electrode sites using the difference in magnitude of the N2a and P3 peaks; and (c) from the single medial-frontal channel Fz. All of these alternate indices correlated >0.95 with our index based on the six medial-frontal sites over the 200–300 ms window post S2-onset on our key outcome variable (e.g., the averaged unpredicted reward-unpredicted nonreward difference wave), and associations with the personality variables across these alternate indices were very similar.

# **RESULTS**

### **PERSONALITY MEASURES**

Means and standard deviations for the personality measures and the correlations between these measures are shown in **Table 1**. Of note, the TEPS-ANT scale had a significant positive correlation with the BAS-reward responsiveness scale, and a substantial but non-significant positive correlation with the EPQextraversion scale. The TEPS-CON scale also had a significant positive correlation with the BAS-reward responsiveness scale, although of a lower magnitude than the correlation between the TEPS-ANT and the BAS-reward responsiveness scales. The TEPS-CON scale also had a significant negative correlation with the SHPS, reflecting their close (but inverse) conceptual relationship. The TEPS-ANT and TEPS-CON scales were moderately positively correlated (*r* = 0.40). Females scored significantly higher on the TEPS-ANT scale, *t*(36) = −4.51, *p* < 0.0001, the BAS-reward responsiveness scale, *t*(36) = −3.72, *p* = 0.001, and the BIS scale, *t*(36) = −2.64, *p* = 0.012; there were no significant differences across gender for any of the other personality measures.

### **TASK MANIPULATION CHECK**

A 2 (predicted, unpredicted) × 2 (reward, non-reward) repeated measures ANOVA was undertaken to ensure that variation in the FRN was largely driven by ERP responses to unpredicted reward and non-reward trials. Variation in ERP response across the four trial types broadly followed the pattern seen in Smillie et al. (2011). The ANOVA showed that ERP averaged over medialprefrontal areas was more negative for non-reward than reward trials, *F*(1,37) = 9.42, *p* = 0.004, and more negative for unpredicted than predicted trials, *F*(1,37) = 6.19, *p* = 0.017. The interaction between predicted and reward trial was not significant, *F*(1,37) = 3.07, *p* = 0.088. Several studies (e.g., Potts et al., 2006, 2010; Smillie et al., 2011) using this paradigm have found that the greatest waveform difference in this 2 × 2 design is between unpredicted reward and unpredicted non-reward trials. In the present study, the difference between these conditions was significant, *F*(1,37) = 7.40, *p* = 0.01, while the difference between the predicted reward and predicted non-reward trials was close to 0, *F*(1,37) = 0.14, *p* = 0.71. We therefore followed past practice with this paradigm and computed a difference waveform. A difference score was therefore calculated for each participant contrasting the ERP response to unpredicted reward trials and unpredicted non-reward trials (i.e., the mean amplitude of response to unpredicted reward trials minus the mean amplitude of response to unpredicted non-reward trials) as an index of RPE. This pattern of effects is shown in the ERP waveforms by trial type in **Figure 1**.

#### **MAIN ANALYSES**

The main analysis sought to examine whether an index of RPE related to a battery of measures assessing traits with a putative basis in dopaminergic functioning. There was no significant difference across gender for this RPE index, *t*(36) = −0.45, *p* = 0.652. Firstly, we sought to replicate our earlier finding (Smillie et al., 2011), showing that EPQ-R extraversion was related to this index of RPE. We did this in a subset of participants in the current sample on whom we had EPQ-R data (*n* = 25). On the basis of our previous result with EPQ-R extraversion and this RPE index, we expected a positive correlation and so report a one-tailed test for this association with RPE (for the other personality measures we report two-tailed tests, given the more exploratory nature of this testing). The result in this sample showed a significant positive correlation between EPQ-R extraversion and the RPE index, *r* = 0.36, *p* = 0.038 (one-tailed), indicating the difference of response magnitude between unpredicted non-reward and unpredicted reward trial types tended to be larger for participants higher in extraversion, as in our previous study.

**Table 1** shows the correlations between the other trait measures used in the study and the RPE index (i.e., the averaged difference wave across unpredicted reward and unpredicted nonreward trials). As expected, the RPE index was significantly and positively correlated with the TEPS-ANT scale (*r* = 0.39), but the RPE index did not correlate significantly with the TEPS-CON scale (*r* = 0.11). A test of the difference between the two related correlation coefficients was carried out to investigate the prediction that the correlation between the TEPS-ANT and the RPE index would be significantly larger than the correlation between TEPS-CON and the RPE index. This comparison was significant, *Z* ∗ <sup>1</sup> = 1.60, *p* = 0.05, one-tailed (*Z* ∗ 1 is a recommended statistic for this comparison, Steiger, 1980). This indicates the difference of FRN response magnitude between unpredicted non-reward and unpredicted reward trial types tended to be larger for participants higher on trait anticipatory, but not consummatory, pleasure. The correlation between the RPE index and the TEPS-CON should be treated with caution, however, given the low reliability of the TEPS-CON in this sample. **Figure 2** shows scatterplots of these two sets of associations. A model regressing the RPE index on TEPS-ANT and TEPS-CON simultaneously showed that TEPS-ANT, β = 0.41, *t* = 2.39, *p* = 0.02, but not TEPS-CON, β = −0.05, *t* = −0.31, *p* = 0.76, was a significant predictor of the RPE index. For illustrative purposes, we split participants in to high and low TEPS-ANT and show the difference waveforms for the two groups in **Figure 3** (the divergence between these waveforms at around 100–150 ms potentially reflects individual

differences in error-related negativity, with which FRN has been associated).

**Table 1** shows that none of the other trait self-report measures correlated significantly with the RPE index, including measures that primarily assess anhedonia or lack of pleasure (SHPS and BDI-anhedonia), negative affect more generally (BDI, BIS, and EPQ N), impulsivity (BAS-fun-seeking and EPQ-P), and, more surprisingly, the remaining BAS scales (BAS-drive and BASreward responsiveness).

# **DISCUSSION**

The results from this study provide further support for the notion that FRN may be at least partly mediated by RPE signaling (Holroyd and Coles, 2002). Similar to Potts et al. (2006) and Smillie et al. (2011), we found that FRN was significantly more negative for unpredicted non-reward trials when compared with unpredicted reward trials, while there was no significant difference in negativity for predicted reward and predicted non-reward trial types. Further, we replicated the relation between extraversion and FRN reported previously by Smillie et al. (2011), in a subset of the current sample on whom we had extraversion data (*n* = 25). The size of this effect was comparable to that obtained by Smillie et al. (2011) in a sample of extreme high/low scorers

on extraversion. The fact that almost no other trait examined in this study yielded a stronger association with our FRN RPE index offers considerable encouragement to reward-processing theories of extraversion (e.g., Depue and Collins, 1999).

Beyond trait extraversion, our key aim in the current study was to extend research in this area by examining a broader array of putatively BAS-related personality measures in relation to FRN. These measures encompassed a range of constructs that have previously or currently been considered as reflecting variation in the functioning of the BAS, and included measures of impulsivity, reward sensitivity and anhedonia. We were particularly interested in evaluating a relatively recent self-report measure, the TEPS (Gard et al., 2006), which has sought to dissociate the measurement of TEPS-ANT and TEPS-CON. We predicted that the FRN would significantly vary with TEPS-ANT, but not with TEPS-CON. Our findings supported this prediction; for those individuals higher on the TEPS-ANT scale, the RPE difference wave contrasting unpredicted non-reward and unpredicted reward trials was larger compared to those lower on TEPS-ANT. The TEPS-CON scale, on the other hand, did not significantly relate to FRN and this non-significant relationship was significantly weaker than that for the TEPS-ANT. We predicted that a similar dissociation would occur across the BAS scales in the Carver and White (1994) BIS/BAS scales, with positive correlations expected for BAS drive, but not for BAS reward responsiveness or BAS-funseeking. However our findings did not support this, with none of the BAS scales relating significantly to FRN. There were also no significant associations between the FRN and the measures of psychoticism and neuroticism from the EPQ-R, anhedonia (SHPS and BDI), and the BIS scale.

The findings from this study provide further support that variation in personality traits associated with behavioral approach, agency and anticipatory positive emotion are linked with FRN, a potential index of RPE signaling. The replication of our previous result with extraversion is particularly encouraging, given the relative inconsistencies in replication of effects linking extraversion with indices of reward processing (Smillie, 2013). While in both studies we used the EPQ-R measure of extraversion, it would be useful for further research to extend and validate this finding by examining lower order facets/aspects of extraversion, particularly those that potentially distinguish behavioral approach and enjoyment of rewards (e.g., agency vs. affiliation; Depue and Collins, 1999, assertiveness vs. enthusiasm; DeYoung et al., 2007). Our findings in relation to the TEPS are also encouraging and support the validity of the psychometric distinction between anticipatory and consummatory pleasure in this measure. While the TEPS has shown some initial promise, the validity of the distinction between anticipatory and consummatory pleasure remains to be further tested using reward processing paradigms.

None of the BAS scales from the Carver and White (1994) scales correlated with FRN in this study. This runs counter to some previous studies (Lange et al., 2012; Bress and Hajcak, 2013) that have found significant associations between scores on BAS scales and the magnitude of FRN. In one of these studies (Lange et al., 2012), however, an aggregated BAS scale score was used, and so it is unclear how the BAS subscales relate to FRN. We combined the three BAS scales in to a composite BAS score, as in Lange et al. (2012), however this aggregate BAS score also did not significantly relate to our RPE index, *r* = 0.13, *p* = 0.437. As outlined in the introduction, the association between the BAS subscales and the FRN RPE index was not expected to be significant for those subscales (reward responsiveness and fun-seeking), which emphasize reward liking in their item content. Items in these scales might index variation in the tendency to derive pleasure from obtained reward, rather than motivated behavior toward to-be-obtained reward (e.g., *When I get something I want, I feel excited and energized*; *When good things happen to me, it affects me strongly*). We predicted that the BAS drive subscale, which seems least characterized by such pleasure-focused, liking items, would correlate with our FRN RPE index. However, this subscale actually showed numerically the weakest correlation with the RPE index. Of note, the only BAS-relevant scale that BAS drive significantly correlated with in this study was the BAS fun-seeking scale; it did not correlate significantly with EPQ-R extraversion, nor with either of the TEPS scales.

In Bress and Hajcak (2013) a relatively new measure of reward responsiveness was used (Van den Berg et al., 2010). This new measure includes existing items from the Carver and White drive and reward responsiveness scales, and some novel items. The item content overall tends to reflect agency, drive, and anticipatory excitement. On that basis, the results from Bress and Hajcak (2013) are more consistent with the pattern of association we were predicting, while being somewhat at odds with our findings for the drive scale. Bress and Hajcak's findings can perhaps be viewed as being broadly consistent with our findings linking FRN with agency and behavioral approach in EPQ-R extraversion, and anticipatory pleasure from the TEPS.

The results from this study should be considered in light of some potentially important limitations. Firstly, Cronbach's α for the TEPS-CON scale was low (0.50), and this will have attenuated the correlation between this scale and the RPE index. The TEPS-ANT scale, which did show a significant association with the RPE index, did have an acceptable α-value (0.72) in this case. More generally, the TEPS is a relatively recently developed scale and so the psychometric properties of this scale clearly need further exploration. Beyond potential concerns with the reliability of the TEPS-CON scale, Ho et al. (in press) used confirmatory factor analytic modeling of the TEPS to show that while a two-factor structure best represented the data, model fit indices for a twofactor model were less than adequate; this may have been at least partly driven by cross-loading of items across each scale, as has also been shown in other previous studies of this measure (e.g., Gard et al., 2006). Given the importance of having a wellvalidated and reliable self-report measure that dissociates anticipatory and consummatory reward processes, further research and development on the TEPS should be encouraged. It may be that some modification of this measure is required moving forward. Similarly, the anhedonia subscale from the BDI also had very low reliability in this sample, and so the non-significant relationship between this scale and the RPE index may be explained on this basis.

Another potential limitation in the study is the use of a passive associative learning task. Given that participants are not required to make any behavioral responses to the task, it may be that confounding variables related to the passive nature of the task, such as attention or boredom-proneness, became important. It should be noted, however, that the personality variables used in this study were not significantly correlated with the number of trials removed because of movement and other artifacts; these artifacts may partly reflect variables such as lack of attention. Future studies might benefit, however, from using a modified task that includes mixed blocks of active and passive responses to the task contingencies. Indeed, Martin and Potts (2011), using a similar task to that used in the current study, alternated passive and active response blocks of trials in the task. They showed significantly enhanced FRN to outcomes that were worse than expected only in the active condition, although there was a nonsignificant trend in this direction in their passive condition. If a more robust FRN effect is reliably obtained using active responses, then it may be more useful to study personality-based individual differences using FRN tasks that involve an active response. More generally, self-reported level of task interest and engagement are higher in response vs. no response tasks, and the difference in task interest between response vs. no response versions of tasks correlates with FRN magnitude (Yeung et al., 2005), so it would be useful for future personality research in this area to assess task engagement, subjective reward expectation and level of attention more systematically.

This study adds to the literature showing that FRN, as a putative marker of RPE signaling in brain dopaminergic "reward" pathways, is related to scores on self-report personality measures. More specifically, we replicated our previous result (Smillie et al., 2011) showing that trait extraversion, as measured using the EPQ-R and characterized by a focus on behavioral approach and agency, was significantly related to this RPE index. We also showed that the RPE index correlated significantly with the TEPS measure of anticipatory pleasure, but not consummatory pleasure. This provides support for the notion that individual differences specifically in behavioral approach and anticipatory positive affective states are at least partly underpinned by functional variation in dopaminergic systems. This finding might partly be qualified by a lack of dissociation in associations with FRN across the three BAS subscales in the Carver and White (1994) BIS/BAS scales. Nonetheless, it is hoped these findings further contribute to an understanding of how broad-level personality traits, like extraversion, relate to neural responses to rewarding events. In that respect, we also hope these findings provide encouragement for further work examining the separable role that anticipatory and consummatory reward processes may play in personality structure and processes (Smillie, 2013).

### **REFERENCES**


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 21 November 2013; accepted: 04 April 2014; published online: 28 April 2014*. *Citation: Cooper AJ, Duke É, Pickering AD and Smillie LD (2014) Individual differences in reward prediction error: contrasting relations between feedback-related negativity and trait measures of reward sensitivity, impulsivity and extraversion. Front. Hum. Neurosci. 8:248. doi: 10.3389/fnhum.2014.00248*

*This article was submitted to the journal Frontiers in Human Neuroscience*.

*Copyright © 2014 Cooper, Duke, Pickering and Smillie. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

# Extraversion and anterior vs. posterior DMN activity during self-referential thoughts

# *Gennady G. Knyazev\**

*Institute of Physiology, Siberian Branch of Russian Academy of Medical Sciences, Novosibirsk, Russia*

#### *Edited by:*

*Luke D. Smillie, The University of Melbourne, Australia*

#### *Reviewed by:*

*Luke D. Smillie, The University of Melbourne, Australia Mira Chavanon, Philipps-University, Germany*

#### *\*Correspondence:*

*Gennady G. Knyazev, Institute of Physiology, Siberian Branch of Russian Academy of Medical Sciences, Timakova str., 4, Novosibirsk 630117, Russia. e-mail: knyazev@physiol.ru*

Recent studies show that fronto-posterior electroencephalogram (EEG) spectral power distribution is associated with personality. Specifically, extraversion is associated with an increase of spectral power in posterior cortical regions that overlap with the posterior default mode network (DMN) hub and a decrease of spectral power in anterior regions that overlap with the anterior DMN hub. Although there is evidence that dopaminergic neurotransmission may be involved, psychological processes that underlie these associations remain unclear. I hypothesize that these processes may have something to do with spontaneous self-referential thoughts. Specifically, I hypothesize that in extraverts self-referential thoughts may be associated with an increase of spectral power in the posterior DMN hub, whereas in introverts they may be associated with an increase of spectral power in the anterior DMN hub. After spontaneous EEG registration, participants were asked to fill in a questionnaire describing their thoughts during the registration. An item describing self-referential positive expectations (SRPE) was used to measure individual differences in the intensity of these processes. Source localization and independent component analyses were applied to EEG data to reveal oscillatory activity associated with the anterior and the posterior DMN hubs. Hierarchical regression analysis showed a significant interaction between extraversion scores and anterior vs. posterior DMN alpha activity in predicting individual differences in SRPE scores. In extraverts, high SRPE scores were associated with an increase of alpha power in the posterior DMN hub, whereas in introverts they were associated with an increase of alpha power in the anterior DMN hub. Results are discussed in terms of differential involvement of the two DMN hubs in self-related reward processes in extraverts and introverts.

**Keywords: extraversion, default mode network, EEG, alpha oscillations, independent component analysis**

# **INTRODUCTION**

Extraversion is one of a few major dimensions of personality which consistently appear in most personality models. The view, which currently is most popular, links extraversion with the activity of the brain's dopaminergic (DA) reward system (Depue and Collins, 1999; Smillie et al., 2006). However, investigation of DA neurotransmission in humans requires either invasive measurements or expensive neuroimaging techniques. In this connection, the electroencephalographic (EEG) index of DA neurotransmission, which has been suggested by Wacker et al. (2006), seems very attractive. This suggestion is based on observations of an association between extraversion and posterior vs. frontal EEG activity (Hewig et al., 2004, 2006; Wacker et al., 2006). Wacker et al. (2006) demonstrated that the negative association between extraversion and the frontal minus parietal theta activity, which was observed in the placebo group, was completely reversed in the group that received the selective DA D2 antagonist. Similar effect was observed in alpha band. This finding implies that the fronto- posterior distribution of spectral power may reflect trait-like predispositions which depend on the brain DA functioning. This group of researchers has replicated this finding in several studies [see e.g., meta-analysis by Wacker et al. (2010)]. Moreover, they found an association between the posterior minus

frontal slow activity on the one hand and polymorphisms of the DA D2 receptor (Koehler et al., 2011) and enzyme catechol-Omethyltransferase (Wacker and Gatt, 2010) on the other hand. At least one another independent group found a similar association of extraversion with the fronto- posterior spectral power distribution (Knyazev, 2009, 2010; Knyazev et al., 2012a). However, these findings leave unanswered the question about psychological processes that underlie these associations.

It could be noted that the above described associations are in line with some other findings indicating that posterior cortical areas may be more active in extraverts, at least in some circumstances. For example, Yuan et al. (2012) show that posterior cingulate cortices may mediate extraversion-related effect for pleasant stimuli, which essentially results in a decreased threshold for pleasant emotion and an increased threshold for unpleasant emotion. Higher scores on extraversion were found to be associated with higher amplitudes of the P300 component of the ERPs elicited by human faces in parietal cortical regions (Fishman et al., 2011). Phillips et al. (2012) show that monkeys who demonstrate higher levels of exploratory approach behavior have significantly greater gray matter density in the precuneus. On the other hand, there is indirect evidence implying that prefrontal cortical regions might be less active in extraverts. Thus, neural valuation signals in the anterior cingulate cortex and functional coupling of this region with hippocampus and amygdala predict the degree to which future thinking modulates individual preference functions and reduces the rate of delay discounting (Peters and Büchel, 2010); it is known that extraversion is associated with greater discounting (Richards et al., 1999).

#### **THE FRONTO- POSTERIOR SPECTRAL POWER GRADIENT AND THE DEFAULT MODE NETWORK**

Because most of the associations between extraversion and posterior vs. frontal EEG were observed in resting conditions (but see Knyazev, 2009), it seems reasonable to suggest that they may relate to psychological processes and respective brain networks which are most active in these conditions. Recent studies have revealed several networks in the brain which are active in the resting state. Most intriguing findings and ideas are associated with the so-called default mode network (DMN). The DMN is a constellation of brain areas which decrease their activity during a wide number of different goal-oriented tasks as compared to passive "rest" tasks (Raichle and Snyder, 2007). Interestingly, several DMN regions are also related to social cognition (Mitchell, 2006; Gobbini et al., 2007). In line with these findings recent studies have revealed DMN abnormalities in autistic patients (Kennedy et al., 2006; Kennedy and Courchesne, 2008) and in patients with social phobia (Gentili et al., 2009). Although EEG has lower spatial resolution than fMRI and has no direct access to deep cortical regions (e.g., DMN's midline cortices), it could be noted that the anterior and the posterior cortical areas, whose oscillatory activity shows correlations with extraversion, overlap with the anterior and the posterior DMN hubs, respectively. The relation of the posterior vs. anterior spectral power distribution to DMN was also hypothesized by Wacker et al. (2010) and Chavanon et al. (2011).

#### **THE ANTERIOR AND THE POSTERIOR DMN HUBS**

The DMN comprises a set of brain regions that are co-activated during passive states and show intrinsic functional correlations with one another. However, there is clear evidence that the brain regions within the DMN contribute specialized functions that are organized into subsystems that converge on hubs (Buckner et al., 2008). Maps of the intrinsic correlations within the default network show that it comprises at least three interacting subsystems. The medial temporal lobe subsystem functions to provide information from prior experiences in the form of memories and associations that are the building blocks of mental simulation; the medial prefrontal cortex (MPFC) subsystem facilitates the flexible use of this information during the construction of self-relevant mental simulations. These two subsystems converge on the precuneus/posterior cingulate cortex (pC/PCC) (Buckner et al., 2008). Partial correlation network analysis suggests that this latter region may play a pivotal role in the DMN (Fransson and Marrelec, 2008). Indeed, PET studies have shown that the metabolic activity is higher in the pC/PCC than all other regions during rest (Gusnard and Raichle, 2001). It could be suggested that being the place of integration of prior experiences with mental simulations, the pC/PCC sustains a sense of self-consciousness that is engaged in self-referential mental thoughts during rest (for reviews see Cavanna and Trimble, 2006; Buckner and Carroll, 2007) and is commonly associated with positive emotions (Koole et al., 2001).

#### **THE ANTERIOR DMN HUB**

It appears that social salience which reflects the relation between others and oneself is processed by the MPFC (Iacoboni et al., 2004; Schmitz et al., 2004; Seger et al., 2004; Han et al., 2005; Mitchell et al., 2005a,b; Ochsner et al., 2005). MPFC is activated during thinking about the complex interactions among people (Buckner et al., 2008). There is ground to suggest that thinking about the complex interactions among people is frequently accompanied by negative emotion. Besides, the anterior midcingulate cortex is implicated in the integration of negative affect, pain, and cognitive control (Shackman et al., 2011). In general, being a part of the prefrontal cortex, the MPFC is inevitably involved in conscious planning, decision making, and control functions (see e.g., Luk and Wallis, 2009; Alexander and Brown, 2011). Therefore, these cortices are bound to be reciprocally related to motivational centers, such as amygdala and striatum (Quirk and Beer, 2006; Urry et al., 2006; Goldin et al., 2008; Ochsner and Gross, 2008). Indeed, much evidence shows that, the MPFC controls the accumbens dopamine responses to environmental challenges (e.g., Pascucci et al., 2007) and dopamine release in the MPFC exerts an inhibitory influence on dopamine release in the nucleus accumbens, whereas depletion of mesocortical DA facilitates activation of mesoaccumbens DA release (Deutch et al., 1990; Doherty and Gratton, 1996; King et al., 1997).

### **THE POSTERIOR DMN HUB**

The pC/PCC cortices, which constitute the posterior DMN hub, are involved in self-centered cognition (e.g., ongoing selfmonitoring) and self vs. others discrimination (Vogt et al., 2006). First-person-perspective taking in social interaction and in a language task shows differential activation in the medial aspects of the superior parietal lobe and the right temporo-parietal junction (Vogeley et al., 2001; Vogeley and Fink, 2003). PCC, the retrosplenial, and the medial parietal cortices are implicated in putting self-referential stimuli within a temporal context, linking them to past self-referential stimuli (Northoff et al., 2006). Transcranial magnetic stimulation over the medial parietal region caused a decrease in the efficiency of retrieval of previous judgment of mental self as compared to retrieval of judgment of other, confirming that this region may be a nodal structure in selfrepresentation (Lou et al., 2004). Direct appraisals of self as compared to reflected appraisals recruited PCC (Ochsner et al., 2005). Besides, the right inferior parietal cortex and precuneus may be specifically involved in distinguishing self-produced actions from those generated by others (Ruby and Decety, 2001). It should be borne in mind also that the parietal cortex is activated by emotional stimuli that are not the focus of attention and are therefore perceived mostly unconsciously (Iidaka et al., 2001; Knyazev et al., 2009). Moreover, the parietal cortex is a part of the dorsal (non-conscious) processing stream which contributes to visionfor-action (Milner and Goodale, 1995; Goodale and Milner, 2008) and participates in salience detection (Husain and Nachev, 2007). Non-spatial salience detection functions are particularly associated with the inferior parietal lobe, which in humans consists of novel cortical areas not shared with other primates (Husain and Nachev, 2007). Summing up, existing evidence shows that the anterior DMN hub is involved in mostly conscious modeling, planning, and control functions whereas the posterior hub is involved in mostly unconscious processes that include selfrepresentation, emotion, and salience detection. In the context of the present study, it is interesting to note that according to Gray's (1999) theory, salience detection is the main function of the dopaminergic reward system.

#### **THE ANTERIOR AND POSTERIOR DMN HUBS AND DA**

Because the association between extraversion and posterior vs. anterior EEG activity is mediated by dopamine, it is interesting to note that the anterior and the posterior DMN hubs appear to be differently susceptible to dopaminergic influences. Generally, dopaminergic effects appear to be more pronounced in the posterior than in the anterior hub or these effects could be of opposite directions. Many of these observations have been made on Parkinson's disease (PD) patients. Thus, van Eimeren et al. (2009) show that patients with mild to moderate PD (not taking medication) and healthy controls showed comparable deactivation of the MPFC, but different deactivation in the pC/PCC. Compared with controls, PD patients not only showed less deactivation of the pC/PCC, they even demonstrated a reversed pattern of activation and deactivation. Dopamine medication appears to restore the normal pattern of task-related deactivation in the posterior DMN hub. Thus, PD patients taking placebo only deactivated the ventral MPFC during a facial emotion recognition task. They failed to deactivate the posterior midline and lateral parts of DMN. After levodopa administration, this network was restored conjointly with the improvement of motor dysfunction in PD patients (Delaveau et al., 2010). In another study, PD patients were scanned twice, once while on their DA medication (ON condition) and once after medication withdrawal (OFF condition). Higher activation in the precuneus was found in the ON condition (Dusek et al., 2012). Krajcovicova et al. (2012) using the daily levodopa equivalent dose in cognitively unimpaired PD patients as a covariate observed an enhanced functional connectivity of the DMN in the posterior cingulate cortex during a cognitive task. Similar effects were observed not only in PD patients, but also in healthy older adults who compared with younger adults showed diminished fMRI deactivations in pC/PCC during memory recognition. In younger adults, greater task-induced deactivation in this region was associated with higher dopamine synthesis capacity (as measured by the radiotracer 6-[18F]-fluoro-L-*m*-tyrosine). The authors suggest that DA system helps modulate the posterior DMN hub activity in younger adults and that alteration to the DA system may contribute to age-related changes in working memory function (Braskie et al., 2011). Healthy adult subjects that received methylphenidate (a stimulant drug that amplifies dopaminergic signaling in the brain) had increased deactivation during working memory and visual attention tasks in the insula and the PCC (but not in the MPFC) than the group of subjects who received placebo (Tomasi et al., 2011).

Some authors observed opposite dopamine-related effects in the posterior and the anterior DMN hubs. Thus, Tomasi et al. (2009) assessed the relationship between DA transporters (DAT, which regulate extracellular dopamine in the brain) in striatum (measured with positron emission tomography and [11C] cocaine used as DAT radiotracer) and brain activation and deactivation during a parametric visual attention task (measured with BOLD fMRI) in healthy controls. DAT availability in caudate and putamen had a negative correlation with deactivation in ventral parietal regions of the DMN (precuneus, BA7) and a positive correlation with deactivation in the ventral anterior cingulate gyrus (BA24/32). Similarly, Asanuma et al. (2006) show that, levodopa therapy was associated with significant metabolic increases in the precuneus (BA7) but decreases in the MPFC. This evidence appears to suggest that dopamine may exert inhibitory effect on the anterior and excitatory effect on the posterior DMN hub. The former effect is in line with animal data showing that DA increases the threshold for spike firing and exerts an inhibitory action in the prefrontal cortex (Geijo-Barrientos and Pastore, 1995).

In sum, the evidence presented in the previous sections appears to suggest that although both the anterior and the posterior DMN hubs are involved in self-centered and social cognition and are co-activated during passive states, they are associated with rather different functions. The anterior DMN is more involved in integration, planning, and control functions, which are mostly conscious and are reciprocally related to dopaminergic reward processes. The posterior DMN is more involved in self-representation and salience detection. The latter processes are mostly unconscious and are positively related to dopaminergic reward processes. The former processes are less and the latter processes are more pronounced in extraverts than in introverts.

#### **THE PRESENT STUDY**

I hypothesize that the association between extraversion and the resting state posterior vs. frontal EEG activity is mediated by DMN-related spontaneous self-referential processes in such a way that in more extraverted individuals these processes are associated with an increase of spectral power in the posterior DMN hub, whereas in more introverted individuals they could be associated with an increase of spectral power in the anterior DMN hub. In this study, I aimed to obtain EEG records during unconstrained mind-wandering and to test whether extraversion moderates the associations between the prevalence of relevant self-referential thoughts and EEG spectral power within the anterior and the posterior DMN hubs. The existence of an association between self-referential thoughts and EEG spectral power within the DMN has been shown previously (Knyazev et al., 2011, 2012b). The choice of a relevant measure of extraversion and a relevant measure of self-referential thoughts was guided by the hypothesis linking these processes with dopaminergic transmission and social cognition. Depue and Collins (1999) argue that extraversion can be subdivided into two subfactors: affiliation and agency. They propose a dopaminergic basis for the agency facet of extraversion (i.e., a motivational disposition that comprises social dominance, enthusiasm, energy, assertiveness, ambitiousness, and achievement striving). Keeping in mind that the DMN, which is the main focus of this study, is supposedly involved in self-referential processes in the context of interpersonal relationships (e.g., Mitchell, 2006), assertiveness appears to be the facet of extraversion which best captures both the agentic properties of this dimension and its projection onto the space of interpersonal relationships. High assertiveness scorers are independent, dominant, and stand up for their rights. They tend to be at the center of attention at meetings. Low scorers are humble, timid, submissive, and disinclined to take initiative in interpersonal situations, and may be easily imposed upon (Eysenck and Wilson, 2000). With regard to the relevant kind of self-referential thoughts, I intended to capture an aspect of anticipation of a positive reinforcement that is peculiar to extraverts and is supposedly mediated by the dopaminergic reward system.

# **METHODS**

#### **SUBJECTS**

Resting EEG data were collected in 60 healthy volunteers (32 men and 28 women; age range 17–30 years, mean = 20.4, *SD* = 2.5), mostly university students. All applicable subject protection guidelines and regulations were followed in the conduct of the research in accordance with the Declaration of Helsinki. All participants gave informed consent to the study. The study has been approved by the Institute of Physiology ethical committee.

### **INSTRUMENTS AND PROCEDURES**

Participants were seated in a soundproof dimly illuminated room and did not receive any instruction. The spontaneous EEG registration lasted about 6 min and included alternating 2 min intervals with eyes open and eyes closed. Only the eyes closed condition was used in this study because previous research has shown that self-referential thoughts correlate with EEG spectral power in the eyes closed, but not in the eyes open condition (Knyazev et al., 2011). Just after the EEG registration participants were asked to fill in a brief (35 items) spontaneous thoughts questionnaire (STQ) which described different aspects of their state, thoughts, and feelings during the registration. All items were measured on a five-point Likert scale. Factor analysis of all questionnaire items (principal components factor analysis with varimax rotation) showed that a four-factor solution best fitted the data. Accordingly, four scales were created that described nervousness/negative emotion/lack of positive emotion (NE, example items: "felt nervous," "experienced negative emotions," "was calm and relaxed"—reverse scoring, "liked the procedure" reverse scoring, Cronbach's alpha = 0.84); self-referential thought (SRT, example items: "thought about something pleasant that is going to happen to me in the near future," "recollected episodes from my own life," "most of the time, thoughts of my recent past recurred to me," "most of the time, I was absorbed in my private thoughts," Cronbach's alpha = 0.69); arousal level (example items: "was almost asleep"—reverse scoring, "was quiet and relaxed"—reverse scoring, "was somewhat heated," "was very excited," Cronbach's alpha = 0.72); attention to environment (ATT, "my attention was mostly directed to external stimuli," "most of the time, I listened to sounds and skin sensations," "did not pay any attention to external stimuli"—reverse scoring, Cronbach's alpha = 0.65). The SRT scale (SRTS) was used to measure individual differences in mental processes that are presumably related to DMN activity. Besides, for the purpose of this study, I additionally used the first item from this scale (see above), which describes self-referential positive expectations (hereafter SRPE). After filling in the questionnaire subjects participated in experiments which are not described here. After the experiments they filled in a set of personality questionnaires and were debriefed. Facets of Extraversion were measured by respective scales from the Eysenck Personality Profiler (EPP, Eysenck and Wilson, 2000; Knyazev et al., 2004). Assertiveness scale (Cronbach's alpha = 0.78) consisted of 20 items. Example item: "Do you find it difficult to get rid of a salesperson who is persistent and wasting your time?". Activity (Cronbach's alpha = 0.73) and Sociability (Cronbach's alpha = 0.81) scales were additionally used in order to test the specificity of observed effects.

#### **EEG RECORDING**

EEG data were recorded using 32 silver-silver chloride electrodes mounted in an elastic cap on the positions of the international 10–20 system. The signals were amplified with a multichannel biosignal amplifier with a gain of 250 and a bandpass 0.05–70 Hz, −6 dB/octave and continuously digitized at 300 Hz. All recordings were performed using a fronto-central electrode as ground and electronically linked mastoid electrodes as reference. The horizontal and vertical electrooculogram was registered simultaneously. Electrode impedances were at or below 5 k for all electrodes used in the analysis. EEG data were artifact-corrected using ICA via EEGLAB toolbox (http://www.sccn.ucsd.edu/eeglab/) retaining minimally 20 out of 30 components.

#### **3D SOURCE RECONSTRUCTION**

To determine the cortical sources of EEG activity, sLORETA (Pascual-Marqui, 2002) was applied to the data. sLORETA uses a three-shell spherical head model registered to the digitized Talairach and Tournoux (1988) atlas. The solution space is restricted to cortical gray matter and parahippocampal areas. sLORETA yields images of standardized current source density of a total of 6430 voxels at 5-mm spatial resolution. Artifact-free epochs of 1.7 s duration were supplied for cross-spectrum calculation in sLORETA. The number of epochs varied in different subjects from 85 to 210. Subsequently current source densities of delta (2–4 Hz), theta (4–8 Hz), alpha (8–12 Hz), beta (12–30 Hz), and gamma (30–45 Hz) oscillations were estimated in sLORETA. The regularization factor was set at 1/100 (Congedo, 2006).

### **INDEPENDENT COMPONENT ANALYSIS**

In this study, I used the group spatial ICA, because it directly estimates components that are consistently expressed in the population, involves the least amount of user interaction and is straightforward to compare with the existing framework for group ICA of fMRI data (Calhoun et al., 2001). First, current source density estimates for the five EEG frequency bands were calculated using sLORETA. For each frequency band separately, each trial's sLORETA images were converted into the neuroimaging informatics technology initiative (NIFTI) format using modified by the first author LOR2SPM function by Pakhomov (http://www.ihb.spb.ru/∼pet\_lab/L2S/L2SMain.htm). Next, group spatial ICA was applied to sLORETA images in a

fashion that is routinely used in fMRI research. Spatial ICA was performed using the Group ICA for fMRI Toolbox (GIFT, Version 1.3i; http://icatb.sourceforge.net/), using methods and algorithms described elsewhere (Calhoun et al., 2001, 2004). Briefly, a single ICA was performed at the group level after subject-wise data concatenation. Obtained independent components (ICs) were back reconstructed to produce single-subject time courses and spatial maps from the raw data matrix (Calhoun et al., 2001). The minimum description lengths (MDL) criterion was used to estimate the number of extracted components from the data. Basing on these estimates, 20 components were extracted. One-sample T-tests in SPM8 were used to assess the statistical significance of each component. Each subject's respective component image (*z*score spatial map) was entered into a second-level random-effects analysis and assessed statistically using a threshold of PFDR = 0.05 (whole-brain corrected) and minimum cluster size of 8 contiguous voxels. Only the components that were statistically significant across subjects were used in further analyses.

For each respective set of ICA results, ICs were spatially correlated with an anatomically defined template and were ranked according to a "highest correlation" criterion with this anatomy. I created two templates using the Wake Forest Pick atlas toolbox (http://www.fmri.wfubmc.edu/) (Maldjian et al., 2004). The anterior DMN (ADMN) template included the medial frontal and the superior frontal gyrus (BAs 8/9/10) and the anterior cingulate cortex (BAs 11/24/32). The posterior DMN (PDMN) template included the posterior parietal cortex (BA 7), the occipitoparietal junction (BA 39), the posterior cingulate, and the precuneus. For the analysis of associations between SRPE scores and interindividual variation in components' intensity, for each of 20 ICs generated for each frequency band, all positive voxel values in a respective (*z*-score-transformed) independent component image were summed for each subject. These values were further used as is described below. This approach to capturing inter-individual differences is based on the following. After initial decomposition on all concatenated datasets at once, the components are back reconstructed in each individual subject. After that, each component is more pronounced in some subjects and may be weak or absent in others. If a component is strongly pronounced, respective brain areas will have high positive *z*-score values. Therefore, summing and comparing across subjects the positive values in *z*transformed spatial maps allows revealing individual differences in intensity of this component. This method is described in the paper by Allen et al. (2011). For more details see Knyazev et al. (2011, 2012a,b).

#### **STATISTICAL ANALYSIS**

For the analysis of moderation effects a combined measure of PDMN vs. ADMN activity (hereafter P/ADMN) was created. The PDMNIC (hereafter subscript IC denotes the independent component that showed the highest spatial correlation with a respective template) and ADMNIC scores were converted to *z*-scores and ADMNIC scores were subtracted from PDMNIC scores. The resulting measure represented dimension running from low PDMN/high ADMN activity to high PDMN/low ADMN activity.

To test for moderation, regression analyses were specified for the combination of moderator (i.e., extraversion) and factor (i.e., P/ADMN) as predictors of SRTS or SRPE scores. Following guidelines on testing moderator models outlined by Baron and Kenny (1986), predictor variables were entered hierarchically in the following order: (1) main effects for factor tested (i.e., P/ADMN) and proposed moderator variable (i.e., extraversion); (2) the twoway interaction between the factor and the moderator. To test interactions (or moderation effects) involving continuous variables, I converted all continuous variables to *z*-scores, following the suggestion by Aiken and West (1991). To gain an understanding of the overall pattern of the interaction, regression slopes were plotted graphically at high (0.5 SD) and low (−0.5 SD) values of the moderator.

# **RESULTS**

Comparison of EPP scales' means of the current sample with a normative Russian sample (*N* = 576, Knyazev et al., 2004) showed no significant differences on any of the nine personality characteristics. Sociability and Activity correlated positively with SRTS (*r* = 0.32, *p* = 0.017 and *r* = 0.39, *p* = 0.002, respectively) and SRPE (*r* = 0.34, *p* = 0.009 and *r* = 0.38, *p* = 0.003, respectively) and negatively with ATT (*r* = −0.28, *p* = 0.038 and *r* = −0.27, *p* = 0.039, respectively). Assertiveness did not show significant correlations with STQ scales.

There was a significant interaction of Assertiveness with P/ADMN in prediction of SRTS scores, *B* = −0.31, *T*(58) = −2.14, *p* = 0.037. There were no significant moderation effects of Assertiveness in other frequency bands. Moderation effects for Activity and Sociability were not significant for all frequency bands.

Next, moderation analyses were conducted for all SRTS items separately. Only for the first item (i.e., SRPE) its relationship with alpha band P/ADMN activity was significantly moderated by Assertiveness, *B* = −0.71, *T*(58) = −3.52, *p* = 0.001. **Figure 1** shows regression slopes of alpha band P/ADMN on SRPE scores

**FIGURE 1 | Interaction between Assertiveness and P/ADMN alpha activity in their effect on SRPE scores.** Ordinate axis represents SRPE *z*-scores; abscissa axis runs from low PDMN and high ADMN alpha activity to high PDMN and low ADMN alpha activity; solid line represents the group of subjects with high assertiveness (>0.5 SD); dashed line represents the group of subjects with low assertiveness (<−0.5 SD).

in subjects with high (+0.5 SD, *N* = 17) and low (−0.5 SD, *N* = 15) Assertiveness.

As this figure shows, in high Assertiveness scorers, high SRPE scores were associated with a prevalence of alpha activity in the posterior DMN hub, whereas in low Assertiveness scorers they were associated with a prevalence of alpha activity in the anterior DMN hub. *Post hoc* analyses showed that in high Assertiveness scorers (+0.5 SD, *N* = 17), SRPE scores significantly correlated with PDMN (*r* = 0.74, *p* = 0.001), but not with ADMN (*r* = 0.26, *p* = 0.313) alpha activity, whereas in low Assertiveness scorers (−0.5 SD, *N* = 15), SRPE scores significantly correlated with ADMN (*r* = 0.81, *p* < 0.001), but not with PDMN (*r* = −0.01, *p* = 0.969) alpha activity. **Figure 2** shows scatter-plots of the relationships between SRPE scores and alpha activity in the ADMN in low and in the PDMN in high Assertiveness scorers.

**Figure 3** shows anatomy of the ADMN and PDMN alpha components.

# **DISCUSSION**

In line with our hypothesis, in more extraverted individuals, spontaneous self-referential thoughts were associated with an increase of spectral power in the posterior DMN hub, whereas in more introverted individuals they were associated with an increase of spectral power in the anterior DMN hub. In our study, these effects were observed in the alpha band of frequencies. It should be noted that in most recent studies, most prominent

**FIGURE 2 | Scatter-plots of the relationships between SRPE scores and alpha activity in the ADMN in low (left panel) and in the PDMN in high (right panel) Assertiveness scorers.**

extraversion-related differences in cortical distribution of spectral power were noted in low frequencies (predominantly theta) (Wacker et al., 2006, 2010; Wacker and Gatt, 2010; Knyazev et al., 2012a). However, differences in the alpha band of frequencies have also been repeatedly described (Hewig et al., 2004, 2006; Wacker et al., 2006; Knyazev, 2009, 2010). Moreover, substantial evidence has been accumulated indicating that DMN-related processes may be specifically positively associated with alpha and beta oscillations. Thus, Jann et al. (2010) note that BOLD correlates of electrical activity in the alpha and beta frequency bands display striking similarity with the DMN. Mantini et al. (2007) showed that DMN and dorsal attentional network have strong relationship with alpha and beta rhythms, albeit in opposite directions, with the former showing positive and the latter showing negative correlations. Jann et al. (2009) show that the BOLD correlates of global EEG synchronization in the alpha frequency band are located in brain areas involved in the DMN. Jann et al. (2010) report DMN activity to be associated with increased alpha and beta1 band power. Sadaghiani et al. (2010) showed that global field power of upper alpha band oscillations is positively correlated with activity in a network overlapping with the DMN and is negatively correlated with activity in the dorsal attention network. Brookes et al. (2011) identified the DMN using magnetoencephalographic data filtered into the alpha band. Knyazev et al. (2011, 2012b) found that alpha band spatial patterns simultaneously showed a considerable overlap with the DMN and high correlations with presumptive DMN-function-related outcomes. In sum, this evidence suggests that alpha oscillations appear to be positively related to DMN and negatively to attentional networks. Contrary to that, theta oscillations show negative correlations with the DMN (Scheeringa et al., 2008). It could be suggested that extraversion-related differences in cortical distribution of spectral power could be observed in different frequency bands (see e.g., Knyazev, 2009); and different frequency bands contribute to different aspects of extraversion-cortical activity associations. Spontaneous self-referential processes appear to be most prominently related to alpha activity and, hence, extraversion-related differences in cortical distribution are observed in this very band.

As has been described in the section "Introduction," the anterior and the posterior DMN hubs may have different contributions to DMN-related functional outcomes. Specifically, the anterior hub is more involved in modeling the complex social relations. These processes are frequently accompanied by negative emotion and appear to be reciprocally related to nucleus accumbens DA neurotransmission. The posterior hub is more involved in self-centered cognition and salience detection. These processes are commonly associated with positive emotions (Koole et al., 2001) and appear to be positively related to dopaminergic neurotransmission. It appears that spontaneous self-referential thoughts, particularly in the context of anticipation of a positive reinforcement, predominantly engage the posterior DMN hub in extraverts, but the anterior DMN hub in introverts. The two other facets of extraversion (i.e., sociability and activity) did not show significant moderation effects. This could be explained by the fact that sociability is not probably associated with DA neurotransmission (Depue and Collins, 1999), whereas activity is not specifically related to social cognition, which is presumably the main focus of DMN.

Interestingly, a recent study showed that in representatives of a more Western culture (Russia), spontaneous self-referential processes were accompanied by enhanced alpha oscillations in the posterior DMN hub, whereas in representatives of a more Eastern culture (Taiwan) they were accompanied by enhanced alpha oscillations in the anterior DMN hub (Knyazev et al., 2012b). Cross-cultural studies show that Eastern populations are generally lower on extraversion than Western populations (see e.g., Allik and McCrae, 2004), but it has to be revealed in the future whether personality or other cross-cultural variables, such as individualism/collectivism underlie the observed cross-cultural differences.

Some limitations of this study need to be discussed. One methodological limitation is that EEG source localization and ICA were performed on the basis of a somewhat sparse 32 electrodes array. Numerous studies show that localization accuracy improves with increasing the number of recording electrodes (Krings et al., 1999; Laarne et al., 2000; Lantz et al., 2003). ICA decomposition methods generally also require sufficient number of electrodes for reliable and valid component extraction. 32 electrodes may be sufficient, however, so long as there is approximately homogenous scalp coverage (Lantz et al., 2003; Congedo, 2006), as is the case in this study. A simulation study has shown that 32 electrodes array in combination with the method that was used in the current study are sufficient for accurate localization of cortical sources and revealing their time dynamics, frequency characteristics, and between-subject variability (Knyazev et al., 2012a). Moreover, in a recent study (Knyazev et al., 2012b), we replicated the Knyazev et al.'s (2011) findings using denser (64 and 132) electrode arrays.

Another concern relates to the fact that sLORETA produces smooth solutions resulting in many correlated voxels which then are submitted for spatial ICA. The correlated voxels will be combined into one extended component with low spatial resolution. This could be a serious limitation when two closely spaced processes are to be distinguished (see e.g., simulation in Knyazev et al., 2012a). However, this limitation is not important in this study because the anterior and the posterior DMN hubs are situated far apart from each other.

Thus, it could be summarized that in extraverted individuals, spontaneous self-referential thoughts are accompanied by enhanced alpha activity within the posterior DMN hub, whereas in introverted individuals they are accompanied by enhanced alpha activity in the anterior DMN hub. There is a solid ground to suggest that these effects are mediated by dopaminergic neurotransmission, because a number of studies by Wacker and colleagues have shown that extraversion-related posterior vs. anterior EEG asymmetries are associated with the dopaminergic system (Wacker et al., 2006; Wacker and Gatt, 2010; Koehler et al., 2011).

#### **ACKNOWLEDGMENTS**

This work was supported by a grant of the Russian Foundation for Basic Research (RFBR) № 11-06-00041-a. The author is grateful to A. V. Bocharov, A. N. Savostyanov, J. Y., Slobodskoj-Plusnin, and N. V. Dmitrienko for help with data collection.

# **REFERENCES**


from a partial correlation network analysis. *Neuroimage* 42, 1178–1184.


I., Moritz, M., Throop, C. J., et al. (2004). Watching social interactions produces dorsomedial prefrontal and medial parietal BOLD fMRI signal increases compared to a resting baseline. *Neuroimage* 21, 1167–1173.


prefrontal cortex. *J. Neurosci.* 29, 7526–7539.


reward delay discounting through an enhancement of prefrontalmediotemporal interactions. *Neuron* 66, 138–148.


reinforcement sensitivity theory: implications for personality measurement. *Pers. Soc. Psychol. Rev.* 10, 320–335.


integration. *J. Res. Pers.* 44, 167–179.


Neural mechanisms underlying the higher levels of subjective well-being in extraverts: pleasant bias and unpleasant resistance. *Cogn. Affect. Behav. Neurosci.* 12, 175–192.

**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 01 October 2012; accepted: 17 December 2012; published online: 07 January 2013.*

*Citation: Knyazev GG (2013) Extraversion and anterior vs. posterior DMN activity during self-referential* *thoughts. Front. Hum. Neurosci. 6:348. doi: 10.3389/fnhum.2012.00348*

*Copyright © 2013 Knyazev. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any thirdparty graphics etc.*

# Extraversion differentiates between model-based and model-free strategies in a reinforcement learning task

# *Anya Skatova1,2,3\*, Patricia A. Chan3 and Nathaniel D. Daw3*

*<sup>1</sup> The School of Psychology, University of Nottingham, Nottingham, UK*

*<sup>2</sup> Horizon Digital Economy Research, University of Nottingham, Nottingham, UK*

*<sup>3</sup> Department of Psychology, Center for Neural Science, New York University, New York, NY, USA*

#### *Edited by:*

*Luke D. Smillie, The University of Melbourne, Australia*

#### *Reviewed by:*

*Oliver C. Schultheiss, Friedrich Alexander University, Germany Alan Pickering, Goldsmiths, University of London, UK Boon Kiat Quek, Institute of High Performance Computing, Agency for Science, Singapore*

#### *\*Correspondence:*

*Anya Skatova, Horizon Digital Economy Research, University of Nottingham Innovation Park, Triumph Road, Nottingham, NG7 2TU, UK e-mail: anya.skatova@gmail.com*

Prominent computational models describe a neural mechanism for learning from reward prediction errors, and it has been suggested that variations in this mechanism are reflected in personality factors such as trait extraversion. However, although trait extraversion has been linked to improved reward learning, it is not yet known whether this relationship is selective for the particular computational strategy associated with error-driven learning, known as model-free reinforcement learning, vs. another strategy, model-based learning, which the brain is also known to employ. In the present study we test this relationship by examining whether humans' scores on an extraversion scale predict individual differences in the balance between model-based and model-free learning strategies in a sequentially structured decision task designed to distinguish between them. In previous studies with this task, participants have shown a combination of both types of learning, but with substantial individual variation in the balance between them. In the current study, extraversion predicted worse behavior across both sorts of learning. However, the hypothesis that extraverts would be selectively better at model-free reinforcement learning held up among a subset of the more engaged participants, and overall, higher task engagement was associated with a more selective pattern by which extraversion predicted better model-free learning. The findings indicate a relationship between a broad personality orientation and detailed computational learning mechanisms. Results like those in the present study suggest an intriguing and rich relationship between core neuro-computational mechanisms and broader life orientations and outcomes.

**Keywords: extraversion, dopamine, reinforcement learning, personality, decision-making**

## **INTRODUCTION**

It is widely hypothesized that the brain learns from rewards using a prediction error-driven learning rule (Bush and Mosteller, 1953; Rescorla and Wagner, 1972). Prediction errors are thought to drive learning on trial-and-error decision tasks by reinforcing successful actions (a scheme dating back to Thorndike, 1911), and have reliable neural correlates, notably in the firing of neurons containing the neuromodulator dopamine (Houk et al., 1995; Schultz et al., 1997) and in blood oxygenation signals recorded in human functional imaging studies (McClure et al., 2003; O'Doherty et al., 2003). In humans, this mechanism's contribution to learning is evidenced by numerous links between learning performance, neural signatures of reward prediction errors, and/or dopaminergic action (Frank et al., 2004; Pessiglione et al., 2007; Schonberg et al., 2007, 2010; Cools et al., 2009; Voon et al., 2010).

It has also long been suggested that individual differences in reward processing mechanisms such as this one contribute to variations in personality. In an influential review, Depue and Collins (1999) argued for an association between variation in incentive motivation and extraversion, and suggested that this association might be rooted in a dopaminergic mechanism. This work inspired a line of research establishing that extraversion and related personality traits (impulsivity, reward sensitivity, approach motivation, and the behavioral activation system) have links with reward processing (Smillie, 2013). For instance, extraversion and its relatives predict behavioral performance, specifically response bias for rewarded alternatives, on laboratory learning tasks (Corr et al., 1997; Pickering, 2004; Smillie et al., 2007), and more real-world reward-driven behaviors such as eating disorders and drug abuse (Dawe et al., 2004; Dawe and Loxton, 2004). Also, speaking to the relationship between these functions and underlying neural mechanisms, extraversion and similar measures are associated with neural activity related to prediction errors and at dopamine targets (Cohen et al., 2005; Smillie et al., 2011), and genetic polymorphisms related to dopamine expression (Smillie et al., 2010).

Altogether, these experiments suggest that extraversion is associated with reward processing, potentially reflecting variation in a reward prediction error-based learning mechanism (Cohen, 2007; Pickering and Smillie, 2008). However, it has recently become appreciated that such error-driven reinforcement is not exclusive, but instead that the brain contains multiple distinct or even competing pathways for learning from reward (Dickinson and Balleine, 2003; Daw et al., 2005; Rangel et al., 2008). One prominent computational version of this idea (Daw et al., 2005) suggests that the *model-free* reinforcement strategies traditionally associated with error-driven updating are accompanied by an additional system for *model-based* reinforcement learning. Whereas a model-free strategy essentially consists of learning to repeat rewarded actions, model-based algorithms learn a map or model of the structure of the task, and use it to evaluate candidate actions more deliberatively by mental simulations of their consequences. Model-based learning does not rely on reward prediction errors (Gläscher et al., 2010), but it does verifiably contribute to human and even rodent behavior (Dickinson and Balleine, 2003; Daw et al., 2011). The classic example of the distinction between model-free and model-based reinforcement learning is the notion that a rat, when pressing a lever that delivers food, might be doing so for at least two reasons. The first reason, associated with model-free RL, is that the rat has learned that pressing the lever is desirable, because previous leverpresses have been rewarded. The model-based alternative is that the rat might have learned that the lever delivers food, and that the food is desirable, and from this "model" of the action's specific consequences, the rat can conclude that pressing the lever is valuable. This distinction can be tested (an idea going back to Tolman, e.g., Tolman et al., 1946) by examining how subjects adjust their behavior to changes in their goals or the task contingencies: in a way consistent with the model-free reinforcement principle of repeating previously successful actions, or instead in a way that reflects the use of a model of the task contingencies to re-evaluate actions in terms of the newly changed circumstance. In the example of a rat leverpressing, one may ask whether the rat continues to press the lever even if the food is no longer desirable (e.g., if the rat is fed to satiety; Dickinson and Balleine, 2003), as is predicted by model-free but not model-based learning.

Importantly, most laboratory reward tasks do not contain such a manipulation to differentiate which (or what mixture) of these two mechanisms supports learning behavior. Instead, behavior is typically ambiguous as to the underlying learning strategy, and what is apparently the same behavior may reflect different mixtures of their influences in different subjects or circumstances (Dickinson and Balleine, 2003; Daw et al., 2011). In particular, the behavioral tasks so far used to investigate a link between reward learning and extraversion do not establish whether the reward learning behavior is consistent with having been produced by (model-free) reward prediction errors, unconfounded from model-based mechanisms.

This suggests the hypothesis that we test in our present study: that trait extraversion will relate selectively to model-free rather than model-based learning. Such an idea is supported by the links between extraversion, prediction errors, and dopamine, in light of the role of prediction errors in model-free learning. Alternatively, extraversion might not be selective in this manner. For instance, there is some evidence that neural prediction errors (Daw et al., 2011) and dopaminergically mediated learning (Wunderlich et al., 2012) are themselves not entirely selective for model-free learning.

The model-based vs. model-free distinction comes from machine learning (e.g., Sutton and Barto, 1998) and relates most closely to previous theoretical ideas in animal learning and computational neuroscience (Daw et al., 2005). However, this computational distinction may also be related to other dualprocess theories, notably in human cognitive psychology and cognitive neuroscience where researchers have long distinguished between processes that are variously described as automatic, procedural, or incremental vs. deliberative, declarative, or rulebased (e.g., Sloman, 1996; Ashby and Maddox, 2005). In this respect, another previous result suggesting the present hypothesis is a study (Pickering, 2004) that argued that extraversion was selectively linked to procedural rather than rule-based learning (which may parallel model-free vs. model-based; Otto et al., 2013). Specifically, Pickering (2004) reported that in experiments with category learning tasks, performance on conditions requiring integrating information from various stimulus dimensions was linked to extraversion. Conversely, performance on pairedassociate learning tasks was not linked to extraversion. Although these tasks clearly differ on many dimensions, one salient difference is that the former tasks are believed to promote incremental learning and the latter to promote rule-based or memorizationbased solution.

Thus, Pickering's (2004) comparison between the tasks is suggestive, but one advantage of the model-based vs. model-free dichotomy is that the contributions of both processes can be quantified and compared on even ground, in the context of a single task that simultaneously engages both. The present, computational view also substantially refines the more cognitive one, by specifying a quantitative, computational mechanism and situating it in the context of a body of work on animal learning and its mechanistic neural substrates.

Note that whereas in the human literature, the status of learning as explicit vs. implicit has been taken as a key or even defining characteristic of the two processes, the model-based vs. modelfree distinction is defined operationally, in terms of different learning rules, and makes no particular claim about conscious access. However, model-free learning resists (while model-based learning is obliterated by) dual-task interference (Otto et al., 2013) in a manner similar to other signature implicit learning tasks (Nissen and Bullemer, 1987; See Daw and Shohamy, 2008; Daw and O'Doherty, 2013; Otto et al., 2013; for more discussion of the relationships between different dual-process theories).

In the present study we attempt more finely to dissect the relationship between trait extraversion and learning from reward by comparing extraversion to behavior on a two-step decision task which is designed to distinguish model-based from modelfree learning (Daw et al., 2011). The logic of the task, discussed in more detail below, is that the different learning rules predict different patterns of trial-to-trial adjustment of choice preferences in light of the new information given to the participant by each trial's outcome. By examining patterns of switching in this multistep task (where two choices are made in sequence), it is possible to distinguish retrospective, model-free mechanisms (repeating previously successful actions) from more prospective, model-based learning, which evaluates options in terms of their expected consequences at the next step. In previous studies with this task, participants were shown to use a combination of both model-based and model-free decision-making mechanisms, but with substantial individual variation in the balance between them. If the previously reported facilitation of reward learning in extraverts were selective for model-free behavior, this would provide further support for the nexus of function that ties together extraversion and the error-driven learning mechanism.

### **METHODS**

#### **PARTICIPANTS**

We tested two subsamples of participants. All participants were recruited through a New York University message board. Informed consent was obtained from all participants. The first subsample was collected from October to November 2009, *N* = 48 (*M*age = 21*.*7, 68% female). The second subsample was collected from September to October 2012, *N* = 50 (*M*age = 24*.*8, 64% female). There were no significant differences between the two subsamples in terms of age [*t(*95*)* = −1*.*83, *p* = 0*.*07] and gender [χ<sup>2</sup> *(*1*, <sup>N</sup>* <sup>=</sup> <sup>97</sup>*)* <sup>=</sup> <sup>0</sup>*.*182, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*913]. The experimental procedures for both subsamples were identical, with the following exceptions. The first subsample completed 350 trials of the decision-making task, with inter-state and inter-trial intervals of 500 and 300 ms, respectively. The second subsample completed 300 trials of the decision-making task, and the inter-state and inter-trial intervals were 1500 and 1000 ms. (The changes were intended to improve the participants' quality of decisions, as the longer time of the overall procedure, shorter inter-stimuli interval, and inter-trial interval might have imposed a greater cognitive demand on the participants in the first sample.)

One participant was excluded from all analyses because a complete dataset was not obtained due to a software crash. This left 97 participants for the reported results.

### **MEASURES**

Participants began by filling out the extraversion subsection of the EPQ-R questionnaire (Eysenck et al., 1985) via a computer. Next, participants completed 350 or 300 trials (see further explanations in the Participants subsection) of the two-step decision task (Daw et al., 2011). Halfway through the trials, participants took a short break.

The two-step decision-making task was designed to measure the extent to which each individual participant relies on modelbased and model-free learning strategies. The task details were as described by Daw et al. (2011), with the exception that subjects completed more trials separated by shorter inter-event breaks. In the task, participants made a series of two decisions on each trial, and were then given either a single monetary reward, or nothing. The first decision made (i.e., the choice at the first stage) affected the options for the second-stage decision; see the schematic representation of the task in **Figure 1A**.

Specifically, on the first stage, participants were presented with two boxes labeled by Tibetan characters (green boxes, **Figure 1A**). Each box led probabilistically to either of two pairs of secondstage boxes (pink and blue boxes, **Figure 1A**). The two possible second-stage alternatives consisted of another pair of boxes represented by new Tibetan characters (pink and blue boxes, **Figure 1A**). Which of these two pairs of boxes was presented was determined, stochastically, by the first-stage decision. Each option in the second stage was associated with a different probability

**FIGURE 1 | (A)** Schematic representation of the sequential task. Participants are first presented with two first-stage boxes. Each of the boxes (right or left) has a fixed probability of leading to one of the two pairs of second-stage boxes in 70% of cases, and to another pair in 30% of cases. After participants make a first-stage decision, they need to make a second choice, between the second-stage boxes. This leads them to receiving a reward, or none (depicted by a dollar image), based on which second-stage box they choose. **(B)** An example of reward probabilities when choosing the second-stage box. Lines of four different colors represent how probabilities change for four possible second-stage boxes. To encourage participants to learn continually, the reward probabilities diffuse according to independent Gaussian random walks.

of winning a monetary reward (vs. nothing) when chosen. To encourage ongoing learning, the chances of payoff associated with the four possible second-stage options were changed slowly and independently throughout the task, according to independent Gaussian random walks. (At each step, each reward probability was perturbed by adding Gaussian noise with mean zero and *SD* = 0*.*025, with reflecting boundary conditions at 25 and 75%. **Figure 1B** depicts an example of how the win probabilities changed for all four boxes.) The goal of the participants was therefore to earn the most money by tracking which second-stage box was currently most rewarding, and by choosing the first-stage box most likely to lead to it.

The probabilistic coupling between first-stage choices and second-stage options was as follows. Seventy percent of the time (a "common" transition) the choice of each of the first stage boxes led to an associated pair of second-stage boxes. This relationship remained the same over the course of the experiment. The other 30% of the time, however, each first-stage choice led to the other second-stage box not usually associated with it (a "rare" transition). For example, if a participant chose the left green box (**Figure 1A**), in 70% of cases they would experience a common transition and see on stage two a pink pair of boxes, while in 30% of cases they would encounter a rare transition and get the blue pair of boxes. The common/rare transition probabilities were reversed for the right first-stage green box. At the conclusion of the experiment, task winnings were paid in real money at a fractional rate.

#### **ANALYSIS STRATEGY**

Model-free and model-based RL approaches have different consequences for trial-by-trial adjustments in action preferences in light of the events on each trial, which can be assessed by regressing recent rewards on choices (Daw et al., 2011; Wunderlich et al., 2012; Otto et al., 2013). To assess the extent to which a participant relies on a model-based or model-free strategy, we evaluated the effect of events on each trial (trial *n*) of the second-stage choice on the subsequent trial (trial *n* + 1). The two key events on trial *n* were whether or not a reward was received, and whether this occurred after a common or rare transition to the second stage state, given the first-stage choice on trial *n*. We evaluated the impact of these events on the chance of repeating the same first-stage choice on trial *n* + 1.

The logic for this approach (see also Daw et al., 2011) was that model-free RL (e.g., the TD-λ algorithm for λ *>* 0) would tend to repeat a choice that results in reward regardless of in which state that reward occurred, predicting a positive main effect of reward. Model-based RL instead evaluates first-stage actions in terms of the second-stage alternatives they tend to lead to; for this reason, the effect of a reward at the first stage depends in which pair of boxes it was received, and an interaction of reward by transition (common or rare) is predicted. For instance, consider a trial in which a subject chooses the left green box at the first stage, but received the rare (blue) boxes, and was ultimately rewarded for their choice. A model-free learner will be more likely to repeat the first-stage choice following this trial (since it was ultimately rewarded); a model-based learner will, conversely, be more likely to choose the other first-stage choice (since this is the one that is more likely to lead to the blue boxes where the reward was received).

According to this logic, we take the main effect of reward as an index of model-free learning (where larger positive effects indicate more model-free switching) and the reward by transition interaction as an indication of model-based learning (where larger positive effects indicate more model-based switching, since the interaction inverts the sign of the reward effect for rare transitions, which are coded as −1). In previous studies using this task (Daw et al., 2011; Wunderlich et al., 2012; Otto et al., 2013) participants have exhibited a mixture of both effects.

We analyzed these effects using multilevel logistic regression, using the lme4 package (Bates et al., 2012) in the R statistical environment (R Development Core Team, 2011). For each trial after the first, the regression predicted the probability of staying with the previously chosen first stage option (vs. switching) as a function of four population-level predictor variables (which were in later analyses each further interacted with one or two betweensubject covariates). At the level of each subject, the basic model was a 2 × 2 factorial model with factors of reward and transition. This gives rise to four predictors: (1) whether, on the preceding trial, the subject received a reward (1 if rewarded, −1 if unrewarded), (2) whether, the transition from the first-stage to the second-stage choice was common or rare probability (1 if common, −1 if rare), (3) the multiplicative interaction of the reward and transition regressors; (4) an intercept term, which reflects a tendency to perseverate or switch regardless of the events in the task, e.g., regardless whether the previous option was rewarded or not. At the group level, these four effects were all taken as random effects, i.e., each instantiated once per subject from a population distribution. As described below, we also included group-level predictors, such as extraversion, interacted with these factors. Note that only two of these effects—the main effect of reward and its interaction with the transition type—are relevant to the learning model, and only one (the main effect of reward) to our particular hypothesis about model-free learning in this study. The others are included in the model to ensure a more balanced, factorial design.

To assess whether model-based and model-free learning effects covaried with extraversion, the four explanatory variables were each interacted, across subjects, with the participants' extraversion scores. This produced four more group-level coefficients (the main effect of extraversion, its two-way interactions with reward and transition, and the three-way interaction between all factors) characterizing to what extent each of the baseline model parameters changed, across subjects, as a function of their extraversion scores. The extraversion scores were converted to Zscores prior to being entered in the analysis. Again, our main hypothesis concerns the relationship between model-free RL and extraversion (the extraversion by reward effect), with that for model-based RL (the three-way interaction between reward, transition, and extraversion) also of interest, but we estimate a full factorial model with all interactions to ensure that our results are specific to the hypothesized interaction unconfounded by the other, unhypothesized possibilities. In designing and carrying out these analyses, we were guided by Gelman and Hill (Gelman and Hill, 2007; see also Gelman et al., 2003), who tend to advocate against excluding potential explanatory variables, especially in the context of a multilevel model.

Finally, to examine whether the relationship between extraversion and RL task performance was affected or obscured by between-participant variations in task motivation or engagement, we defined a measure of task responsiveness ("engagement"). This overall sensitivity to events in the RL task was measured by fitting the logistic regression described before to each participant's choices individually. At the individual level, this model involves four effects of intercept, reward, transition, and reward by transition, but not the between-subject terms involving interactions with extraversion. We scored each participant's overall sensitivity to the RL task by subtracting the model's deviance from the deviance of a reduced logistic regression model containing only the intercept, i.e., an average tendency to stay or switch but no learning effects at all. The logic of this measure was to characterize the extent to which subjects' choices were responsive to the events in each trial, without assuming either a model-based or modelfree form for this dependence. The difference of deviances is a measure of the relative fit of the two models to the data thus, measuring how much better the choices are explained by assuming the subject adjusts their preferences in light of each trial's outcome according to any combination of the factors of reward and transition, vs. responding at random or with constant preferences. (Specifically, this measure is the test statistic for the likelihood ratio test comparing these models, and is related to the approximate log Bayes factor between them; Kass and Raftery, 1995). In order to be able to obtain unbiased estimates of the relationship between extraversion, the engagement score, and learning strategy, we defined the engagement score using fits to only odd-numbered trials, while we tested the relationship between variables of interest on only the even-numbered trials. This ensured that engagement was defined on a different set of data than those on which its effects were tested; avoiding any bias that otherwise might arise from defining and testing the effect on the same data subset.

We used this engagement score both to define a subgroup of highly responsive participants (the top 20% on this score, across both subsamples) for separate analysis, and also entered it (Z-scored) as a covariate in an additional version of the RL regression, interacted with the basic RL effects and their interactions with extraversion to produce eight more predictors. (Again, the hypothesis concerns the three-way interaction of reward by extraversion by engagement, but we include all factorial interactions to ensure the interpretability of this result.)

The R formulas for these models were:

stay ∼ trans ∗ rew ∗ extra + (1 + trans ∗ rew |subID) and

stay ∼ trans ∗ rew ∗ extra ∗ engage + (1 + trans ∗ rew |subID)

which were estimated using the "glmer" function with family = binomial.

#### **RESULTS**

Subjects completed a two-stage decision task (Daw et al., 2011). They failed to complete a small fraction of trials (average number of missed trials, 1.4%, ±0*.*53 SEM) due to response time limits. They received reward for, on average, 50.8% (±0*.*44 SEM) of their completed trials. **Figure 2** depicts the observed frequency of staying with a top-stage choice as a function of the previous trial's reward and transition, averaged across the sample.

To examine individual differences in subjects' trial-by-trial learning strategies in the RL task, we used a mixed effects logistic regression to explain each trial's first-stage choice in terms of the events on the previous trial (**Table 1**, Daw et al., 2011). As expected, evidence for both model-free and model-based influences on choices was observed at the group level, but with individual variability in their degree. In particular, the reward on a trial significantly predicted the subsequent choice (a marker for model-free RL, see Methods; β = 0*.*198, *Z* = 7*.*36, *p <* 0*.*001), and the interaction of the reward effect with transition (whether the reward was received after a common or rare state transition,

indicative of model-based RL) was also positive (β = 0*.*132, *Z* = 6*.*19, *p <* 0*.*001).

As the data for this study were collected in two subsamples with some variations in task timing (see Methods) we tested for differences between the groups by including an indicator variable for subsample interaction with all effects in the regression. The only such effect that reached significance was the main effect of subsample, indicating that participants in the first subsample tended to switch more often than participants in the second subsample (β = 0*.*22, *Z* = 2*.*62, *p* = 0*.*008). Seeing no differences with respect to the behaviors of interest, we conducted the remaining analyses in this study on data from the combined group of 97 participants.

We examined the relationship between scores on the extraversion scale and RL task performance. The mean score on the extraversion scale for the first subsample was 16.52 with a standard deviation of 4.69, and α = 0*.*85. The mean score for the second subsample was 15.2 with a standard deviation of 5.06, and α = 0*.*85. There was no significant difference in extraversion scores between the two subsamples: *t(*96*)* = −1*.*34, *p* = 0*.*18.

Extraversion scores were included as a covariate in the regression on the RL task. Here, positive interactions with reward or reward by transition would indicate better model-free or model-based RL (respectively) for subjects with higher extraversion scores. This factor interacted significantly with our indicators for both model-free and model-based learning, with higher extraversion indicating a decreased influence of both strategies (**Figures 3A,B**). In particular, the interaction of extraversion with reward (model-free) was negative (*Z* = −2*.*04, *p* = 0*.*041), and the three-way interaction of extraversion, reward, and transition (model-based) was significantly negative (*Z* = −2*.*25, *p* = 0*.*024). Thus, personality scores did not have the hypothesized selective effect on model-free learning, nor even the previously


**Table 1 | RL and extraversion effects for the overall sample of 97 participants, top 20%, and with engagement index: Beta coefficient estimates with standard errors from three mixed effects logistic regression analyses.**

reported facilitatory impact on any sort of reward learning; instead, higher extraversion predicted generally poorer performance on both sorts of RL. In addition, there was a main effect of extraversion: high extraverts tended to switch more than participants low in extraversion (*Z* = −2*.*71, *p* = 0*.*007).

These results raise the possibility that a general disengagement from or unresponsiveness to the experimental task, associated with extraversion, was masking more selective influences of extraversion on learning strategy. Such a generalized relationship between subject performance and extraversion is consistent with reports that under some conditions trait extraversion tends to predict less accurate responses and faster reaction times on certain tasks (Matthews and Gilliland, 1999; Wacker et al., 2006). To examine this possibility, we first repeated our analysis in a sample of 20% of participants (*N* = 20) chosen for the best responsiveness to the RL task (a subject engagement measure measuring the difference in each subject's model fit between the learning model and a null model, with a higher score reflecting higher task engagement; see Methods). For this and the subsequent analysis, we defined task engagement based on the performance on odd trials, and tested behavior by fitting a model to the remaining, even trials. Consistent with the previous finding that higher extraversion predicted worse performance on the RL task, engagement and extraversion trended toward being negatively correlated [*r* = −0*.*19, *t(*95*)* = −1*.*89, *p* = 0*.*062], though this relationship was not significant. Similarly, the extraversion scores for the 20% high performing subjects were on average lower than those for the other 80% [–0.44 vs. 0.11; *t(*28*)* = −2*.*15, *p <* 0*.*05], though well-distributed throughout the range.

Within the subgroup of high performing subjects, higher extraversion scores predicted better learning from reward in a model-free fashion (positive extraversion by reward interaction, Z = 2.02, *p* = 0*.*0432; **Table 1**, **Figure 3C**) together with no significant relationship to model-based learning (*Z* = −0*.*559, *p >* 0*.*5; **Figure 3D**), a result which was in line with the original hypothesis.

Finally, to investigate whether it is indeed the case that in the full sample, the association between extraversion and RL strategy depends on a participant's overall task engagement, we repeated the same regression analysis as previously, but additionally testing the interaction of all factors with the engagement measure (again, defined on a non-overlapping subset of trials to avoid bias). Here, a positive three- or four-way interaction (engagement by extraversion by reward or engagement by extraversion by reward by transition) would indicate a pattern whereby the relationship between extraversion and model free (or, respectively, model-based) learning became more facilitatory for more engaged participants. Indeed, the engagement measure interacted positively with the association between extraversion and modelfree learning (*Z* = 2*.*11, *p* = 0*.*0344) and not significantly with the association between extraversion and model-based learning (*Z* = −1*.*436, *p* = 0*.*1511). Directly comparing these effects using a linear contrast, we verified that the relationship between engagement, extraversion and model-free learning was larger than that for model-based learning (i.e., the effect of engagement is specific to model-free learning; <sup>χ</sup>2*(*1*, <sup>N</sup>* <sup>=</sup> <sup>97</sup>*)* <sup>=</sup> <sup>5</sup>*.*75, *p* = 0*.*01). Thus, to the extent that a participant was more responsive to the task, this was selectively associated with a stronger positive coupling between extraversion and model-free learning.

group level-estimate.

# **DISCUSSION**

Previous studies have shown that extraversion is associated with enhanced reward sensitivity (Pickering, 2004; Smillie et al., 2011). In the current study, we aimed to revisit and refine this association. We assessed the relationship between extraversion and individual differences in the specific, model-free learning strategy most commonly associated with learning from reinforcement in the brain, by using a reinforcement learning task that distinguishes this mechanism from more deliberative, model-based learning that typically confounds it. Contrary to the hypothesis, we found that overall extraversion was associated with poorer reinforcement learning on both model-based and modelfree dimensions, apparently reflecting poor task engagement. However, the hypothesis that extraverts would be better at modelfree RL did hold up in a subset of the more engaged participants, and accordingly, across the full group, higher task engagement was associated (on a different subset of trials) with a shift toward the expected pattern, by which extraversion selectively promoted model-free RL.

**transition type (B), respectively] in the full population and in the top**

At least among the more engaged participants, then, these results demonstrate a relationship between a broad personality orientation and a detailed computational learning mechanism. Moreover, although we are manifestly not in a position to infer any causation and, in this study, did not measure any observables directly related to dopaminergic function, these findings are consistent with other suggestions that both of these aspects of behavior may arise due to a common dopaminergic cause. They also fit well with and sharpen previous results using category learning, which showed a positive association between extraversion and incremental, but not rule-based learning (Pickering, 2004).

At the same time, given such previous reports linking extraversion to improved reward learning, the unhypothesized relationship in our full sample between extraversion and more generically *worse* reinforcement learning performance is puzzling. Especially combined with increased alternation between options from trial to trial in extraverts, the pattern of their choices, which was more weakly sensitive to reward feedback suggests that these participants were simply less engaged with or responsive to the task. It may be that this complex, multistep learning task is more cognitively demanding and/or less engaging than others previously tested with extraversion, promoting a previously subtler tendency among extraverts to disengage. Hints of a tendency toward impatient or careless performance among extraverts might also be seen in previous findings that under certain conditions extraverts tend to be less vigilant and attentive than introverts (Matthews and Gilliland, 1999).

Testing this interpretation remains an important issue for future work. It should be possible to modulate task difficulty (e.g., by manipulating the speed at which options change) within the present task, and/or compare the sequential decision task to traditional one-step tasks so as to examine whether extraverts are sensitive to harder task demands.

In any case, it does appear that overall poor task performance among extraverts in our sample masked the more specific relationship by which (to the extent participants were engaged in the task) extraversion promoted model-free learning. It is also possible that task attentiveness, operationalized by our measure of participant engagement, was capturing the contribution of some additional competing or interacting cognitive or motivational process, which we did not account for in our study. For example, other researchers have pointed out that neuroticism and associated traits can have effects (in some cases, interacting with extraversion) on the performance in learning tasks (Pickering et al., 1995). Further, in tasks similar or identical to the one used here low working memory capacity (Gershman et al., 2012) or concurrent working memory demand (Otto et al., 2013) biases individuals away from model-based choices, leading them to rely on model-free strategy. Future studies can examine cognitive load and personality traits on task performance in RL to investigate whether these other factors mediate or interact with the present results.

Taken together with evidence linking individual differences in pharmacological manipulations of dopaminergic function to performance on learning tasks, our results may indirectly support the idea that individual differences in dopamine are associated with trait extraversion. For instance, individuals with higher baseline synthesis in the striatum demonstrated better learning from rewards in a reversal learning task (Cools et al., 2009), and there are several reports of reward learning deficits in Parkinson's disease that are remediated by dopamine replacement medication (Frank et al., 2004; Bodi et al., 2009). Both in Parkinson's patients (Voon et al., 2010) and healthy participants (Pessiglione et al., 2006), the dopamine precursor L-Dopa promotes learning from reward and reward prediction error-related striatal activity in an instrumental learning task. One note of caution for interpreting the current study's results in dopaminergic terms is that a recent attempt to test the widely hypothesized linkage of dopamine, specifically, to model-free learning using the same task we use here (Wunderlich et al., 2012) instead reported that L-Dopa, paradoxically, promoted model-based over model-free reinforcement

#### **REFERENCES**


et al. (2009). Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinsons patients. *Brain* 132, 2385–2395. doi: 10.1093/brain/awp094

Bush, R. R., and Mosteller, F. (1953). A stochastic model with applications to learning. *Ann. Math. Stat.* 24, 559–585. doi: 10.1214/aoms/ 1177728914

learning. However, (as discussed in that report) there are several interpretations of such result consistent with the otherwise substantial evidence that dopamine subserves a prediction error for model-free temporal difference learning.

Finally, our findings highlight something of a disconnect between the way reward processing in extraversion is viewed through the lens of personality research (which is typically focused on broader life trends and higher level decisions) vs. neuroscientific research (which is typically focused on neural underpinnings of short-term choices in laboratory tasks). In personality psychology there is often a sense that extraversion is beneficial (e.g., associated with positive life outcomes Herringer, 1998; Ryan and Deci, 2001; Williams et al., 2004; Jylhä et al., 2009), whereas the specific model-free learning mechanism linked to extraversion here and to dopamine generally is not necessarily so benign. Notably it is a prominent hypothesis that a dominance of model-free over model-based decisions (or "habitual" over "goal directed" processes) contributes to disorders of compulsion, such as drug abuse (Everitt and Robbins, 2005; Redish et al., 2008). Some complementary results are reported in the personality research field. For instance, Francis (1997) found associations between extraversion and positive attitudes toward substance use in a large sample of pupils between 13 and 15 years old. Further, extraversion positively predicted the number of drugs tried by adolescents whose parents were alcoholics (Conner et al., 2010) and traffic offending in young males (Renner and Anderle, 2000). Although such tentative evidence exists, the possibility that extraversion can predict negative life outcomes and the mechanisms by which it may do so remain largely under-investigated. Results like those in the present study suggest an intriguing and rich relationship between core neurocomputational mechanisms and broader life orientations and outcomes.

# **ACKNOWLEDGMENTS**

We are grateful to Seth Madlon-Kay for useful conversations and assistance and advice on data analysis. This research was funded by a Scholar Award from the McDonnell Foundation, a Scholar Award from the McKnight Foundation, a Young Investigator Award from NARSAD, grant 1R01NS078784 from the National Institute for Neurological Disease and Stroke, and Grant RGP0036/2009-C from the Human Frontiers Science Program. Anya Skatova was funded by a BESTS scholarship and the School of Psychology studentship from University Of Nottingham, and Horizon Digital Economy Research, RCUK grant EP/G065802/1.


*Brain Res*. 25, 851–861. doi: 10.1016/j.cogbrainres.2005.09.018


D'Esposito, M. (2009). Striatal dopamine predicts outcomespecific reversal learning and its sensitivity to dopaminergic drug administration. *J. Neurosci.* 29, 1538–1543. doi: 10.1523/JNEUROSCI.4467-08.2009


habits to compulsion. *Nat. Neurosci*. 8, 1481–1489. doi: 10.1038/nn1579


Eysenck and J.A. Gray: a comparative review. *Pers. Individ. Dif.* 26, 583–626. doi: 10.1016/S0191- 8869(98)00158-5


for statistical computing," in *R Foundation for Statistical Computing* (Vienna). ISBN 3-900051-07-0, URL http://www.R-project.org/


approach. *J. Pers. Soc. Psychol.* 91, 171. doi: 10.1037/0022-3514.91. 1.171


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 17 March 2013; accepted: 13 August 2013; published online: 03 September 2013.*

*Citation: Skatova A, Chan PA and Daw ND (2013) Extraversion differentiates between model-based and modelfree strategies in a reinforcement learning task. Front. Hum. Neurosci. 7:525. doi: 10.3389/fnhum.2013.00525*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2013 Skatova, Chan and Daw. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Perceived stress predicts altered reward and loss feedback processing in medial prefrontal cortex

#### *Michael T. Treadway <sup>1</sup> \*, Joshua W. Buckholtz <sup>2</sup> and David H. Zald3*

*<sup>1</sup> Department of Psychiatry, Center for Depression, Anxiety and Stress Research, McLean Hospital/Harvard Medical School, Belmont, MA, USA*

*<sup>2</sup> Department of Psychology, Harvard University, Cambridge, MA, USA*

*<sup>3</sup> Departments of Psychology and Psychiatry, Vanderbilt University, Nashville, TN, USA*

#### *Edited by:*

*Luke D. Smillie, The University of Melbourne, Australia*

#### *Reviewed by:*

*Kirsten G. Volz, Werner Reichardt Centre for Integrative Neuroscience, Germany James F. Cavanagh, Brown University, USA*

#### *\*Correspondence:*

*Michael T. Treadway, Department of Psychiatry, Center for Depression, Anxiety and Stress Research, McLean Hospital/Harvard Medical School, 234 de Marneffe, Room 231, 115 Mill Street, Belmont, MA 021478, USA. e-mail: mtreadway@ mclean.harvard.edu*

Stress is a significant risk factor for the development of psychopathology, particularly symptoms related to reward processing. Importantly, individuals display marked variation in how they perceive and cope with stressful events, and such differences are strongly linked to risk for developing psychiatric symptoms following stress exposure. However, many questions remain regarding the neural architecture that underlies inter-subject variability in perceptions of stressors. Using functional magnetic resonance imaging (fMRI) during a Monetary Incentive Delay (MID) paradigm, we examined the effects of self-reported perceived stress levels on neural activity during reward anticipation and feedback in a sample of healthy individuals. We found that subjects reporting more uncontrollable and overwhelming stressors displayed blunted neural responses in medial prefrontal cortex (mPFC) following feedback related to monetary gains as well monetary losses. This is consistent with preclinical models that implicate the mPFC as a key site of vulnerability to the noxious effects of uncontrollable stressors. Our data help translate these findings to humans, and elucidate some of the neural mechanisms that may underlie stress-linked risk for developing reward-related psychiatric symptoms.

**Keywords: medial prefrontal cortex (mPFC), perceived stress, reward processing, insula, Monetary Incentive Delay task**

# **INTRODUCTION**

Alterations in reward-seeking and goal-directed behavior are a common symptom of mental illness. In the broadest sense, such alterations usually reflect a shift in how different options in the environment are valued and pursued, resulting in either a reduced motivation for experiences that were previously found to be rewarding (Treadway and Zald, in press), or a heighted sense of craving for particular rewards (e.g., drugs, food) (Volkow, 2004; Everitt and Robbins, 2005). While progress has been made in identifying the neural systems that participate in reward processing behavior, many questions remain as to how these systems become dysfunctional in clinical populations.

Exposure to stress is a central risk factor in the development of psychiatric conditions characterized by prominent abnormalities in reward-related processes, such as depression, schizophrenia, and substance use (Kessler, 1997; Kendler et al., 1999, 2004; Sinha, 2001, 2008; Yuii et al., 2007). The term stress describes physically or emotionally demanding circumstances, frequently involving the real or imagined threat of loss or pain (McEwen, 2007). This can include either physical or emotional pain, and may occur in the context of professional, social and familial relationships. A wealth of data suggests that stress exposure alters how individuals process and make decisions about rewards in their environment (Bogdan and Pizzagalli, 2006; Koob and Kreek, 2007; Pascucci et al., 2007; Pizzagalli et al., 2007; Arnsten, 2009; Dias-Ferreira et al., 2009; Schwabe and Wolf, 2009; Cavanagh et al., 2010; Cabib and Puglisi-Allegra, 2011; Mather and Lighthall, 2012; Shafiei et al., 2012). In particular, stress has been found to blunt sensitivity to new information about future rewards, a phenomenon that has been demonstrated across a variety of experimental paradigms. For example, under conditions of elevated stress, subjects were less sensitive to reinforcement contingencies during a signal-detection paradigm (Bogdan and Pizzagalli, 2006; Pizzagalli et al., 2007; Bogdan et al., 2011). Similarly, subjects show diminished reinforcer devaluation immediately following stress, suggesting that stress can produce habitual response patterns that are resistant to changes in external or internal conditions (e.g., satiety) (Dias-Ferreira et al., 2009; Schwabe and Wolf, 2009; Lemmens et al., 2011).

A variety of evidence highlights a corticostriatal circuit encompassing the striatum and medial prefrontal cortex (mPFC) as a critical neurobiological substrate for stress-borne alterations in reward processing. Data from preclinical studies suggest that stress produces rapid changes in catecholamine levels (Abercrombie et al., 1989; Pascucci et al., 2007), gene expression (Ons et al., 2010; Wang et al., 2010), and local circuit remodeling (Arnsten, 2009) within these areas. Corroborating observations have been made in human neuroimaging studies; where stress has been shown to increase dopamine release (Scott et al., 2006; Soliman et al., 2008; Lataster et al., 2011) and alter neural responses to reward decision-making and anticipation (Ossewaarde et al., 2011; Mather and Lighthall, 2012; Schwabe et al., 2012).

While these studies have helped identify the systems-level mechanisms that underlie responses to an acute stressor, they generally do not address questions regarding the biological basis of individual differences in how stressors are perceived. This issue is critical for understanding how stress confers risk for developing psychopathology, as epidemiological studies reveal that individuals who consider stressful experiences to be uncontrollable and overwhelming are substantially more likely to develop psychiatric symptoms following stress exposure (Kendler et al., 1993, 2004; Kessler, 1997). This is particularly true for symptoms related to impaired reward-reward processing, such as anhedonic symptoms in depression and schizophrenia (Kuiper et al., 1986; Docherty, 1996; Myin-Germeys et al., 2001; Hammen, 2005; Myin-Germeys et al., 2005; Phillips et al., 2005; Rao et al., 2009). Highlighting the importance of this distinction, rodent models suggest that uncontrollable stressors produce a unique pattern of neurobiological changes, particularly in the mPFC (Cabib and Puglisi-Allegra, 1994, 2011; Bland et al., 2003; Amat et al., 2005; Maier and Watkins, 2010). As compared to controllable stressors (i.e., paradigms where instrumental action may alleviate the stressor), repeated exposure to uncontrollable stressors can result in learned helplessness behavior and anhedonia (Seligman et al., 1968; Willner et al., 1992a,b; Amat et al., 2008).

The effects of recent stress perceptions on reward-processing in otherwise non-stressful contexts has not been wellcharacterized. Recent neuroimaging work in humans has focused on the use of experimental paradigms that combine measures of reward processing with laboratory stress manipulations, which can elucidate some of the neural mechanisms underlying changes in reward-related behavior immediately following exposure to stressful stimuli (Ossewaarde et al., 2011; Mather and Lighthall, 2012; Porcelli et al., 2012). However, fewer studies have examined how such networks are affected by perceptions of stress over a longer time period. Consequently, the goal of the current study was to explore associations between reward processing and reported perceptions of stressors in the preceding month. The advantage of this design is its ability to explore the consequences of recent levels in perceived stress on neural networks supporting reinforcement, which may help explain how prior stress exposure can alter reward circuitry so as to confer risk for the subsequent development of psychopathology.

To address this question, we recruited a sample of healthy community volunteers who completed a measure of perceived stress over the past month, and then performed a behavioral rewardprocessing task during a functional magnetic resonance imaging (fMRI) scan. Recent levels of perceived stress were assessed using the Perceived Stress Scale (PSS) (Cohen et al., 1983), a widelyused instrument that measures the frequency, severity, and perceived controllability of daily stressors over the previous 1-month period. The PSS has been previously linked to risk for the development of both physical and mental health symptoms (Kuiper et al., 1986; Cobb and Steptoe, 1996; Culhane et al., 2001), as well as elevations in stress hormones (Pruessner et al., 1999) and inflammation (Maes et al., 1999). More importantly for the aims of the current study, the PSS has been linked to alterations in reinforcement learning assessed using a signal detection task (Pizzagalli et al., 2007). To assess the effects of perceived stress on reward processing, subjects were scanned while performing a monetary-incentive delay (MID) task (Knutson et al., 2000). The MID is a well-validated neuroimaging paradigm that probes neural responses to anticipation of reward (i.e., motor preparation to pursue reward) as well as integration of reward feedback. While the former condition typically engages the ventral striatum, the latter condition recruits mPFC, including aspects of pregenual anterior cingulate cortex (ACC), anterior cingulate sulcus, and Broadmann area 10 (Knutson et al., 2003, 2005). Importantly, the MID has previously been used to identify alterations in neural responses to reward processing in psychiatric populations with reward-related symptoms (Juckel et al., 2006; Pizzagalli et al., 2009).

Given the evidence reviewed above that stress is associated with diminished sensitivity to reward information (Bogdan and Pizzagalli, 2006; Pizzagalli et al., 2007; Schwabe and Wolf, 2009; Bogdan et al., 2011) and that the striatum and mPFC may be particularly critical nodes involved in responses to perceived stress (Cabib and Puglisi-Allegra, 1994, 2011; Amat et al., 2005), the MID appears especially well-suited as a task to probe neural activity in reward-related networks that may be a priori predicted to be affected by levels of perceived stress.

# **METHODS**

#### **PARTICIPANTS**

Participants were 38 volunteers recruited from the community. Subject ages ranged from 18 to 34, with a mean age of 22. Roughly equal numbers of men (*n* = 20) and women (*n* = 18) participated. All subjects were screened for any contraindications for participation in an MRI study, e.g., obesity, claustrophobia, cochlear implant, metal fragments in eyes, cardiac pacemaker, neural stimulator, and metallic body inclusions or other metal implanted in the body, pregnancy.

#### **MEASURE OF RECENT CHRONIC STRESS**

To assess recent levels of chronic stress, all subjects were administered the PSS. The PSS is a well-validated brief self-report measure that has been widely used as an index of current-levels of chronic daily-life stressors (Cohen et al., 1983). Subjects are asked to rate the frequency and intensity of stressful events that have occurred over the most recent one-month period. The PSS also incorporates items that ask subjects to rate the perceived predictability and controllability of these stressors, as well has how overwhelmed they felt. Examples of items from this measure include "In the last month, how often have you felt difficulties were piling up so high that you could not overcome them?" and "In the last month, how often have you felt nervous or 'stressed?"' Each item is rated using a 0–4 scale where 0 is defined as "never" and 4 is defined as "very often," and scores are generated by summing across the total number of items (after appropriate reverse-coding for 4 of the 10 items). Internal reliabilities (Cronbach's-α) across the 10-item scale were recently reported to be 0.91 in two separate national surveys that each included 2000 participants (Cohen and Janicki-Deverts, 2012). The maximum score on this measure is 40, and the minimum is 0. While the PSS is not a clinical instrument and therefore has no "cut-off " score related to diagnostic categories, it has been found to predict mental and physical health outcomes, including vulnerability to infections disease (Cobb and Steptoe, 1996; Culhane et al., 2001) and depression (Kuiper et al., 1986). More specifically to the domain of reward processing, the PSS has been found to predict decreases in reward sensitivity using a signal-detection reinforcement task (Pizzagalli et al., 2007).

#### **MONETARY INCENTIVE DELAY (MID) TASK**

The Monetary Incentive Delay (MID) task is a widely used assessment of neural circuitry associated with reward anticipation and processing of reward feedback (Knutson et al., 2000, 2003, 2005) (see **Figure 1**). Details of our MID task and fMRI scanning protocol have been published previously (Buckholtz et al., 2010). Briefly, during the task participants have the opportunity to win or lose money by pressing a button during the very brief presentation of visual target stimulus. For each trial, participants are shown one of seven cues, indicating that they have the potential to win money (reward magnitude range = \$0*.*20, \$1.00, \$5.00; *n* = 74), the potential to avoid losing money (loss magnitude range = \$0.20, \$1.00, \$5.00; *n* = 69), or that no money was at stake for that trial (No Change trials; *n* = 37). Subjects were instructed to fixate on a cross-hair during a variable interval of 2000–2500 ms (anticipatory delay phase), and then respond to a white target square that appeared for a variable length of time (target phase, 160–260 ms) with a button press. For Potential Win trials, participants were told that if they successfully pressed the button while the target was onscreen (a "hit") they won the amount of money indicated by the cue, while there was no penalty for failing to press the button while the target was onscreen (a "miss"). For Potential Loss trials, participants were told that no money was won or lost for hits, but misses would lead to a loss of the amount indicated by the cue for that trial. A feedback screen (outcome phase, 1650 ms) followed the target's disappearance. The feedback screen notified participants how much money they won or lost during that trial, and indicated their cumulative total winnings at that point. Even though no money was at stake during the No Change trials, participants were instructed to rapidly press the button during the display of the target stimulus.

Before entering the scanner, participants completed a practice version of the task and were shown the money that they could earn by performing the task successfully. Based on reaction times obtained during the prescan practice session, target

reward, or experience no change if they successfully pressed a button before a visual target disappeared on the screen. After the trial cue presentation,

durations were adjusted such that each participant succeeded on approximately 66% of his or her responses. Each MID task session is comprised of 4 functional runs, each approximately 7.73 min long. The MID was programmed in E-Prime (http:// www*.*pstnet*.*com/products/e-prime/) and run off of a dedicated Pentium computer from the scanner control room. The visual display was presented on an LCD panel and back-projected onto a screen positioned at the front of the magnet bore. Subjects lay supine in the scanner and viewed the display on a mirror positioned above them. Manual responses were recorded using a keypad (Rowland Institute of Science, Cambridge MA).

#### **fMRI DATA ACQUISITION**

All fMRI scans were performed on two identically configured 3 Tesla Phillips Achieva scanners located at the Vanderbilt University Institute for Imaging Science (VUIIS). T1-weighted high-resolution 3D anatomical scans were obtained for each participant (FOV 256 × 256, 1 × 1 × 1 mm resolution). Fast spin echo axial spin density weighted (*TE* = 19, *TR* = 5000, 3 mm thick) and T2-weighted (*TE* = 106, *TR* = 5000, 3 mm thick) slices were obtained to exclude any structural abnormalities. Additionally, a field map was additionally collected in order to remove distortion caused by inhomogeneity. Functional (T2∗ weighted) images were acquired using a gradient-echo echoplanar imaging (EPI) pulse sequence with the following parameters: *TR* = 2000 ms, *TE* = 25 ms, flip angle 90◦, FOV 240 × 240 mm, 128 × 128 matrix with 30 axial oblique slices (2.5 mm, 0.25 mm gap) oriented approximately 15 degrees from the AC-PC line. The slice prescription was adjusted for each subject to ensure coverage of the midbrain, ventral striatum, amygdala, mPFC, and orbital gyrus. Higher-order shimming was employed to compensate for magnetic field inhomogeneity in the orbitofrontal/ventral striatal region. fMRI volume acquisitions were time-locked to the offset of each cue and each target, so were thus acquired during anticipatory and during outcome periods. 242 volumes were acquired for each functional run.

#### **fMRI DATA PREPROCESSING AND ANALYSIS**

Prior to random effects analysis in SPM5, all fMRI time series data received conventional preprocessing, including slice-timing correction, spatial realignment, normalization into a standard stereotactic space (MNI) and smoothed with a 6 mm full-width-half

disappeared. Afterwards, subjects received feedback as to whether or not they had been successful, and what the monetary outcome was for the trial.

maximum gaussian kernel. Functional images were slice-time corrected using the middle slice as a reference, motion corrected via spatial realignment (4th degree B-spline) of all images to a mean image after alignment to the first image of each run. Following realignment, the Fieldmap toolbox was used to create voxel displacement maps (VDMs) from static magnetic field (B0) maps acquired during each scan session. These VDMs were used to correct for susceptibility-X-movement-related distortions in the EPI images. These distortion-corrected images were then co-registered to the subject's anatomical image. Images were spatially normalized (4th degree B-spline) into a standard stereotactic space (MNI template), re-sampled into 2 mm isotropic voxels, and smoothed with a 6 mm full-width-half-maximum gaussian kernel. We then applied a high-pass filter (128 s cutoff) to remove low-frequency signal drift. Each subject's data were inspected for excessive motion—only subjects with *<*3 mm motion in every direction across all runs were included in analyses. For single-subject analyses, trials were pooled across the levels of monetary value for a given condition. Onsets for the anticipatory delay period and for the feedback period of each of the three trial types were separately modeled using a canonical hemodynamic response function (HRF) with a time derivative. In addition, six head motion parameter estimates (translation in *x*, *y*, *z*; roll, pitch, yaw) were included as covariates in the design matrix. Each run was modeled separately. We then contrasted the beta-weights of repressors using a *t*-test between trial types to create, for each subject, a contrast image showing voxels that were differentially activated as a function of task conditions.

Based on our *a priori* hypotheses regarding the relationship between perceived stress and corticostriatal function, our group analyses included associations between PSS scores and neural activity during both the anticipatory and feedback phases. For the anticipatory phase, we separately examined the relationship between PSS scores and contrasts of Potential Win Anticipation *>* No Change Anticipation and Potential Loss Anticipation *>* No Change Anticipation. Note that these analyses included all trials for each of the conditions regardless of the outcome of the trial. In contrast, analyses of the feedback phase were dependent upon the outcome of the trial. Because we were primarily interested in responses to gains or losses, analysis of the Feedback phase focused on the contrasts of Win Feedback *>* No Change Feedback and Loss Feedback *>* No Change Feedback. For the Win Feedback *>* No Change Feedback contrast, we only modeled trials in which the subject had successfully achieved a "Hit," meaning they had responded before the target disappeared from the screen, and therefore received feedback indicating a monetary gain of the amount available on the given trial. Potential Win and No Change trials where the subject failed to respond quickly enough (a "Miss") were not included in this contrast because there was no change in money in those trials. Conversely, for Loss Feedback trials, we only modeled trials in which the subject failed to respond before the target disappeared from the screen ("a Miss"), and received feedback informing them that they had lost money. For the Loss Feedback contrast, we did not model Potential Loss or No Change trials in which the subject achieved a "Hit" and avoided a loss of money because there was no change in money on those trials.

Random effects analyses of fMRI data were performed in SPM5 by regressing subjects' perceived stress scores against contrast images with subject sex and scanner as covariates in the model. While effects of sex on reward processing were not the focus of the current study, past studies have suggested the possibility of sex differences in response to stress (e.g., Mather and Lighthall, 2012). Consequently, to control for the possible differences of sex, we included it as a covariate. This approach has been used in a number of prior publications involving individual differences in reward processing from our lab (e.g., Buckholtz et al., 2010).

All analyses were whole-brain, and SPMs were explored using a voxel-wise threshold of *p <* 0*.*005 (uncorrected) and a minimum cluster extent of 20 voxels. Whole-brain correction for multiple comparisons was achieved using a cluster-extent correction procedure as implemented in SPM5. Only results surviving this cluster-correction (*p*cluster *<* 0*.*05) are reported. For contiguous clusters that spread across multiple regions, the automated labeling atlas (AAL) was used to divide clusters so as to differentiate between structures. After significant clusters had been identified, parameter estimates reflecting task-dependent changes in BOLD signal for each subject were extracted and entered into SPSS19 (IBM, Armonk, NY) for the purposes of visualization.

#### **RESULTS**

#### **PSS SCORES**

Subject scores on the 10-item PSS ranged from 0 to 32 (*M* = 14*.*7, *SD* = 7*.*5) out a maximum possible score of 40 on the instrument. These results are consistent with normative data on this instrument for subjects within this age range (*M* = 14*.*2, *SD* = 6*.*2) (Cohen and Williamson, 1988).

#### **NEUROIMAGING DATA: MID RESULTS** *Win and loss feedback*

Consistent with numerous prior studies using the MID task, a contrast of Win Feedback *>* No Change Feedback revealed increased BOLD signal in bilateral mPFC encompassing aspects of pregenual cingulate and medial prefrontal gyrus (Peak: *x* = −6, *y* = 44, *z* = −2; *Z*-score = 6*.*13; *k* = 763; *p*cluster *<* 0*.*001) [all coordinates are given in the stereotactic space of the Montreal Neurological Institute (MNI)]. A similar region of mPFC of was identified in the processing of monetary losses during the contrast of Feedback Loss *>* No Change Feedback, where subjects received feedback that they had missed the target and therefore experienced a monetary loss (Peak: *x* = −8, *y* = 48, *z* = 14; *Z*-score = 4*.*01, *k* = 140, *p*cluster = 0*.*034) (see **Table 1**).

#### *Potential reward and loss anticipation*

Also in keeping with prior findings using the MID, we observed robust activation in the ventral striatum during the contrast of Potential Win Anticipation *>* No Change Anticipation, as well as activity in amygdala, hippocampus, insula, mPFC, thalamus **Table 1 | Brain regions activated during reward anticipation and feedback conditions of the MID task.**


and cerebellum. A similar pattern of activation was obtained during the contrast of Potential Loss Anticipation *>* No Change Anticipation (see **Table 1**).

### **NEUROIMAGING DATA: CORRELATIONS WITH PERCEIVED STRESS** *Reward and loss feedback*

We regressed PSS scores against reward feedback activity during the Win Feedback *>* No Change Feedback contrast, and found a significant inverse association in bilateral mPFC, primarily in pregenual ACC and cingulate sulcus (Peak: *x* = 0, *y* = 50, *z* = 4; *Z*-score = 3*.*53; *k* = 132, *p*cluster = 0*.*023) (see **Table 2**; **Figure 2**). This association suggests that individuals reporting higher levels of stress in the preceding month exhibited diminished amounts of BOLD signal in this region.

We next examined the relationship between perceived stress and reward feedback activation during the Loss Feedback *>* No Change Feedback contrast, and again found a significant **Table 2 | Brain regions showing an association with PSS scores.**


inverse association in mPFC (Peak: *x* = −6, *y* = 46, *z* = 8; *Z*-score = 3*.*62; *k* = 132; *p*cluster = 0*.*041) as well as a region of left anterior insula (Peak: *x* = −44, *y* = 26, *Z*-score = 4*.*17; *k* = 182; *p*cluster = 0*.*009) (see **Table 2**; **Figure 3**). This finding

suggests that higher PSS scores were associated with reduced neural responses in both mPFC and insula when subjects received feedback that they had experienced a monetary loss.

#### *Potential reward and loss anticipation*

There were no suprathreshold voxels showing an association between PSS scores and neural activity during the anticipation phase for either the Potential Win Anticipation *>* No Change or Potential Loss Anticipation *>* No Change contrasts.

# **DISCUSSION**

The present study tested the relationship between individual differences in perceptions of recent life stressors and corticostriatal circuit functioning during reward processing. We found that higher levels of perceived stress were associated with diminished neural responses in the mPFC when subjects received feedback about monetary rewards and losses. These findings support a growing body of evidence implicating the mPFC as a critical region for stress-linked changes in reward processing.

The localization to mPFC is notable for several reasons. First, mPFC is known to be structurally vulnerable to chronic stress. Numerous independent studies in animals have shown that chronic stress incites a retraction of dendritic morphology within the mPFC (Cook and Wellman, 2004; Radley et al., 2005, 2006a,b; Cerqueira et al., 2007); for a review, see McEwen (2007), impairing its capacity to communicate with other striatal and limbic regions involved in reward salience and learning (Dias-Ferreira et al., 2009). While the mechanisms of this susceptibility are not fully understood, strong evidence suggests that prefrontal glucocorticoid elevations play a key role (McEwen, 2007): along with the hippocampus, the mPFC expresses a high number of glucocorticoid receptors (Chao et al., 1989; Ahima and Harlan, 1990; Patel et al., 2000), and participates in negative feedback regulation of glucocorticoid release (Akana et al., 2001; Mizoguchi et al., 2003). Further, site-injections of glucocorticoids have been found to mimic the structural consequences of chronic stress within mPFC (Wellman, 2001; Cerqueira et al., 2005a,b, 2007). Consistent with these preclinical findings, elevated cortisol levels in humans have been found to correlate with reduced gray matter volume in this region (Castro-Fornieles et al., 2009; Treadway et al., 2009).

Such stress-related microdamage in mPFC impacts a variety of cognitive processes (Liston et al., 2006; McEwen, 2007). In the context of reward, stressors can increase habitual response patterns that are insensitive to changing reinforcement context. (Schwabe and Wolf, 2009; Soares et al., 2012). Importantly, this stress-induced shift toward habitual responding has been linked to diminished mPFC activity in response to reward information (Schwabe et al., 2012). Consistent with the current findings, these data suggest that stress-induced shifts in mPFC function—possibly reflecting structural microdamage (Dias-Ferreira et al., 2009)—may impair appropriate encoding of value information. This proposed role for mPFC function is consistent with electrophysiological data recorded in nonhuman primates, where individual neurons within mPFC especially the ACC and cingulate sulcus—have been shown to play a vital role in incorporating reward feedback information as a means of encoding action-outcome relationships and updating values for subsequent behaviors (Wallis and Kennerley, 2010). Our data would appear to corroborate this model, suggesting that elevated stress reduces the capacity to accurately encode the appropriate salience of new information. In keeping with this proposal, individual differences in the PSS have been previously linked to decreased sensitivity to reinforcement information during a signal detection task (Pizzagalli et al., 2007).

Somewhat unexpectedly, we did not observe any associations between perceived stress and neural activity during the anticipation phase. On the surface, this is surprising, as several fMRI studies using acute stressors have observed direct effects on reward anticipation and anticipatory decision-making, rather than reward feedback (Ossewaarde et al., 2011; Mather and Lighthall, 2012; Porcelli et al., 2012). This discrepancy may partly reflect the fact that unlike studies that use an acute, inthe-moment stress manipulation to examine neural responses to stress (Ossewaarde et al., 2011; Mather and Lighthall, 2012; Porcelli et al., 2012), the current study used the PSS to test the association between a recent history of elevated stress perceptions to reward and loss processing. It is increasingly recognized that the neural mechanisms governing acute vs. chronic stressors are somewhat distinct (Cabib and Puglisi-Allegra, 2011). Moreover, animal models suggest that it is chronic stress that is most likely to affect the various forms of structural microdamage in mPFC discussed above. Consequently, the selective associations between PSS scores and feedback-related activity may reflect the duration of stress that is captured by the PSS. In addition to this temporal dimension, the PSS assesses subjects' perceptions of their ability to cope with, control and adapt to stressful experiences. Perceived controllability has marked effects on the neurobiological consequences of stress, and has similarly been localized to mPFC (Cabib and Puglisi-Allegra, 1994; Amat et al., 2005, 2008; Pascucci et al., 2007; Maier and Watkins, 2010). Additional research will be required to fully understand the implications of these divergent responses in mPFC as a function of chronicity and subjective perception. That said, it should be emphasized that it is stressors that are experienced as being chronic, unpredictable and uncontrollable that are most likely to increase risk for psychopathology, rather than acute stressors (Docherty, 1996; Kessler, 1997; Kendler et al., 2004; Hammen, 2005).

It is also worth commenting on the similar pattern of results observed for both the "Win" and "Loss" conditions. This stands in contrast with a number of recent papers showing divergent effects of stress on reward learning and decision-making, where acute stress has been found to selectively facilitate learning about wins while impairing learning about punishment (Petzold et al., 2010; Cavanagh et al., 2011; Mather and Lighthall, 2012; Porcelli et al., 2012). Interestingly, one distinction that emerged between the two conditions was that perceived stress was associated with decreased left anterior insula activity during the Loss trials, but not the Win trials. The anterior insula is increasingly recognized as an important region in value-based decision-making in general (Weller et al., 2009; Treadway et al., 2012) and punishmentavoidance learning in particular (Kim et al., 2006; Pessiglione et al., 2006; Samanez-Larkin et al., 2008; Palminteri et al., 2012). Moreover, alterations in anterior insula activity during reward decision-making have been observed as a consequence of stress (Mather and Lighthall, 2012). Given the apparent valence-specific role of the anterior insula in avoidance-learning, it is intriguing that neural activity in this region showed an association with perceived stress only during the loss condition. As with mPFC, reduced activity in this region during feedback may contribute to decreased encoding of reinforcer information following stress.

In sum, the current findings help identify how variation in perceived stress influence neural circuitry involved in responding to reward feedback information. Understanding how the brain is affected by elevated stress load is important for understanding stress-linked risk for psychopathology. Our findings primarily highlight the mPFC, which is widely implicated in a number of fundamental cognitive processes related to affect regulation (Ochsner and Gross, 2005; Etkin et al., 2006), value-based decision-making (Rushworth et al., 2004; Wallis and Kennerley,

# **REFERENCES**


cortex structure and function. *Nat. Rev. Neurosci.* 10, 410–422.


2010), and self-evaluation and negative self-judgment (Enzi et al., 2009). Importantly, structural, functional, and neurochemical alterations in mPFC have been reported across a number of psychiatric diagnoses (Coryell et al., 2005; Fitzgerald et al., 2008; Goldstein et al., 2009; Koch et al., 2009; Shin et al., 2009; Fineberg et al., 2010; Treadway and Zald, 2011; Gabbay et al., 2012; Keating et al., 2012). Taken together these findings implicate mPFC as a transdiagnostic nexus, wherein dysfunction predisposes diverse forms of psychopathology that, while categorically distinct, may be symptomatically related due to shared deficits in mPFCsubserved cognitive processes (Buckholtz and Meyer-Lindenberg, 2012). While our study did not include a patient sample, the present data indicate that associations with perceived stress can be observed even in samples with no overt psychopathology. Given the well-known link between perceived stress and the risk for developing such disorders, our data support the hypothesis that the mPFC is a critical node of vulnerability for developing stress-linked reward processing symptoms.

# **ACKNOWLEDGMENTS**

This research was funded by the National Institute on Drug Abuse (R01DA019670-04) to David H. Zald and the National Institute of Mental Health (F31MH087015-01) to Michael T. Treadway.

controllable and uncontrollable aversive experiences. *J. Neurosci.* 14, 3333–3340.


Treadway et al. Perceived stress and mPFC

(2005). Subgenual prefrontal cortex volumes in major depressive disorder and schizophrenia: diagnostic specificity and prognostic implications. *Am. J. Psychiatry* 162, 1706–1712.


task in cocaine addiction. *Proc. Natl. Acad. Sci. U.S.A.* 106, 9453–9458.


pathways, and the transition to drug dependence. *Am. J. Psychiatry* 164, 1149–1159.


daily life stress in psychosis. *Arch. Gen. Psychiatry* 58, 1137–1144.


response to rewards in unmedicated individuals with major depressive disorder. *Am. J. Psychiatry* 166, 702–710.


B. (2008). Individual differences in insular sensitivity during loss anticipation predict avoidance learning. *Psychol. Sci.* 19, 320–323.


*Transl. Psychiatry* 2:e131. doi: 10.1038/tp.2012.59


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 12 January 2013; accepted: 22 April 2013; published online: 20 May 2013.*

*Citation: Treadway MT, Buckholtz JW and Zald DH (2013) Perceived stress predicts altered reward and loss feedback processing in medial prefrontal cortex. Front. Hum. Neurosci. 7:180. doi: 10.3389/fnhum.2013.00180*

*Copyright © 2013 Treadway, Buckholtz and Zald. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

# The role of the dorsoanterior striatum in implicit motivation: the case of the need for power

#### *Oliver C. Schultheiss <sup>1</sup> \* and Anja Schiepe-Tiska2*

*<sup>1</sup> Department of Psychology, Friedrich-Alexander University, Erlangen, Germany*

*<sup>2</sup> Centre for International Student Assessment (ZIB), TUM School of Education, Technische Universität München, Germany*

#### *Edited by:*

*Jan Wacker, Philipps-Universität Marburg, Germany*

#### *Reviewed by:*

*Markus Quirin, University of Osnabrueck, Germany Rajendra Badgaiyan, University at Buffalo, USA*

#### *\*Correspondence:*

*Oliver C. Schultheiss, Department of Psychology, Friedrich-Alexander University, Nägelsbachstrasse 49b, 91052 Erlangen, Germany. e-mail: oliver.schultheiss@ psy.phil.uni-erlangen.de*

Implicit motives like the need for power (nPower) scale affective responses to need-specific rewards or punishments and thereby influence activity in motivational-brain structures. In this paper, we review evidence specifically supporting a role of the striatum in nPower. Individual differences in nPower predict (1) enhanced implicit learning accuracy, but not speed, on serial-response tasks that are reinforced by power-related incentives (e.g., winning or losing a contest; dominant or submissive emotional expressions) in behavioral studies and (2) activation of the anterior caudate in response to dominant emotional expressions in brain imaging research. We interpret these findings on the basis of Hikosaka et al.'s (2002a) model of central mechanisms of motor skill learning. The model assigns a critical role to the dorsoanterior striatum in dopamine-driven learning of spatial stimulus sequences. Based on this model, we suggest that the dorsoanterior striatum is the locus of nPower-dependent reinforcement. However, given the centrality of this structure in a wide range of motivational pursuits, we also propose that activity in the dorsoanterior striatum may not only reflect individual differences in nPower, but also in other implicit motives, like the need for achievement or the need for affiliation, provided that the proper incentives for these motives are present during reinforcement learning. We discuss evidence in support of such a general role of the dorsoanterior striatum in implicit motivation.

#### **Keywords: implicit motives, personality, reinforcement, learning, dopamine, power motivation, striatum, caudate nucleus**

Implicit motives represent enduring non-conscious, affect-based preferences that drive humans' behavior toward the attainment of certain types of incentives, such as those related to power/dominance, social affiliation, attachment, achievement/mastery, food, or sex that are fundamental for survival in the social and non-social world (Schultheiss and Wirth, 2008; Schultheiss and Brunstein, 2010). The need for power (nPower) is an implicit motivational disposition to experience one's own impact on others as rewarding and others' impact on oneself as aversive (Winter, 1973; Schultheiss, 2008). Research has accumulated evidence for a critical role of this need in implicit learning of behavior that is instrumental for obtaining rewards and avoiding punishers in the power domain. Other research suggests an involvement of the dorsoanterior striatum in nPower-associated responses to power incentives. In the present paper, we first briefly review these two lines of research and then integrate them into a model of nPower-dependent individual differences in instrumental learning mediated by the dorsoanterior striatum. In closing, we will discuss the role of the striatum in the context of other motivational needs.

#### **nPOWER: MEASUREMENT AND VALIDITY AS A MOTIVATIONAL NEED DISPOSITION**

Measures of nPower were developed and successively finehoned through studies in which researchers studied effects of experimentally aroused power motivation on the content of imaginative stories that research participants wrote about picture cues (Veroff, 1957; Uleman, 1972; Winter, 1973). In this way, content-coding systems for nPower were derived that have causal validity (see Borsboom et al., 2004) and that capture the themes that power-motivated people spontaneously think about and inject into picture stories. Story themes related to nPower can be objectively coded from picture stories, as documented by inter-rater reliabilities of typically *>*0.85 (Schultheiss and Pang, 2007). nPower scores derived from content-coding have good retest reliability (Schultheiss and Pang, 2007) and are particularly suitable for predicting spontaneous behavior in response to non-verbal incentives, long-term behavioral trends, and health outcomes such as immune system functioning and cardiovascular disease (McClelland, 1989; Schultheiss, 2008; Fodor, 2010). Notably, nPower is considered to be an implicit motive, because content-coded nPower does not generally correlate with questionnaire measures of self-attributed (i.e., explicit) power motivation, dominance, or aggression (e.g., Pang and Schultheiss, 2005; Stanton and Schultheiss, 2007), and neither do these explicit measures account for the motivational outcomes and phenomena that nPower predicts (for reviews of the differences between implicit and explicit motive measures, see McClelland et al., 1989; Schultheiss, 2008; Stanton et al., 2010).

Like other implicit motives (e.g., the needs for achievement, affiliation, or intimacy), nPower determines the degree to which a person finds pleasure in, or*likes* (cf. Berridge, 2003), a particular class of rewards, which, in the case of nPower, consist of episodes in which the person has impact on others or dominates others. It also determines the degree to which a person experiences as aversive, or dislikes, a particular class of punishments, such as failing to have impact on others or being the object of others' dominance. Individual differences in nPower thus correspond to individual differences in the reward and punishment value of such episodes and, as a consequence, in the intensity and frequency with which a person strives for them or *wants* them in the case of power-specific rewards or wants to avoid them in the case of power-specific punishments.

Evidence for differential pleasure responses to dominance success or failure come from studies using objective indicators of affect as represented in facial expressions. Assessing activity of the corrugator muscle, which is involved in frowning and assumed to reflect hedonic responses to objects and events (Larsen et al., 2003), Fodor and colleagues have demonstrated that individuals high in nPower respond with increased corrugator activation when confronted with dominant others and with decreased activation when dealing with non-dominant interaction partners (Fodor et al., 2006, 2012; Fodor and Wick, 2009). Other studies have used subjective ratings of hedonic well-being to show that nPower predicts individuals' emotional well-being in response to success and failure in the everyday pursuit of power goals (Brunstein et al., 1998; Schultheiss et al., 2008a).

Research on autonomic responses to power incentives shows that nPower predicts distinct hormonal release patterns to dominance and defeat. Men high in nPower respond to a victory in a one-on-one competition against another man with an increase in testosterone, whereas they respond to a defeat with a decrease in this hormone (Schultheiss and Rohde, 2002; Schultheiss et al., 2005b). Women high in nPower show a parallel response pattern to victory and defeat in their estradiol levels (Stanton and Schultheiss, 2007). Power-motivated men and women both respond with increased adrenal catecholamines to power-arousing situations (McClelland et al., 1980, 1985) and with increased cortisol to defeat in such situations (Wirth et al., 2006). These studies suggest that the hypothalamus, a key interface between motivation, endocrine regulation, and behavior (Iversen et al., 2000), is involved in nPower (see Schultheiss, 2013, for a review) and that nPower thus has many of the hallmarks of power/dominance motivation as studied by biopsychologists and neuroscientists (e.g., Sapolsky, 1987; Albert et al., 1992; Johnson et al., 2012).

More evidence that nPower is associated with core motivational processes comes from a brain imaging study in which Schultheiss et al. (2008b) used an oddball detection task to test effects of facial expressions of emotion (FEE) on activation of brain areas that are critically involved in motivational regulation of behavior (striatum, amygdala, insula, orbitofrontal cortex). This work was based on the notion that FEEs represent interpersonal incentives whose reward and punishment value depends both on the emotion displayed by the sender and the motivational needs of the perceiver (Stanton et al., 2010). More specifically, Schultheiss et al. (2008b) expected that for high-power individuals, but not for low-power individuals, angry expressions signal high dominance and thus represent an aversive stimulus and that surprised expressions signal low dominance and thus represent a rewarding stimulus. Except for the amygdala, in which the signal was in the expected direction but too weak to pass a stringent significance threshold (see Hall et al., 2010), individual variations of nPower predicted enhanced brain activation responses in all motivational-brain areas to angry expressions, relative to neutral expressions, and to a lesser extent also to surprised expressions. Notably, nPower-dependent activation increases to dominance-related FEEs were strong and extensive in the dorsoanterior striatum, particularly the caudate head, a key structure for reinforcement learning (Delgado, 2007). This observation plays a key role in our explanation of phenomena associated with nPower-dependent implicit learning, an issue to which we turn next.

# **nPOWER-DEPENDENT IMPLICIT LEARNING**

Implicit learning occurs when a person picks up a regularity in the patterning of environmental cues and uses it to increase response efficiency, above and beyond performance changes unrelated to learning and without being able to explicitly state the regularity (Reber, 1989; Berry, 1996). Although implicit learning is a phenomenon usually studied from the perspective of cognitive psychology, some researchers have extended its range of validity to the social domain (e.g., Lewicki et al., 1989). Lieberman (2000) in particular argued that implicit learning is the basis of social intuition, that is, complex, yet largely automatic behavioral adjustments in response to social feedback that individuals need to make to succeed in their interactions with others.

This social-adjustment view of implicit learning also guided a series of studies we and our collaborators conducted on nPower-moderated responses to dominance contest outcomes. The research was based on the hypothesis that because individual differences in nPower determine to what extent dominating another person is rewarding or being dominated by another person is aversive, implicit learning of behavior that results in these situational outcomes should likewise depend on individual differences in nPower. For instance, because a person high in nPower can enjoy beating an opponent in a direct competition, this person should also better learn whatever he or she has done during the competition to be victorious. In contrast, a person low in nPower should not enjoy a victory against a competitor and therefore also fail to get reinforced for whatever behavior has led to this outcome.

We have tested this hypothesis in a series of studies that combined a dominance-contest paradigm, in which the outcome (victory, defeat) was experimentally manipulated, with implicitlearning tasks that participants worked on during the contest. In all studies, gains in implicit learning were assessed after the contest, or, in the parlance of learning psychology, during extinction, when reinforcement (beating the opponent; being beaten by the opponent) was no longer provided. In all studies, explicit awareness of learning was assessed at the end of data collection, and participants generally had no declarative knowledge of the stimulus-response pattern they had learned. Moreover, when those few participants who showed explicit knowledge of the pattern were excluded from analyses, the results reported in the following remained unchanged, suggesting that explicit awareness of the pattern was not critical for its acquisition and execution.

Using a paper-and-pencil task that allowed participants to learn a repeating pattern of connections between successive numbers, Schultheiss and Rohde (2002) found in a study with male German participants that nPower significantly predicted better learning among contest winners, and worse learning among contest losers, who were also low in activity inhibition, a measure of brain lateralization during stress (Schultheiss et al., 2009). Schiepe-Tiska (2012), who used a computer-administered variant of Nissen and Bullemer's (1987) serial-response-task (SRT) paradigm for the assessment of implicit learning in a similar contest paradigm, recently replicated these results in another study with male German participants. Like Schultheiss and Rohde (2002), Schiepe-Tiska found a joint effect of nPower and contest outcome on implicit learning among participants low in activity inhibition, with nPower predicting better learning among winners, but not among losers.

These findings were replicated and extended to both genders by Schultheiss et al. (2005b) in two studies with US students using the SRT paradigm for the assessment of implicit learning. In these studies, nPower predicted better learning among winners and worse learning among losers in men and women alike and regardless of participants' activity inhibition levels. The findings reported by Schultheiss and Rohde (2002), Schiepe-Tiska (2012), and Schultheiss et al. (2005b) are all consistent with the notion that winners should learn and utilize the fixed sequence inherent in implicit learning tasks only to the extent that they experience the outcome as rewarding (victory) or punishing (defeat), which in turn depends on participants' nPower.

Going beyond the dominance-contest paradigm, Schultheiss et al. (2005a) tested whether individual differences in nPower also predict implicit learning when the action-contingent outcome is the presentation of an FEE. Like Schultheiss et al. (2008b), these authors argued that facial expressions of anger, joy, surprise, and neutrality can be aligned on a dominance dimension, with anger and joy signaling someone else's high dominance and thus being aversive for a power-motivated perceiver, surprise signaling someone else's low dominance, and thus being rewarding for a power-motivated perceiver, and neutrality representing a midpoint on the dominance dimension. Schultheiss et al. (2005a) tested the validity of this proposition by having each of their participants learn three distinct SRT sequences. One sequence was always followed by an emotional face, one always by a neutral face, and one always by no reinforcing stimulus. Emotion (anger, surprise, joy) was varied between subjects. Learning was tested in extinction, that is, when SRT fixed-sequence execution was no longer reinforced by faces. Results showed that compared to learning on neutral-face or no-face sequences, nPower predicted enhanced learning of surprise-face SRT sequences and impaired learning of joy-face sequences. For participants in the angry-face condition, nPower predicted impaired implicit learning overall. These findings suggest that, as proposed by Lieberman (2000), implicit learning is indeed sensitive to social signals such as brief emotional expressions. But like the contest studies, it shows that the meaning of social dominance signals and dominance-related outcomes as hedonically charged rewards and punishers depends on individuals' nPower.

One surprising but very consistent finding in the studies using the SRT paradigm by Schultheiss et al. (2005a,b) and Schiepe-Tiska (2012) was that the effect of nPower on learning emerged for the *accuracy* with which participants executed the fixed sequence (relative to random sequences), but not for a more commonly used indicator of implicit learning, that is, the relative *speed* with which participants executed the fixed sequence. (The task used by Schultheiss and Rohde, 2002, did not allow to differentiate between accuracy and speed effects.) Effects of nPower on speed-based learning emerged only in the FEE-reinforcement study by Schultheiss et al. (2005a). However, these effects were considerably weaker and more dependent on additional factors (e.g., FEE presentation time) than the effects observed for accuracy. Across all studies, the specificity of the effect of nPower on learning accuracy was particularly striking because speedand accuracy-based indicators of learning were positively correlated (up to *r* = 0*.*50). How can the differential sensitivity of implicit-learning accuracy and speed for nPower-dependent reinforcement be explained?

# **A STRIATAL BASIS OF nPOWER-DEPENDENT IMPLICIT LEARNING**

We propose that insights from more than a decade of research on the role of the dorsoanterior striatum in early sequence learning, action-outcome learning, and the modulation of learning by dopamine (DA) input to the striatum may help answer this question. Using a serial-response task that could be adapted for use with both primates and human research participants, Hikosaka and colleagues demonstrated, by transient blockade of learning through transmitter antagonists (Miyachi et al., 1997) and by augmentation of learning through electrical stimulation of neuron populations (Nakamura and Hikosaka, 2006; see also Williams and Eskandar, 2006), that the anterior portion of the caudate nucleus is critically involved in the *implicit learning of new visuomotor sequences*, and that such learning is reflected by an increase in sequence execution *accuracy*. In contrast, experimental manipulation of neuronal activity in more posterior parts of the striatum specifically altered the performance of *well-learned sequences* and was reflected in changes in sequence execution *speed* (Miyachi et al., 1997, 2002).

Applied to the previously reviewed findings relating nPower and implicit learning accuracy, this suggests that nPowerdependent modulation of instrumental learning occurs early, during the acquisition of action-outcome contingencies, and is mediated by the dorsoanterior striatum. Such an interpretation would be consistent with the observation of nPower-dependent activation of the caudate head in response to perceived dominance signals (Schultheiss et al., 2008b; Hall et al., 2010), which may have reflected a process related to the recruitment of suitable responses for dealing with the emotional stimulus. It would also be consistent with a hypothesis presented by Hikosaka and colleagues (Hikosaka et al., 1999; Balleine et al., 2007; see also Balleine and O'Doherty, 2010), who have argued that the acquisition of stimulus-guided behavioral sequences in the dorsoanterior striatum, and particularly the head of the caudate, is a form of action-outcome contingency learning that depends on the motivational value of the outcome at the time of acquisition: the higher the reward value of the outcome, the steeper the learning (see, for instance, Delgado et al., 2003). Moreover, Balleine et al. (2007) point out that action-outcome learning mediated by the dorsoanterior striatum is particularly likely to be observed in tasks that have a strong social-interaction component, such as punishing others for transgressing a social norm (de Quervain et al., 2004) or learning to trust another person in an economic exchange (King-Casas et al., 2005). This, too, fits the studies on nPower and learning, which featured "strong" social interactions by using actual face-to-face contest situations to make victory and defeat salient (Schultheiss and Rohde, 2002; Schultheiss et al., 2005b; Schiepe-Tiska, 2012).

Our interpretation of nPower-dependent implicit learning also fits well with Lieberman's (2000) neurocognitive model of social intuition. Like Balleine et al. (2007), Lieberman (2000) argues that intuition based on implicit learning of socially adaptive behavior depends critically on the striatum—effective and sophisticated adaptation of social behavior is possible only to the extent that an intact striatum supports implicit learning processes. Frequently, power-motivated individuals are socially successful not because they try to have impact on others through blunt dominance and aggression—a strategy that is prone to backfire—but by picking up on "behaviors that work," such as appearing competent and intelligent to others (Schultheiss and Brunstein, 2002), being perceived as charismatic (De Hoogh et al., 2005), or even learning to execute an arbitrary sequence of key presses, as in our contest studies. We argue that the diverse range of sophisticated behaviors that power-motivated individuals learn to employ in their quest for impact depend on striatum-mediated implicit learning that gives rise to such intuitive and successful behavioral strategies. Following Lieberman's (2000) lead, we would therefore predict that a loss of a functional dorsoanterior striatum would equal a loss of sophisticated pursuit of power-related incentives in power-motivated individuals. This is illustrated by a case study of a young woman with bilateral damage of the caudate head, reported by Richfield et al. (1987). Before the damage, the woman had graduated from high school with high honors, held a job, and was happily engaged. Although the woman did not complete a measure on nPower, one can surmise that she expressed whatever degree of nPower she had in well-adjusted ways. After the damage, however, her behavior became socially inappropriate and included vulgarity and violent outbursts, which can be recognized as the prototypical, unsocialized forms of power seeking typically observed in young children (see McClelland and Pilon, 1983).

Learning of stimulus-response contingencies in the striatum depends on the release of DA by projections of cells located in the brainstem (substantia nigra [SN] and ventral tegmental area [VTA]). Animal and human studies of implicit sequence learning<sup>1</sup> show that experimental enhancement and inhibition of DA release lead to corresponding enhancements and impairments of sequence learning (Kumari et al., 1997; Miyachi et al., 1997; Dunnett et al., 2012; for a review, see Udden et al., 2010). Moreover, human research participants show increased DA release in the striatum, including the caudate head, during implicit learning on the SRT (Badgaiyan et al., 2007). Individuals suffering from Parkinson's disease, which is associated with reduced DA levels, show worse implicit sequence learning performance than healthy control participants (Smith and McDowall, 2004). While these studies suggest that the availability of DA at the synapse is a critical requirement for implicit learning to occur, other research, reviewed in Bromberg-Martin et al. (2010), shows more specifically that phasic bursts of DA in the striatum drive action-outcome learning, depending on the motivational value of the outcome generated by the response. Some DA neurons code for the rewarding consequences of an action, marking the event with a brief increase (i.e., spike) of DA release at striatal synapses, whereas other DA neurons code for punishment, as reflected by a brief reduction (i.e., trough) of DA at striatal synapses (Matsumoto and Hikosaka, 2009). If the outcome has no positive or negative motivational value, DA release is neither increased nor reduced. Thus, at the synaptic level, phasic DA changes reinforce action-outcome learning in the case of reward or suppress it in the case of punishment (see Bromberg-Martin et al., 2010). We suggest that in the context of power-relevant person-environment transactions, nPower determines the magnitude of phasic DA release changes in response to action outcomes, because it determines the motivational value of success or failure at having impact on others. Thus, in the dominance contest studies reviewed above, we would have expected high-power individuals, but not low-power individuals, to show marked DA spikes in the dorsoanterior striatum in response to winning a round on a dominance contest. These DA spikes could in turn have reinforced the stimulus-response contingencies inherent in the implicit visuomotor learning task the contest was based on. Conversely, we would have expected high-power individuals, but not low-power individuals, to show marked DA troughs in the dorsoanterior striatum in response to losing a round. These DA troughs could in turn have suppressed the acquisition of stimulusresponse contingencies in the learning task (see **Figure 1**). We propose that this represents the neurophysiological mechanism by which nPower, in interaction with dominance-related rewards and punishments, drives implicit learning in power-relevant situational contexts.<sup>2</sup>

## **A BROADER PERSPECTIVE ON THE ROLE OF THE DORSOANTERIOR STRIATUM IN IMPLICIT MOTIVATION**

In closing, we want to briefly address the question of where in the brain nPower-associated motivational valuation of an action outcome is encoded and also discuss the broader implications

<sup>1</sup>By necessity, sequence learning tasks in animal studies—particularly when rats or mice are used—often differ from those used in human studies. However, because researchers aim to model the animal tasks on the human tasks, they also share key features, such as the association of motor responses with spatially patterned stimuli (e.g., Dunnett et al., 2012). Moreover, when primates are compared to humans, it is even possible to use the same sequence learning tasks (e.g., Hikosaka et al., 2002b).

<sup>2</sup>Other brain areas, such as the amygdala, contribute to emotional processing in general (e.g., Sergerie et al., 2008) and instrumental learning in particular (e.g., Killcross et al., 1997), and DA projections to the amygdala are known to influence emotional-motivational processing (for an overview, see for instance Cardinal et al., 2002). While we think that a broader network of brain areas including the amygdala and other structures is involved in implicit power motivation (see **Figure 1**; see also Schultheiss et al., 2008b; Hall et al., 2010), we focus selectively on the role of the striatum in nPower-dependent implicit learning in the present paper.

of the model for other implicit motivational needs. Although we have argued that the magnitude of phasic changes in striatal DA release in response to dominance incentives and disincentives reflects nPower-dependent valuation of action outcomes, we do not want to suggest that they represent nPower-dependent neuronal representations of reward evaluation (i.e., liking) or that *all* phasic variations in dopaminergic neurotransmission are driven by nPower. For one, DA responses have been shown to dissociate from liking responses to rewards and punishers and to become associated with incentive-*predicting* cues over time and with the monitoring of prediction accuracy (Schultz, 1998; Bromberg-Martin et al., 2010). Moreover, DA release in the striatum is involved in striving for many different types of incentives, including food, sex, and money, and thus represents a common currency of motivational valuation, not a process specifically linked to one motive, such as nPower. However, DA neurons in the SN and VTA receive inputs from other brain areas that may represent more specific motivational needs and need-specific outcome evaluations and thus may drive DA release in the striatum via their projections to the SN/VTA area. One brain site with particularly extensive projections to this area is the hypothalamus (Gonzalez et al., 2012), which represents physiological and social needs in a domain-specific manner in distinct nuclei (see Schultheiss, 2013) and, as we have pointed out previously, is involved in the nPower-associated release of testosterone in men and estradiol in women. The hypothalamus may also transmit to the SN/VTA domain-specific and topographically distinct hedonic liking signals encoded by the orbitofrontal cortex (OFC; see Kringelbach, 2005), with which it has extensive reciprocal connections (Öngür and Price, 2000). Schultheiss et al. (2008b) and Hall et al. (2010) have argued that these brain sites are particularly likely candidates for representing individual differences in liking responses to motive-specific rewards and punishments, and we suggest that these specific liking responses to power-related rewards and punishments drive responses of DA neurons in the SN/VTA.

It is also conceivable that nPower-specific outcome evaluations influence striatal functions more directly by, for instance, direct projections from the OFC to the dorsoanterior striatum (Öngür and Price, 2000), which may modulate synaptic learning driven by phasic DA changes in a specific manner, or by the effects of nPower-associated testosterone and estradiol, which broadly augment striatal DA effects (e.g., Becker and Rudick, 1999; Frye et al., 2002). The latter suggestion is consistent with the observation by both Schultheiss and Rohde (2002) and Schultheiss et al. (2005b) that in male contest winners and losers, effects of nPower on implicit learning were mediated by changes in testosterone.

Both the notion that need-specific outcome evaluations take place elsewhere in the brain and the fact that the striatum is active during the pursuit of many different motivational incentives suggest that the dorsoanterior striatum, and DA-based learning happening there, may also play a role in other implicit motives, such as the needs for achievement (nAchievement; Pang, 2010) and affiliation (nAffiliation; Weinberger et al., 2010). In support of this notion, Bäumler (1975; reviewed in Schultheiss and Brunstein, 2005) has shown that experimental pharmacological manipulation of DA levels effects changes in a measure of nAchievement, with DA agonists leading to an increase and DA antagonists leading to a decrease of achievement imagery in the stories that research participants write about picture cues related to achievement. Moreover, Hall et al. (2010) report that nAchievement assessed with a picture-story test positively predicts activation of the caudate nucleus in response to anger FEEs in an fMRI study. This observation supports the notion that the striatum plays a role in other implicit motives besides nPower. Interestingly, Hall et al. (2010) also report a *negative* association between nAffiliation and caudate activation in response to angry faces. This suggests that this motive, too, can influence striatal processing of motivational incentives, but perhaps in a different manner than nPower or nAchievement, which were both associated with increased caudate activation in response to anger FEEs (see also Schultheiss et al., 2008b). However, this difference may be due to the fundamentally different meaning of perceived anger expressions as rewards or punishments in the context of power, achievement, or affiliation (see Stanton et al., 2010). Further research is necessary to determine whether nAffiliation, in interaction with positive affiliation-related incentives (e.g., smiling expressions), can also predict increases in anterior striatal activation. Although some evidence already suggests that nAchievement and nAffiliation predict implicit learning that is followed by motive-specific incentives (Schultheiss et al., 2005a; Pang, 2010), more research is also needed to clearly demonstrate

#### **REFERENCES**


when and how these motivational needs influence the implicit acquisition of instrumental behavior. Such evidence would make it appear even likelier that these motives recruit the type of actionoutcome-contingency learning associated with the dorsoanterior striatum that we have postulated here.


of acute administration of damphetamine and haloperidol on procedural learning in man. *Psychopharmacology (Berl.)* 129, 271–276.


electrical stimulation in the primate caudate. *J. Neurosci.* 26, 12885–12895.


Robins, and L. A. Pervin (New York, NY: Guilford), 603–633


Effects of implicit power motivation on men's and women's implicit learning and testosterone changes after social victory or defeat. *J. Pers. Soc. Psychol.* 88, 174–188.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 10 December 2012; accepted: 31 March 2013; published online: 19 April 2013.*

*Citation: Schultheiss OC and Schiepe-Tiska A (2013) The role of the dorsoanterior striatum in implicit motivation: the case of the need for power. Front. Hum. Neurosci. 7:141. doi: 10.3389/ fnhum.2013.00141*

*Copyright © 2013 Schultheiss and Schiepe-Tiska. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

# Motivational salience and genetic variability of dopamine D2 receptor expression interact in the modulation of interference processing

*Anni Richter 1†, Sylvia Richter 1,2†, Adriana Barman1,3, Joram Soch1, Marieke Klein1,4,5, Anne Assmann1,6, Catherine Libeau1, Gusalija Behnisch1, Torsten Wüstenberg7, Constanze I. Seidenbecher 1,8 and Björn H. Schott 1,6,7,8\**

*<sup>1</sup> Department of Behavioral Neurology and Department of Neurochemistry and Molecular Biology, Leibniz Institute for Neurobiology, Magdeburg, Germany*


#### *Edited by:*

*Jan Wacker, Philipps-Universität Marburg, Germany*

#### *Reviewed by:*

*Erik M. Mueller, Giessen University, Germany Hao Yang Tan, Lieber Institute for Brain Development, USA*

#### *\*Correspondence:*

*Björn H. Schott, Leibniz Institute for Neurobiology, Brenneckestr. 6, 39118 Magdeburg, Germany e-mail: bschott@neuro2.med.unimagdeburg.de*

*†These authors have contributed equally to this work.*

Dopamine has been implicated in the fine-tuning of complex cognitive and motor function and also in the anticipation of future rewards. This dual function of dopamine suggests that dopamine might be involved in the generation of active motivated behavior. The DRD2 TaqIA polymorphism of the dopamine D2 receptor gene (rs1800497) has previously been suggested to affect striatal function with carriers of the less common A1 allele exhibiting reduced striatal D2 receptor density and increased risk for addiction. Here we aimed to investigate the influences of DRD2 TaqIA genotype on the modulation of interference processing by reward and punishment. Forty-six young, healthy volunteers participated in a behavioral experiment, and 32 underwent functional magnetic resonance imaging (fMRI). Participants performed a flanker task with a motivation manipulation (monetary reward, monetary loss, neither, or both). Reaction times (RTs) were shorter in motivated flanker trials, irrespective of congruency. In the fMRI experiment motivation was associated with reduced prefrontal activation during incongruent vs. congruent flanker trials, possibly reflecting increased processing efficiency. DRD2 TaqIA genotype did not affect overall RTs, but interacted with motivation on the congruency-related RT differences, with A1 carriers showing smaller interference effects to reward alone and A2 homozygotes exhibiting a specific interference reduction during combined reward (REW) and punishment trials (PUN). In fMRI, anterior cingulate activity showed a similar pattern of genotype-related modulation. Additionally, A1 carriers showed increased anterior insula activation relative to A2 homozygotes. Our results point to a role for genetic variations of the dopaminergic system in individual differences of cognition-motivation interaction.

#### **Keywords: DRD2, TaqIA, dopamine, genetic, motivation, interference processing, flanker, fMRI**

# **INTRODUCTION**

The ability to adapt oneself to uncertain, changeable needs of the environment is considered as an outstanding human skill (Collins and Koechlin, 2012). These competences comprise the decision making based on exploration, adaptation to found conditions, anticipation of expected results or risks of a given action and a suitable choice from a variety of possible responses to a stimulus (Royall et al., 2002; Gilbert and Burgess, 2008; Collins and Koechlin, 2012). This complex set of skills is often subsumed under the term *Executive Functions* (EF), a somewhat diffuse umbrella term that attempts to capture the heterogeneity of the psychological processes involved.

Despite their apparent heterogeneity, the brain processes typically considered as EF can be subdivided into three core functions: inhibition (including the control of interference), working memory, and cognitive flexibility (Miyake et al., 2000; Diamond, 2013). These core functions support more complex cognitive functions like planning or problem solving and thus have a broad impact on human behavior and social interactions, affecting quality of life, job success as well as physical and mental health (Diamond, 2013).

Behavioral and neural manifestations of EF can be investigated in an experimental setting using a variety of well-established paradigms. For example, inhibitory processes can be investigated with the flanker task (Eriksen and Eriksen, 1974), the Simon task (Simon and Berbaum, 1990) or the Stroop task (MacLeod, 1991). In addition to inhibitory processes, successful performance of the flanker task also depends upon selective attention (Posner and Petersen, 1990; Diamond, 2013). Numerous variations of the flanker task exist, but their common feature is that participants are required to focus on a centrally presented target stimulus while ignoring flanking distractor stimuli. The performance of incongruent trials, during which the target stimulus and the flanking stimuli activate different possible reactions (i.e., responding to the central arrow in *>>><>>>*), is typically contrasted to the performance of congruent trials, during which the target and the distractors jointly activate one single choice of action (i.e., responding to the central arrow in *>>>>>>>*). Behaviorally, such interference in flanker tasks is characterized by concomitantly occurring slower reaction times (RTs) and higher error rates in incongruent as compared to congruent trials (Botvinick et al., 1999; Casey et al., 2000; Botvinick et al., 2004; Richter et al., 2011; Bugg and Crump, 2012). Further research suggests that the flanker task does not only allow the investigation of inhibition performance, but also action monitoring and error detection (Ullsperger and Von Cramon, 2004).

At the level of neural circuits, the prefrontal cortex (PFC) is widely considered to be the key neuroanatomical structure mediating EF. The intrinsic organization of the frontal lobes is complex, and a growing body of clinical studies provides evidence for heterogeneous effects of lesions in distinct PFC subregions on different subprocesses of executive functioning (Funahashi, 2001; Royall et al., 2002; Elliott, 2003). While most neuroimaging research on EF has focused on frontal brain structures like the anterior cingulate cortex (ACC) and the lateral PFC, these structures typically co-activate with parietal cortical regions (Roberts and Hall, 2008), reflecting large-scale attention networks that also show increased connectivity during rest (Fox et al., 2006). Moreover, the PFC interacts with subcortical structures, most notably the striatum and the thalamus (Casey et al., 2000; Fan et al., 2005; Posner and Rothbart, 2007).

Because EF, at least to a large extent, mediate goal-directed behavior, it is conceivable that stimuli associated with potential positive or negative reinforcers are likely to undergo preferential processing (Adcock et al., 2006; Boksem et al., 2008; Krebs et al., 2009; Richter et al., 2013), but, when task-irrelevant, also interfere with the task at hand and influence its neural underpinnings (Wiswede et al., 2009; Padmala and Pessoa, 2010; Richter et al., 2011). The association of a stimulus with the possibility to obtain a reward or to avoid an aversive outcome typically renders this stimulus highly salient (Boksem et al., 2008). To elucidate how processes of inhibition and error detection are modulated by such salience, the flanker task can be modified by introducing trials in which participants can receive rewards or avoid penalties upon correct performance (Boksem et al., 2008; Engelmann et al., 2009; Hubner and Schlosser, 2010). Boksem and colleagues investigated the relationship between punishment/reward sensitivity (assessed with the Behavioral Inhibition System and Behavioral Activation System questionnaires, BIS/BAS) and electrophysiological correlates of error processing in the flanker task, demonstrating that individual differences in reward and punishment sensitivity affected the amplitude of error-related event-related potential (ERP) components in flanker trials that were associated with reward or punishment, respectively (Boksem et al., 2008).

Additional evidence for a modulation of PFC/dACCdependent inhibitory control by motivation comes from stop signal and Stroop tasks. Padmala and Pessoa (2010) used the stop signal-paradigm to investigate the neural mechanisms of cognition-motivation interactions during response inhibition. Selective rewarding of correct go-reactions was associated with longer inhibitory RTs in the rewarded relative to the control condition and with reduced PFC activation in rewarded trials. Compatibly, Krebs et al. (2010) observed that reward anticipation exerted beneficial behavioral effects on Stroop task performance, but reward-associated stimuli also impaired the processing of neutral stimuli.

Converging evidence from patient studies, psychopharmacology and genetic investigations suggests that variability of prefrontal dopaminergic neurotransmission contributes substantially to the widely observed individual differences in PFCdependent EF (Mattay et al., 2003; Meyer-Lindenberg and Weinberger, 2006; Stelzel et al., 2010; Barnes et al., 2011; Tan et al., 2012). Most studies investigating the impact of the dopaminergic system on PFC function have focused on catechol-O-methyl transferase (COMT), an enzyme primarily involved in cortical, but not striatal dopamine clearance (Tunbridge et al., 2006), but there is increasing evidence for a delicately balanced mutual regulation of prefrontal and striatal dopamine turnover (Meyer-Lindenberg et al., 2002, 2005, 2007). The dopamine receptor D2 (DRD2) is the predominant postsynaptic dopamine receptor in the striatum, but sparsely expressed in the PFC. Presynaptic, autoinhibitory D2 receptors, on the other hand, play an important role in the regulation of dopamine release throughout the brain. Given this dual role of DRD2, it seems plausible that genetically mediated individual differences of DRD2 expression affect both human striatal and prefrontal neural processes. A commonly investigated single nucleotide polymorphism (SNP) linked to the DRD2 gene on chromosome 11q22-23 is the so-called TaqIA polymorphism, which is characterized by a polymorphic restriction site. The TaqIA polymorphism has been repeatedly associated with alterations of striatal dopaminergic neurotransmission. Despite the fact that the underlying molecular mechanisms are yet not fully understood, a number of studies have provided converging evidence for reduced DRD2 expression in homozygous and heterozygous carriers of the less common A1 allele relative to homozygotes of the A2 allele. *Post mortem* investigations and positron emission tomography (PET) suggest that A1 carriers show a 30–40% decrease in DRD2 density compared to A2 homozygotes in the striatum (Thompson et al., 1997; Pohjalainen et al., 1998; Jonsson et al., 1999; Ritchie and Noble, 2003). One study employing single photon emission tomography (SPECT) did not find a difference in D2 receptor binding between A1 carriers and A2 homozygotes (Laruelle et al., 1998), but that study was later criticized for the combination of healthy participants and patients with schizophrenia in a sample and for the low resolution of the SPECT method (Ritchie and Noble, 2003). Moreover, A1 carriers have been reported to exhibit increased striatal dopamine synthesis (Laakso et al., 2005), possibly reflecting reduced autoinhibitory signaling from presynaptic D2 receptors. In healthy human volunteers, DRD2 TaqIA has been shown to affect neural mechanisms of reward processing, compatible with the high levels of DRD2 expression in the striatum (Lee et al., 2007), and similar effects have been observed for other genetic variations that affect D2 receptor availability (Pecina et al., 2013). In light of the above-mentioned structural and functional connectivity between the PFC and the striatum and the regulation of dopamine release via autoinhibitory presynaptic D2 receptors, it seems plausible that DRD2 TaqIA also modulates PFC-dependent EFs. Indeed, DRD2 TaqIA has been demonstrated to affect task switching and working memory-related processes, the latter in epistatic interaction with COMT Val108/158Met genotype (Stelzel et al., 2009, 2010; Garcia-Garcia et al., 2011).

The reported influences of DRD2 TaqIA on individual differences in prefrontal and striatal function are likely to be particularly pronounced when cognitive processes depend directly on fronto-striatal interactions. In line with this notion, motivationbased probabilistic learning or reversal learning have been shown to be affected by the polymorphism at the levels of both behavior and neural correlates, with A1 carriers being less successful in predicting negative outcomes and showing diminished recruitment of PFC and striatum during negative feedback processing and reversal learning (Klein et al., 2007; Jocham et al., 2009).

The tasks employed by Klein, Jocham and colleagues depend upon the direct interaction of the PFC and the striatum. Here we aimed to investigate effects of DRD2 TaqIA genotype on the modulation of primarily PFC-dependent inhibitory control and action monitoring by motivational processes, i.e., the anticipation of monetary gain or loss. We employed a modified flanker task during which, in a subset of the trials, participants could receive a reward, or avoid a punishment, or both. Recent evidence from animal studies suggests that the combination of appetitive and aversive reinforcement is associated with more pronounced improvement of learning performance than either one type of reinforcement alone (Ilango et al., 2010). Aiming to generalize this observation to human EFs, we also included a combined reward and loss condition in the task. Participants were genotyped for the DRD2 TaqIA polymorphism and grouped into A1 carriers and non-carriers. In a first behavioral experiment, we hypothesized that behavioral responses to incongruent flanker trials would be faster, and possibly more accurate, in rewardassociated or punishment-associated flanker trials, and that, in line with their increased risk for reward-related disorders like addiction (Noble, 2003; Wang et al., 2013), A1 carriers would show more pronounced motivation-related modulation of the flanker trials. At a neural level, we hypothesized that A1 carriers and non-carriers would exhibit differential activation patterns in brain regions associated with conflict processing like the dACC and structures associated with motivational processing, like the striatum and the insula.

# **MATERIALS AND METHODS PARTICIPANTS**

Participants were recruited from a cohort of 615 young (behavioral study: age range 18–30 years, mean 23.65 ± 2.86; fMRI study: age range 19–30 years, mean 23.00 ± 2.51), healthy volunteers of a large-scale behavioral genetic study conducted at the Leibniz-Institute for Neurobiology, Magdeburg. Based on the assumption that a possible small effect of genes may not only require a large number of volunteers but also a strict control of non-genetic factors (Lee et al., 2007), participants were assessed for several exclusion criteria. All participants were right-handed according to self-report, not genetically related, and had obtained at least a university entrance diploma (*Abitur*). Importantly, all participants had undergone routine clinical interview to exclude present or past neurological or psychiatric illness, alcohol, or drug abuse, use of centrally-acting medication, the presence of psychosis or bipolar disorder in a first-degree relative, and additionally, given the design of the experiment, frequent gambling. For both studies, the behavioral and the fMRI experiment, two participants were invited for piloting of the paradigm. Their data were not used for subsequent analyses. The final study sample consisted of 46 volunteers in the behavioral study and 32 participants in the fMRI study, with no overlap between the experiments. All participants gave written informed consent in accordance with the Declaration of Helsinki and received financial compensation for participation. The work was approved by the Ethics Committee of the University of Magdeburg, Faculty of Medicine.

### **GENOTYPING**

Genomic DNA was extracted from blood leukocytes using the GeneMole® automated system (Mole Genetics AS, Lysaker, Norway) according to the manufacturer's protocol. Genotyping was performed using PCR followed by allele-specific restriction analysis using previously described primers (Grandy et al., 1989). Briefly, the DNA fragment on Chr 11q23.1 containing the DRD2 TaqIA polymorphism (NCBI accession number: rs1800497) was amplified using the primers DRD2-F: 5- - CCGTCGACGGCTGGCCAAGTTGTCTA-3 and DRD2-B: 5- - CCGTCGACCCTTCCTGAGTGTCATCA-3 and standard Taq polymerase (Qiagen and Fermentas). PCR products were digested with *TaqI* (Fermentas), yielding two fragments (130 + 180 bp) for the A2 allele or a single fragment (310 bp) for the A1 allele. DNA fragments were separated on a 2.5% ethidium bromide-stained agarose gel and visualized under UV light. Because the COMT Val108/158Met polymorphism (NCBI accession number: rs4680) has previously been linked to individual differences in both PFC function and reward processing (Egan et al., 2001; Schmack et al., 2008; Wimber et al., 2011), participants were also genotyped for rs4680 using PCR and restriction with NlaIII (Schott et al., 2006; Wimber et al., 2011; details available upon request).

# **BEHAVIORAL STUDY**

### *Paradigm*

We employed a modified Eriksen flanker task (Eriksen and Eriksen, 1974) with a motivation manipulation (Boksem et al., 2008). Participants were instructed to fixate a central target arrow and to indicate whether it was pointing to the left or to the right by pressing a button with the index or middle finger of the right hand. They had to ignore six distractor arrows with the same (congruent condition), the opposite (incongruent condition) or random (any three left and three right) orientation. Trials were grouped into four types of motivational categories. In reward trials (REW), volunteers were rewarded with 5 ct for fast and correct responses. Conversely, in punishment trials (PUN), they were punished for incorrect, slow or missing responses by the loss of 5 ct. These two conditions were complemented by neutral trials (NEU) in which responses were associated with neither gain nor loss and with trials in which fast and correct responses were rewarded and incorrect, slow or omitted responses were punished (combination trials—COM). Each condition constituted 25% of the trials, and participants were notified about the upcoming trial type before each trial by presentation of neutral, positive, or negative cartoon face (neutral faces, smilies and frownies; see **Figure 1**). RTs were monitored throughout the course of the experiment. RTs exceeding the current mean RT by more than one standard deviation (SD) were considered too slow, and participants received a feedback ("Faster!") whenever it was exceeded. Accuracy feedback was not delivered.

As in a number of previous studies, the flanker task was combined with a stop-signal paradigm (Logan et al., 1984; Krämer et al., 2007; Boehler et al., 2009). Infrequently (on 20% of the trials), a circle instead of the target arrow was presented, signaling the participants to suppress their response. We used an adaptive short stop-signal delay (SSD) to yield an approximately equal number of signal-inhibit and signal-respond trials (Krämer et al., 2007; Boehler et al., 2009). The SSD was calculated online separate for each motivation condition. Participants were informed that rewards and punishments would never be delivered in stop trials, regardless of their inhibition performance. An example trial and the overview of the trial timing are displayed in **Figure 1**.

The experiment consisted of four runs with 144 trials per run (including 24 stop trials). Each run was counterbalanced for experimental conditions and direction of the target arrow. The currently earned amount of money was displayed after each run. Participants were tested alone or in groups of no more than three persons. Before the experiment they were instructed using a standardized written instruction, followed by the opportunity to ask questions. Before the actual experiment, participants performed a training phase consisting of 42 trials (12 stop trials) during which an accuracy feedback was delivered. In this training the starting value of the RT limit was calculated. Data of the training phase were not analyzed further. Participants could earn up to 9 Euros (mean = 5.57 Euros ± 1.53 Euros).

#### *Statistical analysis*

To examine the influence of the DRD2 TaqIA polymorphism on flanker performance and its modulation by reward and punishment, flanker trials were analyzed with respect to the percentage of incorrect responses and the RTs of correct responses. Correct and incorrect reactions between 200 and 1000 ms after stimulus onset were analyzed. As a measure of interference processing the difference between congruent and incongruent trials (congruency effect) was calculated. Analyses of variance (ANOVAs) for repeated measures were calculated for each dependent variable with the motivation condition as within-subject factor and DRD2 TaqIA genotype as between-subject factor. Degrees of freedom were corrected for non-sphericity using the Greenhouse-Geisser correction.

# **FUNCTIONAL MRI EXPERIMENT** *Paradigm*

The design of the task used in the behavioral study was simplified and adapted for the purposes of fMRI. As the random trials yielded accuracy rates and RTs that lay in between those of the congruent and the incongruent condition, we did not include random trials in the fMRI study, thereby increasing the number of congruent and incongruent trials contributing to the fMRI signal. Furthermore, the potential reward and punishment were increased from 5 to 20 ct, and participants received further 6 Euros to compensate for travel expenses, which they were told after the experiment. The trial timing of the events was the same as in the behavioral study, but the inter-trial interval was increased and jittered between 4 and 8 s, using a near-exponential jitter to optimize the estimation of the trial-specific BOLD responses

(Hinrichs et al., 2000). In total, there were four runs with 96 trials each (16 stop trials). The training phase (36 trials including 6 stop trials) was performed outside the MR tomograph. Participants could earn up to 32 Euros (24.42 ± 3.48 Euros; plus 6 Euros).

#### *Image acquisition*

Four runs of 390 T2∗-weighted echo-planar images (EPIs) per run were acquired on a GE Signa 1.5 T magnetic resonance system (General Electric Medical Systems) in an interleaved acquisition order (23 axial slices, odds first; voxel size = 3.13 mm × 3.13 mm × 4 mm + 1 mm gap; *TR* = 2 s; *TE* = 35 ms). Six EPIs were acquired before each run to allow for magnetic field stabilization and discarded from data analysis. Because proton-density (PD)-weighted MR images possess a good contrast for gray vs. white matter in the striatum and midbrain (D'Ardenne et al., 2008; Schott et al., 2008), a co-planar PD-weighted MR image was acquired and used for improved spatial normalization.

#### *Data processing and analysis*

Data analysis was carried out using Statistical Parametric Mapping (SPM8, Wellcome Department of Imaging Neuroscience, Institute of Neurology, London, UK). EPIs were corrected for acquisition delay and head motion. The co-planar PD-weighted image was coregistered to the mean image obtained from motion correction and used to determine normalization parameters for spatial normalization to the Montreal Neurological Institute (MNI) stereotactic coordinate system (voxel size = 3 × 3 × 3 mm). Data were smoothed using a Gaussian kernel of 8 × 8 × 8 mm, and a high-pass filter with a cut-off of 128 s was applied to the data.

Statistical analysis was performed using a two-stage mixed effects model. At the first stage, the hemodynamic response was modeled by convolving a delta function at stimulus onset with a canonical HRF (Friston et al., 1998). The resulting time courses were downsampled to the temporal resolution of fMRI scanning (1*/TR* = 0*.*5 Hz) to form covariates of a general linear model (GLM). The model included separate covariates for each condition of interest (correct responses in the conditions NEU-congruent, NEU-incongruent, REW-congruent, REW-incongruent, PUN-congruent, PUN-incongruent, COMcongruent, and COM-incongruent). The model included also covariates of no interest, namely incorrect responses, a feedback regressor, four stop-trial regressors for each motivation condition, the instruction screen, and the six rigid-body movement parameters determined from motion correction, plus a single constant representing the mean over scans. Model estimation was performed using a restricted maximum likelihood fit.

At the second stage of the model, the conditions of interest separated by genotype were submitted to second level random effect analyses. Specifically, the within-subject factors congruency and motivation were submitted to a full-factorial ANOVA, with genotype as between-subject factor. Because of our strong *a priori* hypotheses regarding brain regions previously implicated in interference processing and motivation, several regions of interest (ROIs) were defined. The ROI of the dorsolateral prefrontal cortex (DLPFC) was generated using the automated anatomical labeling (AAL) of the superior and middle frontal gyrus implemented in the WFU Pickatlas (Wake Forest University), and ROIs

of the ACC, the anterior insula and striatum were generated using a previously described literature-based probabilistic approach (Schubert et al., 2008; Zweynert et al., 2011; see **Figures A1**–**A3**). The *a priori* statistical threshold was set to *p* = 0*.*05 family wise error (FWE)-corrected for all comparisons, with the correction applied to ROI volumes for regions with *a priori* hypotheses, and an additional Bonferroni correction was applied to correct for the number of ROIs (*n* = 8). Coordinates are given in MNI space. To further verify reliability of genetically driven between-group differences and reduce the influence of outliers, confidence intervals were estimated for the local maxima using bootstrap resampling and the percentile-t method (Schott et al., 2006; Wimber et al., 2011). For visualization purposes, activations were superimposed onto the MNI template image provided by MRIcron (http://www.mccauslandcenter.sc.edu/mricro/mricron/).

# **RESULTS**

### **GENOTYPING**

Among the 615 participants in the original cohort who were genotyped for the DRD2 TaqIA polymorphism, we identified 22 A1 homozygotes, 210 heterozygotes, and 383 A2 homozygotes. The distribution was at Hardy-Weinberg equilibrium [χ<sup>2</sup> <sup>=</sup> 1*.*08, *p* = 0*.*298]. Regarding the COMT Val108/158Met polymorphism, the sample included 164 Met homozygotes, 322 heterozygotes, and 129 Val homozygotes, and HWE was not violated [χ<sup>2</sup> <sup>=</sup> <sup>1</sup>*.*57, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*210].

#### **BEHAVIORAL RESULTS**

In the behavioral study the data of 46 young, healthy participants were analyzed (27 women, 19 men). The cohort consisted of one A1 homozygote, 23 heterozygotes and 22 A2 homozygotes. Thirty-two participants took part in the fMRI study (19 women, 13 men), including 15 heterozygote A1 carriers and 17 A2 homozygotes. Given the low number of A1 homozygous subjects (*n* = 1), A1 carriers (A1+: A1/A1 and A1/A2) were grouped together for all subsequent analyses. The groups A1+ and A1− (A2/A2) did not differ in gender distribution, mean age or in percentage of smokers. Because the COMT Val108/158Met polymorphism (rs4680) has previously been demonstrated to affect PFC function and reward processing, participants were also genotyped for this SNP, and the distribution of Val and Met alleles did not differ significantly between groups. For detailed demographic information see **Table 1**. Error rates and RTs across the different conditions are displayed in **Table 2**, separated by DRD2 TaqIA genotype.

### *Effects of congruency and motivation*

Overall, participants responded fast and accurately. To test for genotype-related and task-related differences in behavioral performance, we computed ANOVAs for repeated measures with congruency and motivation as within-subject factors and genotype as between-subject factor. Replicating previous results (Botvinick et al., 1999; Casey et al., 2000; Botvinick et al., 2004; Richter et al., 2011; Bugg and Crump, 2012), we observed a main effect of flanker condition with higher error rates and slower RTs in the incongruent as compared to the congruent condition in both the behavioral [main effect of congruency: RT: *F(*2*,* <sup>88</sup>*)* = 360*.*26, *p <* 0*.*001; error rate: *F(*2*,* <sup>88</sup>*)* = 66*.*00; *p <* 0*.*001] and the fMRI experiment [RT: *F(*1*,* <sup>30</sup>*)* = 387*.*10, *p <* 0*.*001; error rate: *F(*1*,* <sup>30</sup>*)* = 26*.*73, *p <* 0*.*001]. In the behavioral study, error rates and RTs in the random trials lay in between those of the

#### **Table 1 | Demographic data.**


*Gender distribution, age (average* ± *SD), number of smokers and nonsmokers and COMT Val108/158Met occurrence (mm, met homozygotes; vm, val/met heterozygotes; mm, met homozygotes) of the participants.*

congruent and the incongruent trials, suggesting that the congruency effect depended on the number of distractors (**Table 2**). Motivational salience (i.e., the presence of reward, punishment, or both) was associated with shorter RTs in all motivated trials compared to the NEU in both the behavioral [main effect of motivation: *F(*3*,* <sup>132</sup>*)* = 36*.*18, *p <* 0*.*001] and the fMRI experiment [*F(*3*,* <sup>90</sup>*)* = 11*.*00, *p <* 0*.*001], while the error rates did not differ significantly across the different motivation conditions (all *p >* 0*.*074). In the behavioral study, the REW condition elicited the shortest RTs [REW vs. PUN: *t(*45*)* = −3*.*98, *p <* 0*.*001; REW vs. COM: *t(*45*)* = −4*.*12, *p <* 0*.*001; PUN vs. COM: *t(*45*)* = −0*.*07, *p* = 0*.*947].

#### *Genotype-related modulation of cognition-motivation interaction*

Across flanker and motivation conditions there was no genotyperelated difference in overall RTs [*Behavioral experiment:* A1 carriers: 410 ± 48 ms, A2/A2: 419 ± 55 ms; *t(*44*)* = −0*.*58, *p* = 0*.*567; *fMRI experiment:* A1 carriers: 438 ± 38 ms, A2/A2: 440 ± 43 ms; *t(*30*)* = −0*.*12, *p* = 0*.*905], suggesting that there were no genotype-related differences in sensorimotor function.

To specifically test for effects of genotype on interference processing and its modulation by motivational salience, we computed


*Mean reaction times (RT) of correct responses and error rates* ± *standard deviations (SD) are shown. NEU, condition with no reward or punishment; REW, rewarded condition; PUN, punished condition; COM, condition with reward and punishment.*

the behavioral congruency effects, i.e., the differences of error rates and RTs between incongruent and congruent trials, separated by motivation conditions. These values were the dependent variables in ANOVAs for repeated measures with motivation condition (NEU vs. REW vs. PUN vs. COM) as within-subject factor with four levels, and DRD2 TaqIA genotype (A1+ vs. A2/A2) as between-subject factor with two levels. The analysis of error rates revealed no significant effects of either factors motivation or genotype (all *p >* 0*.*120), but in the analysis of congruency-related RT differences, a significant motivation by genotype interaction was observed in the behavioral experiment [*F(*3*,* <sup>132</sup>*)* = 3*.*07, *p* = 0*.*039]. While this interaction effect was not significant in the (smaller) cohort of the fMRI experiment, it remained significant when combining the data of both experiments [*F(*3*,* <sup>225</sup>*)* = 2*.*96, *p* = 0*.*039; because of the differences in experimental design, the experiment—behavioral vs. fMRI—was included as a covariate of no interest in this ANOVA]. To explore the pattern underlying this interaction, we computed *post-hoc* paired *T*-tests on the RT congruency effects in the different motivation conditions, separated by DRD2 TaqIA genotype. Results of the *post-hoc* comparisons are displayed in **Table 3** (both studies combined) and in **Table A1** (both studies separately; note that *post-hoc* comparisons from the fMRI experiment are for illustrative purpose only, given the lack of an interaction effect in the ANOVA). In summary, the results of the *post-hoc* tests, albeit exploratory, suggest that A1 homozygotes showed a reduced congruency effect primarily in the rewarded condition (significant in the behavioral study only, see **Table A1**) and nominally benefitted from all motivated conditions, whereas A2 homozygotes showed smaller congruency-related RT differences in the combined condition relative to the conditions with reward or punishment alone (**Figure 2**, **Tables 3**, **A1**).

#### **FUNCTIONAL MRI RESULTS**

All comparisons were based on a full-factorial ANOVA model with congruency (congruent vs. incongruent), motivation (NEU vs. REW vs. PUN vs. COM), and genotype (A1+ vs. A2/A2) as factors. An overview of the relevant comparisons in the regions


*Results of post-hoc paired T-tests testing for effects of the motivation conditions on the congruency effect of reaction times, separated by genotype group and collapsed across experiments. All p-values are one-tailed, except for the REW vs. PUN contrast for which we had no directed hypothesis. NEU, neutral condition; REW, reward condition; PUN, punishment condition; COM, combined reward and punishment condition. \*p < 0.05.*

of interest (dACC, DLPFC, insula, striatum) is displayed in **Table 4**.

#### *Effects of congruency and motivation*

In line with previous studies (Ridderinkhof et al., 2004; Ullsperger and Von Cramon, 2004), a one-tailed *T*-test comparing BOLD responses of incongruent and congruent trials revealed increased activity in distributed regions of the DLPFC and in the dorsal anterior cingulate cortex (dACC; see **Figure 3A**, **Table 4**).

The effect of motivational salience was tested by means of comparing the three motivated conditions to the neutral condition, using a one-tailed *T*-test. Irrespective of flanker condition and genotype, motivation-associated trials elicited higher BOLD responses in the bilateral striatum (Knutson et al., 2000; Wittmann et al., 2005) as well as in the ACC, the anterior insula, and in the bilateral lingual gyri when compared to neutral flanker (see **Figure 3B**, left panel). The anticipation of (avoidable) monetary punishment was associated with a similar pattern of brain activity increases, albeit of lower magnitude (**Figure 3B**, middle panel). Activations in the combined reward and punishment trials were largely comparable to those in the rewarded trials (**Figure 3B**, right panel).

A trend for a genotype-independent interaction of congruency and motivation was observed in our ROI of the DLPFC [*x, y,z* = −18*,* 56*,* 16; *F(*3*,* <sup>240</sup>*)* = 8*.*37; *p* = 0*.*036, FWE-corrected for ROI volume, but not significant after Bonferroni correction for

**FIGURE 2 | Behavioral congruency effect.** Plots depict the difference between incongruent and congruent RTs for each motivation condition (± standard errors). Data from both experiments (behavioral and fMRI) are combined. Higher values indicate stronger distractor interference. The observed pattern suggests that A1 carries showed a small to moderate reduction of the RT difference (incongruent vs. congruent) in all motivated trials, particularly in the reward condition, whereas the RT difference reduction in A2 carriers was largely restricted to the combined condition [genotype by motivation interaction: *F(*3*,* <sup>228</sup>*)* = 2*.*96; *p* = 0*.*039]. NEU, neutral condition; REW, reward condition; PUN, punishment condition; COM, combined reward and punishment condition.

#### **Table 4 | Peak activation foci in the ROI analyses.**


*(Continued)*

#### **Table 4 | Continued**


*Clusters with peak activation p < 0.05, FWE-corrected for ROIs volume with their cluster extent at p < 0.005, uncorrected. Coordinates are given in MNI space (unit mm). R, right; L, left; \*p-values remained significant after Bonferroni correction for multiple ROIs (N* <sup>=</sup> *8).*

multiple ROIs] where activation related to the incongruent flanker condition was reduced in the motivated trials relative to NEU (**Figure 4**). Further localization of the activation maximum using the BA map provided by MRIcron revealed that the cluster was located in the lateral portion of Brodmann area (BA) 10, bordering BA 46.

#### *Genotype-related modulation of cognition-motivation interaction*

To investigate potential effects of DRD2 TaqIA genotype on the motivational modulation of interference processing, we first computed the *F*-test comparison for the main effect of genotype. Compared to A2 homozygotes, A1 carriers exhibited increased activation of the left anterior insula [main effect of genotype: *x, y,z* = −30*,* 20*,* −8; *F(*1*,* <sup>240</sup>*)* = 17*.*23; *p* = 0*.*002, FWE-corrected for ROI volume] (**Figure 5**). To verify the reliability of the between-group differences, confidence intervals were estimated for the two genotype groups using bootstrap resampling and the percentile-t method (Schott et al., 2006). Between-group differences were reliable as indicated by nonoverlapping 95 per cent confidence intervals in three motivated conditions (congruent REW, incongruent PUN, incongruent COM), but the confidence intervals in the neutral conditions were largely overlapping between genotype groups, raising the possibility that the genotype-related differences might be largely driven by the motivated conditions. To further explore this possibility, we performed an exploratory *post-hoc* masking analysis in which the main effect of genotype was inclusively masked with the genotype by motivation interaction contrast (thresholded at *p <* 0*.*05, uncorrected). The genotyperelated activation difference in the left insula remained significant at *p <* 0*.*05, corrected for ROI volume, in this masking analysis.

In addition to the main effect of genotype in the anterior insula, we observed a three-way interaction (congruency × motivation × genotype) in the ACC [*x, y*, *z* = 9, 38, 28; *F(*3*,* <sup>240</sup>*)* = 8*.*44; *p* = 0*.*006, FWE-corrected for ROI volume; see **Figure 6**, top]. *Post-hoc* two-sample *T*-tests over the contrasts of parameter estimates (incongruent vs. congruent) at the peak voxel in the right ACC revealed that A2 homozygotes showed higher activation in the trials with potential reward when compared to A1 carriers [ACC: *t(*30*)* = −2*.*87; *p* = 0*.*007] while A1 carriers as compared to A2 homozygotes exhibited increased activation of the right ACC in the combined reward and punishment condition [*t(*30*)* = 3*.*12; *p* = 0*.*004]. We also observed a trend for a threeway interaction in the right striatum [*x, y*, *z* = 21, −1, −2; *F(*3*,* <sup>240</sup>*)* = 6*.*02; *p* = 0*.*050, FWE-corrected for ROI volume; see **Figure 6**, bottom], but this did not survive Bonferroni correction for multiple ROIs.

### *Effects of the COMT Val108/158Met polymorphism on flanker-related brain activity*

In an exploratory analysis regarding the effects of the wellcharacterized COMT Val108/158Met polymorphism on neural correlates of the flanker task, we observed a genotype by congruency interaction in the lateral PFC, but outside our *a priori* defined anatomical ROI of the DLPFC

[*x, y*, *z* = 51, 17, 22; *F(*2*,* <sup>232</sup>*)* = 7*.*80; *p <* 0*.*001, uncorrected]. Specifically Val homozygotes showed relatively higher lateral PFC activation in incongruent trials relative to Met carriers. There were, however, no further interaction effects between COMT genotype and motivational salience.

# **DISCUSSION**

In the present study, we investigated the influences of the DRD2 TaqIA polymorphism on the modulation of interference processing by reward and punishment. Motivational salience, i.e., the possibility to obtain a reward, to avoid a punishment, or both, was associated with shorter RTs in both the incongruent and congruent flanker condition. While the congruencyrelated RT difference did not differ between motivation conditions, functional MRI revealed a reduced congruency effect in the DLPFC during motivated trials, possibly reflecting increased processing efficiency. Moreover, we observed a complex interaction effect of motivation and genotype on the congruencyrelated RT differences in the behavioral experiment. This effect was not significant in the behavioral data of the fMRI experiment, but could still be observed when combining both datasets. Nominally, carriers of the less common DRD2 TaqIA A1 allele (A1+) with presumably lower D2 receptor density in striatum showed an, at least trendwise, improvement of interference processing in all motivated conditions (most strongly in the rewarded condition), whereas A2 carriers exhibited pronounced improvement during combined anticipation of reward or punishment as compared to either reward or punishment alone. At a neural level, genotype-related activation differences were observed in the anterior insula where A1 carriers showed increased task-related activation, and in the anterior cingulate, where a complex task by genotype interaction was observed.

#### **EFFECTS OF MOTIVATION ON FLANKER PERFORMANCE AND NEURAL CORRELATES**

The motivation to obtain a reward or to avoid a loss was associated with shorter RTs in both, congruent and incongruent trials, while error rates did not show a significant modulation by motivational salience. Because of the dichotomous nature of accuracy rates and the considerable individual variability, the power to detect significant within- or between-group differences is limited, and RTs with their continuous distribution might be a more

**FIGURE 4 | Congruency by motivation interaction in the PFC.** A genotype-independent interaction of congruency and motivation was observed in the PFC (BA 10, bordering BA 46) where activation related to the incongruent versus congruent flanker condition was reduced in the motivated relative to neutral trials. This interaction effect was significant at *p <* 0*.*05, small-volume FWE-corrected for ROI volume. Activations are superimposed on the MNI template

brain provided by MRIcron. Coordinates are in MNI space. Bar plots depict contrasts of parameter estimates (incongruent-congruent) at the peak coordinate separated by genotypes and motivation conditions. Error bars depict standard errors of the mean. NEU, neutral condition; REW, reward condition; PUN, punishment condition; COM, combined reward and punishment condition; INC, incongruent; CON, congruent.

sensitive measure of motivation-related enhancement of cognitive processing, reflecting enhanced vigilance in motivated trials (Hardin et al., 2006). One might argue, however, that shorter RTs, when accompanied by reduced accuracy, might reflect impulsive responding rather than improved performance (Caldu et al., 2007). In the present study, error rates were nominally higher in the reward-related and punishment-related trials, but not in the combined condition. Given the overall low error rates and high variability, it is not possible to determine whether the RT decrease in motivated trials observed here might be to some degree related to impulsive responding. Reward anticipation has been demonstrated to promote responding, but to impair response inhibition in a probabilistic go/no-go task, but no such pattern has been observed for the anticipation of losses (Guitart-Masip et al., 2011). During interference processing in a Stroop task, on the other hand, accuracy was actually improved for rewarded trials

#### **FIGURE 6 | Interaction of genotype, congruency, and motivation.**

Complex genotype-dependent modulation of interference processing was observed in anterior cingulate cortex (ACC) and striatum. The three-way interaction of congruency × motivation × genotype is displayed, which was significant for the ACC at *p <* 0*.*05, small-volume FWE-corrected for ROI volumes. Activations are superimposed on the MNI template brain provided by MRIcron. Coordinates are in MNI space. Bar plots depict contrasts of parameter estimates at the peak coordinate separated by genotypes and motivation conditions. NEU, neutral condition; REW, reward condition; PUN, punishment condition; COM, combined reward and punishment condition; INC, incongruent; CON, congruent.

(Krebs et al., 2010). Future studies employing more sensitive measures of accuracy are therefore needed to determine whether reward-related reductions of response times during performance of complex tasks reflects actual improvement of performance vs. a speed-accuracy tradeoff.

Despite the lack of a specific modulation of the RT congruency effect by motivation, at a neural level, we observed a congruency by motivation interaction in the PFC where motivational conditions were associated with reduced activation during processing of incongruent relative to congruent flanker trials. This prefrontal fMRI response reduction is well in line with previous studies suggesting that dopamine modulates processing efficiency in the PFC. Decreased PFC activation accompanied by comparable or even superior behavioral performance has previously been suggested to reflect higher processing efficiency, which has been reported in carriers of the (low-activity) COMT 158Met allele (Egan et al., 2001; Meyer-Lindenberg and Weinberger, 2006; Schott et al., 2006; Caldu et al., 2007) and in Parkinson's disease patients who received L-dopa (Mattay et al., 2002). Most studies reporting dopaminergic modulation of processing efficiency focused on the DLPFC. The activation cluster showing a congruency by motivation interaction in our study was located in the lateral portion of BA 10, in close proximity to BA 46. According to a common definition, BA 9 and 46 are referred to as DLPFC (Cieslik et al., 2012), but there is considerable heterogeneity in the literature regarding the precise delineation of the DLPFC, with several authors referring to at least parts of BA 8, 10, and 45 belonging to the DLPFC (Sarazin et al., 1998; Nitschke et al., 2006; Leung and Cai, 2007), while others have listed BA 46 as part of the ventrolateral PFC (Arango et al., 1995). *In vivo* segmentation of PFC subregions is also somewhat problematic, since most definitions are based on *post mortem* cytoarchitectonic mapping. The precise localization of the prefrontal cluster showing a congruency by motivation interaction to a subregion within the PFC remains thus somewhat speculative. It should be noted, though that its presumed position at the intersection of the DLPFC and the frontopolar cortex is in line with a previous study demonstrating joint deactivation of BA 10 during working memory and reward processing (Pochon et al., 2002) and with recent evidence for pronounced functional connectivity between the anterior portion of the DLPFC and the dACC (Cieslik et al., 2012).

Given the lack of a specific behavioral effect of motivation on the RT congruency effect, our results do not allow us to directly infer that the reduced overall RTs in motivated conditions are the result of increased prefrontal processing efficiency. On the other hand, more generally speaking, the co-occurrence of reduced RTs and decreased DLPFC activation to incongruent trials is at least indicative for a relationship between motivational processes, which are known to elicit dopamine release (Koepp et al., 1998; Schott et al., 2008) and PFC-dependent cognitive processing. In line with this notion, an exploratory analysis within the present study suggested that the COMT 158Val allele, which has been linked to lower prefrontal dopamine availability, was associated with increased lateral PFC activation to incongruent flanker trials.

Several previous studies have investigated the influence of reward and punishment on cognitive tasks, but little is thus far known about their combined effects. Recent animal studies on discrimination learning of frequency-modulated (FM) tones (Ilango et al., 2010) suggest that a combination of both reward and punishment might be associated with particularly strong performance enhancement. In a shuttle box paradigm, Mongolian gerbils were motivated by either appetitive reinforcement (brain stimulation reward) or by aversive reinforcement (avoidance of an electrical footshock), or by a combination of both. Compared to either reinforcement condition alone, the combination of both potentiated speed of acquisition and maximum performance while reducing later extinction. In the study by Ilango and colleagues, reward and punishment were qualitatively distinct (brain stimulation reward vs. foot shock), whereas in our study, the difference between the reward conditions was rather a quantitative one (monetary gain vs. loss). Therefore, the COM condition could to some extent be considered as a reward condition, although it would elicit larger prediction errors than the REW condition. On the other hand, the behavioral pattern observed here speaks against a merely quantitative difference. Namely, while the overall RT reduction across conditions was at least nominally less pronounced in the COM relative to the REW and PUN conditions, the COM condition was the one to show the strongest trend of a motivation-related reduction of the RT congruency effect (for a further interaction with DRD2 genotype, see below). One possible reason for this could be that participants might have slowed down their responses to some extent in the combination condition, in order to maximize accuracy. Indeed, accuracy was nominally higher in the COM condition as compared to the REW and PUN conditions, but these differences were not significant, possibly due to lack of statistical power given the overall high accuracy. Further experiments are needed to clarify whether the combination of both appetitive and aversive reinforcement indeed leads to a shift from speed to accuracy. As a potential limitation, it should also be noted that the size of the cue was larger in the combined condition (**Figure 1**), which could have distracted the participants from fixation of the target arrow after the cue (*Note:* Pilot data from a recent follow-up experiment with cues of equal size does not support the latter explanation).

### **GENETIC VARIABILITY OF D2 RECEPTOR AVAILABILITY INTERACTS WITH MOTIVATIONAL MODULATION OF COGNITIVE PERFORMANCE**

DRD2 TaqIA genotype did not affect overall processing speed, but the congruency-related RT differences, suggesting that its effects cannot be explained by genotype-related differences in sensorimotor processing. Group-specific analysis of the congruencyrelated RT differences in each motivation condition revealed that A1 carriers exhibited improved interference processing in motivated, particularly rewarded trials (albeit significantly so only in the behavioral experiment), whereas the A2 homozygotes benefitted primarily from the combined reward and punishment condition. DRD2 TaqIA has been extensively investigated in neuropsychiatric disorders with presumed dopaminergic dysfunction, and the A1 allele has been associated with increased risk for disorders like substance abuse and pathological gambling or obesity, whereas the A2 allele has been implicated in the genetic risk for schizophrenia (Comings et al., 1996; Noble, 2003; Dubertret et al., 2004; Klein et al., 2007; Wang et al., 2013). Moreover, studies in healthy humans have suggested a role of the DRD2 TaqIA A1 variant in approach-related personality traits (Noble et al., 1998; Reuter et al., 2006; Lee et al., 2007; Smillie et al., 2010). While our finding that A1 carriers exhibit a reduction of the congruency-related RT difference in rewarded trials (and nominally in all motivated conditions) is compatible with the notion that A1 carriers might be more sensitive to rewards and losses, the observation that A2 carriers specifically benefitted from the combined condition was unexpected. The A2 allele has been linked to higher D2 receptor expression in the striatum (see, for example, Ritchie and Noble, 2003). Studies in transgenic mice have shown that even transient overexpression of D2 receptors in the striatum leads to persistent alterations of PFC-dependent cognitive functions, particularly working memory and cognitive flexibility (Kellendonk et al., 2006), and electrophysiological investigations further suggest that these alterations might be related to reduced inhibitory neurotransmission and lower prefrontal dopamine sensitivity (Li et al., 2011). Because levels of D2 receptor overexpression are higher than the described genotype-related D2 receptor expression differences in humans, inferences from these transgenic animal studies to effects of DRD2 TaqIA genotype effects must be considered tentative. If prefrontal dopamine sensitivity was reduced in A2 homozygotes, this might indeed provide a potential explanation for our behavioral results, namely, while reward or punishment alone might be insufficient to raise prefrontal dopamine availability to a level that allows improved interference processing, the combined condition might be associated with a further increase of prefrontal dopamine that might in turn enable a performance advantage in the A2 homozygotes. In A1 carriers, on the other hand, the congruency-related RT difference was at least nominally larger in the combined condition relative to either reward or punishment alone and not significantly different from the neutral condition. If the combined condition was indeed associated with higher prefrontal dopamine release than either reward or punishment alone, the resulting dopamine levels in A1 carriers might be too high for optimal performance, compatible with the model of an inverse Ushaped function of prefrontal dopamine (Meyer-Lindenberg and Weinberger, 2006).

At a neural level, a complex task by genotype interaction was observed in the dACC (**Figure 6**, top). Compared to A2 homozygotes, A1 carriers exhibited relatively reduced dACC activation to incongruent vs. congruent flanker trials in the REW condition, while this pattern reversed in the COM condition, meaning that both groups exhibited lower dACC activation in the condition in which they showed their most pronounced reduction of the congruency-related RT difference. In the DLPFC, lower activation accompanied by comparable or superior performance has been suggested to reflect higher processing efficiency (Meyer-Lindenberg and Weinberger, 2006; see also above), and at least one study suggests that a similar pattern can be observed in the dACC during performance of attention tasks similar to the flanker task (Blasi et al., 2005). One limitation here is the lack of a full replication of the behavioral pattern in the fMRI cohort alone (see **Table A1**). It should be noted, though that the sample size of the fMRI experiment was smaller than that of the behavioral experiment and therefore possibly underpowered for detection of genotype-related differences in behavior. Brain activity phenotypes have been suggested to be more directly related to the molecular and cellular effects of genetic variations and might thus be more readily detectable in smaller samples (Mier et al., 2010). Therefore, we tentatively suggest that the activation pattern in the dACC might to some extent mirror the behavioral pattern, although caution is warranted. This does, on the other hand, not explain why there was no clear genotype-related ACC activation difference in the PUN condition. One explanation for this observation could be that aversive reinforcement might be more likely to engage other neuromodulatory systems, like the serotonergic system (Daw et al., 2002) in addition to the dopaminergic system, which might reduce the overall influence of genetically mediated differences in dopaminergic neurotransmission during PUN trials.

In addition to the interaction effect in the dACC, genotyperelated differences in neural activity patterns included increased activation of the anterior insula in A1 carriers, and *post-hoc* analyses employing confidence interval estimation and masking further suggested that this genotype-related activation difference was largely attributable to the motivated trials. The insula has been commonly found to co-activate with the striatum during reward prediction errors and reward anticipation (for a review, see Diekhof et al., 2012), although some studies argue that insuladependent processing of cues and prediction errors is particularly critical for the prediction of losses (Palminteri et al., 2012; Metereau and Dreher, 2013) and negative choices (Knutson et al., 2007). Previous studies have demonstrated extensive dopaminergic innervation of the insula (Seamans and Yang, 2004), and the insula also shows substantial structural and functional connectivity with the striatum (de Wit et al., 2012; Palminteri et al., 2012; Ye et al., 2011). Expression of D2 receptors, though, is sparse in the insula where the D1 receptor is the predominant dopamine receptor subtype (Hurd et al., 2001). Considering the high levels of D2 receptor expression in the striatum relative to cortical structures, including the insula, it seems somewhat counterintuitive why a genotype-dependent modulation of motivational processing was observed in the insula rather than the striatum where a more complex interaction of task, genotype, and motivation was observed instead. One possible explanation would be that insula activity during motivational processing might be affected by reduced presynaptic D2 autoreceptor density in A1 carriers. In line with this notion, Laakso et al. (2005) observed higher striatal dopamine synthesis capacity in A1 carriers, which they attributed to reduced D2-mediated autoinhibition of dopaminergic terminals in the striatum. Moreover, pharmacological stimulation of D2-type receptors by pramipexole during reward anticipation has been shown to elicit increased activation of the ventral striatum during reward anticipation, which is accompanied by increased functional connectivity between the striatum and the insula (Ye et al., 2011). We tentatively suggest that the parallel reduction of postsynaptic D2 receptors and increase release of dopamine from presynaptic sites in A1 carriers might result in increased dopaminergic action outside the striatum, as also proposed by Stelzel et al. (2010), who suggested that adaptively increased dopamine signaling in A1 carriers might evoke a more pronounced gating signal that facilitates PFC-dependent updating processes during task switching.

It must be seen as a limitation of our study that our results do not allow to make a direct connection between the increased motivation-related insula activity in A1 carriers, which could be observed across motivated conditions, including COM trials, and the complex behavioral pattern in which the different motivation conditions showed non-linear genotype-related differences. Constituting a key structure of the human salience network (Cauda et al., 2011), the anterior insula has been implicated in focal attentional processes as well as in goal-directed behavior (Dosenbach et al., 2007; Nelson et al., 2010), we therefore tentatively suggest that the increased anterior insula activation in the A1 carriers might reflect an increased recruitment of stimulus-responsive attentional resources in the motivated trials, although the relationship between the increased insula activation and the observed behavioral pattern remains, as of now, subject to speculation and needs to be addressed by future studies.

### **POTENTIAL MOLECULAR MECHNANISMS UNDERLYING THE EFFECTS OF DRD2/ANKK1 TaqIA GENOTYPE**

Although the TaqIA polymorphism was initially identified during the localization of the DRD2 gene to human chromosome 11q22-23 (Grandy et al., 1989), it has subsequently been pointed out that the SNP is in fact located 10 kb downstream of the DRD2 termination codon on 11q23.1, within coding region of the adjacent ankyrin repeat and kinase domain containing 1 (ANKK1) gene (Dubertret et al., 2004; Neville et al., 2004). Subsequent genetic association studies have since suggested that other genetic variations of ANKK1 might also be associated with addiction disorders (for a review see Ponce et al., 2009). As the DRD2 and ANKK1 gene are closely linked (Neville et al., 2004; Ponce et al., 2009), it has been suggested that genetic variations in linkage disequilibrium (LD) with TaqIA might explain the observed relationship between the SNP and alterations of human dopaminergic neurotransmission. Indeed the DRD2/ANKK1- TaqIA polymorphism is in LD with several polymorphisms on the DRD2 gene (Duan et al., 2003; Ritchie and Noble, 2003; Fossella et al., 2006). Particularly the C957T polymorphism (rs6277) has received considerable attention as it is in LD with TaqIA and affects stability of the DRD2 mRNA (Duan et al., 2003). However, evidence from *in vivo* D2 receptor binding studies is not conclusive and also in apparent conflict with the *in vitro* data (Hirvonen et al., 2004, see also erratum by Hirvonen et al., 2004, 2009a,b). On the other hand, the TaqIA polymorphism, despite being located on the ANKK1 gene, has been repeatedly associated with reduced striatal D2 receptor density in A1 carriers as evident from three *post mortem* studies (Noble et al., 1991; Thompson et al., 1997; Ritchie and Noble, 2003) and two out of three conducted *in vivo* binding studies (Pohjalainen et al., 1998; Laruelle et al., 1998; Jonsson et al., 1999). Moreover, the A1 allele has been associated with increased striatal dopamine synthesis, presumably due to reduced expression of presynaptic autoinhibitory D2 receptors, whereas no association was found between C957T and dopamine synthesis capacity (Laakso et al., 2005). In line with these findings, Stelzel et al. (2010) reported a generally increased striatal BOLD signal in A1 carriers. As striatal BOLD signal has been shown to correlate with dopamine release (Schott et al., 2008), this increased striatal activation might be related to higher presynaptic dopaminergic activity in A1 carriers.

In light of the converging evidence that TaqIA seems to be most reliably associated with lower D2 receptor density further investigations directed at the interaction of DRD2 and ANKK1 is warranted. The predicted ANKK1 protein is an unselective serine/threonine and tyrosine kinase with 11 ankyrin repeats located at the C-terminal end. TaqIA is located in exon 9 of the ANKK1 gene and leads to a glutamate to lysine substitution in the 11th ankyrin repeat. While a direct interaction of DRD2 and ANKK1 has not yet been confirmed, the ontogenetic pattern of ANKK1 expression strongly resembles that of DRD2 and shows upregulation after D2 receptor stimulation by apomorphine (Hoenicka et al., 2010). Strikingly, a genetic variation in close LD with TaqIA, the ANKK1 Ala239Thr polymorphism differentially modulates constitutive and apomorphine-induced ANKK1 expression *in vitro* (Garrido et al., 2011). While D2 receptor-dependent regulation of ANKK1 expression is therefore likely, future research is required to establish whether ANKK1 in turn can also regulate DRD2 expression.

#### **LIMITATIONS AND DIRECTIONS FOR FUTURE RESEARCH**

A key limitation of our study is the relatively small sample size, particularly with respect to the behavioral results that reached significance in the behavioral study alone and in the combined cohort, but not in the fMRI experiment alone. Therefore, relating the behavioral and fMRI data to each directly remains to some extent speculative. Another limitation is that, while our results are generally in line with previous studies that have demonstrated effects of DRD2 TaqIA genotype on motivational processes and EFs, one must consider that genetic variations within the dopaminergic system do not exert their effects in isolation. Regarding the flanker task, a human electrophysiological study could demonstrate relatively general effects of a DRD4 genetic variation on error processing, with a further modulation by COMT genotype specifically during stop-signal errors. While,

#### **REFERENCES**


in the ventrolateral prefrontal cortex of suicide victims. *Brain Res.* 688, 121–133. doi: 10.1016/0006- 8993(95)00523-S


in the present study, we could replicate previous observations of (inefficient) increased prefrontal activation in Val homozygotes (Meyer-Lindenberg et al., 2006), the sample size did not allow us to systematically investigate the combined effects of COMT and DRD2 genetic variations. Future studies should thus further consider the possibility of both additive (Bertolino et al., 2006) and non-linear (Yacubian et al., 2007) gene-gene interactions within the dopaminergic system on human cognitive and motivational processing.

# **CONCLUSIONS**

Taken together our results provide further evidence for a modulation of PFC-dependent EFs by motivational salience. Behaviorally, motivation was associated with overall RT reduction across flanker conditions. At a neural level, we observed a motivation-related reduction of DLPFC activation specifically during the incongruent vs. congruent flanker trials, suggesting that motivational salience might result in higher processing efficiency. A genetic variation that has previously been linked to striatal dopamine D2 receptor availability did not affect overall performance as indexed by RTs, but instead, showed a complex interaction with motivation on interference effects. A1 carriers with presumably lower D2 expression showed (at least nominally) improved interference processing during rewarded trials, while A2 homozygotes primarily benefitted from the combination of appetitive and aversive reinforcement. At a neural level, a compatible pattern was observed in a complex genotype by task interaction in the dACC. Additionally, A1 carriers showed an increased neural response of the anterior insula, an effect mostly driven by motivationally salient stimuli. These findings are in line with previous research linking prefrontal dopamine to performance of EFs, possibly following an inverse U-shaped function.

# **ACKNOWLEDGMENTS**

The authors would like to thank Maria Michelmann for help with genotyping. We further thank Anna Deibele for help with psychological assessment, Carola Nath for assistance in data processing, and Kerstin Möhring and Ilona Wiedenhöft for assistance with MRI acquisition. This project was supported by the Deutsche Forschungsgemeinschaft (SFB 779, TP A8) and the Leibniz Graduate School (Master stipend to Anni Richter and Marieke Klein, PhD stipend to Adriana Barman). The authors have no conflicts of interest or involvement, financial or otherwise, to report.

L., et al. (2006). Additive effects of genetic variation in dopamine regulating genes on working memory cortical activity in human brain. *J. Neurosci.* 26, 3918–3922. doi: 10.1523/JNEUROSCI.4975-05.2006


doi: 10.1016/j.biopsycho.2008. 04.010


human ventral tegmental area. *Science* 319, 1264–1267. doi: 10.1126/science.1150605


Makam, H., et al. (1989). The human dopamine D2 receptor gene is located on chromosome 11 at q22-q23 and identifies a TaqI RFLP. *Am. J. Hum. Genet.* 45, 778–785.


*Psychon. Bull. Rev.* 17, 821–826. doi: 10.3758/PBR.17.6.821


on neurophysiological markers of performance monitoring. *J. Neurosci.* 27, 14190–14198. doi: 10.1523/JNEUROSCI.4229-07.2007


O-methyltransferase val158-met genotype and individual variation in the brain response to amphetamine. *Proc. Natl. Acad. Sci. U.S.A.* 100, 6186–6191. doi:


*Cogn. Psychol.* 41, 49–100. doi: 10.1006/cogp.1999.0734


B., et al. (2002). The neural system that bridges reward and cognition in humans: an fMRI study. *Proc. Natl. Acad. Sci. U.S.A.* 99, 5669–5674. doi: 10.1073/pnas.082111099


functional magnetic resonance imaging data for related visual and auditory stroop tasks. *J. Cogn. Neurosci.* 20, 1063–1078. doi: 10.1162/jocn.2008.20074


doi: 10.1016/j.neuropsychologia. 2008.08.016


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 15 March 2013; accepted: 20 May 2013; published online: 05 June 2013.*

*Citation: Richter A, Richter S, Barman A, Soch J, Klein M, Assmann A, Libeau C, Behnisch G, Wüstenberg T, Seidenbecher CI and Schott BH (2013) Motivational salience and genetic variability of dopamine D2 receptor expression interact in the modulation of interference processing. Front. Hum. Neurosci. 7:250. doi: 10.3389/fnhum. 2013.00250*

*Copyright © 2013 Richter, Richter, Barman, Soch, Klein, Assmann, Libeau, Behnisch, Wüstenberg, Seidenbecher and Schott. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

# **APPENDIX**

#### **Figure A1 | Literature-based ROI of the dorsal ACC.**


#### **Figure A2 | Literature-based ROI of the anterior insula.**

35 17 −4 TAL Padmala and Pessoa, 2011

33 16 6 TAL Knutson et al., 2000 −32 28 −6 MNI Stoppel et al., 2011 32 20 4 TAL Knutson et al., 2000 30 30 −6 MNI Stoppel et al., 2011 −35 26 5 TAL Padmala and Pessoa, 2011 40 30 4 MNI Stoppel et al., 2011 −31 19 4 TAL Padmala and Pessoa, 2011 38 24 −8 MNI Stoppel et al., 2011 31 17 11 TAL Padmala and Pessoa, 2011 38 34 3 MNI Stoppel et al., 2011 31 19 8 TAL Padmala and Pessoa, 2011 34 26 6 MNI Stoppel et al., 2011

#### **Figure A3 | Literature-based ROI of the striatum.**


#### **Table A1 | Behavioral data (***t***-statistics), separated by experiment.**


*Results of post hoc paired T-tests testing for effects of the motivation conditions on the congruency effect of reaction times, separated by genotype group and experiment. All p-values are one-tailed, except for the REW vs. PUN contrast for which we had no directed hypothesis. NEU, neutral condition; REW, reward condition; PUN, punishment condition; COM, combined reward and punishment condition. Note: As there was no genotype by motivation interaction in the fMRI experiment alone, all t- and p-values are displayed for illustrative purpose only and printed in grey. \*p < 0.05.*

# **REFERENCES**


# Individual visual working memory capacities and related brain oscillatory activities are modulated by color preferences

# *Masahiro Kawasaki 1,2\* and Yoko Yamaguchi <sup>3</sup>*

*<sup>1</sup> Rhythm-based Brain Information Processing Unit, RIKEN BSI-TOYOTA Collaboration Center, Saitama, Japan*

*<sup>2</sup> Laboratory for Advanced Brain Signal Processing, RIKEN Brain Science Institute, Saitama, Japan*

*<sup>3</sup> Neuroinformatics Japan Center, RIKEN Brain Science Institute, Saitama, Japan*

#### *Edited by:*

*Luke D. Smillie, The University of Melbourne, Australia*

#### *Reviewed by:*

*Michael X. Cohen, University of Amsterdam, Netherlands Erik M. Mueller, Marburg University, Germany*

#### *\*Correspondence:*

*Masahiro Kawasaki, 2-1 Hirosawa, Wako-shi, Saitama 351-0198, Japan. e-mail: kawasaki@brain.riken.jp*

Subjective preferences affect many processes, including motivation, along with individual differences. Although incentive motivations are proposed to increase our limited visual working memory (VWM) capacity, much less is known about the effects of subjective preferences on VWM-related brain systems, such as the prefrontal and parietal cortices. Here, we investigate the differences in VWM capacities and brain activities during presentation of preferred and non-preferred colors. To this end, we used time-frequency (TF) analyses of electroencephalograph (EEG) data recorded during a delayed-response task. Behavioral results showed that the individual VWM capacities of preferred colors were significantly higher than those of non-preferred colors. The EEG results showed that the frontal theta and beta amplitudes for maintenance of preferred colors were higher than those of non-preferred colors. Interestingly, the frontal beta amplitudes were consistent with recent EEG recordings of the effects of reward on VWM systems, in that they were strongly and individually correlated with increasing VWM capacities from non-preferred to preferred colors. These results suggest that subjective preferences affect VWM systems in a similar manner to reward-incentive motivations.

**Keywords: working memory, subjective preference, EEG, frontal, theta, beta**

# **INTRODUCTION**

As a well-known proverb says, "Everyone to his taste." there are large individual differences in subjective preferences. Such preferences individually lead to either positive or negative emotions, which seem to influence not only our potential personalities, but also the current behaviors. However, little is known regarding whether subjective preferences directly affect cognitive processing. For example, Are we able to better memorize a preferred visual stimulus? To address this issue, it is useful to measure the visual working memory (VWM) capacity, which refers to our ability to memorize and maintain visual stimuli temporarily (Phillips, 1974; Pashler, 1988; Rensink, 2002). Previous psychological studies have suggested that VWM has a limited capacity and large individual differences, based on parametric load manipulation in a delayed matching-to-sample task, which required the short-term maintenance of several visual items (Luck and Vogel, 1997; Cowan, 2001; Vogel et al., 2001). The VWM capacity can be increased by training (Klingberg et al., 2002; Olesen et al., 2004; Jaeggi et al., 2008; McNab et al., 2009) or enhanced motivation by rewards such as money (Pochon et al., 2002; Gilbert and Fiez, 2004; Krawczyk et al., 2007; Kawasaki and Yamaguchi, in press). However, it not known whether VWM is affected by subjective preferences.

Many previous neuroimaging studies on humans have demonstrated that a vast network of brain regions—including the frontal, parietal, and visual cortices—forms the neural substrates for VWM (Courtney et al., 1997, 1998; Postle and D'Esposito, 1999; Pessoa et al., 2002). In particular, the frontal and parietal regions have been proposed to be associated with maintenance of mental representations. This is because activity in these areas is correlated with individual VWM capacity as demonstrated by electroencephalography (EEG) recordings (Gevins and Smith, 2000; Jensen and Tesche, 2002; Jensen et al., 2002; Vogel and Machizawa, 2004) and functional magnetic resonance imaging (fMRI) studies (Todd and Marois, 2005; Song and Jiang, 2006; Xu and Chun, 2006; Kawasaki et al., 2008; Cowan et al., 2011; Robitaille et al., 2011).

In contrast, the brain regions involved in preference decisionmaking are proposed to be part of the reward-related and emotion-related brain networks, as activities in these areas are enhanced during preference judgments, such as judging the attractiveness of faces or preferences for specific food and beverages. These networks are partially overlapped in the common brain regions including anterior frontal cortex, anterior cingulate, striatum, and amygdala (Aharon et al., 2001; O'Doherty et al., 2002; McClure et al., 2004). However, the emotion-related brain networks are more likely to include hedonic hotspots, such as the nucleus accumbens and ventromedial prefrontal cortex with opioid or cannabinoid neurotransmissions, whereas the reward-related brain networks are involved in the dopaminergic mid-brain and dopamine projected areas such as the medial orbitofrontal cortex (Berridge, 2003; Kringelbach and Berridge, 2009; Dai et al., 2010; Smillie et al., 2011). Indeed, enhanced ventromedial prefrontal activity is strongly correlated with individual subjective preferences, while the dopaminergic mid-brain and orbitofrontal cortex show correlation with the reward values (O'Doherty et al., 2002; Knutson et al., 2005). Moreover, recent fMRI studies have shown that the nucleus accumbens is involved in automatic and first impressions of preferences, whereas the orbitofrontal cortex is involved in decision-making (Kim et al., 2007).

Thus, although there is rich evidence for VWM- and preference-related brain activities individually, few studies have addressed the dynamic interactions between them. That is to say, there is little neurological evidence regarding how VWM-related networks are affected by differences in subjective preferences, although some studies have proposed that there are interactions between VWM- and reward-related brain activities (Pochon et al., 2002; Gilbert and Fiez, 2004; Krawczyk et al., 2007; McNab and Klingberg, 2008; Kawasaki and Yamaguchi, in press).

In this study, we investigated the effects of subjective preferences on VWM capacity and its neural mechanisms, using timefrequency (TF) analyses of EEG data. EEG oscillations are thought to reflect the synchronization of a large number of neurons underlying a particular function (Varela et al., 2001). In particular, low frequency-band activities such as the theta (4–8 Hz) and alpha (8–12 Hz) oscillations are thought to be related to several functions of VWM, such as the manipulation and maintenance of mental representations (Mizuhara et al., 2004; Sauseng et al., 2005; Klimesch et al., 2008; Kawasaki et al., 2010). Theta activity also seems to be related to emotional changes caused by subjective preferences, as theta activities in the frontal and occipital regions that are associated with pleasant stimuli are higher than those associated with unpleasant stimuli (Sammler et al., 2007; Lindsen et al., 2010; Kawasaki and Yamaguchi, 2012). Moreover, frontal beta activity is associated with motivation and the relative evaluation of reward values (Cohen et al., 2007; Marco-Pallares et al., 2008; Kawasaki and Yamaguchi, in press).

In this study, we focused on the individual subjective preferences for colors and compared the oscillatory behavior of the EEG under two conditions in a delayed response VWM task. In one condition, the subjects were required to memorize stimuli presented as colors that they preferred. In the other condition, we asked the subjects to memorize non-preferred colors. We hypothesized that subjective preferences would affect VWM processing, and both VWM capacity and associated EEG oscillation power would be reduced in the non-preferred condition.

#### **MATERIALS AND METHODS**

Nineteen healthy right-handed volunteers (8 women, 11 men; mean age = 21*.*5 ± 0*.*5 years, range 18–27 years) took part in this experiment. They reported having normal or corrected-tonormal visual acuity, normal hearing acuity, and normal motor abilities using subjective questionnaires. All participants gave written informed consent, which was approved by the Ethical Committee of the RIKEN (in accordance with the Declaration of Helsinki), *prior* to participation in this study.

Before the EEG experiments, each participant completed a pretest to identify their personal color preferences (Kawasaki and Yamaguchi, 2012). In the pretest, participants were asked to choose between two colored squares presented in the right and left hemi-fields, relative to a central white fixation point on a 24-in. computer display (ProLite E2410HDS, Iiyama, JAPAN). Each trial consisted of 1 s of stimulus presentation, a 2 s response period for their judgment, and a 2 s inter-trial interval. Two colors were selected from the ten available colors [white (*r* = 255, *g* = 255, *b* = 255), red (255, 0, 0), green (0, 255, 0), blue (0, 0, 255), yellow (255, 255, 0), magenta (255, 0, 255), cyan (0, 255, 255), olive (128, 128, 0), purple (128, 0, 128), and aqua (0, 128, 128)]. Each participant completed 90 trials. All possible color combinations were presented twice, with a reshuffling of the right and left positions. We selected the most and least preferred colors for each individual using the number of times each color was selected.

In the VWM experiments, participants were required to memorize 2, 4, or 6 colored shapes (visual angle; approximately 1◦ × 1◦, shape; circle, square, triangle, star, pentagon, parallelogram, cross, and trapezoid), which were simultaneously presented at random locations in an invisible 3 × 3 cell matrix for 0.2 s (**Figure 1A**, sample display). All colors were defined by the subject's favorite color ("preferred" condition) or least favorite color ("non-preferred" condition) in each trial. After a 2 s retention interval, the participants were required to judge whether a probe shape matched the shape at the same location in the sample display via a button press, while the fixation point was red for 2 s (test display). In half of the trials, the probe shape matched the sample shape. In the other half, the probe shape was replaced with another shape from the sample display. The inter-trial interval (ITI) was 2 s. Each participant completed three separate blocks consisting of 60 trials each, consisting of three shapes (2, 4, or 6) × 2 color preference conditions ("preferred" or "nonpreferred") × 2 change types (change or non-change of the probe shape from the sample shape) × 5 repetitions. Therefore, each condition (number of shapes × color preference) totaled 30 trials. All participants practiced in a behavioral training session before the EEG-measurement sessions. The training sessions were identical to the real sessions in their procedures and both had 60 trials.

EEG recordings were collected from 60 scalp electrodes (Ag/AgCl) embedded in an electro cap (Brain Cap; Brain Products, Germany) in accordance with the extended version of the international 10/20 system. Reference electrodes were placed on the right and left ear lobes. Electrooculography (EOG) measurements were recorded from electrodes above and below the left eye by monitoring eye blinks or vertical eye movements, and from electrodes placed 1 cm from the right and left eyes by monitoring horizontal eye movements. The EEG and EOG data were amplified using Neuroscan equipment (Compumedics NeuroScan Corp., Charlotte, NC, USA). The sampling rate was 500 Hz. The EEG data were filtered in the band-pass range from 0.1 to 50 Hz.

We segmented the EEG data into 3 s epochs (a 2 s retention interval and 0.5 s pre- and post-retention intervals; 1500 time points in total) for each trial. To reduce or eliminate artifacts, we conducted an infomax independent components analysis (ICA) on the EEG data from the correct trials. ICA components that significantly correlated with vertical or horizontal eye movements in the EOG data were rejected, and the ICA-corrected data were

recalculated using a regression of the remaining components. To elucidate the cortical activity with decreasing errors from volume conduction, we applied current source density analysis using the spherical Laplace operator to the voltage distribution on the surface of the scalp (Perrin et al., 1989).

To identify the TF amplitudes during the retention interval, we applied wavelet transforms using Morlet's wavelets having a Gaussian shape in the time and the frequency domains around their center frequency (Tallon-Baudry et al., 1997). We used Morlet's wavelets for their high time and frequency resolutions, which allowed us to observe transitions in both the low and high frequency oscillations (Herrmann et al., 2005). The TF amplitude for each time point in each trial was the squared norm of the results of the convolution of the original EEG signals with a complex Morlet's wavelet function (*f /*σ*<sup>f</sup>* = 7), ranging from 0.5 to 40 Hz in 0.5 Hz steps (i.e., 80 TF values at each time point for each single trial waveform). The TF amplitudes from the ITIs were averaged to generate the baseline amplitudes. The delay-period TF amplitude was calculated by subtracting the baseline data for each trial from each frequency band. The corrected TF amplitude was averaged across single trials for all conditions for each participant.

#### **RESULTS**

Accuracy rates (percentage correct) were higher when we presented fewer objects in both the "preferred" and "nonpreferred" condition (2 objects, preferred: 94*.*0 ± 1*.*5%; 2 objects, non-preferred: 93*.*5 ± 1*.*6%; 4 objects, preferred: 79*.*1 ± 2*.*3%; 4 objects, non-preferred: 77*.*9 ± 2*.*2%; 6 objects, preferred: 71*.*2 ± 2*.*3%; 6 objects, non-preferred: 66*.*7 ± 2*.*1%). A Two-Way analysis of variance (ANOVA) revealed a main effect of the number of objects [*F(*2*,* <sup>108</sup>*)* = 72*.*69, *P <* 0*.*01], but no significant effect of preference [*F(*1*,* <sup>108</sup>*)* = 1*.*54, *P* = 0*.*22] and no significant interaction [*F(*2*,* <sup>108</sup>*)* = 0*.*54, *P* = 0*.*59].

The VWM capacity was estimated using Cowan's *K* formula: *K* = *N* × (hit rate + correct rejection rate—1), where *K* is the estimated number of objects stored in VWM, and *N* is the number of presented objects in the sample display (Cowan, 2001; Todd and Marois, 2004; Kawasaki et al., 2008). For each participant, each *K* value (*K*2 objects, *K*4 objects, and *K*6 objects) was calculated for each conditions for the 2 conditions (preferred or non-preferred) × the 3 different numbers of objects (2 or 4 or 6). To identify the limitation of the VWM capacity of each participant for each condition, we compared *K* values among 3 different numbers of objects and selected maximum *K* value (*K*max) for the preferred and non-preferred conditions. These methods were based on previous studies (Vogel and Machizawa, 2004; Todd and Marois, 2005).

The averaged VWM capacity for each condition is shown in **Figure 1B**. A Two-Way ANOVA revealed a significant interaction [*F(*2*,* <sup>108</sup>*)* = 6*.*87, *P <* 0*.*01], but no significant main effects of the number of objects [*F(*2*,* <sup>78</sup>*)* = 0*.*20, *P* = 0*.*81] or preference [*F(*1*,* <sup>78</sup>*)* = 1*.*00, *P* = 0*.*32]. There was a significant difference between the "preferred" and "non-preferred" conditions for the larger number of presented objects (2 objects, *Z* = 0*.*15, *P* = 0*.*88; 4 objects, *Z* = 0*.*78, *P* = 0*.*43; 6 objects, *Z* = 1*.*93, *P <* 0*.*05). The maximum VWM capacity, which was defined by the maximum values among all VWM capacities for the number of presented objects showed significant differences between the "preferred" and "non-preferred" conditions (*Z* = 2*.*44, *P <* 0*.*02; VWM capacity for preferred condition, 3*.*02 ± 0*.*16 objects; non-preferred condition, 2*.*62 ± 0*.*15 objects). These results suggest that subjective color-preference affected the available VWM capacity in our experiments.

To identify the specific pattern of brain activity representing VWM maintenance, we calculated the TF amplitudes from the 60-channel EEG data during the 2 s retention intervals in comparison with the baseline periods. For the theta (the frequency: 4–8 Hz) and alpha (12 Hz) amplitudes, an ANOVA showed main effects of the number of objects in the frontal and parietal regions which electrodes were shown in **Figure 2A** (*P <* 0*.*05). The theta amplitudes in the frontal and right motor regions were enhanced with increasing number of objects, whereas the occipital alpha amplitudes showed opposite way.

Moreover, in order to investigate the VWM-capacity-related activities, we applied correlation analyses between the limitations of individual VWM capacities (i.e., maximum *K* values; *K*max) and individual differences in the amplitudes between *K*max and *K*2 objects. Significant correlations were observed in the averaged theta (4–8 Hz) and alpha (12 Hz) activities (*P <* 0*.*05). The theta and alpha correlations were mainly found in the frontal/occipital and occipital regions, respectively. The topographic colored scalp maps for the significant statistical values are shown in **Figure 2B**. The frontal theta and occipital theta were positively correlated with individual VWM capacities [electrode measuring the peak statistic value, F4; preferred, *r(*19*)* = 0*.*38, *P <* 0*.*05; non-preferred, *r(*19*)* = 0*.*51, *P <* 0*.*01], whereas the occipital alpha amplitudes showed negative correlations [O1; preferred, *r(*19*)* = −0*.*58, *P <* 0*.*01; non-preferred, *r(*19*)* = −0*.*39, *P <* 0*.*05]. Although these statistical values were not enough for multiple comparisons, the frontal theta and occipital alpha oscillations are likely to be involved in the maintenance of VWM, which is similar to our previous findings.

Next, to investigate the pattern of brain activity representing the effects of color preference on VWM capacity, we compared delay-period oscillatory amplitudes between the "preferred" and "non-preferred" conditions under the high VWM load (6 objects), where the VWM capacity was found to be significantly different between the preference conditions. For the theta amplitudes (4–8 Hz), significant differences in amplitude were found in the frontal, parietal, and occipital brain regions (multiple comparison test with Bonferroni correction for the number of electrodes (i.e., comparison was 61); frontal (AF8 electrodes), *P <* 0*.*05; parietal (Pz), *P <* 0*.*05; occipital (O1), *P <* 0*.*05). In addition, low beta activities (12–20 Hz) under the "preferred" condition were significantly higher than under the "nonpreferred" condition, and the enhancements were distributed across both the right lateral frontal areas (F4, *P <* 0*.*05) and the anterior frontal areas (Fpz, *P <* 0*.*05) (**Figures 3A,B**). However, significant differences were not found in the alpha ranges.

The time-course transitions of the different amplitudes between the "preferred" and "non-preferred" conditions on the right frontal electrode (F4) during the maintenance of 2, 4, and 6 items are shown in **Figure 3C** (top, middle, and bottom, respectively). The frontal low beta amplitudes, including the alpha amplitudes, were discretely higher during the retention intervals under the "preferred" condition. These enhancements were large and long-lasting, increasing with an increasing number of items in the sample display. On the other hand, the frontal theta enhancements were more sustained during the retention interval for six objects under the "preferred" condition in comparison to the "non-preferred" condition.

Finally, to identify the brain oscillations reflecting increased VWM capacity under the "preferred" condition, we examined the activity patterns under the "preferred" condition as compared to those under the "non-preferred" condition. Frontal beta (16 Hz; frequency measuring the peak statistic value) delay-period amplitudes were significantly correlated with the increased VWM capacity in preferred colors vs. non-preferred colors conditions [see **Figures 3D,E**; electrode measuring the peak statistic value, F4: *r(*19*)* = 0*.*55, *P <* 0*.*01]. The frontal areas overlapped the VWM capacity-correlated regions. For the other VWM-capacitycorrelated regions, the frontal theta and occipital alpha amplitudes did not significantly correlate with preference-induced increases in VWM capacity [F4 theta: *r(*19*)* = 0*.*13, *P* = 0*.*41; O1 alpha: *r(*14*)* = −0*.*09, *P* = 0*.*62].

# **DISCUSSION**

The current study clearly shows that, subjective preferences for visual stimuli affect VWM capacity in individuals. VWM capacity was significantly higher with a subject's favorite color compared to their less preferred colors, particularly under conditions involving high VWM loads. This result is in agreement with previous findings showing that VWM capacity is enhanced by anticipation of high monetary rewards for correct answers, in comparison with low or no monetary rewards (Pochon et al., 2002; Gilbert and Fiez, 2004; Krawczyk et al., 2007; Kawasaki and Yamaguchi, in press). Therefore, subjectively preferred stimuli *per se* may affect VWM systems, much like the anticipation of reward.

**FIGURE 2 | (A)** (Left) Statistical topographic colored scalp maps of the theta (4–8 Hz; top panel) and alpha (12 Hz; bottom panel) delay-period activation showing p-values for the main effects of number of objects on ANOVA (*P <* 0*.*05). The drawing shows the top view of the scalp. (Right) Bar graph showing the subject-averaged amplitude of theta frequency bands at the frontal (FCz) electrode and alpha frequency bands at the occipital (O1) electrode under the "preferred" (red) and "non-preferred" (blue) conditions. Error bars represent standard errors of the mean. **(B)** (Left) Statistical

topographic colored scalp maps showing *p*-values for the correlations between individual *K*max values (VWM capacity estimates) and the theta (4–8 Hz; top panel) and alpha (12 Hz; bottom panel) delay-period activation differences between the *K*max and two items (*P <* 0*.*05). (Right) Scatter plot showing the relationships between individual *K*max-values and individual theta and alpha amplitude differences between the *K*max and two items at the frontal (F4) and occipital (O1) electrodes under the "preferred" and "non-preferred" conditions. Black lines represent the regression fit.

In relation to the behavioral results, our EEG results demonstrate that VWM-related brain activities are modulated by subjective preferences. First, we identified delay-period theta and alpha oscillatory activities in the frontal and occipital regions, which are related to number of objects to be remembered (VWM load) and strongly correlated with individual differences in VWM capacities, as patterns of brain activity directly related to VWM. These VWM-load and VWM-capacity-related brain mechanisms were not overlapped. These results might support the previous fMRI finding about functional dissociations between VWM-load and VWM-capacity-related regions in the fronto-parietal networks (Linden et al., 2003), although EEG has spatial limitations in comparison with their fMRI. Moreover, the frontal theta and occipital alpha activities showed opposing relationships to VWM capacity, as frontal theta activity was positively correlated and occipital alpha activity negatively correlated with VWM capacity.

of the beta (16 Hz; frequency measuring the peak statistic value) activation differences between the "prefer" and "non-preferred" conditions. **(B)** Bar graph showing the subject-averaged amplitude of beta frequency bands at the frontal electrode (F4 electrode measuring the peak statistic value) under the "preferred" (red) and "non-preferred" (blue) conditions. Error bars represent standard errors of the mean. **(C)** Different time-frequency amplitudes during the retention intervals for "preferred" and "non-preferred" conditions for the maintenance of 2 (top), 4 (middle),

across successful trials for all participants. The dotted vertical lines indicate the onset of the retention interval. **(D)** Topographic colored scalp maps showing *p*-values for the beta delay-period amplitudes, which were significantly correlated with increasing VWM capacity between the "preferred" and "non-preferred" conditions. **(E)** Scatter plot showing different VWM capacities and F4 electrode beta amplitudes between the "preferred" and "non-preferred" conditions.

These results are somewhat consistent with previous findings using fMRI (Linden et al., 2003; Todd and Marois, 2004, 2005; Song and Jiang, 2006; Xu and Chun, 2006; Kawasaki et al., 2008; Cowan et al., 2011; Robitaille et al., 2011) and EEG recording (Gevins et al., 1979; Jensen and Tesche, 2002; Jensen et al., 2002; Klimesch et al., 2008; Sauseng et al., 2009), even though many fMRI studies reported positive correlations between parietal blood oxygenation level dependent (BOLD) signals and VWM capacities. However, the negative correlations between occipital alpha activity and VWM capacity in the current study may well be because of negative relationships between the BOLD signals and alpha activities (Goldman et al., 2002; Laufs et al., 2003; Moosmann et al., 2003; Meltzer et al., 2007; Michels et al., 2008).

Thus, the frontal and occipital regions are candidate neural substrates for the maintenance of VWM, in agreement with a number of previous electrophysiological studies in non-human primates (e.g., Friedman and Goldman-Rakic, 1994) and many fMRI studies in humans (e.g., Curtis and D'Esposito, 2003). Previous EEG studies have also reported that theta and alpha activities in extended brain regions increase during several VWM tasks, including delayed matching-to-sample, *n*-back, mental manipulation, spatial WM, and mental calculation tasks (Ishihara and Yoshii, 1972; Tesche and Karhu, 2000; Kahana et al., 2001; Raghavachari et al., 2001; Busch and Herrmann, 2003; Cooper et al., 2003; Mizuhara et al., 2004; Klimesch et al., 2005; Kawasaki and Watanabe, 2007; Klimesch et al., 2008; Sauseng et al., 2009; Kawasaki et al., 2010).

VWM-related frontal activities are affected by subjective preferences, as demonstrated by the oscillatory amplitudes correlated with VWM capacity being enhanced under the "preferred" condition as compared to the "non-preferred" condition. The effects of preference on theta amplitudes were strongly and sustainably observed during the retention intervals in this study. These results suggest that the frontal oscillations may reflect the motivational effects of subjective preferences via signals from the rewardand/or emotion-related brain regions during the maintenance of VWM, although these brain regions are proposed to be separated (Berridge, 2003; Kringelbach and Berridge, 2009; Dai et al., 2010; Smillie et al., 2011).

On the other hand, frontal beta activities play an important role in the facilitation of VWM systems by subjective preferences, much as they do with reward incentive motivations. Indeed, beta amplitudes were significantly correlated with increasing VWM capacity from the "non-preferred" to "preferred" conditions, which were similar to previous findings showing improvements in VWM capacity with increasing monetary reward (Kawasaki and Yamaguchi, in press). The beta activities are unlikely to be involved in the enhancements of subjective preferences themselves, because the beta activities did not show any significant differences between the preferred and non-preferred colors during the preference judgment tasks in our recent study (Kawasaki and Yamaguchi, 2012). Moreover, frontal beta activities transiently increased during maintenance of a preferred color, in comparison with non-preferred colors. The modulated activities are distributed not only among lateral parts (F4 and F6 electrodes) but also among anterior parts (Fpz and Fp2 electrodes) of the frontal regions. The anterior and lateral frontal regions are thought to be involved in judgments of preference (Aharon et al., 2001; O'Doherty et al., 2002; McClure et al., 2004).

The preference-related beta activities might be involved in opioid or cannabinoid neurotransmissions which play an important role in processing of emotion (Berridge, 2003; Kringelbach and Berridge, 2009). In contrast, the beta activities were also reported in the similar brain regions during reward predictions (Elliott et al., 2000; Knutson et al., 2001; O'Doherty et al., 2002; Gottfried et al., 2003; McClure et al., 2003; Knutson et al., 2004; Kable and Glimcher, 2007) and the presentation of monetary reward magnitudes and their probabilities, relative to loss feedback for gambling (Marco-Pallares et al., 2008) and reinforcement learning tasks (Cohen et al., 2007). These enhanced frontal beta activities may be related to the mid-brain dopaminergic responses and striatal activities, because the durations of these beta amplitudes are similar to the time-course of frontal dopamine-related activity (Fiorillo et al., 2003; McClure et al., 2003; Schultz, 2007). Moreover, the dopamine-related activity is proposed to be related to the different personality traits which would be tightly linked to subjective preferences in the present study (Depue and Collins, 1999; Zald et al., 2008; Previc, 2009). Considering these data together, frontal beta activities seem to be related to individual different signals of motivation derived from not only reward-related brain regions bus also emotionrelated brain regions. However, it is worth noting that EEG studies have inherent limitations in identifying the precise source of beta activity. Therefore, it is important to identify the detailed neural networks involved in motivation in future studies, possibly by making use of simultaneous fMRI and EEG.

#### **REFERENCES**


In contrast to the frontal regions, occipital regions are involved in VWM maintenance, irrespective of subjective preferences, since the VWM-capacity-related alpha activities showed no differences between the "preferred" and "non-preferred" conditions. However, it is possible that the occipital alpha decrements were affected by the amount of visual processing, since we did not include a control condition requiring participants to merely look at but not to memorize the visual stimuli, as was included in previous studies (e.g., Todd and Marois, 2004). Indeed, the occipital regions contain various visual cortices. However, the occipital theta amplitudes for the "preferred" condition were higher than those for the "non-preferred" condition, even though the activity patterns did not directly represent individual VWM capacities. This may be explained by previous findings that attention-related occipital theta activities were enhanced by subjective preferences (Kawasaki and Yamaguchi, 2012), because VWM tasks require visual attention (Awh and Jonides, 2001).

In this study, it should be noted that there remains the possibility that not only subjective preferences but also other factors such as differences between the discrimination and familiarity of colors may affect VWM capacity and EEG differences (e.g., brightness, RBC dimensions and so on), because the chosen colors differed between the participants. (Preferred color: red, 3; green, 1; blue, 4; yellow, 1; magenta, 1; cyan, 3; olive, 0; purple, 1; aqua, 3; white, 2) (Non-preferred color: red, 3; green, 0; blue, 0; yellow, 1; magenta, 1; cyan, 2; olive, 7; purple, 1; aqua, 1; white, 3). Therefore, future study should rigorously clarify such effects of the color components on VWM capacity and EEG activities.

This study focused on the influence of the color preference on VWM capacity, however, visual stimulus included several features such as shapes. There are possibilities that preferred shapes are better memorized than non-preferred shapes. Unfortunately, it is difficult to judge such preferences of shapes in this study. So, it is necessary to confirm the effects of preference on the VWM capacity by using other visual features in future study.

# **ACKNOWLEDGMENTS**

The research was supported by Grant-in-Aid for Scientific Research on Innovative areas (22118510 & 21120005) and Grantin-Aid for Young Scientists (B) (23700328). We would like to thank Naoko Tsukakoshi for her support of our data acquisitions, Yuuki Koizumi for his support of our data analyses, and Eri Miyauchi for her support in the creation of our task designs.

the role of alpha oscillations during externally vs. internally directed attention and the implications for idling and inhibition hypotheses. *Int. J. Psychophysiol.* 47, 65–74.


Transient and sustained activity in a distributed neural system for human working memory. *Nature* 386, 608–611.


culturally familiar drinks. *Neuron* 44, 379–387.


B., et al. (2002). The neural system that bridges reward and cognition in humans: an fMRI study. *PNAS* 99, 5669–5674.


music. *Psychophysiology* 44, 293–304.


search task in human. *J. Neurosci.* 17, 722–734.


memory for objects. *Nature* 440, 91–95.

Zald, D. H., Cowan, R. L., Riccardi, P., Baldwin, R. M., Ansari, M. S., Li, R., et al. (2008). Midbrain dopamine receptor availability is inversely associated with novelty-seeking traits in humans. *J. Neurosci.* 28, 14372–14378.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 11 July 2012; accepted: 08 November 2012; published online: 27 November 2012.*

*Citation: Kawasaki M and Yamaguchi Y (2012) Individual visual working memory capacities and related brain oscillatory activities are modulated by color preferences. Front. Hum. Neurosci. 6:318. doi: 10.3389/fnhum. 2012.00318*

*Copyright © 2012 Kawasaki and Yamaguchi. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

# The role of D4 receptor gene exon III polymorphisms in shaping human altruism and prosocial behavior

#### *Yushi Jiang1, Soo Hong Chew1 and Richard P. Ebstein2 \**

*<sup>1</sup> Department of Economics, National University of Singapore, Singapore*

*<sup>2</sup> Department of Psychology, National University of Singapore, Singapore*

#### *Edited by:*

*Jan Wacker, Philipps-Universität Marburg, Germany*

#### *Reviewed by:*

*Robert Kumsta, University of Freiburg, Germany Joni Sasaki, York University, Canada*

#### *\*Correspondence:*

*Richard P. Ebstein, Department of Psychology, Faculty of Arts and Social Sciences, National University of Singapore, Block AS4, #02-07, 9 Arts Link, Singapore 117570. e-mail: psyrpe@nus.edu.sg*

Human beings are an extraordinarily altruistic species often willing to help strangers at a considerable cost (sometimes life itself) to themselves. But as Darwin noted ". . . he who was ready to sacrifice his life, as many a savage has been, rather than betray his comrades, would often leave no offspring to inherit his noble nature." Hence, this is the paradox of altruism. Twin studies have shown that altruism and other prosocial behavior show considerable heritability and more recently a number of candidate genes have been identified with this phenotype. Among these first provisional findings are genes encoding elements of dopaminergic transmission. In this article we will review the evidence for the involvement of one of these, the dopamine D4 receptor (*DRD4*) gene, in shaping human prosocial behavior and consider the methodologies employed in measuring this trait, specific molecular genetic findings and finally, evidence from several Gene × Environment (G × E) studies that imply differential susceptibility of this gene to environmental influences.

**Keywords:** *DRD4***, polymorphism, prosociality, altruism, gene × environment interaction, G × E**

# **INTRODUCTION**

Human beings engage in prosocial behavior, sometimes at a considerable personal cost. Charitable giving, volunteer work and even risking life and limb to save others are not uncommon. Such prosocial behavior cannot be easily explained by natural selection viz., the "selfish gene." Not surprisingly then, the paradox of prosociality and altruism have been the subject of speculation, inquiry and even wonder from Adam Smith and Charles Darwin to the present day. Not only are the origins, motivations and mechanisms of such behavior intriguing, but also the causes underlying the remarkable individual differences in prosociality/altruism are the focus of an increasing number of studies.

Evolutionary theories have suggested various mechanisms toward understanding the origins of prosocial behavior and altruism. The *Kin selection* theory (Haldane, 1932; Hamilton, 1964a,b; Smith, 1964), for example, proposes that altruism is maintained because it increases the odds of individual gene transmission to related generations. Although this theory might help to understand altruism toward kin, it does not explain the widely observed altruistic behavior that human beings exhibit toward perfect strangers. Other hypotheses that could account for such phenomena include reciprocity and reputation building (Fehr and Fischbacher, 2003), altruistic punishment (Fehr and Gachter, 2002), and group selection (Eldakar and Wilson, 2011), among others. While these studies attempt to uncover the origins of prosocial behavior, behavioral genetics provides insights on individual differences partially hard-wired by our genomes, while contingent on the varied environmental influences organisms encounter across the life span.

Twin studies demonstrate the considerable heritability of prosocial behavior. An early study by Matthews et al. (1981) estimated the heritability of "empathic responsiveness" from a sample of adult male twins and found an estimated twin correlation at 0.42–0.72. Rushton et al. (1986) showed that ∼50% of variance in altruism can be explained by genes. Although twin studies give us the sense of the genetic landscape of altruism, only molecular genetic approaches can inform regarding specific gene contributions to such behavior.

Dopamine (DA) related genes are plausible candidates for partially encoding prosociality/altruism given the functional involvement of DA transmission in approach behavior and reinforcement learning (Schultz, 2007). Among these genes, the dopamine D4 receptor (*DRD4*) gene has been examined in particular for its association with prosocial behavior, albeit with mixed results. For example, a significant association between *DRD4* and altruism has been found by Bachner-Melman et al. (2005) using the Selflessness questionnaire, and later replicated by Anacker et al. (2013) using the better known NEO-PI-R (altruism subfacet). However, other studies failed to observe a main effect of *DRD4* on prosociality whereas a Gene × Environment (G × E) interaction was demonstrated (Bakermans-Kranenburg and van Ijzendoorn, 2011; Knafo et al., 2011). This review aims to summarize the relationship between *DRD4* and prosocial behavior paying specific attention to differences in methodology and behavioral outcomes. In particular, we address the role of environments that modulate the action of *DRD4* in mediating prosocial and altruistic behavior, and discuss how these G × E interactions are crucial to understanding the behavioral impact of this gene. Importantly, we discuss various evolutionary

interpretations toward a deeper understanding how this gene came into play in human altruism.

# *DRD4* **exonIII VNTR**

The *DRD4* gene is characterized by an unusual 48-bp variable number tandem repeats (VNTR) polymorphism in the exon III coding region that codes for 16 amino acids (Lichter et al., 1993; Rondou et al., 2010). Two to eleven repeats (R) of the VNTR are observed in humans with the 4-repeat (4R) allele being the most common polymorphism (**Figure 1**), followed by the 7R in Caucasian populations (Van Tol et al., 1992) and 2R in East Asians (Chang et al., 1996). The 2R has been speculated as a "displacement" for the 7R in Asian populations and in this group it appears to function as the "risk" allele (Leung et al., 2005; Reist et al., 2007). Intriguingly, whereas the origins of 2R–6R alleles can be explained by simple one-step recombination/mutation events, the origin of 7R is less straightforward. Evidence suggests that this allele originated as a rare mutational event that nevertheless increased to high frequency in human populations by positive selection (Wang et al., 2004). However, more recent analysis using the massive SNP database maintains that there is little evidence for positive selection at this locus (Hattori et al., 2009; Naka et al., 2011).

Functional significance of these repeats has been suggested in many studies (Van Tol et al., 1992; Asghari et al., 1994, 1995; Schoots and Van Tol, 2003; Van Craenenbroeck et al., 2005, 2011). For example, the 7R has been linked to suppressed *DRD4* expression *in vitro* (Schoots and Van Tol, 2003). Moreover, Van Craenenbroeck et al. (2005) showed that there was a difference in the capacity of the *DRD4.2*, *DRD4.4*, and *DRD4.7* variants to be up-regulated through the pharmacological chaperone effect. In a

later study, Van Craenenbroeck et al. (2011)further suggested that the polymorphic repeat variants have different relative affinities to form homo- and heterodimers. Finally, evidence also suggests that the *DRD4*.*7* allele is associated with higher reward-related ventral striatum reactivity (Forbes et al., 2007). These results imply that the repeat lengths of the *DRD4* exon III VNTR are functionally meaningful, albeit they may not be linearly related. Therefore, it is plausible that this polymorphism could reflect complex behavioral phenotypes.

Indeed, a number of studies have reported associations between the 7R (or aggregated long alleles) and increased risk for various disorders including ADHD (Faraone et al., 2001; Maher et al., 2002), Tourette syndrome (Grice et al., 1996), obsessive compulsive disorder (Camarena et al., 2007; Walitza et al., 2008), pathological gambling (Perez de Castro et al., 1997; Eisenegger et al., 2010), substance abuse (Mcgeary, 2009), bulimia nervosa (Kaplan et al., 2008), conduct disorders (Kirley et al., 2004), autism, and schizophrenia (Emanuele et al., 2010; Lung et al., 2011). Moreover, evidence also supports the associations between these *DRD4* risk alleles, especially the 7R, and certain personality traits, including increased novelty seeking (Ebstein et al., 1996), impulsivity (Eisenberg et al., 2007), as well as propensity toward financial risks (Dreber et al., 2009, 2011; Kuhnen and Chiao, 2009).

However, these associations are not always easily replicated, suggesting that the *DRD4* gene may be better conceptualized as a plasticity gene whose effect is contingent on particular environments (Bakermans-Kranenburg and van Ijzendoorn, 2006, 2007, 2011). In this view, the so-called risk alleles are not strictly linked to a definite direction of effects; rather, depend on specific environments these plasticity alleles may show either positive or negative effects. For example, individuals carrying such differential susceptibility alleles may be more prosocial when influenced by one environment, while less prosocial in another environment. In contrast, individuals without differential susceptibility alleles are altogether likely to be less sensitive to environmental influences (Sasaki et al., 2013). These ideas gain support from a recent meta-analysis by Bakermans-Kranenburg and van Ijzendoorn (2011). This study examined the cumulative evidence for association between *DRD4* exon III VNTR and rearing environments and developmental outcomes. The results demonstrated that the seemingly "vulnerable" individuals were actually more susceptible to environments, "for better *and* for worse." The differential susceptibility of the *DRD4* exon III VNTR has been studied for various outcomes including externalizing behavior (Bakermans-Kranenburg and van Ijzendoorn, 2006; Bakermans-Kranenburg et al., 2008), attachment disorganization (Gervai et al., 2007), ADHD (Martel et al., 2011), prosocial behavior (Bakermans-Kranenburg and van Ijzendoorn, 2011; Knafo et al., 2011); unsolved loss or trauma (Bakermans-Kranenburg et al., 2011), and most recently, delay discounting (Sweitzer et al., forthcoming).

# *DRD4* **exonIII VNTR AND PROSOCIAL BEHAVIOR**

To review existing literatures on the association between *DRD4* and altruism/prosocial behavior, we systematically searched the online database of PedMed, with key words *DRD*4+Prosoical behavior; *DRD*4+Prosociality; *DRD*4+Altruism in all fields. The search resulted in a list of seven studies, all conducted within the past decade. These studies are described in **Table 1**.

The first study was conducted by ourselves (Bachner-Melman et al., 2005), and we examined the *DRD4* exon III 4R and 7R alleles for association with altruism, as measured by the Selflessness Scale (Bachar et al., 2002) and TPQ Reward dimension (Cloninger, 1987). The Selflessness Scale "*measures the propensity to ignore ones own needs and serve the needs of others*," thus altruism (Bachner-Melman et al., 2005), whereas the Reward dimension of the TPQ taps into altruism through elements such as empathy. Significant associations have been found between the *DRD4* exon III (D4.4) and higher Selflessness scores, as well as between the 4/4 genotype and higher TPQ Reward scores. That study has recently been replicated by Anacker et al. (2013) among a European sample using the Altruism subscale of Revised NEO Personality Inventory (NEO-PI-R) (Strobel et al., 2011). Consistent with the Bachner-Melman et al. (2005) finding, their results suggested higher altruism scores in the absence of the *DRD4* 7R allele.

A robust alternative to self-report questionnaires is the experimental assessment of human altruism. For example, Bakermans-Kranenburg and van Ijzendoorn (2011) measured children's altruism using experimentally observed donating behavior. The authors hypothesized a G × E interaction between the gene and childhood attachment with parents. Indeed, the results supported the moderating role of *DRD4* exon III repeats in the association between attachment and donating behavior. Secure attachment was significantly related to more donations, but only among children with 7R allele. Interestingly, in the same year, a study by Knafo et al. (2011) used a similar paradigm to examine the interaction between *DRD4* and parenting on children's prosocial behavior. Very similar results to the Dutch study were


*Note:\* Measured in year (y) or month (m).*

*<sup>1</sup> Selfishness scale.*

*<sup>2</sup> TPQ-Reward: reward scale measured by TPQ.*

*<sup>3</sup> NEO-Altruism: altruism subscale measured by NEO-PI-R.*

obtained in a sample of Israeli children. Prosocial behavior in these children was examined using three measures: Compliant (in response to social requests), Self-initiated (enacted voluntarily), and Mother-rated. Parenting measures included maternal positivity, negativity, and unexplained punishment. Although no main effect of *DRD4* was observed, the G × E interaction term was significant. Positive parenting related meaningfully to mother-rated prosocial behavior, and unexplained punishment related positively to self-initiated prosocial behavior, but only among children carrying the 7R allele. To summarize, these two studies independently carried out in distinct ethnic groups strengthen the notion that *DRD4* is a plasticity gene which is sensitive to diverse parenting styles. Notably, the impact of the polymorphism on behavior is constrained by the environment.

The study by Dilalla et al. (2009) was designed to examine the combined effects of the *DRD4* gene, environmental influences due to parents and peers and their interaction. By classifying the children into *DRD4*-L (at least one allele ≥6R) and *DRD4*-S (both alleles *<*6R) groups, they found that *DRD4*-L children are less prosocial in sharing with each other. Moreover, their parents were less sensitive during parent-twin interaction. Additionally, there were significant G × E interactions between *DRD4* and peer behavior/parental sensitivity: children with the high-risk alleles (*DRD4*-L) are more aggressive than the low risk allele (*DRD4*- S) carriers, but only in the low-aggression environment (when peer's behaviors are not aggressive); they are also more likely to be reported as having more externalizing problems than the low risk peers, but only when they have insensitive parents.

An intriguing environmental influence of religious priming and *DRD4* genotype on prosocial behavior was recently reported (Sasaki et al., 2013). In a sample characterized by mixed ethnicity (Caucasian and East Asian), the authors grouped *DRD4*-2R and 7R alleles together as so-called risk alleles, and measured participants' willingness to volunteer (i.e., donating time) as proxy for prosociality. Again, no main effect of *DRD4* was observed, but the interaction between gene and religious priming was significant. Consistent with the concept of differential susceptibility genes, participants with "risk" alleles (2R/7R) were more prosocial than others when primed with religion, whereas they were less prosocial than people without risk alleles in the neutral priming setting.

Finally, the *DRD4* exon III VNTR has also been linked to another aspect of prosociality: the reciprocal fairness preference as measured by an incentivized economic paradigm, the Ultimatum Game (Zhong et al., 2010). In this game two players decide on how to divide an initial endowment, with the proposer states a proposal on how much to give to the responder, and the responder states a minimum acceptable amount. If the proposal is accepted (i.e., the proposer states a higher amount than the responder's minimum acceptable amount), the amount is divided accordingly; otherwise, both would receive nothing. With this Ultimatum Game, reciprocal fairness was inferred from the responders' minimum acceptable amount, with higher amount indicating more concern for fairness. Among a sample of Han Chinese subjects, due to extremely low frequency of 7R alleles, the authors following Kang et al. (2008) considered 2R as the risk allele and combined the 2/2 genotype with 2/4 genotype for comparison with the 4/4 group. A significant main effect of *DRD4* exon III VNTR on responders' behavior was observed; subjects with the 2/2 or 2/4 genotype stated lower minimum acceptable amounts than the 4/4 genotype carriers. Moreover, a three-way interaction effect was observed between gene, gender, and season of birth (SoB): non-winter born male and winter-born female subjects with the 4/4 genotype tend to have a higher minimum acceptable amount than subjects with 2/2 and 2/4 genotype. Although SoB is less clearly interpreted than some other environmental factors such as parenting, these results nevertheless support the argument that the effect of *DRD4* is largely dependent on moderating environments.

In summary, there is modest evidence that the *DRD4* exon III VNTR 7R allele is associated with diminished altruism, especially when assessed with self-report questionnaires. However, the evidence for a role of *DRD4* in altruism is stronger when the genetic effects are examined together with environmental influences. The risk alleles including 2R and the long alleles (≥6R) are shown to be differential susceptibility alleles, which contribute differentially to observed prosocial behavior contingent on environmental characteristics.

# **DISCUSSION**

Based on recent evidences, our brief overview of the involvement of *DRD4* exon III VNTR in shaping human altruism/prosocial behavior underscores the notion of differential susceptibility for this polymorphism (Bakermans-Kranenburg et al., 2008; van IJzendoorn et al., 2008; Belsky et al., 2009; Bakermans-Kranenburg and van Ijzendoorn, 2011; Belsky and Beaver, 2011; Knafo et al., 2011). Whereas a main effect of the gene on prosocial behavior is not consistently observed, nevertheless when the environment is factored into the association a clearer picture appears to emerge. The risk alleles which are thought to be linked with lower prosociality can actually be more prosocial when the environment is supportive.

An evolutionary model for differential susceptibility has been suggested by Belsky (1997), in which he proposed that differential susceptibility is maintained for maximizing reproductive fitness of species in a continually changing and fundamentally uncertain environment. The variation in susceptibility to environmental influences ensures that the changes in environments would lead to diversified reactions among offspring, and thereby increase the probability of transmission of one's gene from generation to generation in an unpredictable world. We conjecture that the early migration out-of-Africa by our species unfolded as a series of unpredictable events, and this creates a favorable environment for selection of plasticity genes such as *DRD4*. Such an evolutionary argument brings us a deeper understanding of the association between *DRD4* and prosocial behavior. As hypothesized by Chen et al. (1999), and later supported by Matthews and Butler (2011), the 2R and 7R alleles of *DRD4* exon III VNTR are associated with population histories of migration. It appears that the serial migration that characterized the human out-of-Africa trek, *selects* for subjects carrying 2R and 7R alleles. Early human society in the Upper and Middle Paleolithic was characterized by small bands of hunter-gatherers, and prosocial behavior and cooperation among con-specifics would likely increase the overall fitness of such groups; this characteristic leads us to speculate that, under strict social norm/rules to promote egalitarian and prosociality within band, altruistic traits encoded in part by the *DRD4* 2R and 7R may have contributed to the remarkably successful out-of-Africa global trek beginning ∼50 k ago. Hence, we hypothesize that, along with risk taking behavior, altruistic traits that are associated with the 2R and 7R exon III repeats under supportive environment partially explains the selection for these two genetic variants in the serial migration out-of-Africa that led to *Homo sapiens'* successful population of the entire planet.

The evidence that *DRD4* polymorphisms differentially contribute to prosocial behavior, can also shed light on the biological roots of human morality. Researchers have long debated regarding the mechanisms and motives underlying prosociality/altruism. Some argue that people behave in a prosocial manner because of the so-called warm glow (Andreoni, 1990), i.e., people feel good by doing good. Others suggest that it is social pressure (Dellavigna et al., 2012) that drives people to engage in prosocial behavior, due to the cost borne by disregarding peer-established norms of behavior. As argued by Sasaki et al. (2013), these two seemingly disparate conjectures may be harmonized by the differential susceptibility hypothesis, based on the role of dopamine in reward-related process (Nemirovsky et al., 2009). Warm-glow individuals, characterized by the DRD4 4/4 repeats, are "born" prosocial irrespective of the environment due to the high dopaminergic tone driven by their genotype. In contrast, carriers of the 7R risk alleles have lower baseline

# **REFERENCES**


C., and Ebstein, R. P. (2005). Dopaminergic polymorphisms associated with self-report measures of human altruism: a fresh phenotype for the dopamine D4 receptor. *Mol. Psychiatry* 10, 333–335.


dopamine tone and hence are only prosocial in the presence of high environmental stimulation such as positive parenting (Wang et al., 2004). These conjectures have salient implications for parenting, moral education, policy-making and even jurisprudence. Individuals with the susceptibility alleles are theorized to be more responsive to moral education and policy interventions; to promote prosociality among this group, positive environments and rewards may be more effective than harsh environments and punishments. Conversely, for individuals without the susceptibility alleles, and thus less responsive to environmental changes, a more disciplined environment might be required to prevent deviations from societal norms of prosocial behavior.

Finally, caution needs to be exercised in interpreting existing G × E studies of *DRD4* and prosocial behaviors, since all studies to date are based on cross-sectional designs and lacking an important dynamic perspective. We do not know for example, how G × E interactions play out across the lifespan from early development to adulthood. As suggested by Bakermans-Kranenburg and van Ijzendoorn (2011), only a longitudinal design can trace the temporal interplay between the gene and the ever-changing environments that characterize our maturation and aging.

# **ACKNOWLEDGMENTS**

This study has been financial supports from Ministry of Education at Singapore (http://www*.*moe*.*gov*.*sg/; Tier 2: Biological Economics and Decision Making), AXA research foundation (http://www*.*axa-research*.*org/; Biology of Decision Making under Risk) and the Templeton foundation (http://www*.* templeton*.*org/; Genes, God and Generosity).

genotype moderates the impact of parental problems on unresolved loss or trauma. *Attach. Hum. Dev.* 13, 253–269.


(2007). Association study between the dopamine receptor D(4) gene and obsessive-compulsive disorder. *Eur. Neuropsychopharmacol.* 17, 406–409.


preschoolers' social behaviors. *Dev. Psychobiol*. 51, 451–464.


(2009). Reward-seeking and discrimination deficits displayed by hypodopaminergic mice are prevented in mice lacking dopamine D4 receptors. *Synapse* 63, 991–997.


D4 receptor biogenesis. *J. Biol. Chem.* 280, 19350–19357.


(2008). Transmission disequilibrium studies in early onset of obsessive-compulsive disorder for polymorphisms in genes of the dopaminergic system. *J. Neural Transm.* 115, 1071–1078.


H. (2010). Dopamine D4 receptor gene associated with fairness preference in ultimatum game. *PLoS ONE* 5:e13765. doi: 10.1371/ journal.pone.0013765

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 30 January 2013; accepted: 27 April 2013; published online: 14 May 2013.*

*Citation: Jiang Y, Chew SH and Ebstein RP (2013) The role of D4 receptor gene exon III polymorphisms in shaping human altruism and prosocial behavior. Front. Hum. Neurosci. 7:195. doi: 10.3389/fnhum.2013.00195*

*Copyright © 2013 Jiang, Chew and Ebstein. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

# The influence of dopaminergic gene variants on decision making in the ultimatum game

# *Martin Reuter 1,2\*, Andrea Felten1,2, Sabrina Penz 1, Anna Mainzer 1, Sebastian Markett 1,2 and Christian Montag1,2*

*<sup>1</sup> Department of Psychology, University of Bonn, Bonn, Germany*

*<sup>2</sup> Center for Economics and Neuroscience, University of Bonn, Bonn, Germany*

#### *Edited by:*

*Jan Wacker, Philipps-Universität Marburg, Germany*

*Reviewed by:*

*Koraly Perez-Edgar, The Pennsylvania State University, USA Songfa Zhong, National University of Singapore, Singapore*

#### *\*Correspondence:*

*Martin Reuter, Department of Psychology, University of Bonn, Kaiser-Karl-Ring 9, D-53111 Bonn, Germany e-mail: martin.reuter@ uni-bonn-diff.de*

One of the most prominent paradigms in neuroeconomics is the ultimatum game (UG) that provides a framework for the study of pro-social behavior in two players interacting anonymously with each other: Player 1 has to split an endowment with player 2. Player 2 can either accept or reject the offer from player 1. If player 2 accepts the offer then the money is split as proposed by player 1. In case of rejection both players get nothing. Until now only one twin study investigated the heritability of the behavior in the UG. Results indicated a strong heritability for the decision behavior of player 2 whereas no genetic influence on player 1 behavior could be detected. Further studies are mandatory to validate these heritability estimates. However, a first candidate polymorphism, the DRD4 exon III, constituting the biological basis of the heritability in the responder behavior has already been identified in a Chinese sample (Zhong et al., 2010). Until now genetic studies in Caucasians on the UG are lacking. The present study wants to fill this gap by investigating the UG in a healthy German sample. Moreover, we intend to find candidate genes that are associated with the first-mover-behavior. In a sample of *N* = 130 healthy participants an online version of the UG was conducted and polymorphisms of the dopamine D2 receptor gene (DRD2) and the DRD4 exon III VNTR were genotyped. We could confirm the DRD4 exon III effect on the responder behavior and the absence of an effect on the proposer behavior reported before. In line with Zhong et al. (2010) carriers of the 4/4 genotype showed a significant higher minimal acceptable offer (*p* = 0*.*023) than subjects with any other genotype. Furthermore, a DRD2-haplotype-block containing the single nucleotide polymorphisms rs1800497 and rs2283265 was significantly associated with the amount player1 offered (*p* = 0*.*005) but not with the decision of player 2. Results support the importance of the dopaminergic system for pro-social behavior.

**Keywords: decision making, ultimatum game, dopamine, DRD2, DRD4, gene, pro-social-behavior, neuroeconomics**

# **INTRODUCTION**

Every day we take numerous decisions that influence our current behavior and often even our future. Sometimes we are confronted to choose between two alternatives that come in the form of an ultimatum: Another person or party makes us an offer that we have either to accept in its present form or that we can reject. In any case we have to bear the consequences. Such a situation can be characterized as a "take it or leave it" situation. Behavioral economists have developed paradigms (so called games) that allow the investigation of human decision making under experimentally controlled conditions [for an overview see Camerer (2003)]. One of these paradigms, the ultimatum game (UG), exactly reflects the above mentioned "take it or leave it" situation. Out of a pool of participants two anonymous players interact in a dyadic situation. One of the players is randomly assigned the role of the first and the other the role of the second mover. Player 1 (also referred to as first mover or proposer) has to split an endowment (e.g., 10 C) between himself and player 2 (also referred to as second mover or responder). If player 2

accepts the offer, the pie is distributed according to player 1's suggestion. If player 2 rejects the offer, both players receive nothing (0 C). According to the assumptions of the economic *Game Theory* player 2 should accept all offers greater than 0 C (Camerer, 1997). However, empirical data from numerous studies contradict this prediction. About half of the offers are declined if they are lower than 30% of the pie (Roth, 1995), i.e., people prefer to dispense with something altogether than being satisfied with at least a small proportion of the pie. Rejecting an unfair offer at one's own cost in order to punish the proposer is not in line with economists' view on man as *homo economicus*. It is stated that the *homo economicus* makes decisions guided by selfinterest (maximization of personal benefit) and that his decisions are completely rational (Persky, 1995). Instead of being rational the responder's action is interpreted as a measure of fairness preference. In contrast, the proposer's offer is interpreted as a mixture between fairness preference (to be a social human being that is able to take the perspective of the responder) and strategic considerations (maximize the own profit while minimizing the risk of being punished for an unfair offer). Empirical data on the first-mover-behavior shows that the average offer ranges between 40 and 50% (Camerer and Thaler, 1995) indicating that people are mostly fair. The UG has been successfully applied in cross-cultural studies revealing variance in the behavioral data across countries and ethnicities (Henrich et al., 2001). Whether an act is judged as fair or not is doubtlessly influenced by environmental effects (e.g., upbringing, moral standards, culture) but also genetic factors are conceivable for the following reasons: (a) empirical data in the UG show variability indicating individual differences, (b) ethnical differences in behavior could be caused by differences in allele frequencies across ethnicities, (c) fairness is a facet of pro-social personality traits (e.g., cooperativeness) and traits are highly heritable (up to 50%, Bouchard et al., 1990). Indeed first evidence based on a Swedish twin study showed that more than 40% of the variation in subjects' rejection behavior in the UG is explained by additive genetic effects (Wallace et al., 2007). These data underline that the etiology of fairness preferences has a strong genetic basis. A first study is available now that has identified the DRD4 gene as one out of several potential gene loci that constitute the molecular basis of this heritability (Zhong et al., 2010). The DRD4 gene consists of 3400 base pairs (bp), is located at chromosome 11p15.5, and codes for the dopamine D4 receptor. In exon III of this gene a highly polymorphic variable number of tandem repeats (VNTR) polymorphism has been identified that is characterized by a repetitive sequence of 48 bp (between 2 and 11 repeats) (Van Tol et al., 1992). Three alleles are most common, the 2-repeat, the 4-repeat, and the 7-repeat, whereas the prevalence of the ancestral 4-repeat allele is highest across ethnicities. In Caucasians the 7-repeat is more frequent than the 2-repeat allele, however in Asians the 7-repeat allele is extremely rare and therefore in Eastern populations the 2-repeat allele is the second most common allele. Besides reported associations between the DRD4 exon III polymorphism and various phenotypes related to decision making behavior like impulsivity, novelty seeking, gambling behavior and attention-deficit hyperactivity disorder (ADHD) the functionality of this polymorphism has been demonstrated (Ebstein et al., 1996; Strobel et al., 1999; Eisenegger et al., 2010; Nikolaidis and Gray, 2010). The VNTR region of the DRD4 gene encodes a portion of the third intracellular loop region of the transcribed receptor protein that spans the nerve cell membrane and mediates interaction with second messenger proteins. The 2-repeat allele shows a 50% reduction in the production of cyclic adenosine monophosphate (cAMP) as compared with the 4-repeat and 7-repeat alleles (Asghari et al., 1995).

Although in the majority of these genetic association studies the 7-repeat allele caused the effects in Caucasian samples, it is the homozygous 4-repeat genotype that turned out to be related to economic decision making: Regarding the UG, Zhong et al. (2010) reported that carriers of the 4/4 genotype stated a 25% higher minimal acceptable offer in the role of the second mover as compared to carriers of the 2/4 and 2/2 genotypes. Notably, these results came from a Chinese sample where the 7-repeat allele is absolutely rare and was therefore not in the focus of our analyses. The authors did not find an association between the DRD4 exon III polymorphism and the UG proposer behavior. This is in line with the fact that there are no heritability estimates for the UG proposer behavior available in the literature until now. Although Zhong et al. reported a significant association between the DRD4 gene and fairness as measured by the UG, the proportion of explained variance is rather small. This is typical for quantitative traits and underlines the necessity to identify further genetic variants influencing the behavior in the UG. In this endeavor we have further focused on the dopaminergic system. Especially the DRD2 receptor gene has been related to various facets of pro-social behaviors like cooperation, attachment style, mentoring, paternal parenting, and positive emotionality to name but a few (Lucht et al., 2006; Reuter et al., 2006; Shanahan et al., 2007; Gillath et al., 2008; Walter et al., 2011). Two polymorphisms for which functionality has been proven are most investigated in genetic association studies the DRD2/ANKK1-Taq Ia (rs1800497) and the DRD2 C957T (rs6277) polymorphism. The DRD2/ANKK1-Taq Ia polymorphism is a restriction fragment polymorphism on chromosome 11 at q22-q23 (Pohjalainen et al., 1998; Reuter et al., 2006). Three genotypes of the dopamine DRD2/ANNK1-Taq Ia locus can be differentiated: The A1A1 genotype (also referred to as TT genotype), the A1A2 genotype (also referred to as TC genotype), and the A2A2 genotype (CC genotype). Due to the small prevalence of the A1A1 genotype (3% of healthy Caucasians), A1A1 and A1A2 subjects are commonly grouped as A1+ subjects, whereas A2A2 subjects are referred to as A1− subjects. The prevalence of at least one A1 allele (A1+ group) leads to up to 30% reduction in D2 receptor density (e.g., Pohjalainen et al., 1998). The direct impact of the DRD2/ANKK1-Taq Ia polymorphism on D2 receptor density has recently been questioned since this SNP is located *<*10 kb downstream of the DRD2 gene within a protein-coding region of the adjacent ANKK1 gene (Neville et al., 2004). Zhang et al. (2007) investigated 23 SNPs within the DRD2 gene and found a decreased expression of the short splice variant of the D2 receptor compared to the long splice variant caused by two intronic SNPs (rs2283265 and rs1076560). Interestingly, in the study by Zhang et al. (2007) the minor alleles of the two SNPs show strong linkage disequilibrium with the A1 allele of the DRD2/ANKK1- Taq Ia polymorphism (*D*- = 0*.*855). These data indicate that due to linkage the DRD2/ANKK1-Taq Ia polymorphism is indeed a marker for dopamine receptor density. The DRD/ANKK1-Taq Ia is the most prominent polymorphism with respect to the DRD2 gene. Mostly the minor A1 allele has been related to problematic or non-normative behaviour (e.g., Shanahan et al., 2007; Gillath et al., 2008).

In sum, the present study wants to (a) replicate the reported association between the DRD4 exon III polymorphism and the responder behavior in the UG reported by Zhong et al. (2010). However, this is more than a replication study since in contrast to Zhong et al. we try to test this association in a Caucasian population where the 7-repeat allele is a common allele in comparison to Asian samples; (b) test other dopaminergic gene variants namely polymorphisms on the DRD2/ANKK1 gene that have been related to decision making or pro-social behaviors. It is expected that these dopaminergic polymorphisms have also an influence on the first-mover-behavior in the UG.

# **METHODS**

# **SAMPLE**

*N* = 130 healthy subjects who are members of the *Bonn Gene Brain Behavior Project* (BGBBP; a gene data bank established with the aim to investigate the genetic underpinnings of human behavior) participated in the present study. The gender distribution was rather skewed [*n* = 105 females (age: *M* = 23.71, *SD* = 6*.*78) and *n* = 25 males (age: *M* = 25*.*32, *SD* = 6*.*63)] which is not surprising because most participants were psychology students at the University of Bonn and most of the psychology students in Germany are female (about 90%). The participants were not familiar with the UG (mainly 1st or 2nd year students participated). Gender groups did not differ with respect to age [*F(*1*,* <sup>129</sup>*)* = 1*.*142, *p* = 0*.*287]. The study was approved by the local ethics committee of the University of Bonn. All participants were completely debriefed on the aim of the study and the rules of the UG in advance of participation.

#### **THE ULTIMATUM GAME (UG)**

The UG was conducted as an online experiment designed in a way that each participant played the game twice, first in the role of the first mover (splitting an amount of 10 C anonymously between himself and another player) and afterwards in the role of player 2 [declaring which minimum amount of money received from player 1 would be accepted by himself (minimal acceptable offer)]. The proposal in the role of the first mover and the minimal acceptable offer in the role of the second mover could be chosen in steps of 0.50 C ranging from 0 to 10 C. Each participant was informed about the consequences of each possible choice in either role: In the role of the first mover, he was instructed that if he for example chooses to send 4 C to the second mover the payoff will be 6 C for himself and 4 C for the interaction partner. Participants were informed that after the end of the study a lottery takes place that randomly builds couples of two players out of the total sample and assigns each participant his actual role in the game (first or second mover). The payoffs are than calculated based on the players' role (first or second mover) and the decisions they had taken before. The payoffs are actually given to the participants after the whole study was completed. There was no additional payment for participation. We contacted about 300 of the BGBBP of whom 130 provided data sets. The duration of the experiment was about 10 min.

### **EXTRACTION OF DNA AND GENOTYPING**

DNA was extracted from buccal cells. Automated purification of genomic DNA was conducted by means of the MagNA Pure® LC system using a commercial extraction kit (MagNA Pure LC DNA isolation kit; Roche Diagnostics, Mannheim, Germany). Genotyping of the three SNPs (rs1800497, rs6277, rs2283265) was performed by real time PCR using fluorescence melting curve detection analysis by means of the Light Cycler System 1.5 (Roche Diagnostics, Mannheim, Germany). The primers and hybridization probes (TIB MOLBIOL, Berlin, Germany) were as follows:

DRD2/ANKK1 Taq Ia (rs1800497):

• Forward primer: 5- -CGGCTGGCCAAGTTGTCTAA-3-


### DRD2 c957t (rs6277):


### DRD2 rs2283265:


# DRD4 exon III:

The DRD4 exon III VNTR polymorphism was amplified from genomic DNA using polymerase chain reaction (PCR) and the primers 5- -TCCTCCGCTTTGGCGCCTCTTCC- (forward) and 5- -TGGGGGTTGCAGGGGAGATCCTG-3- (reverse). In brief, after an initial denaturation for 5 min at 94◦C, 38 cycles of denaturing at 94◦C for 30 s, annealing at 56◦C for 30 s, and extension at 72◦C for 1 min were followed by a final extension at 72◦C for 4 min. PCR amplification was carried out in a final volume of 20 μl consisting of 50 ng genomic DNA, 0.25 mM of each desoxyribonucleotide, 0.5μM of sense and antisense primers, 2.5 mM MgCl2, 10% DMSO, 2 U of Diamond *Taq* polymerase (Eurogentec) and the enzyme supplier's buffer. Amplification products were analyzed by 1.6% agarose gel electrophoresis. The sizes of the common 2-, 4-, and 7-repeats were 379, 475, and 619 bp, respectively. In *n* = 18 subjects genotyping of DRD4 exon III was not possible due to poor DNA quality. The RT-PCR method used for genotyping of SNPs is more sensitive than conventional PCR used for the VNTR. Therefore, valid data for the DRD4 exon III was only available in *n* = 112 subjects. In line with the study by Zhong et al. (2010) subjects with the 4/4 (*n* = 59) genotype were contrasted with the rest of the sample (*n* = 53). Therefore, the DRD4 genotype factor was entered into an ANOVA model with DRD4 as independent factor comprising two levels (4/4 vs. rest).

#### **HAPLOTYPE ANALYSES**

Linkage analyses between SNPs and construction of haplotype blocks were conducted by means of Haploview 3.32 (http:// www.broad.mit.edu/mpg/haploview/index.php). Individual haplotypes were calculated with PHASE, version 2.1. PHASE implements a Bayesian statistical method for reconstructing haplotypes from population genotype data. In simulation experiments it turned out that the mean error rate using PHASE was about half that obtained by the EM (expectation–maximization) algorithm (Stephens et al., 2001).

# **RESULTS**

Descriptive analyses of the UG data showed that the average first-mover-proposals (*M* = 4.23, *SD* = 1*.*53) and the minimal acceptable offers (second mover) (*M* = 3*.*95, *SD* = 1*.*69) were comparable to those reported in numerous other studies (Henrich et al., 2001). There were no gender differences, neither for the first mover [*F(*1*,* <sup>129</sup>*)* = 0*.*494, *p* = 0*.*483] nor for the second-mover-behavior [*F(*1*,* <sup>129</sup>*)* = 1*.*632, *p* = 0*.*204] and therefore gender was not included in the ensuing ANOVA models. It has to be pointed out that the absence of a gender effect may be caused by the small proportion of male subjects in our sample. Due to the homogenous student sample age was also not significantly correlated with the dependent variables. First and second mover behavior was significantly correlated (*r* = 0*.*349, *p <* 0*.*0001) as it is the case in all UG studies. This means subjects who make fair offers in the role of the first mover have also higher minimal acceptance thresholds in the role of the second mover. The size of this correlation is invariant across genotype groups.

#### **GENETIC ANALYSIS**

The observed genotype frequencies for the three SNPs under investigation are all in Hardy-Weinberg-Equilibrium (HWE) and are as follows: DRD2 ANKK1/Taq Ia (rs1800497): A1/A1: *n* = 6, A1/A2: *<sup>n</sup>* <sup>=</sup> 37, A2/A2: *<sup>n</sup>* <sup>=</sup> 87 (HWE: <sup>χ</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>*.*629, *df* <sup>=</sup> 1, *<sup>p</sup>* <sup>=</sup> 0*.*428); DRD2 C957T (rs6277): T/T: *n* = 37, C/T: 61, C/C: *n* = 32 (HWE: <sup>χ</sup><sup>2</sup> = 0.470, *df* <sup>=</sup> 1, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*493); DRD2 rs2283265: G/G: *<sup>n</sup>* <sup>=</sup> 99, G/T: *<sup>n</sup>* <sup>=</sup> 27, T/T: *<sup>n</sup>* <sup>=</sup> 4 (HWE: <sup>χ</sup><sup>2</sup> <sup>=</sup> <sup>1</sup>*.*532, *df* <sup>=</sup> 1, *p* = 0*.*216). The following genotype frequencies—that are also in HWE (χ<sup>2</sup> <sup>=</sup> <sup>9</sup>*.*111, *df* <sup>=</sup> 10, *<sup>p</sup> <sup>&</sup>gt;* <sup>0</sup>*.*05)—were observed for the DRD4 exon III 48bp VNTR: 2/2: *n* = 2, 2/4: *n* = 13, 2/7: *n* = 3, 4/4: *n* = 59, 4/7: *n* = 25, 3/4: *n* = 6, 5/7: *n* = 1, 7/7: *n* = 3.

#### **DRD4 exon III**

We could confirm the DRD4 exon III effect on the responder behavior as reported by Zhong et al. (2010). Carriers of the 4/4 genotype (*M* = 4.305, *SD* = 1*.*831) stated a significant higher minimal acceptable offer [*F(*1*,* <sup>111</sup>*)* = 5*.*329, *p* = 0*.*023; <sup>η</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>*.*046] than subjects with any other genotype [*<sup>M</sup>* <sup>=</sup> <sup>3</sup>*.*557, *SD* = 1*.*571; see **Table 1** and **Figure 1**]. With respect to the proposer behavior the 4/4 genotype group (*M* = 4*.*297, *SD* = 1*.*529) did not differ significantly from the rest of the sample [*M* = 4*.*208, *SD* = 1*.*570; *F(*1*,* <sup>111</sup>*)* = 0*.*092, *P* = 0*.*762; see **Table 2**] a result that is also in line with the Zhong et al. (2010) study. Due to the fact that most Caucasian association studies on DRD4 exon III concentrated on the 7-repeat allele we in addition compared carriers with at least one 7-repeat allele with participants with no 7-repeat allele. There was neither an effect of the 7-repeat allele on the responder [*F(*1*,* <sup>111</sup>*)* = 2*.*595, *p* = 0*.*110] nor on the proposer behavior [*F( )* = 0*.*018, *p* = 0*.*892]. 1, 111

#### **HAPLOTYPE ANALYSIS OF THE DRD2/ANKK1 GENE**

Construction of haplotypes revealed a haplotype block encompassing all three DRD2/ ANKK1 SNPs when using the rather **Table 1 | Descriptive statistics (means and standard deviations) for second-mover-decisions in the UG (minimal acceptable offers) dependent on the DRD4 exon III VNTR polymorphism.**


**Table 2 | Descriptive statistics (means and standard deviations) for UG offers (first-mover-proposals) dependent on the DRD4 exon III VNTR polymorphism.**

**offers in €, mean and SEM.**


liberal *solid spine of LD* method. However, the linkage between DRD2 ANKK1/Taq Ia (rs1800497) and DRD2 C957T was not satisfactory (*D*- = 0*.*52). The more conservative *four gamete rule* resulted in a two SNP haplotype block with the genetic markers DRD2 ANKK1/Taq Ia and rs2283265 spanning a distance of 15 kb (see **Figure 2**). Therefore, individual haplotypes were calculated on the basis of this two SNP haplotype block. The empirical haplotype frequencies are presented in **Table 3**.

An overall ANOVA model with the DRD2/ANKK1 haplotype as the independent variable and the first-mover-proposal as the dependent variable yielded a trend for a significant effect [*F(*4*,* <sup>125</sup>*)* <sup>=</sup> <sup>2</sup>*.*057, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*090; <sup>η</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>*.*062]. An explorative descriptive analysis comparing the mean UG offers dependent on the haplotype genotypes revealed that all participants carrying at least one TT haplotype showed on average lower offers than carriers lacking the TT haplotype completely (see **Table 4**). Therefore, participants were grouped according to the presence or absence of the TT haplotype (testing those with at least on TT haplotype vs. the rest) in the ensuing analyses. An analysis of variance indicated that the TT group offered significantly less money in the UG (first-mover-proposals) than the no TT group [*F(*1*,* <sup>128</sup>*)* = 8*.*102, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*005; <sup>η</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>*.*060; see **Figure 3**].

An overall ANOVA model with haplotype as the independent variable and the second-mover-proposal as the dependent variable yielded no significant effect [*F(*4*,* <sup>125</sup>*)* = 0*.*510, *p* = 0*.*729, see **Table 5**]. Grouping the haplotype groups in the same way as in the analyses of the first-mover-behavior, i.e., comparing the TT

**Table 3 | Empirical haplotype frequencies.**


haplotype carriers with the other haplotypes, did not result in a significant effect [*F(*1*,* <sup>128</sup>*)* = 0*.*119, *p* = 0*.*731].

# **DISCUSSION**

Recent twin studies have demonstrated that human decision making in economic settings has a strong genetic basis (e.g., Wallace et al., 2007; Cesarini et al., 2008). Studies from molecular genetics trying to identify those gene loci that make up this heritability are rather scarce. With respect to the UG, one of the most prominent games in behavioral economics, Zhong et al. (2010) have reported an association between the 4/4 genotype of the DRD4 exon III polymorphism and the second-mover-behavior. Although this study was conducted in an Asian sample and did not consider the 7-repeat allele that is absolutely rare in the Asian population we were able to replicate this finding in a Caucasian sample where the 7-repeat allele is quite common and therefore included in the analyses. In our sample the carriers of the 4/4 genotype stated a 20% higher minimal acceptable offer than carriers without the 4/4 genotype; in the Asian sample the minimal acceptable offer was 25% higher in the 4/4 genotype group. The responder's action is unequivocally interpreted as a measure of fairness preference that is incompatible with the view on man as *homo economicus*.

**Table 4 | Descriptive statistics (means and standard deviations) for UG offers (first-mover-proposals) dependent on haplotypes constituted by rs1800497 and rs2283265.**


**FIGURE 3 | First-mover-proposal (means and SEMs) dependent on the DRD2 ANKK1 haplotype consisting of the two SNPs rs1800497 and rs 2283265.**

**Table 5 | Descriptive statistics (means and standard deviations) for second mover decisions in the UG (minimal acceptable offers) dependent on haplotypes constituted by rs1800497 and rs2283265.**


Interestingly, and complementing our finding Bachner-Melman et al. (2005) found significantly higher self-report scores in altruism in carriers of the DRD4 4-allele and Anacker et al. (2013) in carriers of the 4/4 genotype. In line with the Asian study we could not find an effect of DRD4 exon III on the first-mover-behavior in the UG. Therefore, the present study constitutes an independent replication of the first molecular genetic study on UG behavior this time in a Caucasian sample. Therefore, the hypothesis could be put forward that the DRD4 effect seems to be invariant across ethnicities. In line with a meta-analysis by Camerer and Thaler (1995) the first mover offer in the present study was 42% on average, although our data were collected via the internet. This result indicates that UG data collected via the internet is comparable to data from experimental sessions in the laboratory.

Interestingly we could find an association between a haplotype block, spanning 15 kb of the DRD2/ANKK1 region consisting of the rs18000497 and rs2283265 SNPs, and the first mover offer in the UG. Carriers of at least one TT haplotype offered significantly less money in the UG (first-mover-proposals) than carriers without a TT haplotype. The proposer's offer is interpreted as a mixture between fairness preference (to be a social human being that is able to take the perspective of the responder) and strategic consideration (maximize the own profit while minimizing the risk of being punished for an unfair offer). The TT haplotype indicates that a subject has at least on one chromosome the minor alleles of both gene variants. Both minor alleles have been associated to lower receptor DRD2 density or decreased relative expression of DRD2s mRNA respectively (Pohjalainen et al., 1998; Zhang et al., 2007). On the other hand the second-mover-behavior was not related to genetic variations in the DRD2/ANKK1 region. Due to the restricted sample size we could not test for interaction effects of the DRD2 and DRD4 variants.

In sum, we find a genetic dissociation between DRD2 (first mover) and DRD4 (second mover) related behavior in the UG that needs further clarification. The neuroanatomical differences in receptor distribution qualify as a valuable starting point for this investigation.

D2 receptors are members of the dopamine receptor Gprotein-coupled receptor family that also includes D1, D3, D4, and D5. They are expressed primarily in sub-cortical regions like the nucleus accumbens and caudate putamen where they are involved in the modulation of locomotion, reward, reinforcement, learning, and memory (e.g., Wise, 2004; Klein et al., 2007; Jocham et al., 2009; Frank and Fossella, 2011). Although the DRD4 receptor is also expressed in sub-cortical regions like the amygdala and the midbrain it is also amply located in the frontal cortex (e.g., Oak et al., 2000). The interaction with DRD2 may modulate dopamine- and DA-agonist-induced downstream signaling, i.e., a top-down regulation of emotional processes by central nervous input modulated by DRD4 receptors. First imaging data are available scanning the second movers' brain activity while responding to fair and unfair offers (Sanfey et al., 2003). An increased BOLD response could be detected in response to unfair offers in emotion- (anterior insula) and cognition- (dorsolateral prefrontal cortex) related brain regions. Moreover, Gospic et al. (2011) could demonstrate that also sub-cortical regions, especially the amygdala, are related to the immediate rejection of unfair offers in the UG. These fMRI findings fit perfectly to the behavioral data and the DRD4 gene effects observed in the present study because DRD4 receptors are dominantly expressed in the brain regions triggering the imaging effects. A first fMRI study investigated the brain activity of first movers in the UG (Weiland et al., 2012) and found that fair offers were related to enhanced activity in prefrontal areas, particularly in the subdivisions involved in reward processing and theory of mind. The authors interpreted these findings with the hypothesis that egoistic motives are primarily responsible for fair offers in UG and label this phenomenon as strategic fairness. At first glance, the pronounced role of cognitive aspects in first movers' decision making contradict the DRD2 gene effect reported in the present study because it is assumed to be primarily of sub-cortical nature. However, although strategic, the first-mover decision is not free from affective components, e.g., pity or benevolence for the second mover. Therefore, also sub-cortical effects triggered by sub-cortical DRD2 receptors are likely to influence the proposals in the UG. Nevertheless, it has to be pointed out that the observed gene effects do not allow to directly infer to brain structures related to the UG behavior unless genetic imaging studies have proven such associations.

The strategy to investigate several SNPs on the ANKK1/DRD2 gene simultaneously by means of a haplotype analysis is an elegant method to increase the amount of explained phenotypic variance. Ensuing univariate analyses help to identify the gene variant that drives the genetic effect. In the present study the effect of rs2283265 [*F(*1*,* <sup>128</sup>*)* = 8*.*10, *p* = 0*.*002] on the firstmover-behavior was stronger than that of rs1800497 (DRD2 Taq Ia) [*F(*1*,* <sup>128</sup>*)* = 5*.*44, *p* = 0*.*021] indicating that the association between rs1800497 is probably attributable to a strong linkage with the putative causal effect of rs2283265.

Whereas the second-mover-behavior in the UG is unequivocally interpreted as a measure of fairness preference, the firstmover-proposal is a heterogeneous mixture between strategic considerations and pro-social perspective taking. Future experimental designs investigating the UG and the related dictator game in a within-subject design could disentangle these two components. In contrast to the payoff in the UG that is dependent on the acceptance/rejection of the first mover's proposal by the second mover, the first mover in the dictator game makes a proposal that is implemented independently of the second mover. The identification of distinct gene loci related to the proposals in the UG and the dictator game would contribute to clarify this issue. A shortcoming of the present study is the skewed gender distribution. Although we did not find gender effects there is work pointing to the relevance of gender differences and of sex hormone genes for decision making in the UG (Chew et al., 2013).

In sum, the present study corroborates previous findings demonstrating an influence of the DRD4 exon III polymorphism on second-mover-behavior in the UG and identifies a DRD2/ANKK1 haplotype associated with strategic fairness

# **REFERENCES**


china and israel. *J. Econ. Behav. Organ.* 90, 28–42.


of the first-mover-decision. Results underline the importance of cortical and sub-cortical dopaminergic activity on social decision making. Although the genetic effects explain at the maximum 6% of the variance, such an effect size is rather large for genetic association studies. Nevertheless, it is necessary to search for additional gene variants that are also related to the decision behavior in the UG and human social behavior in general [for a comprehensive review see Ebstein et al. (2010)].


(2012). Neural correlates of fair behavior in interpersonal bargaining. *Soc. Neurosci.* 7, 537–551. doi: 10.1080/17470919.2012.674056

Wise, R. A. (2004). Dopamine, learning and motivation. *Nat. Rev. Neurosci*. 5, 483–494. doi: 10.1038/nrn1406

Zhang, Y., Bertolino, A., Fazio, L., Blasi, G., Rampino, A., Romano, R., et al. (2007). Polymorphisms in human dopamine D2 receptor gene affect gene expression, splicing, and neuronal acitivity during working memory. *Proc. Natl. Acad. Sci. U.S.A.* 104, 20552–20557. doi: 10.1073/pnas.0707106104

Zhong, S., Israel, S., Shalev, I., Xue, H., Ebstein, R. P., and Chew, S. H. (2010). Dopamine d4 receptor gene associated with fairness preference in ultimatum game. *PLoS ONE* 5:e13765. doi: 10.1371/journal.pone.0013765

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 14 March 2013; paper pending published: 12 April 2013; accepted: 17 May 2013; published online: 04 June 2013.*

*Citation: Reuter M, Felten A, Penz S, Mainzer A, Markett S and Montag C (2013) The influence of dopaminergic gene variants on decision making in the* *ultimatum game. Front. Hum. Neurosci. 7:242. doi: 10.3389/fnhum.2013.00242 Copyright © 2013 Reuter, Felten, Penz, Mainzer, Markett and Montag. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

# Dopaminergic foundations of schizotypy as measured by the German version of the Oxford-Liverpool Inventory of Feelings and Experiences (O-LIFE)—a suitable endophenotype of schizophrenia

*Phillip Grant <sup>1</sup> \*, Yvonne Kuepper 1, Eva A. Mueller 1, Catrin Wielpuetz 1, Oliver Mason2 and Juergen Hennig1*

*<sup>1</sup> Personality Psychology and Individual Differences, Department of Psychology, Justus-Liebig-University Giessen, Giessen, Germany*

*<sup>2</sup> Research Department of Clinical, Educational and Health Psychology, University College London, London, UK*

#### *Edited by:*

*Jan Wacker, Philipps-Universität Marburg, Germany*

#### *Reviewed by:*

*Dimitrios Avramopoulos, Johns Hopkins University, USA Ulrich Ettinger, University of Bonn, Germany*

#### *\*Correspondence:*

*Phillip Grant, Personality Psychology and Individual Differences, Department of Psychology, Justus-Liebig-University Giessen, Otto-Behaghel-Str. 10F, D-35394 Giessen, Germany. e-mail: phillip.grant@ psychol.uni-giessen.de*

The concept of schizotypy or "psychosis proneness" captures individual differences in perceptual, cognitive, and affective experiences that may relate to a range of psychotic disorders. The concept is an important way to assess the contribution of pre-existing psychological and genetically based biological features to the development of illnesses such as schizophrenia (so called endophenotypes). The Oxford-Liverpool Inventory of Feelings and Experiences (O-LIFE) is a widely used multi-dimensional measure of the construct and consists of four scales which mirror several groups of psychotic symptoms: Unusual Experiences (UnEx; positive symptoms), Cognitive Disorganization (CogDis; cognitive symptoms), Introvertive Anhedonia (IntAn; negative symptoms), and Impulsive Nonconformity (ImpNon; impulsive and antisocial symptoms). For the purpose of evaluating the suitability of schizotypy as an endophenotype of schizophrenia the current version of the O-LIFE was translated into German: its psychometric properties (including re-test reliability and construct validity) were examined in a large sample (*n >* 1200) and compared to those of the English original. The German version was both highly reliable and consistent with the original. The study aimed to show that schizotypy as measured by the O-LIFE can indeed be regarded as an endophenotype of schizophrenia in terms of genetic associations regarding relevant dopamine-related candidate polymorphisms of schizotypy [i.e., Val158Met-polymorphism of the *COMT* gene, uVNTR of the *MAOA* gene, Taq1A-polymorphism of the *DRD2* gene, VNTR of the *SLC6A3* (*DAT* ) gene]. We also wanted to compare the genetic associations of the O-LIFE to those published using other operationalizations of schizotypy. Our results show a large number of significant associations and borderline-significant trends between the O-LIFE sub-scales and a range of genes, thereby supporting using the O-LIFE in the search for endophenotypic markers.

**Keywords: schizotypy, schizophrenia, endophenotype, O-LIFE, dopamine, genetic associations, psychosis**

# **INTRODUCTION**

According to the World Health Organization, clinical schizophrenia belongs to the most severe disability class (VII), alongside severe depression, severe migraine, quadriplegia, and terminal cancer (WHO, 2008). Ustun (1999) place schizophrenia third in their rank of disabling effects of health conditions by severity behind quadriplegia and dementia. The disorder causes 8.3/8.0 million years lost to disease (YLD), making up for 2.8/2.6% of total YLD with values for males and females respectively (WHO, 2008). First-episode patients are shown to already present with signs of neurodegeneration and loss of brain connectivity which both correlate with the severity of positive symptoms (Suzuki et al., 2005; Lui et al., 2009). Our own research into the mechanisms of neurodegeneration in schizophrenia suggests that the primary pathogenic process underlying the disorder may well be in effect long before the actual onset of psychotic symptoms (Grant, 2011). This is in line both with the current dopamine hypothesis of schizophrenia (Howes and Kapur, 2009) as well as with findings of significant cognitive deficits which predate the first florid psychotic episode (Addington et al., 2003; Van Os and Kapur, 2009). The premorbid IQ of schizophrenic patients, even years before the onset of psychosis, is estimated on average at one-half of a standard deviation below that of healthy controls (Woodberry et al., 2008). Since cognitive deficits are widely accepted to be more enduring than psychotic symptoms (Vinogradov, 2003; Mueser and McGurk, 2004; Van Os and Kapur, 2009) and are considered a better predictor for clinical outcome than response to treatment, it would seem that a possibility for detection of schizophrenic patients before the onset of actual psychosis could become possible in the near future. On such approach could be the investigation of so-called risk alleles of genes thought to be involved in the gene X environment-interaction of the ethiopathogenesis of schizophrenia (Allen et al., 2008; Van Os and Kapur, 2009; Van Os et al., 2010). Both cognitive deficits as well as the presence of such risk alleles are, however, not specific to schizophrenia and are therefore not solely suitable as predictive criteria. It is therefore necessary to search for valid endophenotypes of schizophrenia, which may predict the individual risk of a person to develop psychotic symptoms over the course of time. Endophenotypes are defined as subclinical state-independent characteristics that have a genetic basis and are in concurrence with the biological basis of the actual disorder (Gottesman and Gould, 2003). Such endophenotypes of schizophrenia exist, for example, in deficits in latent inhibition or prepulse inhibition, but the assessment of these as well as genetic testing as a means of screening for endophenotypes are costly and require laboratory work or specialized experimental apparatuses, wherefore they cannot be considered as efficient means within the health care system. The concept of schizotypy according to the original definitions by Rado (1953) or Meehl (1962) meets the criteria for an endophenotype as laid down by Gottesman and Gould (2003).

We therefore examined whether schizotypy/psychosis proneness, as measured by self-report using the "Oxford-Liverpool Inventory of Feelings and Experiences" (O-LIFE) (Mason et al., 1995; Mason and Claridge, 2006), can also be viewed upon as an endophenotype of schizophrenia. Claridge's concept of schizotypy differs from other conceptualizations in that its items are mainly personality-based and not specifically developed on the background of symptoms of clinical schizotypal (personality) disorder or schizophrenia, as are the schizotypyinventories of Rust [1988; Rust Inventory of Schizyotypal Cognitions (RISC), limited to "positive schizotypy"] and Raine [1991; Schizotypal Personality Questionnaire (SPQ)]. Relatedly, the O-LIFE is based on a fully-dimensional approach to schizotypy, which contrasts with taxonic/quasi-dimensional models as proposed by Rado (1953) or Meehl (1962). The fully dimensional approach by Claridge suggests an intra-individually stable array of traits, whereby high schizotypy-values increase the risk of developing psychosis and persons with schizophreniaspectrum disorders show particularly high scores in schizotypy with no clear cut-off or distribution break indicating membership of a risk group. The four subscales of the O-LIFE (Mason and Claridge, 2006) also mirror the groups of symptoms found in psychosis: Unusual Experiences (UnEx; positive symptoms), Cognitive Disorganization (CogDis; cognitive symptoms), Introvertive Anhedonia (IntAn; negative symptoms), and Impulsive Nonconformity (ImpNon; impulsive and antisocial symptoms). The inclusion of the last scale is often questioned, but relates on the one hand to the possible concept of "Einheitspsychose" (Claridge, 1997) and on the other to the often reported increased number of violent offences found in psychotic patients (for a review and meta-analysis, see Large and Nielssen, 2011).

The genetic associations of schizotypy have not been extensively studied and published papers often find negative or ambiguous results. Candidate genes of relevance to dopaminergic neurotransmission with risk-alleles that are also considered in the ethiopathogenesis of schizophrenia that have also been associated with schizotypy are primarily the Val158Metpolymorphism (rs4680; Lachman et al., 1996) of the *COMT*-gene (encoding for the catecholamine-degrading enzyme catechol-*O*methyltransferase, C*O*MT) and the variable number of tandem repeats (VNTR)-polymorphism (Vandenbergh et al., 1992) of the *SLC6A3*-gene (encodes for the dopamine active transporter, DAT). Studies operationalizing schizotypy using the SPQ find higher scores for val/val-homozygotes (rs4680) using healthy male participants (Avramopoulos et al., 2002; Smyrnis et al., 2007) or for samples consisting of, i.a., bipolar patients and firstdegree relatives of schizophrenic patients (Schurhoff et al., 2007), whereas others report highest scores in met/met-homozygotes (rs4680) (Sheldrick et al., 2008) and again others find only weak but non-significant effects of either the rs4680 or the rs6265 in healthy participants (Ma et al., 2007), whereby these results usually only refer to certain subscales of the SPQ. A study performed by Ettinger and co-workers (2006) on a small (*n* = 31) sample of Caucasian males using the RISC found non-significantly higher scores in met/met- (rs4680) and 10/10-homozygotes (*SLC6A3*- VNTR). No studies exist to date examining the genetic associations of these or other dopamine-related polymorphisms and the O-LIFE.

There are several interpretations of the ambiguity or lack of results concerning the genetic associations of schizotypy, especially regarding the *COMT* Val158Met-polymorphism (rs4680): The val-allele is reported to coincide with a higher activity of the expressed enzyme C*O*MT (Lachman et al., 1996), whereby val/val-homozygotes will have the highest and met/methomozygotes the lowest rate of catecholamine-degradation through C*O*MT. Since dopamine is, however, degraded in a two-step reaction through C*O*MT and an enzyme tandem consisting of monoamine oxidase (MAO; both isozymes) and aldehyde dehydrogenase (AD) (for a concise description of dopamine-synthesis and degradation see Grant, 2011), high levels of C*O*MT-activity will not lead to complete degradation of dopamine, but rather to the formation of the toxic metabolite 3-methoxytyramine (3-MT), and low levels of C*O*MTactivity will lead to a relative over-activity of MAOs/AD and thereby to the formation of the equally toxic metabolite 3,4-dihydroxyphenyl-acteic acid (DOPAC). It could therefore be argued that both findings of higher schizotypy in val/valas well as met/met-homozygotes could be valid on the background of dopamine-neurotoxicity in schizophrenia-spectrum ethiopathogenesis (Smythies, 1999, 1997; Grant, 2011). The rate of neurotoxicity would therefore be moderated through the relative activity of MAOs. Since a polymorphism with functional consequences for the activity of MAO-A has been described (*MAOA*-uVNTR, Sabol et al., 1998; Deckert et al., 1999), the effects of the rs4680 may be masked, if the individual genotype of the *MAOA*-uVNTR is not taken into consideration. Also, it could be possible that there is actually a heterosis-effect regarding the rs4680, whereby both val/val- and met/met-homozygotes have a higher risk of psychosis proneness compared to val/metheterozygotes.

Finally, we assume that the fully dimensional model of schizotypy is inherently better suited for endophenotype-research compared to taxonic/quasi-dimensional models, as is leads to more variance in the population compared to those measures with a "quasi-clinical" background containing more items that may be endorsed by fewer individuals. Additionally, due to findings of high heritability of the O-LIFE-scales (Linney et al., 2003), we expect clearer associations to genetic variations with this inventory, especially with the short scales, that have been generated partly on the basis of item-heritability (Mason et al., 2005).

We therefore translated the O-LIFE into German and attempted to assess its suitability for genetic association studies by examining the effects of the aforementioned dopaminerelevant polymorphisms that have previously been related in literature to either the RISC or the SPQ. We additionally examined the association with the *MAOA*-uVNTR and the *DRD2* Taq1A-polymorphism (rs1800497, Pohjalainen et al., 1998), since these polymorphisms have also repeatedly been shown to have significant influences on dopaminergic neurotransmission.

# **MATERIALS AND METHODS SAMPLE**

The main sample for the test-theoretical analysis of the German version of the O-LIFE was acquired via an emailinvitation sent to all members (students, fellows, and administrative/technical employees) of Justus-Liebig-University (JLU), Giessen (Germany), through oral invitations during lectures by Phillip Grant at JLU and THM (Technische Hochschule Mittelhessen, University of Applied Sciences) as well as from a German grammar school (Erftgymnasium Bergheim, North Rhine-Westphalia) through personal contacts of Phillip Grant. The email/personal invitations contained a link to an onlineversion of the inventory programmed by the authors using the platform soscisurvey.de. This online-version consisted of the German O-LIFE and several screening questions regarding somatic and psychological health, drug use (with special regard to alcohol and nicotine) and medication status. The main sample consisted of 1228 participants (341 male, 887 female) with age ranging from 17 to 75 years (*M* = 27*.*1, *SD* = 9*.*47, *MD* = 24).

The sample for the re-test of the O-LIFE was acquired 3 months later in the same fashion as the main sample, whereby in this case all other questionnaires and items except the O-LIFE were omitted in order to reduce the time necessary for participants to answer the items and thereby increase compliance. The re-test sample contained 245 participants (45 male, 200 female) with an age range from 17 to 58 years (*M* = 25*.*83, *SD* = 8*.*6, *MD* = 23).

The sample for genetic associations was acquired through the Giessen Gene Brain Behaviour Project (GGBBP) of the Department of Personality Research and Individual Differences at JLU. The GGBBP contains ca. 1800 datasets of participants including various personality inventories and data on several polymorphisms, whereby for legal reasons only those participants were contacted who had signed a respective consent form within the last 5 years prior to the date of data-acquisition. Therefore, as well as due to a high rate of unreturned invitations to fill in the O-LIFE, only ca. 290 participants could be acquired from the GGBBP. This sub-sample consisted of 288 participants (91 male, 197 female) with an age range from 18 to 51 years (*M* = 22*.*9, *SD* = 4, *MD* = 22).

All genetic and molecular-biological research was approved by the local ethics committee of the psychological faculty at JLU.

## **GERMAN VERSION OF THE OXFORD-LIVERPOOL INVENTORY OF FEELINGS AND EXPERIENCES (O-LIFE)**

The inventory was translated into German by Phillip Grant, a bilingual native-speaker of German and English, and retranslated into English by the native-German co-authors. Most items were considered to be adequately translated and the remaining items were modified in order to meet optimal retranslation criteria.

The full version of the O-LIFE contains 104 items loading on four scales: UnEx, IntAn, CogDis, and ImpNon. For the properties of the original English version see Mason and Claridge (2006). The short scales (Mason et al., 2005) are drawn from the full inventory.

#### **GENOTYPING**

DNA was extracted from buccal epithelia using a standard commercial extraction kit (High Pure PCR Template Preparation Kit; Roche, Mannheim, Germany) in a MagNA Pure LC System (Roche, Mannheim, Germany) in line with participants' entry into the database of the GGBBP.

Genotyping was performed by means of polymerase chain reaction amplification according to standard protocols for the following polymorphisms: Val158Met-polymorphisms (rs4680) of the *COMT* gene (encoding for C*O*MT) (Reuter and Hennig, 2005) and *DRD2* Taq1A-polymorphism (rs1800497) of the *DRD2* gene (encodes for dopamine receptor D2) (Kirsch et al., 2006). For the *MAOA*-uVNTR-polymorphism genotyping was performed using a fluorescently labeled 5- -primer [adapted from Sabol et al. (1998)] and subsequent capillary-electrophoresis on an ABI 310 System (Applied Biosystems, Germany).

Since *MAOA* is an X-chromosomal gene it also has to be noted that men are generally hemizygous, since they only carry a single X-chromosome. In women, heterozygosity is also functionally difficult to interpret, since it cannot be ascertained which individual X-chromosome is inactivated to a Barr-body in each individual neuron. Heterozygous female participants were therefore also excluded from further functional analyses related to the *MAOA*-uVNTR. Due to the absence of methodological points of critique (personal communication from cell-culture expert Dr. Barbara Ahlemeyer, JLU) regarding the functionality-assessments of the *MAOA*-uVNTR-alleles in the study of Deckert et al. (1999), we chose to follow their functional classification of the 5-repeat allele as highly active regarding gene-expression.

In case of the *DAT* 3- UTR-VNTR-polymorphism, which usually consists of 9- or 10-repeat alleles, those participants with other numbers of repeats were omitted, since these alleles are extremely rare (Vandenbergh et al., 1992). Genotyping was performed using primers adapted from Vandenbergh: forward: 5- -TGTGGTGTAGGGAACGGCCTGAG-3and reverse 5- -CTTCCTGGAGGTCACGGCTCAAGG-3- (TIB MOLBIOL, Germany). The 5- -primer was fluorescently labeled, and amplification was followed by capillary-electrophoresis on an ABI 310 System (Applied Biosystems, Germany).

#### **RESULTS**

#### **PROPERTIES OF THE GERMAN OXFORD-LIVERPOOL INVENTORY OF FEELINGS AND EXPERIENCES (O-LIFE)**

The final sample for the German O-LIFE consisted of 1228 participants (341 male, 887 female). The relative imbalance between the sexes can be ascribed to the high prevalence of women in the student-body at JLU, especially within the psychological faculty. The mean age of the participants was 27.1 years with a standard deviation of 9.47, ranging from 17 to 75 years.

Of these 1228 persons, 245 followed the invitation to participate in a re-test 3 months after the initial invitation. These 245 (45 male, 200 female) had an average age of 25.83 years (*SD* = 8*.*6), ranging from 17 to 58 years.

The statistical values for the O-LIFE sub-scales are shown in **Table 1**, including the re-test-reliability coefficients. Although the authors of the O-LIFE suggest not to evaluate or overly interpret the total O-LIFE-score (Mason and Claridge, 2006), we calculated this score for reasons of evaluating re-test-reliability of the whole inventory. It should be noted that this total O-LIFE-score was, however, not used for any further analyses. Due to the fact that this publication is in the English language we chose not to show exemplary items of the German version. Interested parties are welcome to contact the corresponding author in this regard. For exemplary items of the O-LIFE in English see Mason and Claridge (2006) and Mason et al. (2005) for the short scales.

It can be seen that the coefficients of internal consistency (α) and of re-test-reliability are comparable to those of the original version with exception of the scale Impulsive Nonconformity (ImpNon). The mean scores and standard deviations of the scale sum-scores are, however, usually slightly higher than in the British sample.

In order to assess if higher scores, especially in the scale UnEx, resulted from disingenuous answering behavior of some of the participants, we compared the consistency indices of different datasets and subsamples. In all of these analyses α-values were similar and acceptable (data not shown here).

As O-LIFE values are reported to be sexually dimorphic, we performed *t*-tests by sex to see if similar differences could be measured in the German translation (**Table 2**).

Since values are different for the whole sample, it is not surprising that mean values for the male and female subsamples are also not identical to those estimated by Mason and Claridge (2006). It could, however, be shown that the direction of the sexual dimorphism is in line with the original (Mason et al., 1995).

Although the sub-scales of the O-LIFE represent clearly distinguishable facets of psychosis proneness or schizophreniaspectrum disorders, which do not necessarily need to manifest themselves to an equal degree in all individuals, the scales do, nonetheless, show significant inter-correlations in both the original and the German versions (**Table 3**).

We analyzed the effects of ageing on the O-LIFE scores to find weak but mostly significant (*p <* 0*.*05) negative correlations between all four scales and age, whereby only IntAn correlated positively (**Table 4**). In the female subsample the scales UnEx and IntAn did, however, not correlate significantly with age. The directions of these correlations mirror those found for the English O-LIFE (Mason and Claridge, 2006).

#### **GENOTYPE-DISTRIBUTIONS**

Two hundred and eighty-eight participants that had been genotyped for the GGBBP also followed the invitation to fill in the O-LIFE. Note that for the single polymorphisms the numbers of participants do not always add up to 288. This is mainly due to the fact that some participants had joined the GGBBP before this research and respective genotypings were

#### **Table 2 | Sex-differences in O-LIFE scale sum-scores.**


*Abbreviation: M, arithmetic mean; SD, standard deviation of M; df, degrees of freedom; p, conditional probability.*


*Abbreviation: i, number of variables; M, arithmetic mean; SD, standard deviation of M; Engl., respective values of the original O-LIFE; re-test interval was 3 months.*

conducted and DNA-samples were no longer available, even though the participants were still willing to fill in the German O-LIFE online. Also, due to the argumentation regarding the repeat polymorphisms of the *SLC6A3*- and *MAOA*-genes in the methods section, certain participants with extremely rare or functionally not clearly attributable genotypes were omitted from the analyses. Finally, one participant could not be genotyped regarding the *COMT* VAl158Met-polymorphism, probably due to an unexpected individual variation within the amplified fragment.

**Table 5** shows that all examined genotypes are in Hardy– Weinberg equilibrium (Court, 2005–2008).

#### **ANALYSES OF VARIANCE AND HOMOGENOUS SUBGROUPS**

Due to the rationale mentioned in the introduction we performed multivariate analyses of variance to examine genotype-associations regarding the O-LIFE scales without prior classification of expected "risk-alleles." In order to assess, if either the genetic principle of dominance or recessivity was relevant for a given polymorphism, we performed Bonferroni-corrected *post-hoc* tests to examine, if heterozygotes could be considered as (proximately) equal to either of the homozygous groups (data not shown here). These respective groups were then contrasted and compared to the remaining group in single ex-post-facto *t*-tests. This procedure was performed for the whole sample as well as individually for the two sexes. Only relevant data are shown here for reasons of conciseness. Group means and standard deviations can be viewed in **Table 6**.

#### **Table 3 | Inter-correlations of the O-LIFE scales.**


*\*\*p < 0.01.*

#### **Table 4 | Correlations between age and the O-LIFE scales.**


*\*p < 0.05; \*\*p < 0.01.*

#### **Table 5 | Genotype distributions and fit to the Hardy–Weinberg principle.**

# **GENETIC ASSOCIATIONS FOR THE** *COMT* **Val158Met-POLYMORPHISM**

The analyses of variance regarding the effects of the *COMT* Val158Met-polymorphism showed associations with both the full (*p* = 0*.*092) and the short scales (*p* = 0*.*031) for UnEx in the whole sample.

In ex-post-facto *t*-tests we found a recessive effect of the valallele in that the val/val-group showed higher values compared to carriers of the met-allele. This effect was significant for the whole sample (full scale: *T*<sup>278</sup> = 2*.*057; *p* = 0*.*041 and short scale: *T*<sup>278</sup> = 2*.*639; *p* = 0*.*009) and could also be seen in the male and female subgroups, whereby only the values for the short scale of UnEx reached borderline-significance (males: *T*<sup>83</sup> = 1*.*964; *p* = 0*.*053 and females: *T*<sup>193</sup> = 1*.*881; *p* = 0*.*061).

In a general linear model-analysis of the UnEx full and short scales with sex entered as a covariate the recessive effect of the valallele could also be found (full scale: *F*<sup>1</sup> = 4*.*337; *p* = 0*.*038 and short scale: *F*<sup>1</sup> = 7*.*027; *p* = 0*.*008).

#### **GENETIC ASSOCIATIONS FOR THE** *DRD2* **Taq1A-POLYMORPHISM**

Significant effects of the *DRD2* Taq1A-polymorphism could neither be shown for the whole nor the female sample. Within the group of male participants a significant effect of the A1 allele could be found with A1/A1-homozygote males showing significantly lower values in ImpNon (*T*5*.*<sup>367</sup> = −3*.*785; *p* = 0*.*011). Due to the fact that this group, however, only consisted of four individuals, this effect shall not be interpreted further.

#### **GENETIC ASSOCIATIONS FOR THE** *MAOA***-uVNTR-POLYMORPHISM**

Due to the aforementioned argumentation that heterozygous men do not exist and heterozygous females were excluded from the analyses, since their MAO-A-functionality cannot be clearly ascertained, no analyses of variance were performed for this polymorphism.

Within the whole sample a trend could be seen, whereby the low-functional genotype-group showed higher values in CogDis compared to the high-functional group (*T*<sup>170</sup> = 1*.*697; *p* = 0*.*091). No such effect was found in the female sample, but the males showed the same effect in a significant fashion (*T*<sup>80</sup> = 2*.*030; *p* = 0*.*046).

Additionally, the male subsample showed significantly higher values in both the full and short IntAn-scales in the low-activity group (full scale: *T*29*.*<sup>5</sup> = 2*.*557; *p* = 0*.*016 and short scale: *T*<sup>80</sup> = 2*.*359; *p* = 0*.*021).

In GLM-analyses of the whole sample with sex as a covariate there were still no significant effects of the *MAOA*-uVNTR. When sex was, however, entered as a second factor, a significant interaction with the polymorphism could be found for the full



**Table 6 | Significant (***p <* **0***.***05) genetic associations and borderline-significant trends for O-LIFE-scales.**

*Abbreviation: n, sample size; M, arithmetic mean; SD, standard deviation of M; df, degrees of freedom; p, conditional probability.*

and short IntAn-scales (full scale: *F*<sup>1</sup> = 5*.*91; *p* = 0*.*016 and short scale: *F*<sup>1</sup> = 3*.*890; *p* = 0*.*05). Where males with a low-activity genotype showed higher IntAn-scores, females showed equal or even lower (for the full scale) scores in the low-activity group (see **Figure 1**).

#### **GENETIC ASSOCIATIONS FOR THE** *DAT* **3- UTR-VNTR-POLYMORPHISM**

Analyses of the main genotype-groups of the *DAT*-VNTR found in our sample (9/9- and 10/10-homozygotes as well as 9/10 heterozygotes) indicated that the relatively rare 9-repeat-allele could have a recessively protective effect on UnEx. Due to this allele's rarity, however, the whole sample only had 13 individuals in this group, 10 of which were female and 3 male. Like in the case of the *DRD2* Taq1A-polymorphism this effect should therefore not be overly interpreted. Lower scores for 9/9-homozygotes were found in the whole sample (*T*19*.*<sup>91</sup> = −2*.*133; *p* = 0*.*048) and in both sexes, whereby only the female subsample reached borderline-significance (*T*12*.*<sup>95</sup> = −2*.*147; *p* = 0*.*051). The relatively small group of three 9/9-homozygous males did, however, show an indication that the 9-repeat-allele could be unfavorable regarding the CogDis-scale (*T*40*.*<sup>27</sup> = 3*.*348; *p* = 0*.*002).

# **DISCUSSION**

#### **PRELIMINARY PROPERTIES OF THE GERMAN O-LIFE**

Our first results from a sample of 1228 individuals (341 male, 887 female) show that the German translation of the "O-LIFE" may be considered internally comparable to the original English version. Cronbach's coefficients of internals consistency (α) for the sub-scales UnEx, CogDis, and IntAn are almost equal to those reported by Mason and Claridge (2006). Research on the effects of sample size on Cronbach's alpha suggests that values may rise in samples larger than 130 (Javali et al., 2011). It could therefore be possible that the slight decrease in values of 0.01–0.03 may be attributable to the comparatively larger sample (*n* = 1926) used for the extended norms of the English O-LIFE. The consistencyvalue for the scale ImpNon, however, is merely 0.68 compared to 0.77 in the English version. A possible explanation herefore may be of a philological and/or psycholinguistic nature: While Old English as well as German were both Germanic languages, Modern English has been influenced massively by both invasions into Britain (notable the Norman Conquest in 1066) as well as through the expansion of the British Empire and the resulting influx of other non-Germanic vocabulary (Shippey, 2000, 2003; Lamb, 2010). Modern English therefore has an extensively larger vocabulary than Modern German, wherefore "finer points" may be slightly lost when using a rather literal approach to item-translation. Several items of the ImpNon-scale consist of words of which the German translations may imply slightly different meanings to different users, as they are not as specific as their Modern English equivalents. For example words like "urge," "cheat," "annoy," "take advantage of," or "overindulge" were translated as "Drang," "betrügen," "ärgern," "ausnutzen," and "übertreiben," respectively. Although these words are quite literal translations, they lack the finer nuances of their Modern English equivalents. The word "overindulge," for example, does not have a literal equivalent in German and can only be circumscribed or translated as "übertreiben" which in term would literally re-translate as "exaggerate." It would therefore seem necessary to re-examine specific items (not only from the scale ImpNon) and find less literal but therefore possibly more contentually unambiguous translations. Unfortunately, less literal translation would necessarily mean moving away from the original item.

The average scale sum-scores are also similar to the English original with slightly higher means in UnEx, CogDis, and IntAn. Standard deviations were marginally equal to the original, albeit higher in ImpNon. This is likely to be a sampling effect and would also explain the weaker correlations with age compared to the extended norms of the English O-LIFE (Mason and Claridge, 2006). Our sample consisted mainly of students and is relatively young on average, which explains the higher means for UnEx, IntAn, and CogDis in this case. The fact that CogDis does not increase with age may be explained through considerations that cognitive disorganization is firstly unlikely to be found high in the older participants in our sample, who are mainly fellows of JLU, and secondly due to findings that CogDis is negatively related to creativity (Batey and Furnham, 2008), which is also likely to be higher than average in university-students and probably even more so in university-fellows. Nettle (2006) found that mathematicians show higher scores in IntAn and lower scores in CogDis compared to non-mathematicians. If this is extrapolated for (natural) scientists in general (e.g., psychologists, physicists, statisticians, technicians, etc.), who comprised an over-proportionate part of our sample—especially in the higherage groups—compared to the average population, the negative correlation for CogDis and positive correlation for IntAn with age is not surprising in our sample. We are currently trying to increase our sample in the context of students' bachelor's theses to include more non-academics as well as more young (*<*20) and older (*>*60) individuals before attempting to publish a more valid set of norms for the German O-LIFE.

Apart from these differences, reliabilities (both internal consistencies and test–retest-correlations) as well as inter-correlations between scales are more than acceptable and mirror the findings regarding the original O-LIFE. The reliability and validity results here suggest that the German O-LIFE is a reasonable approximation to the original regarding its capability of measuring schizotypy or "psychosis proneness." It can therefore be used in our effort to show that the fully dimensional model of schizotypy is well suited as an endophenotype of psychosis or schizophrenia.

#### **GENETIC ASSOCIATIONS WITH THE O-LIFE SUB-SCALES**

We found a number of significant associations and borderlinesignificant trends regarding individual scales of the O-LIFE for various dopaminergic polymorphisms in the genes *COMT*, *DRD2*, *MAOA*, and *SCL6A3* (*DAT*). We consciously chose not to predict effects of specific "risk"-alleles firstly due to partly ambiguous reports from literature and secondly because we deliberately did not want to exclude the often-overlooked possibility of heterosis-effects, which refer to a general finding that hybrid-species as well as heterozygosity on the genetic level tend to show higher values in advantageous (positive heterosis) and/or lower values in disadvantageous (negative heterosis) traits (for a review on molecular heterosis, see Comings and Macmurray, 2000). Our results did not, however, show significant heterosiseffects, even though we could find possible indications thereof for the rs4680 (see end of following paragraph).

Higher scores in both UnEx-scales for participants homozygous for the val-allele of the *COMT* Val158Met-polymorphism (rs4680) are in line with previous findings (Avramopoulos et al., 2002; Schurhoff et al., 2007; Smyrnis et al., 2007). The val-allele of the rs4680 leads to a higher-active and more thermostable form of the resulting enzyme C*O*MT (Lachman et al., 1996), which is primarily responsible for the degradation of dopamine in the frontal cortex (Karoum et al., 1994). At first, the finding of higher degradation of dopamine leading to higher scores in positive schizotypy may seem contra-intuitive regarding the dopamine-hypothesis of schizophrenia and has also been noted by the authors of the aforementioned papers finding this result. Considering the newest version of the dopamine-hypothesis (Howes and Kapur, 2009), however, this effect does not seem surprising, but actually logically explainable: Under the assumption that schizophrenic and maybe also schizotypal traits do not result from a general over-abundance of dopamine in the frontal cortex and basal forebrain (mainly the accumbens nucleus), but rather from an increased firing rate of the mesolimbic system in the sense of aberrant salience, which in turn is caused by or at least mediated through a reduction in reciprocal prefrontal inhibition of the ventral tegmental area (VTA) (Grant, 2011), pieces start falling into place. Within the prefrontal cortex (PFC) the rate of production of 3-MT, the *O*-methylated metabolite of dopamine, is extremely high, suggesting a disequilibrium of C*O*MT and MAOs (Karoum et al., 1994). Since 3-MT is highly neurotoxic, an increase in its production by the high-active and thermostable variant of C*O*MT, encoded by the val-allele of the *COMT* gene, will lead to increased levels of dopamine-neurotoxicity, atypical frontal neurodegeneration and thereby loss of frontal inhibition of the VTA (Grant, 2011). The question why the effect of val/val-homozygosity is less pronounced in the male and female subgroups and only reached borderline-significance for the short UnEx-scale is harder to explain. Firstly, the O-LIFE short scales were developed based on heritability findings regarding the over 100 O-LIFE items (Linney et al., 2003) and consist of mainly those items with the highest heritability (Mason et al., 2005). It is therefore not surprising that genotypal associations are partly more pronounced in the short compared to the full scales, although this was not the case for all of our findings. Secondly, we also found UnEx-scores to be (non-significantly) lowest in val/methomozygotes for the whole sample as well as both the male and female subsamples, whereby the sex-difference for UnEx was most pronounced in this group, indicating that men profit more from rs4680-heterosis than women. This effect was increased when the ambiguous group of (female) *MAOA*-heterozygotes was excluded from the analysis. We therefore performed an ex-post-facto GLManalysis under the exclusion of *MAOA*- and *COMT*-heterozygotes

and found a trend-interaction of the two genes on UnEx (*F*<sup>1</sup> = 2*.*617; *p* = 0*.*11) (**Figure 2**).

Although this effect is not significant, likely due to the sample being too small for analyses of gene by gene interactions, especially under the exclusion of a major number of *MAOA*and rs4680-heterozygotes, the differences in group means appear to support the role of dopamine-neurotoxicity in the development of (in this case positive) schizophrenic and schizotypal traits and could help explain the findings of Sheldrick et al. (2008) and Ettinger et al. (2006) in whose studies met/methomozygotes of the rs4680 had the highest schizotypy-scores. The hypothetical model would be that increased C*O*MT-activity (val/val-group) in the frontal cortex might lead to increased dopamine-neurotoxicity and thereby to reduced inhibition and increased firing of the VTA, which in turn would lead to increased presynaptic accumulation of DOPAC (the dopamine-metabolite of MAOs), especially in the high-activity *MAOA*-group. This also leads to an increase in intracellular H2O2-production [a byproduct of MAO-activity (Maker et al., 1981)], thereby exacerbating dopamine-neurotoxicity and possibly leading to a reduction of presynaptic auto-regulatory mechanisms through the DAT, the vesicular monoamine transporter 2 (VMAT2) and the D2 autoceptor (Grant, 2011). The met/met- + high-activity *MAOA*group would have the lowest levels of toxic dopamine metabolites due to the disequilibrium of C*O*MT and MAOs in the frontal cortex. In the group with high C*O*MT but low MAO-A-activity, presynaptic auto-regulation would likely be better functioning than in the high *MAOA*-group, wherefore schizotypal traits and schizophrenic symptoms would be less pronounced than in the group with high activity in both enzymes but still more pronounced than in the met/met-group of the rs4680 with high MAO-A activity. The last group with low activities in both C*O*MT and MAO-A would have the highest levels of synaptic dopamine and thereby moderately high UnEx-scores due to the abundance of dopamine itself as well as to an increased rate of dopamine-quinone formation (Graham, 1978; Graham et al., 1978), a secondary mechanism in dopamine-neurotoxicity (Grant, 2011). It has to be noted that this model is based solely on neurochemical properties of dopamine-metabolism/-catabolism and has not yet been entirely proven in schizophrenic patients, merely in animal- and *in-vitro*-studies. It does, however, fit to and best explain the findings regarding the genetic associations of *COMT*- and *MAOA*-polymorphisms in this and other studies. Larger studies with sufficient sizes of all relevant groups as well as research using different methods are necessary to examine the verisimilitude of this model further.

Analyses of the singular association of the *MAOA*-uVNTRpolymorphism show a trend toward higher CogDis-values for the low-functional group in the whole sample. The association became significant in the male sample, but not in the female sample, although the direction of this trend was the same in all samples. This effect is explainable due to the enhancing effect of dopamine on cognitive functions and the neurotrophic effect of dopamine during brain ontogeny (for a review, see Nieoullon, 2002). Overall, *MAOA*-genotype had no effects in the female, yet additional effects in the male sample in the full and short IntAn-scales. We therefore examined the possibility of genotype by sex interactions and found these for both scales (full IntAn: *F*<sup>1</sup> = 5*.*91; *p* = 0*.*016 and short IntAn: *F*<sup>1</sup> = 3*.*89; *p* = 0*.*05; q.v. **Figure 1**). While men with a low-activity *MAOA*-genotype show significantly higher levels than those in the high-activity group for both IntAn-scales, this effect is not found (for the short scale) or even reversed (for the full scale) in women, whereby the differences between genotype-groups was not significant in the female sample. In an animal model, early postnatal MAO-A-inhibition using clorgiline lead to significant reduction in total ambulatory time, rearing behavior and a general increase in neophobia (e.g., in a novelty-suppressed feeding paradigm) compared to vehicletreated animals. That is, behavioral changes that can be considered upon as similar to an increase in introversion and anhedonia, as well as increased levels of striatal dopamine/DOPAC and decreased levels of serotonin (5-HT) and its primary metabolite 5-hydroxyindoleacetic acid (5-HIIA), whereby these effects were not found, when MAO-A was inhibited in adolescent or adult animals (Yu, 2012). Similar effects on behavior were found by Bortolato et al. (2011) in an incomplete (hypomorphic) knockout of the *Maoa*-gene in mice (the gene *MAOA*/*Maoa* is highly conserved in many Eutheria, i.e., *Homo sapiens*, *Pan troglodytes*, *Rattus norvegicus,* and *Mus musculus*). These animals were distinctly different from complete *Maoa*-knockouts in that they still showed low, yet detectable enzymatic activity and showed dysphoria- or depression like behavior (e.g., reduced locomotion, grooming, and social interaction) but not higher levels of aggression. Human studies linking the *MAOA*-uVNTR to affective disorders are rare, but a study in healthy female Korean nursing students found a non-significant increase in Beck's Depression Inventory scores when comparing 4/4 repeats (high activity), 3/4 repeats (classified as low activity in this study) and 3/3 repeats (low activity) (Yang et al., 2007). Unfortunately, the classification of the 3/4-group is highly questionable due to the argumentation mentioned in the materials and methods section of this paper. If the resulting means, standard deviations and sample sizes (low activity: *M* = 8*.*46, *SD* = 6*.*74, *N* = 79; high activity: *M* = 6*.*49, *SD* = 6*.*77, *N* = 43) are used for a one-tailed *t*-test, a *p*-value of 0.063 is found. These results do not, however, explain the interaction with sex in our sample, which is primarily attributable to the high IntAn-scores for low-activity males. We therefore performed exploratory hypothesis-testing on the single item level to find that there was not one item which showed a significant difference in group means for the female sample. In men, however, we found that all items with significant or borderline-significant differences between the high- and low-activity *MAOA*-uVNTRgroups related to social closeness (e.g., making new friends, going out with others, being touched by friends or having intense relationships with others etc.) and behaviors that some men might consider "effeminate" (e.g., enjoying dancing, singing, promenading). It seems therefore that high IntAn-values in males may actually be caused by negation of items referring to social closeness and "unmanly" behavior, which could be in line with findings linking the low-activity variants of the polymorphism in combination with negative life events (in this case possible and severe childhood maltreatment) to higher scores in antisocial personality disorder (Caspi et al., 2002).

A fundamental basis of antisocial personality or psychopathy is the incapability of experiencing fear and learning from errors. The latter has been shown by members of our group to be associated with the A1-allele of the *DRD2* Taq1A-polymorphism (Klein et al., 2007). This allele is linked to reduced density of postsynaptic D2-receptors. A study by Hamidovic et al. (2009) found two other SNPs in the *DRD2*-gene as well as their combined diplotype to be associated with high impulsivity and poor behavioral control in those groups with reduced *DRD2*-expression in healthy subjects. The finding that the group of A1/A1-homozygous men had significantly lower results in ImpNon-scores is therefore surprising. Due to the small number of participants in this group (*n* = 4) we chose, however, not to interpret this result further and only report it for the sake of completeness.

Regarding the *SLC6A3* (*DAT*) 3- UTR VNTR-polymorphism, we found lower scores in UnEx in the whole sample and in the female subsample for persons homozygous for the 9-repeat-allele compared to carriers of one or two 10-repeat-alleles. Studies comparing carriers of the 9-repeat-allele to 10/10-homozygotes find weak but non-significant effects regarding lower RISCscores (Ettinger et al., 2006) and weaker startle magnitudes to various affective stimuli in older adults (Armbruster et al., 2011), which could be indicative of higher affective processing and possible increased salience of affective stimuli in 10/10 homozygotes. Again others report no significant association between the *SLC6A3*-VNTR and schizophrenia (Hauser et al., 2002). In our sample a comparison between 10/10-homozygotes and carriers of one or two 9-repeat-alleles yielded no significant differences. On the other hand, the 9/9-genotype is very rare and was only found in 10 women and 3 men within our sample, wherefore these results are also to be interpreted with caution. While Prata et al. (2009) found a significant interaction between the *SLC6A3*-VNTR and the *COMT* Val158Met on brain activity in healthy subjects (*<sup>n</sup>* <sup>=</sup> 44) and schizophrenic patients (*n* = 41), independently of diagnosis, we could not find this interaction in our sample. We did find, however, that differences between *COMT* Val158Met-genotypes were more pronounced in 10/10- and 9/9-homozygotes compared to heterozygotes, although this observation was not statistically significant.

# **SUMMARY AND CONCLUSION**

Using the German translation of the O-LIFE, which appears to measure the same underlying dimensional trait of schizotypy or psychosis-proneness as the original, we found a large number of significant associations and borderline-significant trends between various dopaminergic and schizophrenia-related genes and the facets of the O-LIFE.

We expected to find generally better associations for the short scales compared to the full scales, due to the fact that these were created partially on the basis of heritability studies. It must be admitted, however, that longer scales (of any trait) probably afford better assessment and reduce error variance. It is possible that alternative item selection and exploratory single-item analyses can identify those individual items with greater relevance for genetic associations.

Most importantly, all genetic associations and trends found between the examined genes and the respective O-LIFE-facets are fully explainable on the basis of neuroanatomical, -chemical, -pathological and -developmental findings. These explanations can also be used interchangeably for schizotypal traits as well as for schizophrenia, wherefore we conclude that the fully dimensional schizotypy-model, as measured with the O-LIFE, is a valid endophenotype of schizophrenia or psychosis-in-schizophrenia.

The limitations of this study are mostly inherent to the genotype- and allele-frequencies of the genes we examined. It is therefore necessary to increase groups with extremely low numbers of individuals (e.g., the *DRD2* Taq1A-group A1/A1) in order to make more generalizable statements. Furthermore, our approach to translate the O-LIFE-items as literally as possible may also have lead to decreases in internal scale-consistencies (especially for the scale ImpNon), wherefore single items might need to be re-examined and slightly altered to better fit the intention of the original. Finally, our sample appears to be selective due to a high prevalence of university-students and -fellows. Whether or not this influences the genetic associations cannot be said at this point, although it would appear improbable, since all effects were clearly explainable and in line with the relevant state of the art. Nonetheless, we are currently increasing our sample to include participants not involved in academia and with a wider age-range, in order to generate genuine norms for the German O-LIFE.

We also realize that not all found effects would survive stringent correction for multiple testing, wherefore the results should be interpreted cautiously. Since, however, each statistical test was performed in independent groups due to our ex-post-facto approach, rather than *post-hoc* comparisons of all individual groups with each other, no group entered into any test twice. Therefore, strictly speaking, it could be argued that no multiple comparisons were reckoned. In any case, it is necessary to assess, if these effects can be replicated in the future in other samples as well as by other researchers.

Our findings add to the growing body of evidence that schizotypy (seen as a set of personality dimensions) and schizophrenia share a common biological basis related to genetic susceptibility or risk as well as shared pathological processes in the sense of dysregulation of dopamine-functioning. We are currently establishing a paradigm to hopefully unequivocally assess aberrant salience and incapability of adequate gating and extinction of irrelevant stimuli, which we want to use on healthy volunteers and schizophrenic patients in combination with genetic analyses, analyses of gene-expression and schizotypy-measurements using the O-LIFE. Our aim hereby is to better understand the pre-clinical development of schizophrenia, the transition from high schizotypy to clinical schizophrenia as well as examine the

#### **REFERENCES**


effects of neurodegenerative processes and their attenuation using respective palliative drugs in the hope of adding to the possibility of hopefully soon being able to detect high schizophrenia-risk in (still) healthy patients and thereby allowing for the possibility of clinical intervention before the first florid episode.

### **ACKNOWLEDGMENTS**

The authors would like to thank Cornelia Meineke for her excellent technical assistance as well as Frieder Gabriel, Dorothea Borchert, Anne Beerbohm, and Marion Krey-Leiden for their help in acquiring participants for this study.

schizophrenia]. *Psychiatr. Pol.* 36, 403–412.


psychosis: a systematic review and meta-analysis. *Schizophr. Res.* 125, 209–220.


artists, and mathematicians. *J. Res. Pers.* 40, 876–890.


changes associated with Schneider's first-rank symptoms in schizophrenia: a MRI study. *Psychol. Med.* 35, 549–560.


Yu, Q. (2012). *Developmental Monoamine Signaling Impacts Adult Affective and Aggressive Behaviors.* Doctoral thesis, Columbia University.

#### **ONLINE REFERENCES**

Court, M. H. (2005–2008). Court lab – HW calculator. www.tufts. edu/∼mcourt01/Documents/Court lab - HW calculator.xls

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 16 November 2012; accepted: 05 January 2013; published online: 24 January 2013.*

*Citation: Grant P, Kuepper Y, Mueller EA, Wielpuetz C, Mason O and Hennig J (2013) Dopaminergic foundations of schizotypy as measured by the German version of the Oxford-Liverpool Inventory of Feelings and Experiences (O-LIFE)—a suitable endophenotype of schizophrenia. Front. Hum. Neurosci. 7:1. doi: 10.3389/fnhum.2013.00001*

*Copyright © 2013 Grant, Kuepper, Mueller, Wielpuetz, Mason and Hennig. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

# Dopaminergic basis of the psychosis-prone personality investigated with functional magnetic resonance imaging of procedural learning

# *Ulrich Ettinger 1,2, Philip J. Corr 3, Ardeshier Mofidi 4, Steven C. R. Williams <sup>2</sup> and Veena Kumari 4,5\**

*<sup>1</sup> Department of Psychology, University of Bonn, Bonn, Germany*

*<sup>2</sup> Department of Neuroimaging, Institute of Psychiatry, King's College London, London, UK*

*<sup>3</sup> Department of Psychology, City University, London, UK*

*<sup>4</sup> Department of Psychology, Institute of Psychiatry, King's College London, London, UK*

*<sup>5</sup> NIHR Biomedical Research Centre for Mental Health, The Institute of Psychiatry, South London and Maudsley NHS Trust, London, UK*

#### *Edited by:*

*Luke D. Smillie, The University of Melbourne, Australia*

#### *Reviewed by:*

*William A. Cunningham, University of Toronto, Canada Philip R. Corlett, Yale University, USA*

#### *\*Correspondence:*

*Veena Kumari, Department of Psychology, Institute of Psychiatry, King's College London, De Crespigny Park, PO 78, London SE5 8AF, UK. e-mail: veena.kumari@kcl.ac.uk*

Previous evidence shows a reliable association between psychosis-prone (especially schizotypal) personality traits and performance on dopamine (DA)-sensitive tasks (e.g., prepulse inhibition and antisaccade). Here, we used blood oxygen level-dependent (BOLD) fMRI and an established procedural learning (PL) task to examine the dopaminergic basis of two aspects of psychosis-proneness (specific schizotypy and general psychoticism). Thirty healthy participants (final *N* = 26) underwent fMRI during a blocked, periodic sequence-learning task which, in previous studies, has been shown to reveal impaired performance in schizophrenia patients given drugs blocking the DA D2 receptor subtype (DRD2), and to correspond with manipulation of DA activity and elicit fronto-striatal-cerebellar activity in healthy people. Psychosis-proneness was indexed by the Psychoticism (P) scale of the Eysenck Personality Questionnaire-Revised (EPQ-R; 1991) and the Schizotypal Personality Scale (STA; 1984). EPQ-R Extraversion and Neuroticism scores were also examined to establish discriminant validity. We found a positive correlation between the two psychosis-proneness measures (*r* = 0*.*43), and a robust and unique positive association between EPQ-R P and BOLD signal in the putamen, caudate, thalamus, insula, and frontal regions. STA schizotypy score correlated positively with activity in the right middle temporal gyrus. As DA is a key transmitter in the basal ganglia, and the thalamus contains the highest levels of DRD2 receptors of all extrastriatal regions, our results support a dopaminergic basis of psychosis-proneness as measured by the EPQ-R Psychoticism.

**Keywords: schizotypy, sequence learning, dopamine, striatum, thalamus**

# **INTRODUCTION**

The psychosis-prone personality has been measured in a number of different ways. One approach has been to measure clinicallyinformed specific traits, as seen in various schizotypy questionnaires (Claridge and Broks, 1984). The second approach has been to measure a more general construct as developed, principally, by psychoticism (P; Eysenck, 1992). Sometimes, these constructs are combined into a single measurement instrument (e.g., Claridge et al., 1996). Debate continues concerning the precise roles played by these different factors in the psychosis-prone personality.

Most work has focused on the construct of schizotypy (Rado, 1953) which is a trait placed within, or in close proximity to, the schizophrenia spectrum (e.g., Meehl, 1990; Claridge, 1997; Gruzelier, 2002; Raine, 2006). It overlaps with schizophrenia at various levels of measurement, including the descriptive clinical level (Lenzenweger, 2010), the cognitive level (e.g., Cochrane et al., 2012; for review, see Giakoumaki, 2012) and the level of neurobiology (e.g., Bollini et al., 2007; Aichert et al., 2012). The enduring hypotheses of, and consistent evidence for, abnormal dopamine (DA) function in schizophrenia (Gray et al., 1991; Di Forti et al., 2007; Howes and Kapur, 2009; Eyles et al., 2012), combined with continuum models of psychosis (Johns and van Os, 2001), suggest that schizotypal personality, or at least some dimensions of it, should also have a dopaminergic basis.

Pharmacological studies provide direct evidence of an association between schizotypy and alterations in DA neurotransmission (Mohr et al., 2005; Woodward et al., 2011; Koychev et al., 2012). Several studies have also shown schizophrenia-like performance on DA-sensitive tasks, for example reduced latent inhibition (review, Kumari and Ettinger, 2009; Granger et al., 2012), reduced prepulse inhibition (Evans et al., 2005; Kumari et al., 2005), increased antisaccade errors (O'Driscoll et al., 1998; Ettinger et al., 2005; Gooding et al., 2006), reduced Kamin blocking effect (Moran et al., 2003), aberrant salience related to dysfunctional reward learning (Roiser et al., 2009) and altered salience attribution (Galdos et al., 2011) in association with various dimensions of (high) schizotypy. Further support comes from functional imaging, for example, in showing a negative relationship between psychometric schizotypy and activity in the striatum and thalamus during antisaccade task (Aichert et al., 2012), and a positive relationship between fronto-striatal prediction error signal and delusion-like beliefs in healthy people (Corlett and Fletcher, 2012), compatible with what has been found in schizophrenia (Raemaekers et al., 2002; Corlett et al., 2007).

Much less work has been conducted using the more general trait of psychoticism, but where such studies exist there is evidence of a DA basis. For example, individuals scoring high on psychoticism show reduced latent inhibition (Kumari and Ettinger, 2009); they also show lower prepulse inhibition (Kumari et al., 1997, 2008a) and less striatal-thalamic activity during prepulse inhibition (Kumari et al., 2008a) in line with what has been found in schizophrenia (Kumari et al., 2003, 2007). There are negative correlations between psychoticism and cerebral perfusion in the basal ganglia (putamen and caudate) and thalamus (O'Gorman et al., 2006)—cerebral perfusion is a fundamental physiological quantity reflecting the rate of delivery of oxygen and other nutrients to an organ or tissue. Psychoticism has also been associated with decreased metabolic rate in the basal ganglia and thalamus (Haier et al., 1987). The psychoticism–DA relationship is consistent with the negative association between psychoticism and DA D2 binding (Gray et al., 1994) and resting fMRI signal in the basal ganglia and thalamus (Kumari et al., 2004). In relation to the experimental manipulation of DA, Corr and Kumari (2000) reported an interaction of psychoticism with (5 and 10 mg) d-amphetamine challenge on self-reported mood.

The motivation for the present study was to compare the validity of these two forms of psychosis-proneness personality constructs in relation to the functional neuroanatomical basis of a strongly DA-sensitive procedural learning (PL) task [a variant of the serial reaction time task (SRT) which involves learning of sequences]. PL is a type of rule-based learning in which performance facilitation occurs with practice on task without the need for conscious awareness (Cohen and Squire, 1980; Squire and Zola-Morgan, 1988). PL is generally independent of intelligence and performance on tests of declarative learning and memory (Feldman et al., 1995). PL is sensitive to changes in the DA system (Foerde and Shohamy, 2011), with most prominent effects seen in the dorsal striatum (improved by moderately elevated DA levels and impaired by decreased DA levels), and there is no clear evidence so far for its sensitivity to serotonergic, noradrenergic, and cholinergic systems (Uddén et al., 2010). The performance on the PL task we used in this study has been shown previously to improve and worsen in healthy people following the acute administration of a DA-agonist, d-amphetamine, and a DA-antagonist, haloperidol, respectively (Kumari et al., 1997). Further supporting a strong dopaminergic basis of PL, patients with schizophrenia given DRD2 blocking typical antipsychotics (e.g., Green et al., 1997; Kern et al., 1998; Kumari et al., 2002), but not atypical antipsychotics (Purdon et al., 2002, 2003; Kumari et al., 2008b), show significant PL impairment.

Neurally, the basal ganglia, in particular the striatum, and the cerebellum are known to play important roles in PL, based on the observations of impaired PL on variants of the SRT in patients with Parkinson's disease (Knowlton et al., 1996; Foerde and Shohamy, 2011), Huntington's disease (Heindel et al., 1989; Knopman and Nissen, 1991; Willingham et al., 1996) and damage to the cerebellum (Pascual-Leone et al., 1993; Molinari et al., 1997; Gomez-Beldarrain et al., 1998). With the striatum (Alexander and Crutcher, 1990) and cerebellum (Schmahmann, 1991) both projecting to the frontal lobe via the thalamus, the frontal cortex is also thought to be a component of the circuit subserving PL (Doyon et al., 1996; Honda et al., 1998; Gomez-Beldarrain et al., 1999). Neuroimaging evidence confirms the involvement of these regions in PL (Jenkins et al., 1994; Doyon et al., 1997; Kumari et al., 2002) and, in addition, shows involvement of the thalamus and cingulate gyrus (Kumari et al., 2002). Considering the various regions involved in PL, DA appears to be a key neurotransmitter given its prominence in the basal ganglia, frontal lobe and the thalamus which contains the highest levels of DRD2 receptors out of all extrastriatal brain regions (Kessler et al., 1993; Hall et al., 1996).

This is the first study, to our knowledge, to examine psychosisproneness personality as well as the discriminant validity of schizotypy and psychoticism in an fMRI study of PL. Based on separate strands of previous evidence concerning the behavioral effect of DA-agonists and antagonists on PL (Kumari et al., 1997), and the imaging literature on PL, we hypothesized that psychosisproneness personality, given its overlap with positive (hyperdopaminergic) symptoms of psychosis, would correlate positively with PL and related brain activity, especially in dopamine-rich regions such as the striatum and the thalamus.

# **MATERIALS AND METHODS PARTICIPANTS**

Thirty healthy individuals (15 men, 15 women) were recruited from the general population using advertisements, flyers and mailing lists. All participants were right-handed and were screened for a history of substance and alcohol abuse, anorexia, mental illness, and regular medical prescriptions. A semistructured interview was conducted to rule out the presence of a mental disorder and the presence of psychosis in their first-degree relatives. Of 30 individuals recruited initially, 26 individuals (13 men, 13 women) were included in the final sample. Of four individuals who were not included in the final sample, two did not fully complete the personality questionnaires, one had missing online performance data due to problems with the button box, and one provided unusable fMRI data.

The study procedures were approved by the Ethics Committee of the Institute of Psychiatry and the South London and Maudsley NHS Trust. All participants provided written informed consent after the study procedures had been fully explained to them.

#### **PSYCHOMETRIC ASSESSMENT**

A number of self-report rating scales can be used to assess psychosis-prone personality factors. In this study, the level of psychosis-proneness in each participant was assessed using two questionnaires: (1) the Psychoticism scale of the Eysenck Personality Questionnaire-Revised (EPQ-R; Eysenck and Eysenck, 1991) and (2) The Schizotypal Personality Scale (STA; Claridge and Broks, 1984). The EPQ-R P scale is proposed to be a general measure of the putative liability to the psychosis spectrum. The scale also includes items on antisocial, criminal and impulsive behaviors. The STA (Claridge and Broks, 1984) is a specific measure of schizotypal personality based on clinical observation. The scale focuses on positive schizotypy, including items on magical and delusional thinking as well as perceptual distortions. On each of the administered scales, higher scores indicate higher levels of self-reported psychosis-proneness. In addition to P, the EPQ-R measures two other major dimensions of personality, namely Extraversion (E) and Neuroticism (N), which were used in additional analyses, as described further.

#### **PROCEDURE**

Participants performed a 5-min sequence learning task in a blocked AB design, as described previously by Kumari and colleagues (2002, 2008b), while undergoing fMRI.

The task consisted of two 30-s alternating conditions: blocks of random trials (OFF, control condition) and blocks of pattern trials (ON, experimental condition). In total, there were five blocks of random trials and five blocks of pattern trials. Participants were presented with a white target stimulus (an asterisk) on a black screen, viewed via a prismatic mirror fitted in the radiofrequency head coil, as they lay in the scanner. This target moved between four locations on the screen, which was divided into four equal quadrants by two intersecting white lines. The target movements during the pattern trials were predictable for 75% of cases, i.e., determined following three specific rules: (1) a horizontal target movement was followed by a vertical target movement; (2) a vertical target movement was followed by a diagonal target movement; (3) a diagonal target movement was followed by a horizontal movement. The fourth movement of the target during the pattern trials was unpredictable, which then was followed by the above mentioned three specific rules (**Figure 1**).

Participants were not told of the existence of specific rules governing the target movements during the pattern blocks, and the beginning of random and pattern blocks ware not marked in any way. They were asked to follow each target movement with their right hand as fast as possible using a MR compatible key pad with four keys, each key corresponding to one of the four quadrants. The movement of the target was initiated by the participants' touching the target key. Reaction times (RTs) were recorded on-line.

Prior to scanning, all participants underwent a practice session during which they practiced on five 30-s blocks of random trials and five 30-s blocks of pattern trials, both alternated with 30-s rest periods, in order to familiarize themselves with task requirements and the use of the MR compatible key pad. The practice session was identical for all participants.

### **IMAGE ACQUISITION**

Echoplanar MR brain images were acquired using a 1.5 T GE Signa system (General Electric, Milwaukee WI, US) at the Maudsley Hospital, London. Daily quality assurance was carried out to ensure high signal to ghost ratio, consistent high signal to noise ratio, and excellent temporal stability using an automated quality control procedure. A quadrature birdcage head coil was used for radio frequency (RF) transmission and reception. In each of 16 near-axial, non-contiguous planes parallel to the intercommissural (AC-PC) plane, 100 T2∗-weighted MR images depicting blood oxygenation level-dependent (BOLD) contrast (Ogawa et al., 1990) were acquired over the 5-min experiment with echo time (TE) = 40 ms, repetition time (TR) = 3 s, in-plane resolution = 3.1 mm, slice thickness = 7.0 mm, and interslice gap = 0.77 mm. Head movement was limited by foam padding within the head coil and a restraining band across the forehead. At the same session, a high resolution 3-D inversion recovery prepared spoiled GRASS volume dataset was acquired in the AC-PC plane with TE = 5.3 ms, inversion time (TI) = 300 ms, TR = 12.2 s, in-plane resolution = 0.94 mm, and slice thickness = 1.5 mm.

### **DATA ANALYSIS**

### *Behavioral data analysis*

To examine the task effect (i.e., the presence of PL), mean RTs to blocks of random and pattern trials were subjected to a repeated measures analysis of variance (ANOVA) with Trial Type (random, pattern) and Block (1–5) as within-subjects factors. The amount of PL was calculated as the difference between the mean RTs to random and pattern trials. The possible association between the amount of PL and personality scores was examined with correlational analysis (Pearson's r). Statistical analyses were performed using SPSS for Windows (version 20.0). The alpha level of testing significance was kept at *p* = 0*.*05, unless stated otherwise.

#### *Image processing*

All images were processed and analyzed using Statistical Parametric Mapping software (SPM8; http://www*.*fil*.*ion*.*ucl*.*ac*.* uk/spm/). For each participant, the 100 volume functional time series was motion corrected (Friston et al., 1996), transformed into standard stereotactic Montreal Neurological Institute (MNI) space, spatially smoothed with an isotropic Gaussian kernel of 8 mm full width height maximum and band pass filtered (highpass filter with cut-off at 128 s) using statistical parametric mapping software.

#### *Models and statistical inferences*

Functional MRI data were analyzed using a random effect procedure (Friston et al., 1999). This analysis consisted of a 30-s boxcar design (convolved with the haemodynamic response function) modeling the experimental condition (pattern trials). The control condition (random trials) formed the model's implicit baseline. Motion parameters were included as covariates at this stage. The second stage of the random effect model tested for generic activations across all participants' images using a one-sample *t*-test. The relationship of EPQ-R P scores with neural activity across the whole brain was identified using a multiple regression model (Psychoticism, Extraversion, Neuroticism, and PL scores entered into the model) within SPM8. In this analysis, the effects that survived *p <* 0*.*05, after correction for multiple comparisons at the cluster level (height threshold *p <* 0*.*01), were considered significant. A similar analysis strategy was used to examine possible association of STA schizotypy scores with BOLD signal in a separate model (i.e., a multiple regression model with STA schizotypy, EPQ-R Neuroticism and PL scores).

# **RESULTS**

### **BEHAVIORAL MEASURES**

The sample characteristics are described in **Table 1**. Intercorrelations between various personality and PL measures are presented in **Table 2**.

There was a highly significant effect of Trial Type (*F* = 29*.*84, *df* = 1*,* 100, *p <* 0*.*001) demonstrating strong PL over the entire session (i.e., shorter RTs on pattern relative to random trials; **Figure 2**). The Block main effect and Trial Type × Block interaction effect were not significant (*p*-values *>* 0.20). The data (**Figure 2**) obtained during the first block (30-s OFF and 30 s ON) of trials suggest that our participants were able to gain from the practice session they had prior to entering the scanner as they showed evidence of learning in the very first block of trials.

The two putative measures of psychosis-prone personality, the EPQ-R P and STA schizotypy scores, correlated significantly positively with each other. The STA schizotypy scores also correlated significantly positively with the EPQ-R Neuroticism scores (**Table 2**). None of the EPQ-R dimensions or STA schizotypy


*EPQ-R, Eysenck Personality Questionnaire-Revised; STA, Schizotypal Personality Scale.*



*EPQ-R, Eysenck Personality Questionnaire-Revised; STA, Schizotypal Personality Scale.*

scores correlated significantly with PL scores. However, the relationship between STA schizotypy and PL became significant in the expected direction (i.e., positive) when we controlled for EPQ-R Neuroticism scores (partial correlation = 0.366, 1-tailed *p* = 0*.*036) (**Figure 3**).

# **FUNCTIONAL MRI**

#### *Group activation*

The generic activation across all participants in association with PL is shown in **Figure 4**. Areas of stronger BOLD signal during PL than control blocks included a large cluster (number of contiguous voxels = 7558; FWE-corrected *p* = 0*.*001) with

calculated as the difference in reaction time between blocks of random trials and blocks of pattern trials. The schizotypy score are derived from the STA.

peak in the inferior frontal gyrus [BA9; (*x*, *y*, *x*) 40, 6, 30; voxel *T* = 4*.*19], subpeaks in the anterior cingulate (BA24; 4, 6, 28; *T* = 4*.*16), putamen (28, 16, 4; voxel *T* = 3*.*65 and −16, 8, 6; *T* = 3*.*34), middle frontal gyrus (BA6; 46, 6, 48; *T* = 3*.*39), and extending to the caudate (bilateral) and insula (left) (**Figure 4**).

# *Psychosis-proneness and brain activity*

The EPQ-R P scores correlated significantly positively with activity during PL in three clusters: (1) the right transverse temporal gyrus extending to the putamen, caudate, thalamus and insula; (2) the inferior frontal and precentral gyri; and (3) middle frontal gyrus extending to the precentral gyrus and anterior cingulate (**Table 3**, **Figure 5**).

STA schizotypy score correlated significantly positively with activity in only one cluster (number of contiguous voxels = 1450; FWE-corrected cluster *p* = 0*.*005) located in the right middle temporal gyrus (BA21; peak: 42, −6, −20; *T* = 4*.*36; sub-peaks: BA21; 54, −46, 2; *T* = 3*.*91; BA22; 62, −18, −12; *T* = 3*.*90). The extent of this cluster is displayed in **Figure 6**.

No brain area showed a significant negative correlation with EPQ-R P or STA schizotypy scores, and no area correlated significantly positively or negatively with the EPQ-R Extraversion or Neuroticism scores.

# **DISCUSSION**

This study replicates the oft-repeated observation of PL in a motor sequence learning task (e.g., Corr et al., 1997). In comparison to blocks of trials where the target moved in a random pattern, there was a significant reduction (i.e., the main effect of Trial Type) in reaction time in blocks where the target moved in


**Table 3 | Brain regions demonstrating positive associations with EPQ-R Psychoticism.**

*BA, Brodmann Area; MNI, Montreal Neurological Institute; FWE, Family Wise Error.*

a repetitive, and thus predictable, sequence. This pattern suggests that the motor sequence was (implicitly) learnt and thus validates the paradigm for use in the current study in order to explore the association between brain function during PL and schizotypy. However, in this study as well as in healthy participants of our previous fMRI study with this paradigm (Kumari et al., 2002), there was evidence of learning in the very first block of trials, likely resulting from the practice session prior to fMRI experiment, and, as a result, no significant Block × Trial Type interaction. This study, therefore, is likely to have identified brain regions associated with recall, and not acquisition, of implicit knowledge about the sequences.

The pattern of brain activation that was observed during PL at the group level is mostly consistent with previous fMRI studies using this task (Kumari et al., 2002, 2008b). We observed increased BOLD signal during blocks involving patterned sequences than during blocks involving random sequences in the inferior frontal gyrus, anterior cingulate, middle frontal gyrus, insula, and striatum. The involvement of dopaminergic regions such as the striatum is to be expected given the previously described link between DA (a prominent neurotransmitter in the striatum), Parkinson's disease (a neurological condition involving loss of nigrostriatal dopaminergic neurons) and PL impairment (Foerde and Shohamy, 2011). As described in the introduction, there is also evidence of dopaminergic influences on PL from pharmacological studies in healthy individuals and patients with a diagnosis of schizophrenia (Green et al., 1997; Kumari et al., 1997, 2002; Kern et al., 1998).

At the behavioral level there were no significant first-order correlations between the amount of PL and different measures of psychosis-prone personality factors. However, when covarying for neuroticism, the correlation between positive schizotypy STA and PL became significant in the expected direction, indicating more PL in higher schizotypy. Neuroticism is known to be correlated with measures of schizotypy (e.g., Eysenck and Barrett, 1993; Lipp et al., 1994) and increased levels of neuroticism are observed in patients with a diagnosis of schizophrenia (Berenbaum and Fujita, 1994; Catts et al., 2000). A recent twin study showed that the overlap between neuroticism and positive schizotypy is largely of genetic origin (Macare et al., 2012).

Due to the observation of a relationship between schizotypy and neuroticism, a number of previous studies have investigated the relationship between schizotypy and cognitive performance whilst covarying for individual differences in neuroticism (Braunstein-Bercovitz, 2000; Ettinger et al., 2005; Völter et al., 2012). Significant partial correlations in those studies indicate an association of schizotypy with cognition over and above any contributions from neuroticism. Here we observed that the correlation between positive schizotypy STA and PL became significant only after including neuroticism as a covariate.

At the brain functional level there were significant positive correlations between psychosis-prone personality factors and BOLD signal in a number of areas. Higher EPQ-R P scores were associated with higher brain activity during PL in temporal cortex, striatum, thalamus, inferior frontal areas, middle frontal gyrus, and anterior cingulate. Higher STA scores on the other hand were associated with higher brain activity only in the right middle temporal gyrus.

On the basis of previous evidence of a relationship between DA and PL, we had expected that individual differences in PL would be associated primarily with differences in brain activity in dopaminergic regions in the striatum and thalamus. The current findings, at least in terms of individual differences as assessed by EPQ-R P scale, are in agreement with this expectation. Additionally, the same relationship was found between psychosisprone personality factors and clusters in the frontal lobe. It should be emphasized that the direction of the relationship between psychosis-proneness and fronto-striatal-thalamic activity was *positive* in the present study but *negative* in our previous fMRI studies that involved tasks requiring involuntary (prepulse inhibition; Kumari et al., 2008a) or voluntary inhibition (antisaccade; Aichert et al., 2012). Taken together such results, although not surprising given that increased DA activity is known to disrupt performance on inhibitory tasks such as prepulse inhibition (Swerdlow et al., 2008) but to increase PL (Kumari et al., 1997), indicate that the direction of the association of psychosis-prone personality factors with frontostriato-thalamic activity is situation specific (i.e., negative during inhibitory tasks; positive during automatic tasks facilitated by practice) rather than static. There is further support for this position from other studies (e.g., Szymura et al., 2007) showing that psychosis-proneness, as assessed with EPQ-R P, facilitates performance of simple tasks but leads to impairment on complex ones requiring flexibility and effortful control (review; Corr, 2010).

Our findings in relation to correlates of the STA schizotypy and EPQ-R P scales were not identical at either the behavioral or neural levels. Although the EPQ-R P and STA schizotypy scales had a modestly positive association (*r* = 0*.*431) with each other, only the STA schizotypy scale had a positive correlation with EPQ-R Neuroticism (*r* = 0*.*471). Only the EPQ-R P, and not the STA schizotypy, had an association with activity in the basal ganglia and thalamus (with or without the EPQ-R Neuroticism in the model). Given the pattern of effects we observed in this study, it seems sensible to conceptualize the EPQ-R P and STA schizotypy scales as measuring related but distinct constructs (Pickering, 2004; Corr, 2010).

Another aspect of the present study deserving discussion is that, unlike the findings of a recent fMRI study (Corlett and Fletcher, 2012) showing a correlation between non-clinical schizotypal experiences and aberrant frontal and striatal prediction error signal, consistent with the deficits found in early psychosis (Corlett et al., 2007), the behavioral/brain effects we observed in relation to psychosis-proneness in this study are not in line with what we found earlier in unmedicated first episode patients (Kumari et al., 2008b). Although in our previous study (Kumari et al., 2008b) we had found somewhat faster PL (i.e., greater PL in earlier blocks) in unmedicated first episode patients than the healthy group, this had resulted from longer RTs to random trials, rather than faster RTs to pattern trials. As we discussed previously (Kumari et al., 2008b), this might have reflected a conscious or unconscious search on the part of patients for, or imagining, "specific patterns" in the random trials condition driven by the presence of paranoia and other positive symptoms, as can be inferred from some of the neurobiological models of positive symptoms (e.g., Kapur, 2003; Corlett et al., 2010). The pattern of results we find in relation to psychosisproneness in this study (further confirmed by absence of any correlation between reactions times to random trials and schizotypy measures; data not shown) is however consistent with what we observed in healthy people following acute administration of 5 mg d-amphetamine (Kumari et al., 1997), i.e., faster RTs to pattern trials. It is also worth pointing out that while the effects of DA-blocking antipsychotics in schizophrenia patients seem fairly consistent (resulting in poor PL), this is not the case for a relationship between symptoms of schizophrenia and PL (e.g., Exner et al., 2006; Reiss et al., 2006). Thus, whilst our data suggest a link between DA, given previous data showing strong sensitivity of SRT to dopaminergic manipulations, and schizotypy/broader "psychosis-proneness," consistent with the DA hypothesis of schizophrenia (Gray et al., 1991; Carpenter and Koenig, 2008; Howes and Kapur, 2009; Howes et al., 2012), they cannot be viewed as supporting the continuum between schizotypy and psychosis at the symptom levels. The study did not directly address the continuum between psychosis-prone personality and psychosis at particular symptom level, e.g., delusional beliefs.

An important feature of this study was the investigation of specificity of the observed effects. As we had collected data not only on psychosis-prone personality factors (psychoticism and STA) but also on neuroticism and extraversion, we were able to explore whether those traits were associated with PL or brain function. No such correlations were found (but see above for the role of neuroticism in the relationship between STA and PL), suggesting some specificity of the current findings across different personality traits. Future work will be required to further probe specificity within the spectrum of clinical phenotypes and their subclinical expressions, for example given evidence of altered motor sequence learning in attention deficit/hyperactivity disorder (ADHD) (Adi-Japha et al., 2011; Prehn-Kristensen et al., 2011) and autism (Mostofsky et al., 2000).

Limitations of the study include the relatively modest sample size. Therefore, replication of the design in independent, larger samples will be important in order to validate the current findings. A further limitation concerns the fact that this study is unable to address the continuum between psychosis-proneness personality and schizophrenia at specific symptom/dimension level. We did not include a separate specific measure for negative schizotypy which is measured with scales such as the Physical Anhedonia or Social Anhedonia scales (Chapman et al., 1976) or the Introvertive Anhedonia scale from the Oxford Liverpool Inventory of Feelings and Experiences (Mason et al., 1995). Negative schizotypy shows distinct cognitive, affective, genetic, and neural correlates from positive schizotypy (Suhr and Spitznagel, 2001; Ettinger et al., 2005; Holahan and O'Driscoll, 2005; Lewandowski et al., 2006; Soliman et al., 2008; Macare et al., 2012). Accordingly, it would have been of interest to investigate whether negative and positive schizotypy also differentially relate to the brain functional response during PL. Finally, it should of course be mentioned that this study did not include direct examination of DA function in relation to psychosis-proneness personality factors.

In conclusion, the present study shows that the well replicated PL effect in a motor sequence learning task is associated with increased activation in frontal and striatal areas. Individual differences in psychosis-prone personality factors are found to relate both to the amount of PL (when neuroticism is considered as covariate) and the brain functional response in frontal, striatal and thalamic brain areas. These data are interpreted as being supportive of a dopaminergic involvement in psychosisproneness, at least when measured using EPQ-R P scale.

## **REFERENCES**


# **ACKNOWLEDGMENTS**

Ulrich Ettinger acknowledges financial support from the Deutsche Forschungsgemeinschaft (DFG grant Et 31/2-1). Veena Kumari acknowledges financial support from the Wellcome Trust, UK (067427).

lobes in the learning of a visuomotor sequence. *Brain Cogn.* 34, 218–245.


and schizotypy. *Schizophr. Res.* 54, 95–103.


learning in treatment-resistant schizophrenic patients. *Biol. Psychiatry* 44, 726–732.


in *Latent Inhibition: Cognition, Neuroscience, and Applications to Schizophrenia,* eds R. Lubow and I. Weiner (New York, NY: Cambridge University Press), 419–447.


contribution to higher function. *Arch. Neurol.* 48, 1178–1187.


attentional flexibility. *Pers. Individ. Dif.* 43, 2033–2046.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 15 December 2012; accepted: 25 March 2013; published online: 15 April 2013.*

*Citation: Ettinger U, Corr PJ, Mofidi A, Williams SCR and Kumari V (2013) Dopaminergic basis of the psychosisprone personality investigated with functional magnetic resonance imaging of procedural learning. Front. Hum. Neurosci. 7:130. doi: 10.3389/fnhum. 2013.00130*

*Copyright © 2013 Ettinger, Corr, Mofidi, Williams and Kumari. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

# More creative through positive mood? Not everyone!

# *S. Akbari Chermahini † and Bernhard Hommel\**

*Institute for Psychological Research and Leiden Institute for Brain and Cognition, Leiden University, Leiden, Netherlands*

#### *Edited by:*

*Jan Wacker, Philipps-Universität Marburg, Germany*

#### *Reviewed by:*

*Ian Tharp, University of Greenwich, UK Darya Zabelina, Northwestern University, USA*

#### *\*Correspondence:*

*Bernhard Hommel, Department of Psychology, Cognitive Psychology Unit, Leiden University, Wassenaarseweg 52, 2333 AK Leiden, Netherlands. e-mail: hommel@fsw.leidenuniv.nl*

#### *†Present address:*

*Soghra A. Chermahini, Faculty of Human Science, Department of Psychology, Arak University, Arak, Islamic Republic of Iran.*

It is commonly assumed that positive mood improves human creativity and that the neurotransmitter dopamine might mediate this association. However, given the non-linear relation between dopamine and flexibility in divergent thinking (Akbari Chermahini and Hommel, 2010), the impact of mood on divergent kinds of creativity might depend on a given individual's tonic dopamine level. We tested this possibility in adults by assessing mood, performance in a divergent thinking task [the Alternate Uses Task (AUT)], and eye blink rates (EBRs), a well-established clinical marker of the individual dopamine level, before and after positive mood or negative mood induction. As expected, the association between flexibility in divergent thinking performance and EBR followed an inverted U-shape function (with best performance for medium levels), positive mood induction raised EBRs and only individuals with below-median EBRs, but not those with above-median EBRs, benefited from positive mood. These observations provide support for dopamine-based approaches to the impact of mood on creativity and challenge the generality of the widely held view that positive mood facilitates creativity.

**Keywords: divergent thinking, creativity, dopamine, emotion, eye blink rate**

Creativity is arguably the most potent human resource both for the advancement of mankind in general and people's individual progress and success in daily life in particular. And yet, the cognitive and neural mechanisms underlying creative behavior are poorly understood. Researchers agree that at least some forms of creativity vary with mood and two recent meta-analyses have concluded that performance in tasks tapping divergent (brainstormlike) thinking can be reliably improved by inducing positive mood (Baas et al., 2008; Davis, 2009). This conclusion fits with earlier considerations of Isen (1987), who claimed that positive affect (PA) impacts cognitive processing by (1) increasing the number of cognitive elements available for association; (2) defocusing attention so to increase the breadth of those elements treated as relevant to the problem; and (3) increasing cognitive flexibility.

Exactly how positive mood manages to improve creativity is not clear yet, but in approaches that tackle this issue the neurotransmitter dopamine (possibly in concert with other neurotransmitter systems: Cools et al., 2008) plays a major role. Notably, Ashby et al. (1999) have pointed out that phasic changes in dopamine levels, mood changes, and changes in creativity (especially in cognitive flexibility) may be strongly interrelated. Their approach is inspired by insights into the neurobiology of reward, the encounter of which has been shown to induce both PA and phasic increases of dopamine levels (e.g., Beninger, 1991; Bozarth, 1991; Phillips et al., 1992; Schultz, 1992). Accordingly, Ashby and colleagues (1999) suggest that improved mood states are accompanied by phasic increases in dopaminergic supply provided by frontal and striatal pathways. These phasic increases might facilitate switching from one task set or item to another, thereby increasing cognitive flexibility in creativity tasks. This scenario is consistent with results from neural-network modeling (Cohen and Servan-Schreiber, 1992; Ashby et al., 1999) and the observation that divergent thinking performance interacts with individual differences in the DRD2 TAQ IA gene—which affects receptor density in the striatal dopaminergic pathway (Reuter et al., 2006). Moreover, the personality trait of "seek," which has been claimed to rely on dopaminergic pathways (Panksepp, 1998), has been reported to be positively related to creativity (Reuter et al., 2005).

To assess the connection between creativity and dopamine, Akbari Chermahini and Hommel (2010) related individual performance in a divergent thinking task to spontaneous eye-blink rates (EBRs), an indirect but well-established clinical marker of the individual dopamine level (Karson, 1983; Blin et al., 1990; Kleven and Koek, 1996). Flexibility in divergent thinking (or cognitive flexibility for short) did in fact covary with EBR but the function relating these two measures was non-linear and followed an inverted U-shape1 . As indicated in **Figure 1**, an idealized function modeled after Akbari Chermahini and Hommel's findings, individuals with medium EBR were performing better than individuals with low or higher rates did (individuals with particularly

<sup>1</sup>Akbari Chermahini and Hommel (2010) assessed divergent thinking by means of Guilford (1967) Alternate Uses Task, which was also used in the present study. Responses in this task are commonly scored according to flexibility, originality, fluency, and elaboration, to quantify the number of different categories used, the uniqueness and number of responses, and the amount of detail, respectively. As Akbari Chermahini and Hommel (2010) found systematic, reliable findings for flexibility only (an observation that we replicated in the present study), and because most theoretical claims relate to flexibility (e.g., Ashby et al., 1999; Hommel, 2012), the present study was focusing on this variable.

high rates were not tested in this study,2 ). If we take EBRs as a marker of the current dopamine level (presumably integrating tonic and phasic levels), this has a number of rather serious implications that we set out to test in the present study.

First, it suggests that EBR can be used to monitor the impact of mood manipulations. If it is the case that inducing positive mood increases the current dopamine level and if EBR indeed reflects this level, we should be able to demonstrate that inducing positive mood leads to an increase in EBR. Whether we should expect the induction of negative mood to decrease EBR was less clear. On the one hand, there is evidence that dopamine neurons are activated by events that are more rewarding than predicted and depressed by events that less rewarding than predicted (Schultz, 1992), suggesting that positive and negative mood might increase and decrease EBR in a symmetric fashion. On the other hand, however, numerous findings suggest that negative mood is not just the opposite of positive mood (e.g., Baas et al., 2008), which fits with the increasing evidence that while positive mood is heavily affected by dopamine, negative mood is more strongly linked to serotonin (e.g., Cools et al., 2008; Dayan and Huys, 2008). Considering this possibility, EBR and cognitive flexibility might be more impacted by positive than by negative mood.

Second, if we take both mood and EBR changes as reflections of phasic dopaminergic changes, the *amount* of mood and EBR changes should be systematically related to the *degree* of change in cognitive flexibility. That is, elevated mood and increased EBRs should be associated with improved flexibility, whereas negativegoing mood might rather be associated with decreased EBRs and impaired flexibility.

Third, Akbari Chermahini and Hommel (2010) observation that cognitive flexibility relates to EBR in an inverted U-shaped fashion suggests that the impact of increasing (or decreasing) the individual dopamine level on flexibility should depend on the basic level of the corresponding individual. Consider, for instance, an individual with a relatively low level of dopamine, as the hypothetical person "X" in **Figure 1**. In view of Akbari Chermahini and Hommel's findings, this individual would be expected to perform rather poorly with respect to cognitive flexibility. Inducing positive mood would be expected to increase the dopamine level by some hypothetical amount and thereby move this individual more toward the central zone of the performance function, which is associated with the best performance. Hence, positive mood induction should be beneficial for individuals with low EBR. Positive mood should also be beneficial for individuals with higher EBRs, as long as the rate falls on the ascending flank of the function. Accordingly, the hypothetical person "Y" would show a benefit, which however would be smaller than that shown by "X." However, for individuals with even higher EBR, such as person "Z" in **Figure 1**, positive mood should no longer improve flexibility but have no effect or even impair performance. Hence, we would expect that people with a low pre-experimental EBR would be expected to benefit from positive mood more than people with medium or relatively high pre-experimental EBRs do.

We assessed these three hypotheses in the following way: Participants were first tested on general, pre-experimental mood (for both their general and their current mood state), on performance in divergent thinking, and on their pre-experimental EBR. Then two subgroups of participants underwent a positive mood and negative mood induction, respectively, before again being tested on mood, divergent thinking, and EBR.

#### **METHODS**

Eighty-one native Dutch students of Leiden University volunteered in exchange for course credit or pay. The study consisted of three phases, which together took 45 min to complete. First, all participants filled out an inventory assessing their general mood Positive and Negative Affect (NA) Scales (PANAS) and a mood inventory assessing their current mood state (MI1), before performing a divergent-creativity task (Alternate Uses Task: AUT1); finally, their spontaneous EBR were measured (EBR1). In the second phase, 43 participants received a positive mood induction while 38 participants received a negative mood induction. In the third phase, another version of the mood inventory (MI2) was filled out, EBR2 was measured, and another version of the divergent thinking task was performed (AUT2). The order of the two versions of the mood inventory and the divergent thinking task was counter-balanced across participants. EBR2 was measured after mood induction while subject continually was thinking about either happy or sad memory.

<sup>2</sup>Informal observations from our lab revealed that people with very high EBR levels are rare in our student population and more often than not report to have family members with schizophrenia. This fits with the distribution of EBRs in Akbari Chermahini and Hommel (2010) and in the present study, where the EBRs of the majority of participants falls on the left, ascending part of the inverted U-shaped function relating EBR to divergent thinking. If we in the following distinguish between below- and above-median EBRs, it should therefore be kept in mind that even above-median EBRs in the present study are actually representing medium EBRs in the general population. In other words, even though we will compare individuals with low vs. high EBRs, the present study actually compares individuals with low vs. medium EBRs.

#### **POSITIVE AND NEGATIVE AFFECT SCALES (PANAS)**

The PANAS (Watson et al., 1988) is 20-items self-report mood scale that measures general ("how do you feel generally?") PA and NA. It comprises of 10 positive and 10 negative adjectives rated on a Likert scale from 1 (very little or not at all) to 5 (very or extremely). We used a Dutch version of the scale with high internal consistencies for the PA (Cronbach's alpha = 0.84) and the NA (Cronbach's alpha = 0.80) subscale (c.f., Hill et al., 2005).

#### **MOOD INVENTORY (MI)**

Instead of presenting the PANAS repeatedly (which would have invited memory biases), we used two Dutch versions of a mood inventory (developed by Phillips et al., 2002, and similar to the scale of Isen et al., 1987) to assess current mood in the first and the third phase of the experiment. Three of the five items of this inventory assess the hedonic quality of affect (Phillips et al., 2002). One version (Cronbach's alpha = 0.75) used the following adjective pairs (Dutch words are given in parentheses) to measure valence: happy–sad (*blij-verdrietig*), peaceful–anxious (*vredig-angstig*), and carefree–serious (*zorgeloos-serieus*). The second version (Cronbach's alpha = 0.85) used the pairs: positive– negative (*positief-negatief*), calm–uptight (*kalm-opgewonden*), and bright–dispirited (*helder-serieus*). Positive and negative words were presented on the left and right side of a page, respectively. Nine-point Likert scales separated the words of each pair and participants were asked to rate their current mood state (following Phillips et al., 2002). For analytical purposes the mood scores were reversed and then totaled, so that higher scores indicate more positive mood.

#### **ALTERNATE USES TASK (AUT)**

Following Guilford (1967), participants were asked to write down as many possible uses for a common household item as they can within 5 min. Two different items were used: *cup* and *pencil*. The order of the two items was balanced across participants, so that half of the participants received the *cup* item before and *pencil* after mood induction, while the other half received the opposite sequence. Responses were scored with respect to flexibility, originality, fluency, and elaboration (Guilford, 1967). However, given that flexibility is most strongly and reliably related to EBR measures (Akbari Chermahini and Hommel, 2010), we focused on the flexibility score1, which is derived from the number of different categories being used for each item.

#### **EYE BLINK RATE (EBR)**

A BioSemi ActiveTwo system (BioSemi Inc., Amsterdam) was used to record the EBR. We recorded with two horizontal (one left, one right) and two vertical (one upper, one lower of left eye) Ag-AgCl electrodes, for 6 min eyes-open segments under resting conditions. The vertical electrooculogram (EOG), which recorded the voltage difference between two electrodes placed above and below the left eye, was used to detect eye blinks. The horizontal EOG, which recorded the voltage difference between electrodes placed lateral to the external canthi, was used to measure horizontal eye movements. As spontaneous EBR is stable during daytime but increases in the evening (around 8:30 pm, see Babarto et al., 2000), we never registered after 5 pm. We also asked participants to avoid smoking before the recording. Participants were comfortably sitting in front of a blank poster with a cross in the center, located about 1 m from the participant. The participant was alone in the room and asked to look at the cross in a relaxed state to record EBR1. After mood induction (either positive or negative) EBR2 was recorded. The individual EBR was calculated by dividing the total number of eye blinks during the 6 min measurement interval by 6.

#### **MOOD INDUCTION**

We used the common mental-imagination procedure (e.g., Strack et al., 1985; Bodenhausen et al., 1994; Phillips et al., 2002; DeSteno et al., 2004; Baas et al., 2008) to induce positive and negative mood. Participants were asked to write down a couple of sentences about an event of their life that made them happy (in a calm, relaxed way) or sad (in a calm, non-angry way), respectively, for 5 min. Calmness was emphasized to keep the two emotional states comparable regarding activation and arousal. EBR2 was recorded right after the mood induction; participants were asked to stop writing but to keep thinking about the event during the measurement interval. The session was completed by filling in the MI2 and the AUT2.

# **RESULTS**

Before assessing our three experimental hypotheses, we tested whether the experimental groups were comparable before the different moods were induced (see *Comparability of groups*), whether the mood manipulation actually worked (see *Manipulation check*), and whether performance in the creativity task related to individual EBR like it did in the study of Akbari Chermahini and Hommel (2010) [see *Replication of* Akbari Chermahini and Hommel (2010)]. All reported *p* values are for two-tailed testing unless indicated otherwise (one-tailed tests were used for predicted correlations).

#### **COMPARABILITY OF GROUPS**

A set of independent *t*-test were conducted to check whether the two experimental groups were comparable before undergoing the mood induction. There was not any hint to any pre-experimental difference between the two groups with respect to either the positive or negative subscale of PANAS, and the hedonic valence scores computed from the MI1, nor did any of these scales correlate with EBR1, all *p*s > 0.05. **Table 1** provides the relevant information about the mood states in two experimental groups and the four subgroups. Interestingly, the lack of a correlation between EBR1 and pre-experimental mood suggests that mood does not depend on the tonic dopamine level but, if anything, on phasic changes. There was also no hint to a pre-experimental group difference with regard to pre-experimental EBR1 (*p* = 0.14) or flexibility (*p* = 0.88).

#### **MANIPULATION CHECK**

Another set of paired sample *t*-tests on the hedonic valence score in MI1 and MI2 served to check whether the mood manipulation worked. As expected, participants were significantly more happy after positive-mood induction than before (*M* = 20.95 vs. 18.11), *<sup>t</sup>*(42) <sup>=</sup> <sup>5</sup>.74, *<sup>p</sup>* <sup>&</sup>lt; <sup>0</sup>.001,η<sup>2</sup> <sup>=</sup> <sup>0</sup>.44, and significantly less happy after negative mood induction (*M* = 13.07 vs. 19.65), **Table 1 | Means and standard deviations for pre-experimental general mood states (PANAS: positive and negative scales), and current mood states (only hedonic valence score) before (MI1) and after (MI2) mood induction in the two experimental groups, and four subgroups, as a function of low vs. (relatively) high pre-experimental eye blink rate (EBR).**


*Note: PANAS-PA, PANAS positive affect subscale; PANAS-NA, PANAS negative affect subscale.*

*<sup>t</sup>*(37) <sup>=</sup> <sup>7</sup>.76, *<sup>p</sup>* <sup>&</sup>lt; <sup>0</sup>.001. <sup>η</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>.62. This suggests that the mentalimagery procedure was effective in inducing the respective mood states.

#### **REPLICATION OF AKBARI CHERMAHINI AND HOMMEL (2010)**

The relationship between flexibility in the divergent thinking task (AUT1) and EBR1 followed an inverted U-shaped function (**Figure 2**, quadratic fit = 0.36, *p* = 0.005), whereas the linear fit was poor (0.06, *p* = 0.62)—a pattern that replicates our previous observation (Akbari Chermahini and Hommel, 2010). As in our previous study, there was no significant relation between EBR and one of the other scores of divergent thinking.

#### **HYPOTHESIS 1: ARE INDUCED POSITIVE (OR NEGATIVE) MOOD CHANGES REFLECTED IN CORRESPONDING INCREASES (AND DECREASES) IN EBR?**

Paired sample *t*-tests revealed systematic changes in EBR after mood induction: As expected, the induction of positive mood led to a significant increase in EBR (*M* = 18.79 vs. 14.1), *t*(42) = 3.8, *<sup>p</sup>* <sup>&</sup>lt; <sup>0</sup>.001, <sup>η</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>.26. Negative mood induction reduced EBR numerically (*M* = 16.78 vs. 17.39) but this effect was far from significance, *<sup>t</sup>*(37) <sup>=</sup> <sup>0</sup>.64, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>.53, <sup>η</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>.01. To summarize, positive-going mood changes are systematically reflected in corresponding EBR changes, while negative-going mood changes are not.

#### **HYPOTHESIS 2: ARE POSITIVE-GOING (NEGATIVE-GOING) MOOD AND INCREASED (REDUCED) EBRs ASSOCIATED WITH INCREASED (REDUCED) FLEXIBILITY?**

Paired sample *t*-tests assessed the impact of mood induction on performance in the creativity task by comparing flexibility scores before and after the mood manipulation. As expected, the induction of positive mood enhanced flexibility (*M* = 7.1 vs. 5.7), *<sup>t</sup>*(42) <sup>=</sup> <sup>3</sup>.26, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>.002, <sup>η</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>.20. The induction of negative mood reduced flexibility numerically (*M* = 5.26 vs. 5.52), but this effect was far from significance, *<sup>t</sup>*(37) = 0.84, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>.41, <sup>η</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>.02.

**FIGURE 2 | Flexibility in the divergent thinking task as a function of spontaneous eye blink rate (EBR) per min.** Regression line for best (quadratic) fit.

Overall, the correlation between change in cognitive flexibility (AUT2-AUT1) and change in mood (MI2-MI: hedonic valence) was positive and reliable, *r* = 0.24, *p* < 0.018, onetailed. However, separate analyses revealed that the correlation was positive and pronounced in the positive mood induction group, *r* = 0.44, *p* < 0.001, one-tailed, but negative and unreliable in the negative mood induction group, *r* = −0.18, *p* = 0.28.

Correlations between change in EBR (EBR2-EBR1) and change in flexibility (AUT2-AUT1: flexibility score) showed a similar pattern. Overall, the correlation was positive and reliable, *r* = 0.19, *p* = 0.047, one-tailed. Separate analyses of the two mood induction groups showed that individuals were becoming more flexible in divergent thinking to the degree that positive mood induction increased their EBR, *r* = 0.29, *p* = 0.03, onetailed (**Figure 3**, line: P)—the pattern we expected. In contrast, however, participants in the negative mood group tended to become *more* (rather than less) creative to the degree that negative mood induction decreased their EBR, *r* = −0.23, *p* = 0.17 (**Figure 3**, line: N)—a pattern that we did not expect.

To summarize, the changes in EBR induced by positive mood induction were systematically related to changes in cognitive flexibility. Although the negative mood induction produced (negative) mood shifts of even greater magnitude, it did not cause significant changes in either EBR or cognitive flexibility. Also, changes in EBR and cognitive flexibility as well as changes in mood and cognitive flexibility were unrelated in the negative mood induction group.

#### **HYPOTHESIS 3: DO INDIVIDUALS WITH LOW PRE-EXPERIMENTAL EBR BENEFIT MORE (IN TERMS OF FLEXIBILITY) FROM POSITIVE MOOD MORE THAN INDIVIDUALS WITH HIGHER EBR DO?**

We assessed this hypothesis by categorizing participants according to their pre-experimental EBR (EBR1): participants with EBRs below the median were considered low-EBR individuals while participants with EBRs above the median were considered (relatively) high-EBR individuals (which actually represent median-EBR individuals2). As expected, and shown in **Figure 4**, the induction of positive mood improved flexibility only in low-EBR individuals (from 5.8 to 8.0 categories, *t*(21) = 3.54, *p* = 0.002, <sup>η</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>.37) but not in high-EBR participants (5.7 vs. 6.1), *<sup>t</sup>*(20) <sup>=</sup> 0.87, *p* = 0.4).

# **DISCUSSION**

The aim of the present study was to investigate the relationship between mood, flexibility in divergent thinking, and EBR—a marker of individual dopamine levels. Importantly, we were able to fully replicate the inverted U-shaped function relating flexibility to pre-experimental EBR, first reported by Akbari Chermahini and Hommel (2010), which reinforces the notion that flexibility relates to individual dopamine levels.

Our first hypothesis assumes that mood changes are reflected in corresponding changes of the EBR: positive-going mood should increase EBR while negative mood might either reduce EBR or leave it unaffected. Mood changes and EBR changes were indeed correlated and positive clearly increased EBRs; negative mood, in turn, had no reliable impact. This suggests that EBR is a sensitive measure of (some of) the neural processes underlying (positive) mood changes, presumably changes in the individual dopamine level. Even though the functional connection between dopaminergic activity and EBR is not yet well understood and even though the exact quantitative relationship between dopamine level and EBR is not yet known, the finding of a reliable correlation between mood and EBR changes has substantial methodological implications. At the moment, not many ways to assess the current dopamine level of individuals are available: Apart from EBR, dopaminergic activity can be assessed by means of Positron Emission Tomography (Volkow et al., 1996), a rather invasive method, and the advent of high-field MRI may make it possible to scan the current activity level of dopamineproducing nuclei. Hence, in comparison, measuring EBR is a relatively simple, cheap, and non-invasive method that provides at least some insight into dopaminergic activity.

Our second hypothesis assumes that experimentally-induced changes in perceived mood and EBR predict corresponding changes in the flexibility of divergent thinking. As expected, flexibility was improved through the induction of positive mood but not reliable affected by the induction of negative mood.

Moreover, the degree of this improvement was predicted by the individual degree to which the mood induction manipulation was successful. Likewise, EBR increased through the induction of positive mood but was not reliable affected by negative mood induction. Finally, the experimentally-induced positive changes in EBR reliably predicted the increase of flexibility. If we assume that EBR reflects changes in dopaminergic activity, this suggests that cognitive flexibility is systematically affected, and perhaps even driven by phasic changes in dopamine. In any case, mood, EBR, and flexibility are related to each other exactly as predicted from dopamine-based approaches to creativity. Interestingly, the predicted relationship was found for the impact of positive mood only, but not for negative mood effects, which provides support for the notion that the functional (e.g., Baas et al., 2008) and neuromodulatory (e.g., Cools et al., 2008; Dayan and Huys, 2008) mechanisms underlying positive and negative mood are different.

According to our third hypothesis, this interrelationship in view of the fully replicated inverted U-shaped relationship between EBR and flexibility—suggests that individuals with low tonic dopamine levels might benefit more from the induction of positive mood than individuals with medium or high levels do. Indeed, mood-induced improvement of flexibility was only observed in individuals with a pre-experimentally low EBR and a presumably corresponding low tonic dopamine level. Not only does this fit with the non-linear relation between EBR in flexibility reported by Akbari Chermahini and Hommel (2010), it is also likely to explain why unreliable findings and failures to replicate are still abundant in studies on the connection between mood and creativity (Baas et al., 2008; Davis, 2009). Indeed, depending on the particular characteristics and the corresponding distribution of individual dopamine levels in a given sample, the exact same mood-related manipulation can produce significant effects or null results alike, especially if the sample size is small.

Taken together, our findings support the assumption that phasic changes in dopamine levels might provide the common currency underlying the relationship between mood and creativity, as suggested by Ashby et al. (1999) and others, and they provide the hitherto most direct evidence for the underlying interrelationship between mood, creativity, and dopamine. In particular, our findings suggest that elevated mood indeed increases the individual dopamine level and improves aspects of human creativity, as assessed by the flexibility score in our divergent thinking task. At the same time, however, we were able to demonstrate that the reliability and, presumably, the direction of the impact of mood and associated phasic dopamine changes depend on the individual tonic dopamine level (but not the basic mood level!). This questions the generality of claims regarding the positive impact of mood on creativity and calls for closer consideration of individual differences. As our findings show, better mood may or may not facilitate (and may in some cases even impair) creative performance of a given individual. Depending on the specific characteristics of a given sample, this complication may well conceal the true connections between creativity, mood, and dopaminergic activity in empirical studies and applied settings.

In the light of our findings, a number of further questions present themselves. For instance, it remains to be seen whether a comparable interrelationship exists between mood, dopamine, and convergent thinking—which apparently relates to tonic dopamine levels in different, and in some sense opposite, ways than divergent thinking does (Akbari Chermahini and Hommel, 2010). Recently we observed that engaging in convergent thinking leads to more negative mood (Akbari Chermahini and Hommel, 2011), which would fit with this expectation. Moreover, it seems important to clarify the functional relationship between mood and phasic dopaminergic changes. After all, mood is a concept that relates to a personal level of description and relates to a person having and experiencing it. In contrast, changes in dopaminergic activity refer to the systems level of description, which may or may not correspond to personal-level concepts in a one-to-one fashion. Hence, it would be important to understand whether and to what degree dopaminergic changes are the neural reflection of being in a particular mood, or whether they are mere by-products of particular mood states.

#### **ACKNOWLEDGMENTS**

We thank the two reviewers and the guest editor Jan Wacker for constructive criticism. S. Akbari Chermahini was supported by a post-graduate scholarship (Ph.D.) of the Iranian Ministry of Science, Research and Technology.

#### **REFERENCES**


*Behav. Hum. Decis. Process.* 108, 25–38.


stroop and fluency tasks. *Emotion* 2, 12–22.


validation of brief measures of positive and negative affect: the PANAS Scales. *J. Pers. Soc. Psychol.* 54, 1063–1070.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 23 June 2012; accepted: 08 November 2012; published online: 26 November 2012.*

*Citation: Akbari Chermahini S and Hommel B (2012) More creative through positive mood? Not everyone! Front. Hum. Neurosci. 6:319. doi: 10.3389/ fnhum.2012.00319*

*Copyright © 2012 Akbari Chermahini and Hommel. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

# The neuromodulator of exploration: A unifying theory of the role of dopamine in personality

# *Colin G. DeYoung\**

*Department of Psychology, University of Minnesota, Minneapolis, MN, USA*

#### *Edited by:*

*Luke D. Smillie, The University of Melbourne, Australia*

#### *Reviewed by:*

*Richard Depue, Cornell University, USA Linh C. Dang, Vanderbilt University, USA*

#### *\*Correspondence:*

*Colin G. DeYoung, Department of Psychology, 75 East River Rd., Minneapolis, MN 55455, USA e-mail: cdeyoung@umn.edu*

The neuromodulator dopamine is centrally involved in reward, approach behavior, exploration, and various aspects of cognition. Variations in dopaminergic function appear to be associated with variations in personality, but exactly which traits are influenced by dopamine remains an open question. This paper proposes a theory of the role of dopamine in personality that organizes and explains the diversity of findings, utilizing the division of the dopaminergic system into value coding and salience coding neurons (Bromberg-Martin et al., 2010). The value coding system is proposed to be related primarily to Extraversion and the salience coding system to Openness/Intellect. Global levels of dopamine influence the higher order personality factor, Plasticity, which comprises the shared variance of Extraversion and Openness/Intellect. All other traits related to dopamine are linked to Plasticity or its subtraits. The general function of dopamine is to promote exploration, by facilitating engagement with cues of specific reward (value) and cues of the reward value of information (salience). This theory constitutes an extension of the entropy model of uncertainty (EMU; Hirsh et al., 2012), enabling EMU to account for the fact that uncertainty is an innate incentive reward as well as an innate threat. The theory accounts for the association of dopamine with traits ranging from sensation and novelty seeking, to impulsivity and aggression, to achievement striving, creativity, and cognitive abilities, to the overinclusive thinking characteristic of schizotypy.

**Keywords: dopamine, personality, extraversion, openness, impulsivity, sensation seeking, depression, schizotypy**

# **DOPAMINE AS DRIVER OF EXPLORATION**

Personality neuroscience is an interdisciplinary approach to understanding mechanisms in the brain that produce relatively stable patterns of behavior, motivation, emotion, and cognition that differ among individuals (DeYoung and Gray, 2009; DeYoung, 2010b). Dopamine, a broadly acting neurotransmitter, is one of the most studied and theorized biological entities in personality neuroscience. Dopamine acts as a neuromodulator; relatively small groups of dopaminergic neurons in the midbrain extend axons through much of the frontal cortex, medial temporal lobe, and basal ganglia, where dopamine release influences the function of local neuronal populations. Despite the extensive attention paid to dopamine in personality neuroscience, no comprehensive theory exists regarding its role in personality, and it has been implicated in traits ranging from extraversion to aggression to intelligence to schizotypy.

The present article attempts to develop a unifying theory to explain dopamine's apparently diverse influences on personality, linking it to all traits that reflect variation in processes of exploration. Exploration is defined as any behavior or cognition motivated by the incentive reward value of uncertainty. (This definition will be explored in more detail below, in the section titled *Exploration, Entropy, and Cybernetics*.) Personality traits can be explained as relatively stable responses to broad classes of stimuli (Tellegen, 1981; Gray, 1982; Corr et al., 2013). Personality traits associated with dopamine, therefore, are posited to be those that reflect individual differences in incentive responses to uncertainty.

Before discussing personality traits in detail, it will be necessary to have a working model of dopaminergic function. In my attempt to develop a unifying theory of the role of dopamine in personality, I also posit a unifying theory of the function of dopamine in human information processing. One might think it naïve to assume that complex neuromodulatory systems have any core function unifying their diverse processes. Dopamine is involved in a variety of cognitive and motivational processes; dopaminergic neurons originate in multiple sites in the midbrain; and dopaminergic axons extend to multiple regions of the striatum, hippocampus, amygdala, thalamus, and cortex. Finally, there are five different dopamine receptors, in two classes (D1 and D5 are D1-type, whereas D2, D3, and D4 are D2-type), with very different distributions in the brain. Why should not this diversity have evolved to serve several independent functions, with no unifying higher-order function? The simple reason this seems unlikely is evolutionary path-dependency. If dopamine served a particular function in a phylogenetically early organism, then it would be easier for evolution to co-opt the dopaminergic system to perform additional functions if they were not incompatible with the first function, and easier still if the new functions were influenced by some broad selective pressure that also influenced the older function, which is to say, if they shared some more general function. This is because any factor that affects synthesis of dopamine, whether genetic, metabolic, or dietary/digestive, is likely to influence all aspects of dopaminergic function, no matter how diverse, as it will tend to increase or decrease available dopamine in all branches of the system. The maintenance of some overarching consistency of dopaminergic function by evolution is likely because it would avoid conflict between different branches of the system when global levels of dopamine are raised or lowered. Note that this is an argument about what is evolutionarily *likely*, not what is evolutionarily necessary; it is intended merely as preliminary evidence for the plausibility of the unifying theory that follows.

The nature of evolutionary path-dependency suggests a hierarchical organization of functions of the dopaminergic system. The different functions carried out by different branches and components of the dopaminergic system are posited, in the present theory, to have one higher-order function in common, and that function is exploration. The release of dopamine, anywhere in the dopaminergic system, increases motivation to explore and facilitates cognitive and behavioral processes useful in exploration. <sup>1</sup>

Different forms of exploration exist, however, and these are governed by different subsystems of the dopaminergic system. Further, different branches of the dopaminergic system are likely to have different effects on different brain regions (e.g., cortical vs. subcortical regions) in order to adjust neural populations in those regions to particular functional demands. Thus, the dopaminergic system can be considered to carry out multiple distinct functions, which may appear extremely diverse or even incompatible when considered at the level of specific brain structures, but which nonetheless possess a larger functional unity.

#### **EXPLORATION, ENTROPY, AND CYBERNETICS**

Before providing evidence that this functional unity reflects exploration, the definition of exploration as "any behavior or cognition motivated by the incentive reward value of uncertainty" must be explained. To explore is to transform the unknown into the known or the known into the unknown (Peterson, 1999). More formally, what is unknown is what is uncertain or unpredicted, and what is uncertain or unpredicted can be defined in terms of psychological entropy <sup>2</sup> . The theory I present here is an extension of the entropy model of uncertainty (EMU), which posits that anxiety is a response to psychological entropy (Hirsh et al., 2012). Entropy is a measure of disorder, originally developed to describe physical systems (Clausius, 1865; Boltzmann, 1877) but later generalized to all information systems (Shannon, 1948). It can be most simply defined as the number of microstates possible in a given macrostate. For example, the entropy of a shuffled deck of cards is a function of the number of possible sequences of cards in the deck; in contrast, the entropy of a new, unopened deck of cards is much lower, because decks of cards ship with their suits together in numerical order. Entropy, therefore, describes the amount of uncertainty or unpredictability in an information system. Human beings are complex information systems, and, specifically, they are cybernetic systems—that is, goal-directed, self-regulating systems (Carver and Scheier, 1998; Peterson and Flanders, 2002; Gray, 2004; Van Egeren, 2009; DeYoung, 2010c). Wiener (1961), the founder of cybernetics, noted that the entropy of a cybernetic system reflects the uncertainty of its capacity to move toward its goals at any given time.

As a cybernetic system, the human brain must encode information about (1) desired end states or goals, (2) the current state, largely comprising evaluations and representations of the world as it is relevant to those goals, and (3) a set of *operators* potentially capable of transforming the current state into the goal state; operators are skills, strategies, and plans that aid one in moving toward one's goals (Newell and Simon, 1972; DeYoung, 2010c). (All of these may be encoded both consciously and unconsciously. In psychology, the term "goal" is sometimes reserved for explicit, conscious, specific formulations of goals, but the term is used here in the broader, cybernetic sense.) The amount of uncertainty in these three cybernetic elements of a person constitutes *psychological entropy*, which reflects the number of plausible options or affordances available to the individual for representation (both perceptual and abstract) and for behavior, at any given time (Hirsh et al., 2012). In other words, the harder it is for the brain to answer the questions, "What is happening?" and "What should I do?" the higher the level of psychological entropy. Again, the brain addresses these questions both consciously and unconsciously; thus, they need not be explicitly framed in language to be a constant feature of human psychological functioning.

In explicating EMU, Hirsh et al. (2012) described anxiety as the innate response to increases in psychological entropy. Entropy is necessarily aversive to a cybernetic system because it renders the function of that system (progress toward its goals) more difficult. In other words, uncertainty is threatening. The crucial extension of EMU developed in the present theory is that, although entropy is innately aversive, it is simultaneously innately incentively rewarding. In fact, what is uncertain or unpredicted is unique as a class of stimuli in being simultaneously threatening and promising (Peterson, 1999; Peterson and Flanders, 2002). This unusual, ambivalent property of unpredicted or novel stimuli has been well-established in research on reinforcement learning (Dollard and Miller, 1950; Gray and McNaughton, 2000), and can be grasped intuitively by considering instances in which people seek out uncertainty for the excitement it provides, despite attendant risk or even the expectation that loss is more likely than gain (e.g., gambling).

In cybernetic terms, rewards are any stimuli that indicate progress toward or attainment of a goal, whereas punishments

<sup>1</sup>This claim may raise a red flag for those familiar with the conceptual distinction between *exploration* and *exploitation* (e.g., Frank et al., 2009). In the section *Exploration: Motivation and Emotion Associated with Dopamine*, I argue that exploratory processes, facilitated by dopamine, occur during behavior typically described as "exploitation."

<sup>2</sup>In the decision-making literature, uncertainty is sometimes distinguished from ambiguity, where uncertainty describes any outcome with a known probability less than 100% and ambiguity describes events in which the exact probability of a given outcome is unknown. In the present work, I do not distinguish uncertainty from ambiguity; situations in which probabilities are unknown are more uncertain than situations in which probabilities are known. Further, from the perspective of psychological entropy, a situation can contain observable uncertainty or ambiguity that is deemed neutral or irrelevant and is, therefore, *not* uncertain from the perspective of the cybernetic system because it is predicted. For example, one might observe that a particular event of no consequence takes place with uncertain frequency. That event would often be treated as minimally (if at all) unpredicted. (Consider, as an example, the variability in the noises made by one's refrigerator).

are any stimuli that disrupt progress toward a goal. These definitions are generally compatible with the behaviorist definition of rewards and punishments as stimuli that increase or decrease, respectively, the frequency of the behaviors leading up to them. Two classes of reward can be distinguished: consummatory rewards, which represent the actual attainment of a goal, and incentive rewards, also called cues of reward or promises, which indicate an increase in the probability of achieving a goal. Similarly, one can distinguish between punishments, which represent definite inability to reach a goal, and threats, or cues of punishment, which indicate a decrease in the likelihood of achieving a goal. (Note that goals can be of any level of abstraction, ranging from concrete goals like avoiding pain to abstract goals like succeeding in business, falling in love, or understanding Joyce's *Ulysses*.) Importantly, because of the nested nature of goals, in which superordinate goals are achieved through the accomplishment of more immediate subgoals, a single stimulus can be simultaneously a punishment and a threat (of further punishment) or simultaneously a consummatory reward (attainment of a subgoal) and an incentive reward (cuing increased likelihood of attaining the superordinate goal).

The reason that increases in psychological entropy are threatening is relatively obvious, whereas the reason that they are simultaneously promising is probably not. How could an increase in entropy simultaneously indicate decreased and increased likelihood of meeting one's goals? The most basic and general answer is that an unpredicted event signals uncertainty about the likelihood of meeting one's goals. This likelihood may be increased or decreased depending on the as-yet-undetermined implications of the unpredicted event. (Remember, as well, that people have multiple goals, and an unpredicted event may increase the likelihood of reaching one goal even as it decreases the likelihood of reaching another.) Another way to say this is that everything both good and bad comes initially out of the unknown, so that an unpredicted event may signal an obstacle or an opportunity (or it may simply be neutral, signaling nothing of relevance to any goal), and which of these possibilities is signaled is often not immediately evident (Peterson, 1999). What this implies is that the organism should have two competing innate responses to an unpredicted event—caution and exploration—and this is exactly what has been demonstrated (Gray and McNaughton, 2000). (Here it is important to note that "unpredicted" can refer to any aspect of an event, such that an event of interest can be unpredicted, even if it is strongly expected, as long as its timing is not perfectly predicted). Animals have evolved a suite of behaviors useful in situations in which they do not know exactly what to do or what to think—in other words, when prediction fails. Some of these behaviors are defensive, as what you don't know *can* hurt you, and some are exploratory, as an uncertain situation might always include some as yet undiscovered reward.

#### **TYPES OF UNCERTAINTY AND THE REWARD VALUE OF INFORMATION**

Unpredicted events are unified functionally by the fact that they increase psychological entropy. Nonetheless, they vary widely in the degree and manner in which they do so, and this variation helps to determine whether caution or exploration will predominate in response to any given anomaly. For many unpredicted stimuli, it will be quickly evident that they signal a specific reward or punishment (or something definitely neutral, which requires no response beyond learning the irrelevance of the stimulus). In the case of reward, psychological entropy may be increased relatively little, and the optimal response is often straightforward: First, in all cases of unpredicted reward, learning should take place, both so that the behavior that led to the reward is reinforced and so that environmental cues that may predict the reward are remembered. This learning constitutes a very basic form of cognitive exploration, transforming the unknown into the known and the unpredictable into the predictable. Second, if the unpredicted stimulus is an incentive reward rather than a consummatory reward, additional approach behavior will often be necessary to attempt to attain the consummatory reward that is signaled. The effort expended in this attempt is exploratory (and accompanied by heightened dopamine release) to the degree that attainment of the reward remains uncertain following the cue (Schultz, 2007). The one condition—a fairly common occurrence—that makes the increased entropy accompanying unexpected incentive reward more than minimal is when pursuing the reward would disrupt the pursuit of some other currently operative goal. As discussed in the next section, one division of the dopaminergic system appears to potentiate both reinforcement learning and approach behavior in response to unpredicted reward.

In the case of unpredicted stimuli that signal a specific punishment, determination of what to do is more complicated, primarily because punishments or negative goals are repulsors rather than attractors (Carver and Scheier, 1998). Attractors are goals that require a cybernetic system to minimize distance between current state and desired state. Repulsors, in contrast, require increasing the distance of the current state from the undesired state, but they do not inherently specify a concurrent attractor that could guide behavior. Thus, psychological entropy is typically increased more by unexpected punishment than by unexpected reward. As a general rule, the greater the increase in entropy, the more likely aversion is to predominate over exploration (Peterson, 1999; Gray and McNaughton, 2000). Nonetheless, the present theory argues that all uncertainty has incentive value, and unpredicted threat or punishment is the crucial test case. What is the incentive reward value of an unexpected event that clearly signals a specific punishment? Put simply, one potential consummatory reward signaled by any unpredicted event is information, which is identical to a decrease of psychological entropy. Exploration is worthwhile, even in the case of an unexpected punishment, because it may lead to an increase of information, which will allow the person to better represent the world or select behavior in future, which in turn increases the likelihood of goal attainment (and the relevant goal may simply be avoiding the punishment in question). In other words, any unpredicted event, including unpredicted threat or punishment, signals the possibility that exploration may lead to a rewarding decrease in psychological entropy. In the case of threat, cognitive exploration (searching for relevant patterns in perception and memory) is more likely to be adaptive than approach-oriented behavioral exploration because a known punishment should usually be avoided rather than approached. As discussed below, the other major division of the dopaminergic system appears to potentiate exploration in response to the incentive value of the possibility of gaining information—that is, it drives curiosity or desire for information.

Information potentially relevant for optimal adjustment of the parameters of a cybernetic system logically has reward value for that system. Empirical evidence is consistent with this assertion. Bromberg-Martin et al. (2010) cite several studies that have shown both humans and other species to have a preference for environments in which rewards, punishments, and even neutral sensory events can be predicted in advance—in other words, environments with greater available information (Badia et al., 1979; Daly, 1992; Chew and Ho, 1994; Herry et al., 2007). Further, they have shown that dopaminergic activity tracks this preference in monkeys (Bromberg-Martin and Hikosaka, 2009). This preference is adaptive for any cybernetic system that can utilize information about its environment to predict an effective course of action in any given situation. The fact that a preference exists even for neutral events to be predictable is of interest because it illustrates the fact that information is rewarding even if it is not immediately connected to a known reward or punishment. This is sensible because, in any naturalistically complex environment, what is neutral or irrelevant at present may become motivationally relevant in future. Thus, the information about the present state maintained by the cybernetic system is likely to include some potentially extraneous detail, not inherently linked to a currently operative goal. Another demonstration of the reward value of information comes from two studies of curiosity, utilizing trivia questions (Kang et al., 2009). A functional magnetic resonance imaging (fMRI) study showed that neural reward signals in the dorsal striatum, upon seeing the answer to trivia questions, were correlated with the amount of curiosity about the answer. Thus, desired information triggers the brain's reward system in much the same way that monetary, social, or food rewards do. A second study showed that people are willing to expend limited resources to acquire answers to trivia questions, much as they are to acquire more concrete rewards.

The third important category of unpredicted stimuli is also clearly linked to the reward value of information; these are stimuli in which what is signaled is itself uncertain. Whether they are threatening, promising, or neutral is ambiguous, at least initially. When such stimuli are proximal or otherwise particularly salient (e.g., a loud, unexpected noise nearby), they trigger an alerting or orienting response, which involves the involuntary direction of attention toward the stimulus, so as to aid in identifying its significance (Bromberg-Martin et al., 2010). This is a reflexive form of exploration, aimed at acquiring information (and potentially capturing fleeting reward). Obviously, unpredicted stimuli of ambiguous value are not a discrete category but exist on a continuum with the unpredicted stimuli (described above) that quickly and clearly signal specific rewards or punishments. The more ambiguous the unpredicted stimulus, the more strongly it should drive both cognitive and behavioral exploration. However, the larger its magnitude as an anomaly—that is, the more psychological entropy it generates, which is a function of which goals and representations it disrupts—the more strongly it will also drive defensive aversion responses, including caution, anxiety, fear, or even panic (Peterson, 1999; Gray and McNaughton, 2000). Severely anomalous events, which have highly uncertain meaning, constitute one of the most motivating but also the most conflict-generating, and thus stressful, classes of stimuli. They trigger massive release of neuromodulators, including both dopamine, to drive exploration, and noradrenaline (also called "norepinephrine"), to drive aversion and to constrain exploration (Robbins and Arnsten, 2009; Hirsh et al., 2012).

Although dopamine is the focus of the present theory, it will be necessary to refer occasionally to noradrenaline, which is posited by EMU as the major neuromodulator of anxiety (Hirsh et al., 2012). Noradrenaline has been described as a response to "unexpected uncertainty" that acts as an "interrupt" or "stop" signal following increases in psychological entropy (Aston-Jones and Cohen, 2005; Yu and Dayan, 2005). The release of noradrenaline in response to uncertainty leads to increased arousal and vigilance and to slowing or interruption of ongoing goal directed activity. Noradrenaline is released in both phasic and tonic firing patterns. Short phasic bursts of noradrenaline are necessary for appropriate flexibility within a task, allowing switching between different strategies and representations when the need arises (Robbins and Roberts, 2007). Tonic elevations in noradrenaline, however, appear to indicate a more persistent increase in psychological entropy and increase the likelihood that performance in a task will be slowed or interrupted, often with concurrent anxiety (Aston-Jones and Cohen, 2005; Hirsh et al., 2012). Whereas dopamine is posited to signal the incentive value of uncertainty, noradrenaline signals the aversive value of uncertainty (which, in a cybernetic framework, is equivalent to the degree that uncertainty should disrupt ongoing goal-directed action). Thus, the present theory holds that dopamine and noradrenaline act in competition in response to uncertainty, setting the balance between exploration and aversion.

#### **FUNCTIONAL NEUROANATOMY OF THE DOPAMINERGIC SYSTEM**

The dopaminergic system appears to be largely organized around two classes of incentive motivation: the incentive reward value of the possibility of specific goal attainment, and the incentive reward value of the possibility of gains in information. The theory developed here is based heavily on a model of the dopaminergic system proposed by Bromberg-Martin et al. (2010), who reviewed and synthesized a great deal of what is known about dopamine into a coherent model positing two distinct types of dopaminergic neuron, which respond to three different types of input. The two types of dopaminergic neuron they label *value coding* and *salience coding*. Value coding neurons are activated by unpredicted reward and inhibited by unpredicted aversive stimuli (including omission of expected reward). The magnitude of their activation reflects the degree to which the value of the stimulus over- or under-shoots expectations. They thus provide a signal of the value of unpredicted stimuli. Salience coding neurons are activated by unpredicted punishments as well as unpredicted rewards and thus provide an index of the salience, or degree of motivational significance, of stimuli. In addition to value and salience signals, a third type of input, consisting of *alerting signals*, excites both value coding and salience coding neurons (there do not appear to be any distinct "alerting neurons"). Alerting signals are responses to any "unexpected sensory cue that captures attention based on a rapid assessment of its potential importance" (Bromberg-Martin et al., 2010, p 821) and correspond to the third category of unpredicted stimuli discussed above, in which the value of a stimulus is initially unclear.

Where the present theory extends the theory of Bromberg-Martin et al. (2010) is in positing that both value coding and salience coding dopaminergic neurons are driven by unpredicted incentives specifically, and that all dopamine release potentiates exploration designed to attain the rewards signaled by those incentives. The hypothesis that the dopaminergic system responds to unpredicted incentive rewards is not new (e.g., Schultz et al., 1997; Depue and Collins, 1999); however, previous theories of incentive reward applied only to value coding dopaminergic neurons. According to the present theory, salience coding neurons respond to incentive cues for the value of information that can potentially be obtained following any increase in psychological entropy, regardless of whether this increase stems from an unexpected reward, an unexpected punishment, or a stimulus of unknown value. The recognition that information itself has incentive value for a cybernetic system allows the integration of both divisions of the dopaminergic system into a unified theoretical framework, in which the overarching function of the whole dopaminergic system can be identified as the potentiation of exploration. Despite this abstract functional commonality, however, the differences between the value and salience coding divisions of the dopaminergic system are extensive and crucial for understanding dopaminergic function and its role in personality. Thus, I next summarize the functional neuroanatomy of the two divisions of the dopaminergic system, as described primarily by Bromberg-Martin et al. (2010).

Dopaminergic neurons are primarily concentrated in two adjacent regions of the midbrain, the ventral tegmental area (VTA) and the substantia nigra pars compacta (SNc). (In the primate brain, dopaminergic neurons have recently been discovered that project to the thalamus from several regions other than VTA and SNc, but much less is known about these; Sánchez-González et al., 2005.) The distribution of value coding and salience coding neurons forms a gradient between VTA and SNc, with more value coding neurons in the VTA and more salience coding neurons in the SNc. Nonetheless, populations of both types of neurons are present in both areas. From the VTA and SNc, dopaminergic neurons send axons to release dopamine in many brain regions, including the basal ganglia, frontal cortex, extended amygdala, hippocampus, and hypothalamus. Bromberg-Martin et al. (2010) present evidence that value coding neurons project preferentially to the shell of the nucleus accumbens (NAcc) and the ventromedial prefrontal cortex (VMPFC), whereas salience coding neurons project preferentially to the core of the NAcc and the dorsolateral PFC (DLPFC). Both value and salience coding neurons project to the dorsal striatum (caudate and putamen). For other brain structures, it is currently unclear whether they are innervated by value or salience coding neurons. Dopamine release in the amygdala increases during stress (the presence of aversive stimuli), which is likely to indicate activity of the salience system specifically (Pezze and Feldon, 2004). The anatomical distribution of projections from value vs. salience neurons renders each type of neuron appropriate to produce different types of response to uncertainty, which can be described as different forms of exploration. This is particularly evident in relation to the neuroanatomical structures currently known to be uniquely innervated by each type of dopaminergic neuron.

Value coding neurons are described by Bromberg-Martin et al. (2010) as supporting brain systems for approaching goals, evaluating outcomes, and learning the value of actions. These processes are involved in exploration for specific rewards. The VMPFC is crucial for keeping track of the value of complex stimuli, and the shell of the NAcc is crucial to engagement of approach behavior and reinforcement of rewarded action. Additionally, in the dorsal striatum, a detailed model exists describing how the value system signals values both better and worse than predicted. Dopaminergic neurons have two primary modes of firing: a tonic mode, in which, as their default, they fire at a relatively constant, low rate, and a phasic mode, in which they fire in bursts at a much higher rate in response to specific stimuli. Value coding dopaminergic neurons have also been demonstrated to show phasic reductions in firing, below the tonic baseline, in response to outcomes that are worse than predicted (as in omission of expected reward), which enables them to code negative as well as positive values. Whereas phasic responses in the value system signal the value of unpredicted stimuli, shifts in tonic level have been hypothesized to track the long-run possibilities for reward in a given situation and to govern the vigor or energy with which an individual acts (Niv et al., 2007); in the present theory, the tonic level would correspond to the general strength of the exploratory tendency, in contrast to the exploratory responses to specific stimuli produced by phasic bursts of dopamine. Phasic increases and decreases in firing by the value system interact with two different dopamine receptor subtypes in the dorsal striatum to transform the value signal into either facilitation or suppression of exploratory approach behavior, depending on the presence of unpredicted rewards or punishments (Bromberg-Martin et al., 2010; Frank and Fossella, 2011).

Salience coding neurons are described by Bromberg-Martin et al. (2010) as supporting brain systems for orienting of attention toward motivationally significant stimuli, cognitive processing, and increasing general motivation for any relevant behavior, processes that are involved in exploration for information. The DLPFC is crucial for working memory, which involves the maintenance and manipulation of information in conscious attention and is thus central to most complex cognitive operations. Adequate dopamine in DLPFC is crucial for maintaining representations in working memory (Robbins and Arnsten, 2009). The core of the NAcc is important for overcoming the cost of effort, for enhancement of general motivation, and for some forms of cognitive flexibility (Bromberg-Martin et al., 2010). The theory presented here hinges on the premise that, whereas the value system is designed to potentiate behavioral exploration for specific rewards, the salience system is designed to potentiate cognitive exploration for information.

In considering individual differences in personality related to the dopaminergic system, I argue that the most important distinction is between value and salience coding dopaminergic neurons. Of course, the dopaminergic system contains many further complexities that are likely to have important consequences for individual differences in behavior, motivation, emotion, and cognition. These include the difference between tonic and phasic firing patterns, different receptor types, and differences in mechanisms of reuptake and synaptic clearance in different brain regions, among many others. Regarding how these differences influence specific traits, however, too little evidence exists to be of much use. At the level of resolution with which personality neuroscience has been studied to date, the difference between the value and salience coding systems appears to be sufficient to create a relatively unified account of how dopamine is involved in personality. Hopefully, future research will flesh out the framework presented here with a more detailed model of how more fine-grained differences within each of the two major divisions of the dopaminergic system influence personality.

#### **EXPLORATION: MOTIVATION AND EMOTION ASSOCIATED WITH DOPAMINE**

With a basic understanding of dopaminergic neuroanatomy, we can now turn to the question of how dopaminergic function is manifest in human behavior and experience. To say that it is manifest in exploration is likely to be misleading without a thorough understanding of the pervasive influence of the exploratory tendency. Some might argue that my use of "exploration" to describe all cognition and behavior in response to the incentive reward value of uncertainty is problematically broad, but this breadth is crucial to the theory. The assertion that all dopaminergic function is in service of exploration hinges on the observation that dopamine is not released in response to all motivationally relevant stimuli (e.g., all cues of reward), but only to those that are unpredicted or uncertain. Thus, dopamine is not simply an energizer of all behavior. Indeed, Ikemoto and Panksepp (1999, p 24) argued that "the effects of [dopamine] agonists may be better characterized as elevations in general exploration rather than general motor activity."

Following Peterson (1999), I argue that all psychological function is either engaged with the unknown (adapting to increases in psychological entropy through exploration), or it is concerned with stabilizing ongoing goal pursuit (engaging in activities aimed at preventing increases in psychological entropy)<sup>3</sup> . This observation highlights the continual necessity of exploration, as uncertainty arises frequently across a wide range of magnitudes of implication for representation and behavior. For minor uncertainties, processes of exploration are unlikely to be conscious or explicitly noted using the colloquial vocabulary of "exploration," but they are nonetheless importantly exploratory in their function. For example, many processes of learning can be considered exploration. (To equate all processes of learning with exploratory processes potentiated by dopamine would be too broad, however. Learning from punishment, for example, often involves contraction of the cybernetic system, abandoning a particular goal or subgoal and avoiding it in future. This kind of learning as pruning of the goal system is specifically punishment-related and probably facilitated by noradrenaline rather than dopamine.) Any kind of expansive rather than contractive learning, in which new associations are being formed, is exploratory and probably facilitated by dopamine (Knecht et al., 2004; Robbins and Roberts, 2007).

Another case in which some might consider my use of the term "exploration" too broad comes in contexts where exploration has been contrasted with exploitation (Cohen et al., 2007; Frank et al., 2009). These are situations in which the individual must choose between continuing to pursue a strategy with a reward value that is at least partly predictable (exploitation), or switching to some other strategy with an unknown reward value that may be greater (but may be less) than that of the current strategy (exploration). This is an important distinction, but I would argue that, even in exploitation mode, some forms of dopaminergically mediated exploration take place, unless the reward in question and its associated cues are entirely predictable, in which case no dopaminergic activity will be evoked. This exploration includes not only learning about the reward and its cues but also any effort exerted to ensure the delivery of the reward, as long as that delivery is at all uncertain. One crucial fact about the dopaminergic system is that its tonic activity increases following a cue of reward, in proportion to the degree that delivery of that reward remains uncertain, and this increase is distinct from the phasic bursts that accompany unpredicted reward or cues of reward (Schultz, 2007). This tonic elevation seems likely to occur to potentiate effort that could increase the likelihood of acquiring uncertain rewards, and, given the premise the dopamine always potentiates exploration, it supports the existence of exploratory processes during most cases of "exploitation." Finally, although the switch from exploitation mode to exploration mode may be accomplished by noradrenergic interruption of goal directed activity (Cohen et al., 2007), once the individual is in exploration mode, dopaminergic activity in both value and salience systems should increase to facilitate exploratory behavior (Frank et al., 2009).

What are the motivational states that accompany exploration? Activity in the value coding system should be accompanied by motivation (conscious or unconscious) to learn how stimuli and actions predict reward and to exert vigorous effort to reach goals. Activity in the salience coding system should be accompanied by motivation to learn what predicts reward or punishment and to engage cognitive effort to understand the correlational and causal structure of relevant stimuli. When both systems are activated together by an alerting stimulus, they should produce strong motivation to learn what just happened and to exert cognitive and motor effort to classify the unpredicted event.

Note that in the case of unexpected reward, both value and salience coding dopaminergic neurons will typically be activated. This is sensible because of the potential benefit from exploring both the possibility of acquiring the specific reward in question (signaled by value neurons) and the possibility of gaining information about the reward and its context (signaled by salience neurons). In the case of unexpected punishment,

<sup>3</sup>The neuromodulators dopamine, noradrenaline, and acetylcholine all appear to govern elements of adaptation to increases in psychological entropy (Yu and Dayan, 2005; Hirsh et al., 2012), whereas serotonin appears to govern the stabilization of goal-directed behavior that allows avoidance of increased entropy; the latter is accomplished by serotonin's suppression of disruptive impulses and facilitation of goal-congruent behavior (Gray and McNaughton, 2000, Appendix 10; Carver et al., 2008; DeYoung, 2010a,b; Spoont, 1992).

however, salience neurons will be activated, whereas value neurons will be suppressed. This should facilitate general motivation to cope with the threat and cognitive and perceptual exploration of the situation, while suppressing behavioral exploration that might be risky. The general motivation produced by the salience system may, in the presence of aversive stimuli, aid in overcoming the cost of effort to explore possible coping strategies for dealing with the threat. Overcoming the cost of effort appears to be an important function of dopamine, probably attributable to the value system as well as the salience system. This was demonstrated by a recent study showing that individual differences in dopaminergic function in the striatum and VMPFC predicted willingness to expend effort to seek reward, particularly when probability of receiving the reward was low (Treadway et al., 2012).

Dopamine produces motivation to exert effort to seek reward or information, but this does not entirely clarify what emotions accompany dopamine release. Because of its role in response to reward, dopamine has often been erroneously described as a "feel-good" chemical. There is no doubt that dopamine can make people feel good; drugs that increase dopaminergic function, like cocaine or amphetamine, are abused in part because they produce feelings of excitement, elation, and euphoria. In neuroimaging studies, degree of self-reported elation in response to cocaine was associated with dopaminergic response and levels of neural activity in the striatum (Breiter et al., 1997; Volkow et al., 1997). Increasingly, however, research shows that positive hedonic tone, the pleasure or liking felt for reward, is not directly due to dopamine, but rather to other neurotransmitters, including endogenous opiates, and a critical distinction has been made between the *wanting* that is produced by dopaminergic activity and the *liking* produced by the opioid system (Berridge, 2007). This distinction has been demonstrated extensively through pharmacological manipulation in rodents, but relevant human studies exist as well. For example, administering an opiate antagonist together with amphetamine eliminated the pleasure otherwise associated with amphetamine (Jayaram-Lindström et al., 2004).

Dopamine most purely seems to produce desire to seek reward (i.e., to achieve some goal) or to discover information. This desire is not necessarily pleasant. When working hard for a reward that is highly uncertain, for example, or when progress is frustratingly slow, the desire that is driven by dopamine may involve little pleasure in and of itself, and may even be experienced as unpleasant. This is true as well of the desire for information associated with the salience system. People sometimes describe themselves as "dying of curiosity" or "dying" to reach a particular goal it is safe to assume that the use of "dying" as a metaphor rarely signals straightforward enjoyment. To be extremely eager can be emotionally painful. Of course, the desire for specific rewards or information can be accompanied by intense pleasure when progress toward the goal is satisfactory (cf. Carver and Scheier, 1998), but that particular type of pleasure is likely to be due to the combination of dopamine release by the value coding system with release of endogenous opiates.

The role of the opioid system in pleasure does not mean that high-arousal pleasure states like elation and excitement should not be considered dopaminergic emotions, because they are probably never experienced due to opioid activity alone but rather require dopaminergic activity as well. (Opiate related pleasure without dopaminergic activity is likely to be experienced as a more relaxed pleasure, involving satisfaction or bliss, rather than elation and excitement.) However, the importance of the opioid system for pleasure does highlight the fact that dopaminergic emotions are not simply pleasant and that they reflect wanting more specifically than liking. They are likely to include a variety of emotions oriented toward future acquisition of reward or information: desire, determination, eagerness, interest, excitement, hope, curiosity (cf. Silvia, 2008). (This list is not intended to be exhaustive.) At present, we can only speculate about the difference between emotions associated specifically with the value system vs. the salience system. Emotions related to specific rewards, like elation or craving, seem likely to be driven primarily by the value system, whereas curiosity seems likely to be driven primarily by the salience system. Surprise seems likely to be an emotion tied to the alerting signal (Bromberg-Martin et al., 2010). The full range of emotions related to dopamine should be a fruitful topic for future research.

#### *Involuntary versus voluntary encounter with the unknown*

Up to this point, increases in psychological entropy have been described primarily as the result of stimuli to which individuals are involuntarily exposed. This framing glosses over one of the most important facts about exploration, namely that it frequently entails voluntary efforts to increase psychological entropy, to put oneself in situations where one is uncertain of what to do or how to understand what is happening. This is a relatively straightforward consequence of the fact that uncertainty has innate incentive reward value, but its implications must not be overlooked. People seek incentive rewards just as they seek consummatory rewards; thus, people are motivated to seek increases in psychological entropy. Individual differences in dopaminergic function influence not only what people do when confronted with the unknown but also the degree to which they will eagerly seek out the unknown. Individual differences in exploration are evident in everything from mountain climbing to reading. Why there is some value in exploring in the presence of anomaly is obvious. What is more complicated is why there is value in unprompted exploration, the creation of additional psychological entropy even when no threat to any particular goal is evident.

A mechanism that supplies psychological entropy with reward value not only serves to encourage learning when anomaly is encountered, it also drives the organism to look for anomaly even when this is not necessary. From an evolutionary perspective, unnecessary exploration may be advantageous, despite attendant risk, because it tends to increase potentially useful knowledge about the environment, which may sooner or later facilitate either acquisition of reward or avoidance of punishment. EMU posits the evolutionary function of voluntary exploration to be a longterm decrease in entropy—that is, a more effective strategy for pursuing the goals of the organism (Hirsh et al., 2012), and my extension of EMU does not alter that assumption. However, evolution does not need to instantiate a particular goal directly, as long as the goals it does instantiate serve that function; for example, evolution does not need to instill a desire for offspring as long as it instills a desire for sex. Because of the innate incentive value of uncertainty, people desire exploration for its own sake (i.e., they treat it as a goal in itself) and engage in it even at times when exploration will not obviously further their goals. The exploration theory of dopamine posits that, although human beings are indeed "motivated to reduce the experience of uncertainty to a manageable level" (Hirsh et al., 2012, p 4), they are also motivated to increase the experience of uncertainty to an interesting level—in other words, to a level at which some previously unknown reward or information may be discovered. Thus, exploration is used not only to transform the unknown into the known, but also the known into the unknown (Peterson, 1999). The value system seems likely to drive unprompted, but potentially fruitful, behavioral exploration of the social and physical world, whereas the salience system seems likely to drive spontaneous innovation and cognitive exploration.

#### **DOPAMINE AND PERSONALITY**

With a working model of the role of dopamine in the human cybernetic system, we can now turn to personality. How do individual differences in the functioning of the dopaminergic system relate to individual differences in personality traits? Personality traits are probabilistic descriptions of the frequency and intensity with which individuals exhibit particular behavioral, motivational, emotional, and cognitive states (Fleeson, 2001; Fleeson and Gallagher, 2009; DeYoung, 2010b; Corr et al., 2013). The major goal of personality neuroscience is to identify the mechanisms that produce those states and the parameters of those mechanisms that vary to influence personality traits (DeYoung, 2010b). In the previous sections, I have elaborated on the exploratory states that are associated with dopaminergic function. In what follows, I develop a theory of the traits related to those states.

Three broad dopaminergic parameters seem likely to be centrally important for determining personality traits: (1) global levels of dopamine, determined by genetic and metabolic processes that influence availability of dopamine throughout the dopaminergic system, (2) level of activity in the value coding dopaminergic system, and (3) level of activity in the salience coding dopaminergic system. Obviously, some individual differences in behavior and experience are likely to be associated with additional parameters more fine-grained than these three, such as the density of different dopaminergic receptors in different brain structures, or the efficiency of different mechanisms of synaptic dopamine clearance. Nonetheless, the extent of available evidence is not yet conducive to compelling theory at that level of detail, and I will only occasionally speculate about such effects, when it is particularly relevant to the evidence in question.

An important premise in many theories of the biological basis of personality is that traits reflect relatively stable responses to broad classes of stimuli (Gray, 1982; Corr et al., 2013). (Note that this should alleviate any concern that personality trait constructs are inadequate to describe human behavior because they are not context sensitive. They are indeed context sensitive, but the broader the class of stimuli in question, the more contexts to which they will be relevant.) With this in mind, we can identify uncertain or unpredicted stimuli as the very broad class to which all traits influenced by dopamine are responses. Other traits (e.g., Neuroticism) may also reflect stable patterns of response to uncertainty, but they reflect different types of response (aversive or defensive responses in the case of Neuroticism). Dopaminergic traits reflect individual differences in incentive responses to uncertainty. Global level of dopamine should influence typical exploratory responses to the incentive value of all kinds of uncertainty. Activity level in the value system should influence typical exploratory responses to cues of specific reward, and activity level in the salience system should influence typical exploratory responses to cues of information.

#### **PERSONALITY STRUCTURE: DOPAMINE IN THE BIG FIVE HIERARCHY**

The core of the present theory is that activity level in the value system is reflected in *Extraversion*, activity level in the salience system is reflected in *Openness/Intellect*, and global levels of dopamine are reflected in the metatrait *Plasticity*, which represents the shared variance of Extraversion and Openness/Intellect (DeYoung, 2006). All other traits influenced by dopamine are hypothesized to be related to these three traits or one of their subtraits (although not every trait related to these three traits is presumed to be influenced by dopamine). To understand why these are the primary traits of interest requires some discussion of personality structure. The goal of the present theory is to link a theory of dopamine to what is already known about the structure of personality in general. One might instead ignore the history of research on personality structure and posit a trait of exploration, or interest, or curiosity, or engagement, and then develop a questionnaire scale specifically targeting that trait (e.g., Kashdan et al., 2004). Indeed, if the present theory is correct, such a scale would be likely to correspond well to the trait manifestation of dopaminergic function in personality, but, additionally, it should be very strongly related to Plasticity, due to the comprehensiveness of the Big Five as a taxonomy.

Extraversion and Openness/Intellect are two of the Big Five personality traits, which also include Conscientiousness, Agreeableness, and Neuroticism (John et al., 2008). The Big Five system (also known as the Five-Factor Model) was developed empirically, through factor analysis of patterns of covariance among ratings of personality using trait-descriptive adjectives taken from the lexicon (Goldberg, 1990). Very similar five-factor solutions have been found in many languages<sup>4</sup> . Importantly, the Big Five appear not only in lexical research, but also in factor analysis of many existing personality questionnaires, even when those questionnaires were not designed to measure the Big Five (Markon et al., 2005). Additionally, factors closely resembling the Big Five appear in factor analysis of symptoms of personality disorder (Krueger et al., 2012; De Fruyt et al., 2013).

The major premise of the Big Five as a taxonomy is that the same five latent factors are present in any sufficiently comprehensive collection of personality assessments. This means

<sup>4</sup>A six-factor solution may be somewhat more replicable across languages (Ashton et al., 2004), but this system is not very different from the Big Five because the major change is simply to split Agreeableness into two factors (DeYoung et al., 2007; McCrae et al., 2008; De Raad et al., 2010). At any rate, the primary traits of interest for the present theory, Extraversion and Openness/Intellect, remain essentially the same in the six-factor solution.

that five major dimensions underlie most variation in human personality, and personality neuroscience should focus on explaining the mechanisms and parameters that are responsible for the coherence of these dimensions. Extraversion, for example, represents the shared variance of diverse traits including gregariousness, assertiveness, positive emotionality, and excitement seeking. Personality neuroscience needs to explain what these traits have in common in their underlying neurobiological processes. Given that the brain controls all behavior, personality traits must proximally be produced by variation in brain function, regardless of their distal sources in genetic and environmental influences (DeYoung, 2010b). Because the brain is a single unified cybernetic system, biological theories for all specific traits should be compatible and ultimately unified. Thus, theories of specific, theoretically-derived personality traits (e.g., exploration or curiosity) should not stand alone, but should rather be integrated with theories based on the Big Five.

The other crucial fact about personality structure for the present theory is that traits are organized hierarchically (**Figure 1**). Traits near the top of the personality hierarchy represent broad regularities in psychological functioning, encompassing many different types of behavior and experience that tend to vary together. Narrower traits lower down in the hierarchy represent more limited sets of behavior and experience that tend to vary together. Important traits exist both above and below the Big Five in the personality hierarchy (Markon et al., 2005; DeYoung, 2006; DeYoung et al., 2007). Although the Big Five were originally assumed to be orthogonal and the highest level of the personality hierarchy, they have been demonstrated to have a regular pattern of intercorrelation that reveals the existence of two higher-order personality factors (Digman, 1997; DeYoung, 2006; Chang et al., 2012), and these higher-order factors or *metatraits* are also evident in genetic correlations derived from samples of twins (McCrae et al., 2008). We labeled the metatraits *Stability* (the shared variance of Conscientiousness, Agreeableness, and reversed Neuroticism) and *Plasticity* and hypothesized that they reflect the primary manifestations in personality of individual differences in serotonergic and dopaminergic function, respectively (DeYoung et al., 2002; DeYoung and Gray, 2009).

Below the Big Five in the personality trait hierarchy are two additional levels of structure. The bottom level of the hierarchy is described as containing *facets*, many narrow traits that form the constituent elements of all broader dimensions. No consensus exists as to the number and identity of the facets, and different instruments assess different collections of facets. Recently, a level of personality structure has been discovered between the many facets and the Big Five domains, appearing first in behavioral genetic research in twins, which found that two genetic factors were necessary to explain the covariance among the six facets in each Big Five domain as measured by the popular NEO Personality Inventory-Revised (NEO PI-R; Costa and McCrae, 1992b; Jang et al., 2002). If the Big Five were the next level of the personality hierarchy above the facets, only one genetic factor would be necessary for each domain. This finding was extended by a non-genetic factor analysis of 15 facet scales within each Big Five domain that found evidence for the existence of exactly two factors in each of the Big Five (DeYoung et al., 2007). These factors corresponded sufficiently closely to the previously reported genetic factors to suggest that both studies might be describing the same intermediate level of structure within the Big Five hierarchy. Traits at this level were described as *aspects*, with each of the Big Five having two aspects, and the aspect factors were characterized by correlating them with over 2000 items from the International Personality Item Pool. This procedure enabled the construction of an instrument to measure the aspects, the Big Five Aspect Scales (BFAS; DeYoung et al., 2007).

The aspect level of personality structure is important in part because it is empirically derived, whereas most lists of facets have been rationally derived. The 10 aspects of the Big Five provide a less arbitrary system than the facets for investigating personality traits below the Big Five, and they seem likely to represent the most important differentiations for discriminant validity within each of the Big Five (e.g., DeYoung et al., 2013a). As well as discussing evidence for the relation of dopamine to Extraversion, Openness/Intellect, and Plasticity, I argue that the aspect-level of the personality hierarchy is important for understanding the full extent of dopamine's influence on personality, as depicted in **Figure 1**. Crucially, traits at lower-levels of the hierarchy contain

unique genetic variance, not shared with traits at higher levels (Jang et al., 2002). Thus, dopamine may influence aspect level traits without influencing the traits above them in the hierarchy.

#### **EXTRAVERSION**

The dimension identified as Extraversion in the Big Five represents the shared variance among traits including talkativeness, sociability, leadership, dominance, activity level, positive emotionality, and excitement seeking. The various facets of Extraversion group into two related but separable aspects, *Assertiveness* and *Enthusiasm*, with Assertiveness encompassing traits like leadership, dominance, and persuasiveness, and Enthusiasm encompassing sociability or gregariousness and positive emotionality. Some traits, like talkativeness, are shared by both Assertiveness and Enthusiasm. One facet of Extraversion that does not fit neatly into either major aspect of the trait is excitement seeking, which will be discussed in the section *Impulsivity and Sensation Seeking* with related constructs like sensation seeking and novelty seeking (DeYoung et al., 2007; Quilty et al., 2013).

Extraversion is the trait most commonly linked to dopamine in the existing personality literature, and Extraversion is believed to reflect the primary manifestation in personality of sensitivity to reward (Depue and Collins, 1999; Lucas and Baird, 2004; Smillie, 2013). A number of studies have found evidence of a link between Extraversion and dopamine using pharmacological manipulation of the dopaminergic system (Depue et al., 1994; Rammsayer, 1998; Wacker and Stemmler, 2006; Wacker et al., 2006, 2013; Depue and Fu, 2013). Although Extraversion is often viewed as a social trait, it encompasses more than just social behavior, including physical activity level and positive emotion even in non-social situations. Further, its social component can be seen as the direct result of the fact that many human rewards are social; among the most potent human rewards are social status or dominance and interpersonal affiliation. Sensitivity to the reward value of status appears to be associated primarily with Assertiveness, whereas sensitivity to the reward value of affiliation appears to be associated primarily with Enthusiasm (DeYoung et al., 2013a).

In a similar vein, Depue and colleagues (Depue and Collins, 1999; Depue and Morrone-Strupinsky, 2005) have distinguished between *Agentic Extraversion* and *Affiliative Extraversion*, which correspond reasonably well to Assertiveness and Enthusiasm, respectively. However, they have tended to lump traits related to Agreeableness together with Affiliative Extraversion, which can be misleading because Enthusiasm appears to entail finding affiliation rewarding, whereas Agreeableness appears to be related to affiliation for other reasons (such as the ability to empathize). Agreeableness reflects differences in the various forms of altruistic social behavior. The relations among Extraversion and Agreeableness can be clarified by noting that these two traits define the interpersonal circumplex (IPC), a two-dimensional model widely used to describe social behavior (DeYoung et al., 2013a). The two aspects of Agreeableness are Compassion, describing empathy and concern for the feelings and desires of others, and Politeness, describing suppression of rude or aggressive behavior. Assertiveness and Compassion correspond to the vertical and horizontal axes of the IPC, and Enthusiasm and Politeness correspond to the diagonal axes at 45 and 315◦ (**Figure 2**). Because Enthusiasm and Compassion are adjacent axes of the circumplex, they are as strongly correlated with each other as with the other aspect of their respective Big Five trait, and this has led some researchers to blur the distinction between Compassion and Enthusiasm. Such blurring is likely to be problematic for personality neuroscience, given the hypothesis that Enthusiasm is related to reward sensitivity but Compassion is not (DeYoung et al., 2013a).

In previous work, we have hypothesized that Assertiveness and Enthusiasm reflect *wanting* and *liking* respectively, which would suggest that only Assertiveness should be directly related to dopaminergic function (DeYoung, 2010b; Corr et al., 2013; DeYoung et al., 2013a). This would be consistent with the hypothesis of Depue and Collins (1999) that Agentic Extraversion, specifically, is related to dopamine. This contrast is probably overly simplistic, however. Based on the emotional content associated with Enthusiasm and a study by Smillie et al. (2013), the current theory proposes that Enthusiasm reflects a combination of wanting and liking, whereas Assertiveness is a purer reflection of wanting. The most explicitly emotional items in the BFAS assessment of Enthusiasm are, "Rarely get caught up in the excitement," "Am not a very enthusiastic person," and "Show my feelings when I'm happy" (DeYoung et al., 2007). These are the sort of eager, vigorous emotional responses that suggest dopaminergic activation in response to the promise or delivery of reward. Of course, they are also suggestive of hedonic pleasure in the receipt or imagination of reward, and the present theory maintains the hypothesis that variance in Enthusiasm reflects variation in the opioid system but proposes that it is also influenced by the dopaminergic value system. This would be consistent with the finding that both Assertiveness and Enthusiasm similarly predicted high levels of activated positive affect (e.g., feeling "energetic" and "active") in response to an appetitive film clip depicting vigorous goal-directed behavior (Smillie et al., 2013). These findings suggest that both Assertiveness and Enthusiasm predict individual differences in emotional response to the kind of incentive cues that trigger dopaminergic activity in the value system. Nonetheless, because Enthusiasm is assumed to reflect liking as well as wanting, variance in Assertiveness is hypothesized to be

more strongly related to dopamine than is variance in Enthusiasm (cf. Wacker et al., 2012).

No discussion of the relation of Extraversion to dopamine could be complete without reference to the work of Jeffrey Gray, who was one of the first researchers to develop a biological personality model based on the premise that traits represent consistent individual differences in responses to different classes of stimuli (Gray, 1982). Gray developed a "conceptual nervous system" that included a Behavioral Activation or Approach System (BAS) to respond to cues of reward and a Behavioral Inhibition System (BIS) and Fight-Flight-Freeze System (FFFS) to respond to threats (Gray and McNaughton, 2000). Personality traits are proposed to result from individual difference in the sensitivity of these systems. The biological basis of the BAS was never fleshed out as thoroughly as that of the BIS and FFFS, but its core was always presumed to be the dopaminergic system and its projections to the striatum (Pickering and Gray, 1999). Panksepp (1998) has posited a similar system centered around dopaminergic function, which he labeled the SEEKING system.

Gray (1982) originally considered the trait associated with BAS sensitivity to be different from Extraversion and suggested that it could be characterized as *Impulsivity*. More recent research, however, suggests that measures of BAS sensitivity assess the same latent trait as measures of Extraversion and that impulsivity is a distinct trait (Zelenski and Larsen, 1999; Elliot and Thrash, 2002; Pickering, 2004; Smillie et al., 2006; Wacker et al., 2012). One of the most popular measures of BAS sensitivity includes three subscales, Drive, Reward Sensitivity, and Fun Seeking (Carver and White, 1994). Drive appears to be a reasonably good indicator of Assertiveness, whereas Reward Sensitivity may be more related to Enthusiasm (Quilty et al., 2013), although one study found that it loaded with Drive on an Agentic Extraversion factor (Wacker et al., 2012). Fun Seeking is similar to Excitement Seeking and will be discussed below in the section *Impulsivity and Sensation Seeking*. Total BAS sensitivity scores from this instrument have been shown to predict pharmacological responses to a dopaminergic drug (Wacker et al., 2013).

If Extraversion is the primary manifestation of reward sensitivity in personality, a major contributor to that sensitivity seems likely to be the tendency to seek and learn about possible rewards, which is driven by the value coding dopaminergic system. Most of the behaviors associated with Extraversion function as forms of exploratory behavior designed to pursue rewards. (Note that speech is an important mode of behavior in social interactions, often used to pursue rewards related to status and affiliation.) Extraversion has been shown to predict better learning under conditions of reward in reinforcement learning paradigms (Pickering, 2004; Smillie, 2013), as well as to predict facilitation of reaction times and accuracy following rewarding stimuli (Robinson et al., 2010). A recent study showed that Extraversion predicted the tendency for Pavlovian conditioning to take place when subjects were given a dopamine agonist rather than a placebo (Depue and Fu, 2013).

In addition to the pharmacological studies of dopamine mentioned above, neuroimaging studies provide evidence of the link between Extraversion and the brain systems involved in reward. Several structural MRI studies have found that Extraversion is associated with greater volume of VMPFC, a region known to be innervated by the value coding dopaminergic system and involved in coding the value of rewards (Omura et al., 2005; Rauch et al., 2005; DeYoung et al., 2010; but see Kapogiannis et al., 2012, for a failure to replicate). A few fMRI studies have shown that brain activity in response to monetary rewards or pleasant emotional stimuli is associated with Extraversion, but their samples sizes have typically been very small (*N <* 20), rendering their findings inconclusive (Canli et al., 2001, 2002; Cohen et al., 2005; Mobbs et al., 2005). Nonetheless, on the whole, a compelling body of evidence suggests that Extraversion may reflect the primary manifestation of individual differences in the value coding dopaminergic system as it interacts with other elements of the brain's reward systems. Extraversion has been described in a cybernetic context as an energizer of behavior (Van Egeren, 2009), precisely the role ascribed to tonic levels of dopamine (Niv et al., 2007). This description is congruent with the present theory, as long one specifies that it is exploratory behavior specifically that is energized by dopamine, and that behavior energized by the value coding system corresponds primarily to Extraversion, whereas behavior energized by the salience system corresponds primarily to Openness/Intellect.

#### **OPENNESS/INTELLECT**

Openness/Intellect describes the general tendency to be imaginative, curious, perceptive, creative, artistic, thoughtful, and intellectual. The psychological process unifying these traits has been identified as "cognitive exploration," with cognition conceived broadly to include both reasoning and perceptual processes (DeYoung et al., 2012; DeYoung, in press) <sup>5</sup> . The trait's compound label stems from an old debate, with some researchers favoring "Openness to Experience" and others "Intellect" (e.g., Goldberg, 1990; Costa and McCrae, 1992a). In fact, these two labels capture the two distinct (but equally important) aspects of the trait, with Intellect reflecting engagement with abstract information and ideas and Openness reflecting engagement with perceptual and sensory information (Saucier, 1992; Johnson, 1994; DeYoung et al., 2007). When I refer to "Openness/Intellect," I am referring to the Big Five dimension; when I refer to either "Intellect" or "Openness" alone, I am referring just to one subtrait within Openness/Intellect. Traits within Intellect include intelligence, perceived intelligence or intellectual confidence, and intellectual engagement, whereas traits within Openness include artistic and aesthetic interests, absorption in sensory experience, fantasy proneness, and apophenia or overinclusive pattern detection (DeYoung et al., 2012; DeYoung, in press). (The inclusion of intelligence within Intellect is controversial and will be discussed further below.) The present theory posits that variation

<sup>5</sup>Note that the reward learning associated with the dopaminergic value system, which the present theory associates primarily with Extraversion, can be considered a basic form of "cognitive exploration." However, the potentiation of exploration that would more typically be considered "cognitive," involving the search for correlational or causal patterns in perception and memory, is posited to be the function of the salience system and hence associated with Openness/Intellect.

in Openness/Intellect reflects, in part, variation in the salience coding dopaminergic system.

The evidence for involvement of dopamine in Openness/Intellect is more circumstantial than the evidence for Extraversion, with the exception of two molecular genetic studies showing associations with the *DRD4* (dopamine D4 receptor) and *COMT* genes in three samples (Harris et al., 2005; DeYoung et al., 2011). COMT (catechol-*O-*methyltransferase) is an enzyme that degrades dopamine and is important for synaptic clearance. Because D4 receptors are localized primarily in the cortex (Meador-Woodruff et al., 1996; Lahti et al., 1998), and because COMT is believed to be more influential on dopaminergic levels in the cortex than in the striatum (Tunbridge et al., 2006), these associations seem particularly likely to be related to cognitive exploration and the salience coding dopaminergic system. Nonetheless, molecular genetic studies are notoriously difficult to replicate, and the circumstantial evidence is, therefore, additionally important.

We originally hypothesized that dopamine is involved in the biological substrate of Openness/Intellect based on four lines of evidence (DeYoung et al., 2002, 2005). First, as noted above, the involvement of dopamine in curiosity and exploratory behavior is well-established. Given the centrality of curiosity to the Openness/Intellect factor, and its relation to exploratory traits like novelty seeking and sensation seeking (Costa and McCrae, 1992a; Aluja et al., 2003), the conceptual link to dopamine is obvious. Second, dopamine is involved in the mechanisms that support cognitive exploration specifically, being necessary for working memory function and also contributing to learning. Openness/Intellect is the only Big Five trait positively associated with working memory ability, and its Intellect aspect has been shown to predict neural activity in the PFC that is correlated with working memory performance (DeYoung et al., 2005, 2009). These findings suggest that variations in salience coding dopaminergic function in PFC might be partly responsible for the cognitive attributes associated with Openness/Intellect. Third, Openness/Intellect appears to be associated with reduced latent inhibition (Peterson and Carson, 2000; Peterson et al., 2002). Latent inhibition is an automatic pre-conscious process that blocks stimuli previously categorized as irrelevant from entering awareness. Dopamine appears to be the primary neuromodulator of latent inhibition, with increased dopaminergic activity producing reduced latent inhibition (Kumari et al., 1999). Finally, the correlation of Openness/Intellect with Extraversion, which reveals the metatrait Plasticity, is itself suggestive that dopamine may be one cause of their covariance, given the evidence for dopamine's involvement in Extraversion.

Highlighting the fact that the division of the dopaminergic system into salience and value coding systems is coarse, and that each system has multiple subcomponents, the salience coding dopaminergic system seems likely to play somewhat different roles in Intellect vs. Openness. Intellect rather than Openness is uniquely associated with general intelligence and working memory (DeYoung et al., 2009, 2013b; Kaufman et al., 2010) and seems likely to reflect dopamine's facilitation both of voluntary reasoning processes that rely on DLPFC and of motivation to reason about experience. Openness, in contrast, appears likely to reflect dopamine's facilitation of the detection of patterns in sensory experience (Wilkinson and Jahanshahi, 2007). One study found a double dissociation in which Intellect predicted working memory, but Openness predicted implicit learning, the automatic detection of patterns (Kaufman et al., 2010). Implicit pattern detection is likely to be modulated by dopamine's action in the striatum rather than the prefrontal cortex, and different branches of the salience system project to these two brain regions. Additionally, Openness may be particularly influenced by dopaminergic projections to the thalamus, which are likely to play an important role in controlling the flow of sensory information to the cortex and basal ganglia (Sánchez-González et al., 2005). Finally, Openness, like Enthusiasm, seems likely to be influenced by the opioid system as well as by dopamine, because aesthetic pleasure (the enjoyment of sensory patterns) is one of its key features (DeYoung, in press). On the whole, Intellect seems likely to be more strongly linked to dopamine than Openness.

#### *Intelligence*

The inclusion of intelligence within Intellect is controversial. I have made the case for it elsewhere (DeYoung, 2011, in press; DeYoung et al., 2012) and will not reiterate all the arguments here because, for the present theory, it is irrelevant whether one considers intelligence to be a facet of Intellect or a separate but related trait. In either case, the pattern is maintained that all traits influenced by variation in dopaminergic function are related to Plasticity and/or its subtraits. Intelligence has traditionally been separated from most personality traits by its method of assessment, performance tests as opposed to questionnaires. Intelligence scores are therefore more specifically an index of ability than are any scores derived from questionnaires. Nonetheless, integrating intelligence mechanistically with the rest of personality is important to further the development of a coherent neurobiological explanation of individual differences. Because the brain is a single system of interacting elements, mechanistic theories for all specific traits should be compatible and ultimately unified. One of the mechanisms that may link intellectual confidence and engagement with intellectual ability or intelligence is the function of the salience system as it facilitates working memory and explicit learning. Considerable evidence implicates working memory capacity as one of the major contributors to general intelligence (Conway et al., 2003; Gray et al., 2003), although other factors, like processing speed, and the ability to learn associations voluntarily are likely to contribute as well (Kaufman et al., 2009). Given the importance of dopamine for working memory, dopamine's link to intelligence is highly likely.

Nonetheless, the evidence directly linking dopamine to tests of intelligence is not extensive. Some of the best evidence comes from research on cognitive aging, which has been associated with the variation in the normative decline in dopamine with age. Even controlling for age, dopaminergic function assessed by positron emission tomography (PET) has been found to predict intelligence in these studies (Volkow et al., 1998; Erixon-Lindroth et al., 2005). Different components of the salience system may influence intelligence differently, with binding at D1-type receptors facilitating reasoning and binding at D2-type receptors facilitating cognitive flexibility (Wacker et al., 2012).

#### *Creativity*

Whereas the inclusion of intelligence within the general Openness/Intellect factor is controversial, the inclusion of creativity is not. The general tendency toward innovation, originality, and creativity is common to both aspects of the trait and is the facet most central to Openness/Intellect as a whole (Johnson, 1994; DeYoung, in press). Indeed, Johnson (1994) proposed *Creativity* as an alternative label for the Openness/Intellect factor. This proposal was based primarily on the relation of various trait-descriptive adjectives to the Openness/Intellect factor, but it has been amply demonstrated that Openness/Intellect is the best Big Five predictor of creativity, whether creativity is measured through performance tests in the lab or by creative achievement in real life (McCrae, 1987; Feist, 1998; Carson et al., 2005; Chamorro-Premuzic and Reichenbacher, 2008). Creativity is typically defined as the ability to generate products (abstract or material) that are simultaneously novel and useful or appropriate (Mumford, 2003; Simonton, 2008).

Creative achievement, like Openness/Intellect, is associated with reduced latent inhibition, which presumably allows the creative person to perceive possibilities that others would automatically ignore and suggests the importance of dopamine for creativity (Carson et al., 2003). More directly, both genetic and neuroimaging studies have linked dopamine to performance on creativity tests (Reuter et al., 2006; de Manzano et al., 2010). Finally, multiple studies have found that creative performance is predicted by eye-blink rate, which is a marker of dopaminergic activity that also predicts Extraversion (Depue et al., 1994; Chermahini and Hommel, 2010, 2012).

#### *Positive schizotypy or apophenia*

Schizotypy is a personality trait (more precisely, a cluster of traits) that reflects subclinical levels of symptoms of schizophreniaspectrum disorders in the general population, and it is a major liability factor for those disorders. Dopamine has long been implicated in schizophrenia, and most anti-psychotic medications are dopamine antagonists. Importantly, excess dopamine seems to be involved specifically in the psychotic, or *positive*, symptoms of schizophrenia, which include magical ideation, perceptual aberrations (e.g., hallucination), and overinclusive thinking (Howes et al., 2009, 2011). All the symptoms of positive schizotypy can be described as *apophenia*, the tendency to perceive meaningful patterns and causal connections where none in fact exist, and these symptoms are predicted by Openness (DeYoung et al., 2012; Chmielewski et al., in press). The tendency to detect covariance patterns, which is associated with Openness as well as apophenia (Kaufman et al., 2010), may lead to over-interpretation of coincidences and sensory noise as meaningful patterns. Indeed, apophenia as a trait is positively correlated with identification of meaningful patterns in noisy or random visual stimuli (Brugger et al., 1993; Blackmore and Moore, 1994). Apophenia may be caused, at least in part, by the low levels of latent inhibition that have been demonstrated repeatedly in psychosis and schizotypy (Lubow and Gewirtz, 1995; Gray et al., 2002). (Occasional failures to detect associations of latent inhibition with schizotypy may be due to the confounding of positive and negative symptoms. The latter comprise anhedonia—that is, lack of pleasure in sensory and social experience—and may actually be positively related to LI (Cohen et al., 2004), which is consistent with the association of anhedonia with dopamine, per the section *Depression and Anxiety* below.) In neuroimaging studies, schizotypy has predicted D2 receptor density and dopamine release in response to amphetamine (Woodward et al., 2011; Chen et al., 2012). Excess dopamine has been described as producing "aberrant salience" in schizophrenia-spectrum disorders (Kapur, 2003). The association of apophenia with Openness suggests that both may be influenced by level of activity in the salience system (DeYoung et al., 2012), although apophenia seems likely to be more specifically related to dopamine than is Openness more generally.

Inclusion of positive schizotypy or apophenia as a facet of Openness is nearly as controversial as inclusion of intelligence as a facet of Intellect, in part because apophenia is weakly negatively correlated with intelligence and nearly uncorrelated with questionnaire measures of Intellect. Nonetheless, we have shown that both apophenia and intelligence load positively on the general Openness/Intellect factor, and that when Openness and Intellect are separated, then apophenia loads strongly with Openness (DeYoung et al., 2012). The negative association of apophenia with intelligence suggests it could be caused in part by an imbalance of dopaminergic function in different branches of the salience system. If striatal dopamine is highly active in response to salient events, encouraging the assignment of meaning to correlational patterns, but dopamine levels in DLPFC are either too high or too low to support working memory and intelligence, this could lead to difficulty differentiating likely from unlikely patterns (cf. Howes and Kapur, 2009). (Of course, deficits in intelligence with causes entirely unrelated to dopamine could also produce apophenia in conjunction with high levels of activity in the salience coding system.) Apophenia is clearly linked to Openness and can be well-described as "openness to implausible patterns" (DeYoung et al., 2012).

In the Personality Inventory for the DSM 5 (PID-5; Krueger et al., 2012) and in the Personality Psychopathology Five model (PSY-5; Harkness et al., 1995), positive schizotypy or apophenia is labeled *Psychoticism*. The construct measured by the PID-5 and other scales assessing apophenia should not be confused with the construct measured by Eysenck's Psychoticism scale, which most personality psychologists agree was mislabeled, as it measures antisocial and impulsive behavior (sometimes called "impulsive non-conformity") rather than positive schizotypy (Goldberg and Rosolack, 1994; Pickering, 2004; Zuckerman, 2005). Some have considered impulsive non-conformity to be a facet of schizotypy, but it is distinct from the positive psychotic symptoms that are characterized by apophenia. Eysenck's Psychoticism does not appear to predict risk for schizophrenia diagnosis (Chapman et al., 1994; Vollema and van den Bosch, 1995). Studies linking Eysenck's Psychoticism to dopamine (e.g., Kumari et al., 1999) are thus most relevant to the sections *Impulsivity and Sensation Seeking* and *Aggression* below, which discuss impulsivity and aggression.

### **PLASTICITY**

Plasticity, the shared variance of Extraversion and Openness/Intellect, in a sense forms the core of the present theory. This very broad trait should be influenced by forces that alter global dopaminergic tone and thus increase or decrease activity of both the value and salience systems. For now the only evidence for this hypothesis is the evidence, described above that dopamine is involved in both Extraversion and Openness/Intellect. In future, the hypothesis that Plasticity should predict global levels of dopamine may be tested directly.

The label "Plasticity" has the potential to be confusing because the term is more often applied to brain function than to personality. Psychologists are probably most familiar with it in the context of the phrase "neural plasticity," which refers to the ability of the brain to alter many aspects of its neural architecture in response to experience. *Plasticity*, as a personality trait, is not intended to be synonymous with "neural plasticity," regardless of the degree to which neural plasticity plays a role in the exploratory processes associated with Plasticity. Similarly, *Stability*, as a personality trait, is not synonymous with "neural stability." Rather, the terms refer to the stability and plasticity of the cybernetic elements that constitute the individual psychologically (DeYoung, 2010c). Recall that the cybernetic system encompasses (1) desired end states or goals, (2) knowledge and evaluations of the current state, and (3) operators potentially capable of transforming the current state into the goal state. As a parameter of this system, the metatrait Stability is hypothesized to reflect the degree to which the individual resists disruption of ongoing goal-directed functioning by distracting impulses, maintaining stable goal-representations and relevant evaluations of the present, and selecting appropriate operators <sup>6</sup> . Plasticity is hypothesized to reflect the degree to which the cybernetic system is prone to generating new goals, new interpretations of the present state, and new strategies to pursue existing goals (this is a description of exploration in cybernetic terms). As personality traits, Stability and Plasticity reflect between-person variation in the processes that fulfill two basic needs of any cybernetic system in an environment that is not fully predictable: first, to be able to maintain the stability of its own functioning so that goals may be accomplished, and second, to be able to explore complex, changing, and unpredictable circumstances, thereby increasing the adaptive effectiveness of its goal pursuit.

Stability and Plasticity may seem conceptually opposed, but it would be more accurate to describe them as in tension. Of course, heightened Plasticity may make Stability a challenge, but without adequate adaptation enabled by Plasticity, the individual will not long remain stable in an unpredictably changing environment. Because of the nested nature of subgoals within goals, processes associated with Plasticity can generate new subgoals in the service of a higher-order goal that is being maintained by processes associated with Stability. Further, without adequate Stability, the magnitude of psychological entropy is likely to be great enough that aversion wins out over exploration, leading to reduced Plasticity. When the Big Five are measured using ratings from multiple informants, Stability and Plasticity appear to be uncorrelated (DeYoung, 2006; Chang et al., 2012). The opposite of "stability" is "instability" not "plasticity," and the opposite of "plasticity" is "rigidity" or "inflexibility" rather than "stability." A well-functioning cybernetic system must be both stable and plastic.

In short, the function associated with Plasticity is posited to be precisely that which dopamine facilitates: to explore and thus to achieve the rewards inherent in the positive potential of uncertainty. Several studies have supported predictions based on this theory. (For an effect to be considered associated with Plasticity, it should be associated with both Extraversion and Openness/Intellect with roughly similar magnitude, so that it is truly their shared variance driving the effect, rather than variance at the Big Five level.) For example, Plasticity was found to predict self-reported moral conformity negatively, based on the premise that those who conform to societal moral expectations are less likely to be exploratory or to rely on their own adaptive capacity (DeYoung et al., 2002). Plasticity was also found to positively predict Externalizing (a factor indicating the general tendency toward impulsivity, aggression, antisocial behavior, and drug use), following the premise that externalizing behavior is driven in part by motivation to explore behaviors that are socially unacceptable, and the fact (discussed below) that externalizing behaviors have been associated with dopamine (DeYoung et al., 2008). Stability also predicted conformity and Externalizing, in the opposite direction from Plasticity. In fact, Stability was the primary correlate of both of these characteristics, and the association with Plasticity was not evident unless one controlled for Stability<sup>7</sup> .

It is particularly of interest to identify behaviors that are primarily associated with Plasticity rather than Stability. The general tendency to explore may not be most purely manifested in behaviors that are most strongly associated with common colloquial meanings of "exploration," such as pursuing experiences that are extremely novel to the individual or unusual or novel in society as a whole. Such particularly dramatic forms of exploration, especially when not socially sanctioned, may be predicted not only by Plasticity, but also by low Stability, as implied by the studies of conformity and externalizing behavior mentioned above.

<sup>6</sup>Based on this description of the psychological meaning of Stability, one might expect it to be influenced by dopamine, given dopamine's role in the maintenance of the stability of goal representations in DLPFC. Dopamine in DLPFC is certainly important for the neural stability of representations in working memory (Robbins and Arnsten, 2009). However, no direct or indirect evidence of the sort cited for other traits in the present theory exists to suggest that dopamine influences the personality trait Stability. Traits from the Openness/Intellect domain are the only traits in the Big Five hierarchy that are consistently related to working memory performance (DeYoung et al., 2005, 2009). It may be that representations in working memory (even when they are well-stabilized by dopamine) are present for too short a time to be relevant to the kind of motivational stability reflected by the broad Stability trait. Only information currently in the field of conscious attention is maintained and manipulated by working memory. Additionally, the distractions suppressed in Stability are impulses related to reward or punishment and thus not identical to the cognitive distractions that must be suppressed for good working memory function.

<sup>7</sup>The path from Plasticity to Externalizing reported by DeYoung et al. (2008) was actually slightly greater than the path from Stability. However, this is likely to be a quirk of this sample and not to generalize, because externalizing behavior has typically been found to be associated considerably more strongly with Neuroticism, low Agreeableness, and low Conscientiousness than with either Extraversion or Openness/Intellect.

What then are the best specific markers of Plasticity in the general population? In one large, middle-aged, middle-class sample (DeYoung, 2010c), the personality items that specifically characterized Plasticity were dominated by content reflecting leadership, skill, and expressiveness in social situations (e.g., "Have a natural talent for influencing people," "Have a colorful and dramatic way of talking about things") with some additional items also clearly reflecting innovation and curiosity (e.g., "Am able to come up with new and different ideas," "Look forward to the opportunity to learn and grow"). In the same sample, we examined how Plasticity and Stability uniquely predicted the self-reported frequency, over the past year, of 400 behaviors (Hirsh et al., 2009). We found that Plasticity was almost universally a positive predictor of behavioral frequency, consistent with dopamine's role as a motivational energizer, and the behaviors it most strongly predicted were an intriguing collection, which included planning a party, attending a public lecture, attending a city council meeting, giving a prepared talk or public recital, writing a love letter, going dancing, and making a new friend, among others. Here we see the manifestation of a general exploratory tendency among middle-aged, middle-class Americans. (In contrast, Stability was almost universally a negative predictor of behavioral frequency, with the strongest effects on various impulsive or disruptive behaviors.) In the present theory, all of these behaviors associated with Plasticity should be among those most facilitated by increasing dopaminergic activity in both the value and salience systems simultaneously.

It should be noted that other interpretations and labels have been offered for the factor we label Plasticity. Digman (1997), who discovered the metatraits, labeled them simply *Alpha* (Stability) and *Beta* (Plasticity) and proposed that the latter reflects a tendency toward personal growth. Olson (2005, p 1692) labeled the Plasticity factor *Engagement* and argued that it reflects "the extent to which individuals actively engage their inner and outer worlds." Further, the metatraits of the Big Five resemble the two-factor solution that has been reported in lexical studies, in which the trait containing content from both Extraversion and Openness/Intellect has been labeled *Dynamism* (Saucier et al., 2013). All these interpretations seem compatible with each other. A general tendency toward exploration will lead to active engagement with novel and interesting phenomena and should produce behavior that others find dynamic and that is likely to lead to personal growth.

#### *Lack of simple structure and the relation of Plasticity to Industriousness and achievement striving*

In order to understand the full extent of the probable role of Plasticity and dopamine in personality, it is important to understand one additional thing about the personality trait hierarchy namely that it is an over-simplification. If the personality hierarchy were exactly as schematically depicted in **Figure 1**, none of traits located under Stability would be related to any of the traits located under Plasticity. However, it has long been known that personality does not have simple structure, in which each variable loads on one and only one factor (Costa and McCrae, 1992b; Hofstee et al., 1992). Attempting to fit the model depicted in **Figure 1** to data from the BFAS, using confirmatory factor analysis, will yield a poor fit because of cross-loadings at the aspect-level (e.g., Ashton et al., 2009). Many lower-level traits are related to more than one higher level trait, and this is true even across the two sides of the hierarchy defined by the metatraits. I have already alluded to one example in the section on Extraversion (also depicted in **Figure 2**): although Extraversion and Agreeableness are unrelated, their aspects are systematically related, such that Enthusiasm is positively related to Compassion, and Assertiveness is negatively related to Politeness. Examining the pattern of correlation among the 10 aspects of the Big Five, and their lack of simple structure, suggests two important points regarding Plasticity. First, the shared variance of Extraversion and Openness/Intellect (i.e., Plasticity) appears to be due primarily to the association of Assertiveness and Intellect. These two traits are correlated with each other at about *r* = 0*.*5, at least as strongly as they are with the other aspect of the Big Five trait to which each belongs (DeYoung et al., 2007). Openness is considerably more weakly associated with the two aspects of Extraversion, and Enthusiasm is considerably more weakly associated with both aspects of Openness/Intellect. Second, there are two other aspect-level traits that are strongly correlated with Assertiveness and Intellect, as well as with each other; these are the Industriousness aspect of Conscientiousness and the Withdrawal aspect of Neuroticism. The latter encompasses anxiety and depression and predicts the other traits negatively.

This cluster of traits has been detected in slightly different guises in previous personality research. First, these aspect-level traits are all related to the lexical Dynamism factor (Saucier et al., 2013). Second, an attempt to discredit the existence of the metatraits, using the BFAS, purported to show that the metatraits could be rendered unnecessary by allowing aspect traits to cross-load on other Big Five factors—in other words, by taking into account their lack of simple structure (Ashton et al., 2009). Interestingly, however, the pattern of cross-loadings created an "Extraversion" factor that had similarly strong loadings not only for Enthusiasm and Assertiveness, but also for Intellect, Industriousness, and Withdrawal. Clearly, this is no longer just an Extraversion factor but rather a broader trait. In essence, a metatrait resembling Plasticity was recreated directly from the covariance of the aspect-level scales. Finally, in the Multidimensional Personality Questionnaire (MPQ), an Achievement scale that is strongly related to Conscientiousness and Openness/Intellect in the Big Five is grouped with scales reflecting Extraversion in a higher-order *Agentic Positive Emotionality* factor (Markon et al., 2005; Tellegen and Waller, 2008). In previously unpublished analysis of the BFAS and the MPQ in the Eugene-Springfield community sample (ESCS; Goldberg, 1999; *N* = 445), the Achievement scale showed its strongest correlations with Industriousness (0.30), Assertiveness (0.32), and Intellect (0.35). (The Achievement Striving scale from the NEO PI-R shows a similar pattern of correlations with the BFAS in this sample,*r* = 0*.*56, 0.46, and 0.31, respectively—the stronger correlation with Industriousness is not surprising, as this Achievement Striving scale was engineered as a facet of Conscientiousness). Confidence, ambition, and agency seem to be at the core of manifestations of Plasticity, and they are related not only to Extraversion (particularly Assertiveness), but also to Intellect and Industriousness and to a lack of Withdrawal. (The link between Withdrawal and dopamine is discussed below in the section *Depression and Anxiety*) The present theory posits that all of these traits are influenced by dopamine.

If the shared variance of Assertiveness and Intellect represents what is most central to Plasticity, one can understand the relation of Industriousness to Plasticity as reflecting the contribution that dopaminergic drive, in both value and salience systems, makes to the motivation for sustained hard work and the accomplishment of tasks. As noted above, dopamine appears to be crucial for overcoming the cost of effort when deciding to initiate behavior aimed at reward, especially as the probability of attaining the reward declines (Treadway and Zald, 2013). Industriousness is primarily an aspect of Conscientiousness, which reflects the capacity for top-down effortful control over impulses and distractions and is probably determined largely by characteristics of the prefrontal cortex (DeYoung et al., 2010), but Industriousness appears to have an important secondary contribution from Plasticity. To the extent that Industriousness reflects the enactment of a drive to achieve (rather than just dutifully doing what one is told), dopamine is likely to be an important influence. Achievement striving specifically is, therefore, posited to be strongly influenced by dopamine. Although at present there is little direct evidence for this hypothesis, one study found MPQ Achievement to be associated with dopamine receptor density in the midbrain and NAcc in a sample diagnosed with ADHD (Volkow et al., 2010).

#### **IMPULSIVITY AND SENSATION SEEKING**

We now turn to traits related to dopamine that are negatively rather than positively related to Conscientiousness, and which are all related to Externalizing. Nonetheless, they are all positively related to Extraversion, and sometimes to Openness/Intellect as well. The terminology and exact definitions of these traits have been a source of confusion for decades, suffering from both the jingle fallacy (different traits called by the same name) and the jangle fallacy (the same trait called by different names). Perhaps the most confusion has been created by use of the word "impulsivity" to refer to a number of related but importantly distinct traits. Impulsivity-related constructs have been substantially clarified by the development of the UPPS model (Whiteside and Lynam, 2001; Smith et al., 2007), which identifies four distinct types of impulsivity: Urgency, lack of Perseverance, lack of Premeditation, and Sensation Seeking. Urgency, the tendency to act impulsively in ways that have negative consequences under conditions of emotional arousal, currently appears least relevant to dopamine; its major correlate in the Big Five hierarchy is low Stability (DeYoung, 2010a). Perseverance is essentially identical to Industriousness (discussed above), and thus the current theory would imply that lack of perseverance might stem in part from low global levels of dopamine (although it is also possible that a specific profile of dopaminergic responding in the value system to cues of immediate reward rather than cues of more distant reward could be responsible for lack of perseverance). The clearest evidence links lack of premeditation and sensation seeking to dopaminergic function.

Premeditation refers to "the tendency to think and reflect on the consequences of an act before engaging in that act" (Whiteside and Lynam, 2001, p 685). It is associated primarily with Conscientiousness, in the Big Five, but is more peripheral to that trait than is Industriousness/perseverance and appears to be associated almost as strongly (negatively) with Extraversion as with Conscientiousness (DeYoung, 2010a). Lack of premeditation reflects rapid action without consideration of possible negative consequences, which is perhaps the most common meaning of "impulsivity" in psychology. Its link to Extraversion suggests the degree to which Extraversion energizes behavior, presumably through dopaminergic mechanisms (Niv et al., 2007; Van Egeren, 2009). Individuals who tend not to premeditate are prone to act quickly on their exploratory impulses, rather than to engage in preliminary cognitive exploration of the possible consequences of those actions. Thus, lack of premeditation may reflect reduced activity in the dopaminergic salience system, at the same time that it reflects increased activity in the value system.

A negative association of salience system activity with lack of premeditation is plausible because of the negative association of intelligence with impulsivity (Kuntsi et al., 2004). Additionally, variation in the *DRD4* gene has been found to moderate the negative association between intelligence and the general Externalizing factor, of which impulsivity is a component (DeYoung et al., 2006). Differential functioning in value and salience systems might be particularly important in generating symptoms of attention-deficit/hyperactivity disorder (ADHD), which reflects problematic levels of impulsivity, in the form of both lack of premeditation (impulsivity and hyperactivity symptoms) and lack of perseverance (inattention symptoms). ADHD is most commonly treated by dopamine agonists, such as methylphenidate, and these appear to have their salutary effects in part by increasing dopamine in DLPFC—that is, in the salience system (Arnsten, 2006).

Sensation seeking reflects "willingness to take risks for the sake of excitement or novel experiences" (Zuckerman et al., 1993, p 759). Although it has often been considered a form of impulsivity and is associated with externalizing behavior in general (Krueger et al., 2007), a reasonable case can be made that sensation seeking is not necessarily impulsive. It may involve planning, perseverance, accurate assessment of risks, and steps taken to keep risk below a desired level (consider mountain climbing or hang gliding, for example). Indeed, although sensation seeking predicts frequency of behaviors like gambling and alcohol and drug use, it does not appear to predict problematic levels of engagement in those behaviors, whereas urgency and lack of premeditation do (Smith et al., 2007).

Although *Sensation Seeking*, *Novelty Seeking*, *Fun Seeking*, and *Excitement Seeking* all appear to reflect the same latent trait, some scales with these labels are broader than others. Zuckerman's (1979) Sensation Seeking Scale, for example, contains not only Thrill-and-Adventure-Seeking and Experience-Seeking subscales, but also Disinhibition and Boredom Susceptibility subscales, which have been found to reflect lack of perseverance more than sensation seeking in the UPPS system (Whiteside and Lynam, 2001). Cloninger's (1987) Novelty Seeking scale is similarly broad, containing subscales labeled Exploratory Excitability, Extravagance, Impulsiveness, and Disorderliness. The more pure measures of Sensation Seeking include the version from the UPPS scales (Whiteside and Lynam, 2001), Excitement Seeking from the NEO PI-R (Costa and McCrae, 1992b) and Fun Seeking from the BIS/BAS scales (Carver and White, 1994). Regardless of their breadth, all of these measures have in common that they are associated positively with Extraversion and negatively with Conscientiousness, though the balance is shifted more toward Extraversion in the purer scales (DeYoung and Gray, 2009; Quilty et al., 2013). As noted by Depue and Collins (1999), variation in impulsivity-related traits is likely to be the result not only of variation in the strength of impulses to approach rewards (related to Extraversion), but also of variation in the strength of top– down control systems that constrain those impulses (related to Conscientiousness).

Using PET to assess the binding potential of dopamine D2 autoreceptors in the SNc and VTA, Zald and colleagues have produced compelling evidence for the importance of increased dopaminergic function for lack of premeditation and sensation seeking. They have shown that both Cloninger's Novelty Seeking scale and the Barratt Impulsiveness Scale (which primarily assesses lack of premeditation; Whiteside and Lynam, 2001) predict reduced D2 binding in the midbrain, which in turn predicts greater dopaminergic release in the striatum in response to amphetamine (Zald et al., 2008; Buckholtz et al., 2010b). Because the D2 autoreceptors in the midbrain inhibit dopaminergic neurons, reduced binding potential translates to greater dopaminergic activity. These results are consistent with previous research associating dopaminergic function with sensation seeking and impulsivity (Zuckerman, 2005).

Whether the salience system, as well as the value system, is involved in sensation seeking seems likely to depend on exactly what type of sensation is being sought. If sensation seeking involves planning and forethought (e.g., mountain climbing, hang gliding), then it may be associated with increased activity in the salience system, whereas more spontaneous sensation seeking seems less likely to be related to salience. The effect of dopamine on behavior can either facilitate long-term goal pursuit or hinder it, depending on other factors that are likely to include not only the ability of DLPFC to maintain a stable focus on long-term goals but also differential influence of different parts of the dopaminergic system (value vs. salience, striatal vs. cortical, tonic vs. phasic). This observation may account for the fact that some Extraversion-related traits are positively related to Conscientiousness, whereas others are negatively related.

#### **AGGRESSION**

Aggression is another trait, like lack of premeditation, that might be influenced in opposite directions by the value and salience systems. Salience system deficits are suggested by the negative association of working memory and intelligence with aggression (Seguin et al., 1995; Koenen et al., 2006; DeYoung et al., 2008; DeYoung, 2011). However, more direct evidence is available for the positive association of the value system with aggression. Buckholtz et al. (2010a) found that a trait of Impulsive Antisociality (combining rebelliousness, impulsivity, aggression, and alienation) was associated with dopaminergic response to amphetamine, even after controlling for impulsivity, novelty seeking, and Extraversion (notably, this was in the same sample in which they also showed associations of dopaminergic function with novelty seeking and impulsivity). These results are reasonably congruent with animal studies linking dopamine to aggression (Seo et al., 2008), and to studies reporting high levels of dopaminergic metabolites (and low levels of serotonin metabolites) in highly aggressive populations (Soderstrom et al., 2001, 2003). Like most externalizing behaviors other than sensation seeking, aggression is probably more strongly related to serotonergic than dopaminergic function, but dopamine nonetheless seems likely to be an important secondary influence.

Aggression is an excellent indicator of the low pole of Agreeableness, and specifically of the Politeness aspect of Agreeableness that is negatively related to Assertiveness, such that they form adjacent axes of the interpersonal circumplex, as depicted in **Figure 2** (DeYoung et al., 2013b). This link to Assertiveness suggests that aggression is facilitated by activity in the value coding dopaminergic system. Assertive people may be more willing to take aggressive action to pursue rewards. One important consideration in the possible association of dopamine with trait levels of aggression is the difference between reactive and proactive aggression, which have different biological substrates (Lopez-Duran et al., 2009; Corr et al., 2013). Reactive or defensive aggression is aimed at eliminating a threat, often appears with panic, and is controlled by low-level defense systems in the brain that are inhibited by serotonin (Gray and McNaughton, 2000). Proactive or offensive aggression is aimed at acquiring resources, dominance status, or revenge and seems more likely to be influenced by dopamine. (Of course, individual acts of aggression may reflect a blend of reactive and proactive that is difficult to disentangle.) A study comparing rats bred to be either high or low in threat sensitivity found that both groups were more aggressive than normal rats, but that dopaminergic antagonists applied to the NAcc reduced aggression only in the low threat-sensitivity rats whose aggression seems likely to be offensive rather than defensive (Beiderbeck et al., 2012).

#### **DEPRESSION AND ANXIETY**

The next traits considered are those that may be negatively related to dopaminergic function in both value and salience systems. These fall within the aspect of Neuroticism labeled Withdrawal, which is one of two traits strongly linked to Plasticity that fall outside of Extraversion and Openness/Intellect in the Big Five hierarchy (the other being Industriousness). The grouping of depression and anxiety in a single trait dimension is consistent with clinical research showing that risks for diagnosis of depression and generalized anxiety disorder overlap very strongly, forming a more general factor that has been labeled "Distress" (Wright et al., 2013). In the Big Five hierarchy, Distress is equivalent to Withdrawal. (Note that, in the PID-5, a slightly different factor is labeled Withdrawal, which represents *social* withdrawal specifically, rather than anxiety and depression; De Fruyt et al., 2013.) The connection of the Withdrawal aspect of Neuroticism with low Plasticity is consistent with lexical research, in which the Dynamism factor that appears when only two factors are extracted is related to Withdrawal (Saucier et al., 2013). An absence of depressed or anxious affect appears to be importantly related to Plasticity.

Neuroticism is considered to reflect the primary manifestation in personality of sensitivity to threat and punishment. In Gray's system, Neuroticism is the result of the joint sensitivities of the BIS and the FFFS (Gray and McNaughton, 2000; Corr et al., 2013). The FFFS produces active avoidance (panic, defensive anger, and flight) in response to threats where the only motivation is avoidance. Variation in FFFS sensitivity is not hypothesized to be related to dopamine. The BIS produces passive avoidance, inhibiting behavior and increasing vigilance and arousal when there is conflict between multiple possible goals or representations—in other words, in response to increases in psychological entropy. The prototypical activator of the BIS is an approach-avoidance conflict, in which the possibility of some reward is juxtaposed with the possibility of punishment (for example, when the desire to meet a potential mate is in conflict with the fear of rejection). The BIS operates by inhibiting approach toward the goal in question. In other words, it is antagonistic to the BAS, suggesting BIS sensitivity may be negatively associated with activity in the dopaminergic system. The BAS is inhibited by the BIS in order to produce caution that can prevent encountering the danger potentially associated with the current goal (Gray and McNaughton, 2000). In the Big Five hierarchy, BIS sensitivity seems to correspond to Withdrawal (DeYoung et al., 2007; Corr et al., 2013). Gray and McNaughton (2000) subdivide the passive avoidance states associated with the BIS into anxiety and depression, based on whether the danger in question is perceived to be avoidable or unavoidable. Passive avoidance in general is a response to dangers that must be approached in order to achieve some goal. When one is anxious, approach is slowed, caution and vigilance are increased, and arousal increases to prepare for a possible switch to flight or panic controlled by FFFS, if danger becomes too great. Anxiety is a state in which the possibility of punishment has not entirely overcome the possibility of reward, such that the goal in question is still potentially attainable. In contrast, depression is a state in which punishment is perceived to be unavoidable, which can be described cybernetically as a state in which a goal (and therefore reward) is perceived to be unattainable. Anxiety can be alleviated either by determining that no real threat is present or by acting in such a way as to eliminate the threat or at least to reduce the likelihood of punishment. Alternatively, anxiety can be alleviated by abandoning the operative goal and turning to some other goal (cf. Nash et al., 2011). If the previously operative goal is not soon replaced by another goal, this abandonment becomes equivalent to entering a state of depression. Depression is typically identified when this amotivated state is persistent across situations and generalizes to multiple goals. When depression is used to describe a clinical condition, then the abandonment of goals has been inappropriately generalized. Depression has been described as "learned helplessness" to reflect the fact that motivation has been extinguished in the face of threat and the perceived difficulty of achieving goals generally (Miller and Norman, 1979).

Degree of motivation to explore the possibilities for attaining a goal, during or after passive avoidance, may be the core contribution of individual differences in dopamine to depression. That dopaminergic function is diminished in depression is well-established (Dunlop and Nemeroff, 2007). The symptom of depression most often linked to dopamine is *anhedonia*, loss of interest or pleasure in one's usual activities, and this is the feature of depression that is most clearly negatively associated with Extraversion (e.g., De Fruyt et al., 2013). Because Extraversion is the trait that reflects variation in the energetic enjoyment and pursuit of rewards, anhedonia may be essentially equivalent to low Extraversion (or perhaps low Plasticity) in conjunction with high Neuroticism. Like Extraversion, depression is related to reward sensitivity, though of course negatively rather than positively (Pizzagalli et al., 2009; Bress et al., 2012). The loss of interest associated with anhedonia is particularly likely to be associated with reduced dopaminergic function (Treadway and Zald, 2013). Loss of interest might be best described as *amotivation*, reserving "anhedonia" to describe loss of pleasure, which seems likely to be more related to the opioid liking system than to dopamine. In the present theory, the amotivation associated with depression reflects a reduction in dopaminergically driven exploration of possibilities either for reward or for information that might allow the creation of viable new goals or strategies. Both the value and salience systems thus seem likely to be influential in depression. In relation to salience, depression is associated not only with reduced motivation in general but also with cognitive deficits that may stem from reduced dopaminergic tone in DLPFC (Murrough et al., 2011).

#### *Anxiety is probably related to noradrenaline but not dopamine*

The association of anxiety with dopaminergic function is more uncertain than that of depression, and any associations found between anxiety and dopamine may be due to the high correlation between anxiety and depression. Future research needs to disentangle these related traits carefully (cf. Weinberg et al., 2012). Little evidence links dopamine to trait anxiety or anxiety disorders specifically. Several candidate gene studies have reported associations of various dopaminergic genes with anxiety or the broader trait of Neuroticism, but, in addition to the fact that they typically did not control for depression, they may be false positives, given the lack of confirming evidence from genome-wide association studies (e.g., de Moor et al., 2010). Amotivation, which provides the clearest evidence for dopamine's involvement in depression, is not a central feature of anxiety. The present theory takes the position that anxiety, as a trait distinct from depression, is unlikely to be related to individual differences in dopaminergic function.

As preliminary and indirect evidence for this hypothesis, **Table 1** presents analyses of associations between depression and anxiety and traits from the Big Five hierarchy depicted in **Figure 1**, assessed in 481 members of the ESCS. Anxiety and **Table 1 | Associations of NEO PI-R Anxiety and Depression (Costa and McCrae, 1992b) with the Big Five aspect scales (DeYoung et al., 2007) and Plasticity and Stability scales (DeYoung, 2010c) in the Eugene-Springfield community sample.**


*<sup>N</sup>* <sup>=</sup> *481, \*p <sup>&</sup>lt; 0.05*

depression were measured using the NEO PI-R, which has no items identical to those in the questionnaires used to measure the Big Five and their aspects (BFAS) or the metatraits, which were assessed using the 40 items previously identified as specific markers of Stability or Plasticity (DeYoung, 2010c). Although at the zero order anxiety was correlated with most of the traits hypothesized to be influenced by dopamine, this was due to the variance anxiety shares with depression. After controlling for depression, anxiety was not significantly correlated with any of the traits in question (except of course Withdrawal, of which it is a facet). Depression, in contrast, remained correlated with those traits after controlling for anxiety. (The only exceptions for depression were Openness/Intellect and Openness, which are to be expected because Openness is positively related to Neuroticism, despite the fact that Intellect is negatively related; DeYoung et al., 2012). What this pattern suggests is that, although dopaminergic function may be negatively associated with Withdrawal, which represents the general tendency toward passive avoidance, only depression is likely to be associated with dopamine once one examines variance specific to anxiety or depression. If one considers anxiety without controlling for depression, however, anxiety may appear negatively associated with dopaminergic function.

Having staked out the position that trait anxiety is unrelated to dopamine, except inasmuch as it is related to trait depression, I now discuss potential evidence against this position, with the caveat that it comes from rodent research, so generalization to humans is uncertain. One study showed decreased exploration and increased postural indicators of anxiety in rats following depletion of dopamine in medial PFC (Espejo, 1997). A more recent study in mice provides evidence that the salience system specifically might be influential in trait anxiety: A manipulated genetic deactivation of the dopaminergic system in response to aversive events was found to lead to failure to learn about specific threats, which in turn led to an overgeneralized threat-sensitivity analogous to generalized anxiety (Zweifel et al., 2011). Thus, failure to learn, due to reduced salience system activity, might lead to anxiety due to increased psychological entropy (i.e., increased uncertainty).

Nonetheless, it is possible that dopaminergic activity in the salience system under aversive conditions is orthogonal to anxiety if the latter is considered independently of depression (which would be difficult to accomplish in rodents). In this case, variation in the salience system in response to threat would merely influence the likelihood that someone who responds with anxiety will engage in active or "problem-focused" coping (cf. Carver and Connor-Smith, 2010). Individuals high in anxiety with relatively high levels of dopamine should be more likely to overcome the inhibition that accompanies anxiety, in order to explore the threat in question, to explore possible solutions to the problem posed by the threat, and to rapidly begin approaching some other goal if their anxiety is great enough to produce complete passive avoidance of the goal in question. On the whole, they should have better outcomes following stress and should be less likely to transition from anxiety to depression, but they should not necessarily feel any less anxious about threat. Both noradrenaline and dopamine are released in response to stress (Schultz, 2007; Robbins and Arnsten, 2009), and the current theory proposes that proneness to anxiety under stress is related to variation in noradrenergic function, whereas proneness to active coping vs. depressive response to stress is related to variation in dopaminergic function. Under this hypothesis, higher levels of dopaminergic activity will not make people feel less anxious but will make them more likely to engage in active coping (which may lead to better outcomes and hence, indirectly, to less anxiety in the long run).

In a previous article, I proposed that the exploration associated with Plasticity "is distinct from the kind of exploration, triggered by threat that consists of vigilance and rumination designed to scan for further threat" (DeYoung, 2010c, p 27), but I now suspect that this statement needs to be qualified. Although it is likely to be the noradrenaline associated with anxiety that primarily triggers vigilance and rumination, the type of exploration associated with Plasticity may nonetheless be evoked by threat, inasmuch as the dopaminergic salience system is activated. In fact, it may be precisely those high in Plasticity who are likely to be resilient in the face of threat because increased dopaminergic activity will incline them to engage in active coping. Further, if the dedication of cognitive resources to exploring a problem (presumably driven by the dopaminergic salience system) is experienced as rumination, then salience system activity might be positively related to rumination specifically. Anxiety certainly interrupts the function of the higher cognitive systems that are facilitated by the salience coding system, but that does not necessarily mean it inhibits them (Fales et al., 2008). It may simply redirect them to consider threat, which would be consistent with the fact that the salience coding system is triggered by unpredicted aversive stimuli.

While considering the role of dopamine in depression, it is important to consider hypomania, a personality trait specifically involved in bipolar or manic depression. Much as "depression" can be used to describe a personality trait as well as the more severe and typically more time-limited pathological episodes that receive a clinical diagnosis of depression, "hypomania" can be used to describe the milder and more stable personality trait that constitutes risk for episodes of mania (the prefix "hypo" indicates behavior less severe than full-blown mania). Mania is linked to heightened exploratory behavior (Perry et al., 2010), positive emotion (Gruber, 2011), and dopaminergic function (Park and Kang, 2012), and individuals described as hypomanic show behavioral signs of frequent intense activation of both value and salience systems, vividly illustrated by items from the Hypomanic Personality Scale (Eckblad and Chapman, 1986): "I have often been so excited about an involving project that I didn't care about eating or sleeping" (value); "Sometimes ideas and insights come to me so fast that I cannot express them all" (salience).

Consistent with involvement of both divisions of the dopaminergic system, trait hypomania is positively associated with both Extraversion and Openness/Intellect (Meyer, 2002; Schalet et al., 2011). Similarly, diagnosis of bipolar disorder is associated with elevated Extraversion and Openness/Intellect, a very unusual pattern among psychiatric disorders (Tackett et al., 2008). The link to general dopaminergic function is additionally consistent with the fact that mania has been linked to achievement striving (Johnson, 2005). Finally, for the salience system to be hyperactive in hypomania would be consistent with the former's apparent role in positive schizotypy, given that bipolar and schizophreniaspectrum disorders share considerable genetic risk (Craddock and Owen, 2010). Whereas unipolar depression and depression as a personality trait are posited to be associated with a general reduction in dopaminergic function, mania and hypomania are posited to reflect a strong general increase in dopaminergic function. The neurobiological dynamics that induce alternating episodes of reduced and hyperactive dopaminergic function constitute one of the most important topics for future research on bipolar disorder and related traits.

#### **SUMMARY OF DOPAMINERGIC TRAITS AND CONCLUSION**

**Table 2** presents the list of traits hypothesized to be influenced by dopamine, noting whether each is hypothesized to be primarily or secondarily associated with the value or salience coding dopaminergic systems. A primary association indicates that variation in the particular dopaminergic subsystem is hypothesized to be one of the largest determinants of variation in the trait. A secondary association indicates that other biological systems are hypothesized to determine more variance in the trait than does the particular dopaminergic subsystem. The sign of the association indicates whether dopaminergic activity is positively or negatively related to trait level. Activity in the value system influences traits that mainly involve behavioral exploration, whereas activity in the salience system influences traits that mainly involve cognitive exploration (taking a broad definition of "exploration" as any process that functions to transform the unknown into the known or vice versa). Traits linked to the value coding system **Table 2 | Traits hypothesized to be related to the value coding and salience coding dopaminergic systems.**


++*, Primary positive influence;* +*, Secondary positive influence;* −*, Secondary negative influence; parentheses indicate association conditional on different forms of the trait in question.*

are related to Extraversion and its subtraits; traits linked to the salience coding system are related to Openness/Intellect and its subtraits. Aggression and some forms of impulsivity (particularly lack of premeditation) are unusual in that they are posited to be positively associated with activity in the value system but negatively related to activity in the salience system.

The present theory has several implications for research on the role of dopamine in personality. First, the difference between value and salience systems clarifies one major reason why not every measured parameter of dopaminergic function must be related to every dopaminergic trait. Some traits will be related to parameters specific to one or the other system. Second, even within each system, different parameters may be related to different traits (because of the complexity of each system and their interactions with each other). For example, a dopaminergic valuesystem parameter that predicts sensation seeking need not necessarily predict Extraversion. What should be the case, however, is that some parameter of the value system could be found that is related to both Extraversion and sensation seeking—because the theory presumes that any trait influenced by dopamine will be related to Extraversion or Openness/Intellect partly through dopaminergic mechanisms. Because of the many different parameters that may vary in the dopaminergic system, Extraversion and Openness/Intellect need not account for (or fully mediate) every association of some other trait with dopaminergic function, but any trait associated with dopaminergic function should be associated with Extraversion and/or Openness/Intellect or one of their subtraits.

Because Extraversion and Openness/Intellect are considered to be the primary manifestations of dopaminergic function in personality, one should always test whether an association between a dopaminergic parameter and some other personality trait is mediated by these two traits, and particularly by their Assertiveness and Intellect aspects, which are hypothesized to be most strongly related to dopamine. Further, when demonstrating an association of any phenomenon with Extraversion or Assertiveness, one should always test whether the effect might be due to variance shared with Intellect, and vice versa. For example, any positive association of working memory capacity or intelligence with Extraversion is likely to be merely an artifact, due to the association of these cognitive abilities with Intellect (DeYoung et al., 2005, 2009, 2013b).

The list of traits in **Table 2** is intended to be reasonably comprehensive. Some of these traits may be fractionated further into facets, but all facet-level traits related to dopamine are likely to be facets of one of the traits in the list. If additional traits are identified that cannot be considered a facet of one of the traits in **Table 2**, they should nonetheless be related to Extraversion or Openness/Intellect. One might predict, for example, that sociosexual orientation (i.e., desire for many short-term vs. few long-term sexual relationships; Simpson and Gangestad, 1991a) is likely to be associated with dopaminergic function. Whether or not this trait qualifies as a facet of Extraversion, it is substantially correlated with Extraversion (Simpson and Gangestad, 1991b) and seems likely to be influenced by the dopaminergic value system.

One should not fall victim to the jangle fallacy and assume that because a scale has a different name it cannot be measuring one of the traits already on the list. For example, the MPQ, which is often used in research on dopamine, contains a Social Potency that is a good measure of Assertiveness (DeYoung et al., 2013b). Similarly, Novelty Seeking and Excitement Seeking are not listed because they are subsumed by Sensation Seeking.

Another important caveat is that variations in the dopaminergic system are not presumed to be solely responsible for variation in any of the traits listed here. Even traits like Assertiveness and Intellect that are hypothesized to be strongly influenced by dopaminergic function are undoubtedly influenced by non-dopaminergic neurobiological parameters as well. Further, because multiple biological systems will influence most, if not all, traits, the mere fact that a trait is associated with Extraversion or Openness/Intellect does not guarantee that it is influenced by dopamine. Some other biological system or process may be responsible for the trait associations in question.

In recent years, the most prominent theory of the role of dopamine in personality has linked it to Extraversion, reward sensitivity, and approach behavior (Depue and Collins, 1999). Recognition of the distinction between the value and salience coding systems provides a coherent framework for understanding how traits related to cognitive function, like Openness/Intellect and positive schizotypy, might also be related to dopamine. The most important premise for the development of a unified theory of dopaminergic function is that information has innate reward value, just as do food, warmth, sex, affiliation, and status. This premise allows the identification of exploration—cognition and behavior motivated by the incentive reward value of uncertainty—as the basic function of all dopaminergic activity. In turn, this unity of function may help to explain why Extraversion (sensitivity to specific rewards) and Openness/Intellect (sensitivity to the reward value of information) are sufficiently correlated to allow characterization of a higher-order Plasticity factor. Global variations in dopaminergic tone across the value and salience systems are posited to produce variation in the general exploratory tendency reflected in individual differences in Plasticity.

This theory about the nature of dopaminergic function and its role in personality is an extension of the entropy model of uncertainty (EMU; Hirsh et al., 2012), which characterizes anxiety as a response to uncertainty, defined as psychological entropy. What the initial presentation of EMU left out was an account of the fact that uncertainty is not only innately threatening, but also innately promising (Peterson, 1999). Uncertainty or the unknown is the only class of stimuli to have this inherently ambivalent motivational significance (Gray and McNaughton, 2000). A fully elaborated EMU can account not only for the response to entropy as a threat but also for the response to entropy as a potential source of reward. Traits related to dopamine reflect variation in the ways that individuals respond to the incentive reward value of uncertainty.

## **ACKNOWLEDGMENTS**

I thank a number of people who aided me at various stages of the creation of this article: Oliver Schultheiss for introducing me to the value and salience model of dopamine, Jacob Hirsh and James Lee for early conversations about the theory, and Jordan Peterson, Jacob Hirsh, Alex Rautu, Daniel Hawes, and Steve DeYoung for comments on drafts of the manuscript. Thanks as well to Lew Goldberg for his generosity in making data available from the Eugene-Springfield Community Sample.

## **REFERENCES**


integrative quantitative model of the adult externalizing spectrum. *J. Abnorm. Psychol.* 116, 645–666. doi: 10.1037/0021-843X.116.4.645


comorbidity with other clinical disorders. *Aggression Violent Behav.* 13, 383–395. doi: 10.1016/j.avb.2008.06.003


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 19 July 2013; accepted: 23 October 2013; published online: 14 November 2013.*

*Citation: DeYoung CG (2013) The neuromodulator of exploration: A unifying theory of the role of dopamine in personality. Front. Hum. Neurosci. 7:762. doi: 10.3389/ fnhum.2013.00762*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2013 DeYoung. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*