- 1Department of Psychology, Utrecht University, Utrecht, Netherlands
- 2Cognitive, Perceptual and Brain Sciences, University College London, London, UK
The present paper aims to advance the understanding of the control of human behavior by integrating two lines of literature that so far have led separate lives. First, one line of literature is concerned with the ideomotor principle of human behavior, according to which actions are represented in terms of their outcomes. The second line of literature mainly considers the role of reward signals in adaptive control. Here, we offer a combined perspective on how outcome representations and reward signals work together to modulate adaptive control processes. We propose that reward signals signify the value of outcome representations and facilitate the recruitment of control resources in situations where behavior needs to be maintained or adapted to attain the represented outcome. We discuss recent research demonstrating how adaptive control of goal-directed behavior may emerge when outcome representations are co-activated with positive reward signals.
Introduction
Human goal-directed behavior is supported by a set of mental tools that tune action to dynamic environments. The question how this adaptive control process works has received a lot of attention in the literature (Morsella et al., 2009). Although there exist different conceptualizations, such as executive processes (Smith and Jonides, 1999), working memory operations (Baddeley, 2007), and cognitive control (Miller and Cohen, 2001), they share three basic components of control: active maintenance of goal-relevant information; inhibition of irrelevant information; and shifting of information (Miyake and Shah, 1999).
Most research on the control of human behavior considers the person as the agent of control (Locke and Latham, 1990; Bandura, 2001). People are assumed to control their behavior by setting goals, keeping them active in mind, and adapting their behavior when needed. More recent research adopts a mechanistic account by suggesting that adaptive control processes are self-emergent once a goal is activated (Braver and Cohen, 2000; Postle, 2006; Hazy et al., 2007). In line with this mechanistic account we take the activation of a goal as the starting point of our analysis, and address the question of how the self-emergent process may be modeled to understand how goals instantiate adaptive control.
Basically, two features are central to the control of goal-directed behavior. The first feature pertains to the notion that actions are represented in terms of outcomes. The second feature comprises the rewarding property or value of these represented outcomes. Research on ideomotor theory of action investigates the first feature by examining and explaining how action-effect knowledge is acquired and how outcome representations are implemented in action selection (Hommel, 2013). Research on the second feature investigates how rewarding or positive affective signals, such as positive mood (Aspinwall, 1998; van Wouwe et al., 2011), monetary gains (Muller et al., 2007; Heitz et al., 2008), or positively valenced outcome information (Custers and Aarts, 2005; Gable and Harmon-Jones, 2008) influence perception and cognition in action control.
In essence, both features work in tandem to control behavior adaptively. Whereas outcome representations serve as reference points for perception and action (Powers, 1973; Carver and Scheier, 1982), accompanying positive reward signals assign value or utility to outcomes (Shizgal, 1999) and facilitate the recruitment of executive control processes (Locke and Braver, 2010). However, a theoretical and empirical analysis of the combined role of these features has largely been neglected in the literature. Here, we aim to integrate research on the ideomotor principle and research on the role of reward signals in action control.
The Role of Outcome Representations in the Control of Behavior
Human goal-directed behavior is thought to result from the brain's capacity to predict and represent actions in terms of their outcomes (Suddendorf and Corballis, 2007). Activating an outcome representation prepares action in an offline fashion (i.e., planned ahead). However, engaging in goal-directed behavior requires knowledge about action-effect relationships. Action-effect learning has been extensively studied and provides an explanation for the emergence of outcome representations (Shin et al., 2010). Basically, a link between action and effect is formed when a consequence of a motor movement is observed and further strengthened if this effect occurs consistently. Because the link between action and effect is assumed to be bidirectional, this strengthened link can be used to produce a specific outcome. This is the ideomotor principle: activating an outcome representation readily selects the action (Hommel et al., 2001).
According to this principle, multiple outcome representations can be associated with multiple actions (Hommel, 1996; Kunde et al., 2002). This way, goal-directed behavior is structured around equifinality and multifinality sets. Multiple actions can thus serve one outcome or a single action can produce multiple outcomes, rendering goal-directed behavior adaptive (Kruglanski et al., 2002).
Initially the ideomotor principle explains action selection on a sensorimotor level. However, human behavior is more complex and involves goals that are further removed from direct motor activation. It can be suggested, though, that goal-directed behavior emerges from simple movement goals to complex social goals that are accessed in different contexts by the same mechanisms underlying action-effect learning (Maturana and Varela, 1987). We first learn to orchestrate our motor movements before we can effectively hit a light switch and illuminate a dark room. Eventually certain learned patterns of motor movements become associated with new observable outcomes in terms of sensory/perceptual and semantic/cognitive codes (Pulvermüller, 2005; Kray et al., 2006; Lindemann et al., 2006; Aarts and Veling, 2009). Indeed, it has been demonstrated that sensory-motor goal representations (acquired in goal-directed motor tasks) generalize to abstract features of outcomes, such that outcome representations can become socially meaningful (Beckers et al., 2002).
People rely on these outcome representations during action selection and execution. In cybernetic models of action control outcome representations serve as reference points (Adams, 1971). When an action produces an outcome not matching the pre-activated outcome representation, an action-related error signal is produced (Carter et al., 1998). Control is then necessary and should subsequently result in switching to a new course of action and inhibiting the old one. Active maintenance of the outcome representation thus often operates in concert with other adaptive control processes to attain the outcome.
The Role of Reward Signals in Control
Ideomotor theorizing provides a parsimonious framework to understand how action-effect knowledge is acquired and how outcome representations are involved in the selection of action. However, it does not include specific predictions about when and how outcome representations gain control over behavior. There is a vast literature that does examine the emergence of adaptive control from an affective-motivational perspective.
First of all, there is research on the role of positive mood or emotion in cognitive control (Ashby et al., 1999; Fredrickson, 2004). This literature suggests that positive affect can broaden cognition (e.g., making people more creative) or funnel cognition (e.g., by focusing on local stimuli). Secondly, there is literature showing effects of prospective monetary gains on control processing such that effortful behavior can be boosted or strategically implemented (Bijleveld et al., 2012). Finally, the positive valence of outcome representations (acquired through evaluative conditioning procedures) can enhance effortful control in tasks generating the outcome (Custers and Aarts, 2010). These different lines of research suggest that positive affect, monetary gains and positive outcome representations serve as a general reward signal that acts as a common currency for modulating adaptive control (Shizgal and Conover, 1996), which either results in increased flexibility or more focused processing (Aston-Jones and Cohen, 2005). It remains unclear how the affective-motivational perspective deals with the question of when flexible or focused processing dominates. However, it is assumed that adaptive control processes originate from subcortical output releases of dopamine in the PFC, which is associated with the processing of general reward signals (Aarts et al., 2011; Chiew and Braver, 2011).
From this affective-motivational perspective, reward signals have been found to play a crucial role in each of the three basic components of adaptive control. Reward signals have been shown to (1) cause active maintenance of task relevant information and outcomes (Zedelius et al., 2011); (2) facilitate the inhibition of task-irrelevant information (Veling and Aarts, 2010); and (3) reduce switch costs in task-switching paradigms (Dreisbach and Goshke, 2004). These findings indicate the close relationship between adaptive control of human action and the processing of reward signals.
Reward-driven modulation of executive control is highly adaptive, because it justifies the allocation of limited cognitive resources (Pessoa, 2009). Resource allocation is guided by a principle of conservation such that effort will be expended only if it can be compensated by a significant benefit in the end (Brehm and Self, 1989; Gendolla et al., 2011). Reward signals thus ensure the recruitment of adaptive control processes when behavioral demands are imposed by environmental changes. Indeed, there are several studies that show how task demands and task incentives interact in producing effort intensity (Bijleveld et al., 2009; Silvestrini and Gendolla, 2013). In this research the conditions of demand are often explicitly communicated and it is shown that individuals invest effort only when the goal is attainable (i.e., moderately high demands) and valuable rewards are at stake. Thus, people seem to make trade-offs by weighing explicit information of reward value and demands. This raises the question of whether demand information needs to be explicit or whether such trade-offs also occur in contexts where differences in demands are less clear.
In a recent line of research we addressed this question using a modality shift paradigm (Marien et al., in preparation-a). Participants were instructed to respond to visual or auditory targets as fast as possible. Immediately before presentation of these targets we either presented a preparatory stimulus in the same modality as the target (ipsimodal trials, e.g., visual-visual), or a preparatory stimulus in a different modality (crossmodal trials, e.g., visual-auditory). The latter type of trials requires more resources (i.e., are more demanding) to respond to than the former type, because participants have to switch their prepared visual modality to the auditory modality. This typically results in a delayed response time caused by a modality switch cost, especially when this switch cannot be anticipated (Turatto et al., 2002). On half of the trials participants were presented with a 5 eurocents coin which they could earn; on other trials this reward signal of the coin was absent. Importantly, the preparatory stimuli were not predictive of whether a switch would occur or not. As expected, participants responded significantly faster when a reward was at stake during crossmodal trials, but there was no speeded responding during rewarded ipsimodal trials. Furthermore, the absence of the latter effect could not be explained by physical limits of speed of responding. Reward signals thus specifically reduce switch costs in an instrumental way, even in contexts that are ambiguous about task demands.
However, in most research on reward signals and cognitive control participants are instructed to perform a given action to obtain a specific outcome. Accordingly, research on the impact of reward signals on adaptive control is thus mainly limited to instructed task goals and does not consider how reward signals interact with outcome representations in controlling behavior (Dickinson and Balleine, 1994). We propose that analyzing the interplay between outcome representations and positive reward signals offers a more comprehensive examination of adaptive control of human action. In the next section, we discuss some recent research that examines this interplay in more detail.
The Combined Role of Outcome Representations and Reward Signals
The combined role of outcome representations and reward signals has been examined to explore the building blocks of adaptive control in goal pursuit (Custers and Aarts, 2005, 2010). For instance, the activation of the outcome representation of physical exertion facilitated effortful control in action when this outcome representation was immediately followed by reward signals (i.e., positive words) in an evaluative conditioning procedure (Aarts et al., 2008). Participants resisted the pressure to release but persisted in squeezing a handgrip. Furthermore, this study provided evidence for the distinct roles of outcome representations and reward signals. The mere activation of the outcome representation facilitates initiation of the action, but did not increase control unless positive reward signals were attached to it. Several other studies have also demonstrated the function of reward signals in mobilizing action control (e.g., Capa et al., 2011; Köpetz et al., 2011; Veltkamp et al., 2011).
Building on this line of research, we investigated whether the pairing of positive reward signals with outcome representations translates into adaptive control in terms of making people more flexible in goal-directed behavior (Marien et al., 2012). In a modification of a set-switch paradigm (Dreisbach and Goschke, 2004), participants had to turn on a light by pressing either a left or a right key. On each trial, the correct response was indicated by a dot of a particular color appearing either left or right. A dot of a different color was presented in the opposite location, but had to be ignored. Before each trial, a cue appeared consistently reminding people of the outcome (turn on light). These cues were immediately followed by positive or neutral stimuli. After some trials, participants had to ignore the color they had to attend to earlier and react to a new color. Participants in the positive reward signal condition had significantly lower switch costs than those in the neutral condition. These findings suggest that being able to swiftly switch the course of action to obtain an outcome is dependent on whether the outcome representation of the action was co-activated with reward signals.
Whereas most studies on the combined role of outcome representations and reward signals in facilitating control consider the outcomes as given, from research on ideomotor theory one would expect that these outcome representations are normally acquired in daily life as a result of learning that the outcome follows from an action (Elsner and Hommel, 2001). Thus, according to our present analysis positive reward signals should only increase control when an action is represented in terms of its outcome. Specifically, only when the presentation of a specific stimulus follows an action rather than preceding it, will an accompanying positive reward signal cause people to engage in controlled behavior to obtain the outcome.
In a recent test of this idea (Marien et al., in preparation-b), participants had to execute an action (pressing a key) that was either preceded or followed by a stimulus on the computer-screen (e.g., the word “scissors”). The stimulus was accompanied by a neutral or positive reward signal by presenting a spoken word through headphones (e.g., the word “with” or “nice”). Thus, the stimulus represented an outcome of an action or not, and this outcome representation was co-activated with a reward signal or not. After some pairings, participants were presented with the stimulus on the screen and had to press another key repeatedly to move the stimulus closer to themselves in an easy way (one single key) or a more demanding way (multiple keys). Faster repetitive action in this task implies more control. Results showed that participants were faster in moving the stimulus to themselves only when it represented an outcome of their action and was co-activated with a positive reward signal. This effect was more pronounced when moving the stimulus to themselves was demanding. These findings suggest that adaptive control of goal-directed behavior is more likely to occur when positive reward signals accompany the process of representing action in terms of outcomes. Moreover, resources to control behavior seem to be allocated to obtain the outcome according to a principle of conservation (Silvestrini and Gendolla, 2013).
Conclusion, Implications, and Prospects
We proposed that an integration of ideomotor accounts with affective-motivational accounts of action can shed new light on the control mechanisms underlying human goal-directed behavior. Although ideomotor theorizing offers a framework to understand how action-effect knowledge is acquired and how outcome representations select action, it is less explicit in predicting when and how control of behavior results from the activation of outcome representations. To understand the emergence of adaptive control reward signals should be taken into account. Although there is some research investigating the impact of reward signals on action-effect learning, the analysis is mainly focused on how it affects the binding strength and performance of the associated action (Muhle-Karbe and Krebs, 2012).
We also suggest that motivational accounts of adaptive control should incorporate more insights of ideomotor theory. Adaptive control processes are closely linked with reward processing, but the role of outcome representations is under-investigated in this literature. It is important for reward signals to connect with outcome representations in order for them to have a profound effect on adaptive control. The present analysis suggests that positive signals of different sources denote the value of an outcome and facilitate control of behavior. This implies that the influence of reward signals on recruiting executive control resources might not follow a direct path, but is mediated by the assigned value of the outcome representation. Future research could address (1) how personal value of an outcome representation results from reward signals, and (2) whether personal value mediates the instigation of control.
One way to approach this matter is by analyzing the neurocircuits prioritizing and controlling goals. Specifically, recent work in cognitive neuroscience proposes the involvement of specific neurotransmitter systems that cause people to exploit (being rigid to reach a goal) or to explore (prioritizing other goals) their environment (Aston-Jones and Cohen, 2005). Noradrenergic pathways in the brain are suggested to be associated with exploitation while dopaminergic pathways are supposed to be engaged in exploration.
This neurocircuit analysis of adaptive control can benefit from the present analysis. Adaptive control in terms of flexible or rigid/persistent processes may be dependent on the level of behavioral representation to which reward signals are attached. Goal-directed behavior is hierarchically structured (Botvinick, 2008), and hence the control of behavior may be directed at the level of action (means) representations or outcome (goal) representations depending on context and individual differences (Vallacher and Wegner, 1989). For example, goal-directed control of turning on a light may be identified and guided by the representation of “pressing the button” or “turning on the light.” So when representations of means are paired with reward signals action control is more likely to occur on the means level. Paradoxically, this could lead to more rigidity in control. We found that participants were less prone to switch to another action when the representation of the means was cued and paired with reward signals (Marien et al., 2012). In other words, when an outcome representation can be regarded as a subgoal of another outcome representation higher in the hierarchy (i.e., “pressing the key” in order to “turn on the light”), treating it with reward signals will increase local exploitive focus instead of broad explorative processing (Gable and Harmon-Jones, 2008). Taking the level of behavior representation into account may lead to specific predictions when reward signals produce a flexible or rigid mode of control.
Research on adaptive control of human action can advance by looking at outcome representations in combination with reward signals. It can especially help us to understand how the human mind functions optimally in the ever changing environment that we inhabit.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Aarts, E., van Holstein, M., and Cools, R. (2011). Striatal dopamine and the interface between motivation and cognition. Front. Psychol. 2:163. doi: 10.3389/fpsyg.2011.00163
Aarts, H., Custers, R., and Marien, H. (2008). Preparing and motivating behavior outside of awareness. Science 319, 1639. doi: 10.1126/science.1150432
Aarts, H., and Veling, H. (2009). Do resonance mechanisms in language and action depend on intentional processes? Eur. J. Soc. Psychol. 39, 1188–1190. doi: 10.1002/ejsp.681
Adams, J. A. (1971). A closed-loop theory of motor learning. J. Mot. Behav. 3, 111–150. doi: 10.1080/00222895.1971.10734898
Ashby, F. G., Isen, A. M., and Turken, A. U. (1999). A neuropsychological theory of positive affect and its influence on cognition. Psychol. Rev. 106, 529–550. doi: 10.1037/0033-295X.106.3.529
Aspinwall, L. G. (1998). Rethinking the role of positive affect in self-regulation. Motiv. Emot. 22, 1–32. doi: 10.1023/A:1023080224401
Aston-Jones, G., and Cohen, J. D. (2005). An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 28, 403–450. doi: 10.1146/annurev.neuro.28.061604.135709
Baddeley, A. D. (2007). Working Memory, Thought, and Action. Oxford: Oxford University Press. doi: 10.1093/acprof:oso/9780198528012.001.0001
Bandura, A. (2001). Social cognitive theory: an agentic perspective. Annu. Rev. Psychol. 52, 1–26. doi: 10.1146/annurev.psych.52.1.1
Beckers, T., De Houwer, J., and Eelen, P. (2002). Automatic integration of non-perceptual action effect features: the case of the associative affective Simon effect. Psychol. Res. 66, 166–173. doi: 10.1007/s00426-002-0090-9
Bijleveld, E., Custers, R., and Aarts, H. (2009). The unconscious eye opener pupil dilation reveals strategic recruitment of resources upon presentation of subliminal reward cues. Psychol. Sci. 20, 1313–1315. doi: 10.1111/j.1467-9280.2009.02443.x
Bijleveld, E., Custers, R., and Aarts, H. (2012). Human reward pursuit from rudimentary to higher-level functions. Curr. Dir. Psychol. Sci. 21, 194–199. doi: 10.1177/0963721412438463
Botvinick, M. M. (2008). Hierarchical models of behavior and prefrontal function. Trends Cogn. Sci. 12, 201–208. doi: 10.1016/j.tics.2008.02.009
Braver, T. S., and Cohen, J. D. (2000). “On the control of control: the role of dopamine in regulating prefrontal function and working memory,” in Attention and Performance XVIII, eds S. Monsell, and J. Driver (Cambridge, MA: MIT Press), 713–737.
Brehm, J. W., and Self, E. A. (1989). The intensity of motivation. Ann. Rev. Psychol. 40, 109–131. doi: 10.1146/annurev.ps.40.020189.000545
Capa, R. L., Cleeremans, A., Bustin, G. M., Bouquet, C., and Hansenne, M. (2011). Effects of subliminal priming on nonconscious goal pursuit and effort-related cardiovascular response. Soc. Cogn. 29, 430–444. doi: 10.1521/soco.2011.29.4.430
Capa, R. L., Cleeremans, A., Bustin, G. M., and Hansenne, M. (2011). Long-lasting effect of subliminal processes on cardiovascular responses and performance. Int. J. Psychophysiol. 81, 22–30. doi: 10.1016/j.ijpsycho.2011.04.001
Carter, C. S., Braver, T. S., Barch, D. M., Botvinick, M. M., Noll, D., and Cohen, J. D. (1998). Anterior cingulate cortex, error detection, and the online monitoring of performance. Science 280, 747–749. doi: 10.1126/science.280.5364.747
Carver, C. S., and Scheier, M. F. (1982). Control theory: a useful conceptual framework for personality–social, clinical, and health psychology. Psychol. Bull. 92, 111–135. doi: 10.1037/0033-2909.92.1.111
Chiew, K. S., and Braver, T. S. (2011). Positive affect versus reward: emotional and motivational influences on cognitive control. Front. Psychol. 2:279. doi: 10.3389/fpsyg.2011.00279
Custers, R., and Aarts, H. (2005). Positive affect as implicit motivator: on the nonconscious operation of behavioral goals. J. Pers. Soc. Psychol. 89, 129–142. doi: 10.1037/0022-3514.89.2.129
Custers, R., and Aarts, H. (2010). The unconscious will: how the pursuit of goals operates outside of conscious awareness. Science 329, 47–50. doi: 10.1126/science.1188595
Dickinson, A., and Balleine, B. W. (1994). Motivational control of goal-directed action. Anim. Learn. Behav. 22, 1–18. doi: 10.3758/BF03199951
Dreisbach, G., and Goschke, T. (2004). How positive affect modulates cognitive control: reduced perseveration at the cost of increased distractibility. J. Exp. Psychol. Learn. Mem. Cogn. 30, 343–353. doi: 10.1037/0278-7393.30.2.343
Elsner, B., and Hommel, B. (2001). Effect anticipation and action control. J. Exp. Psychol. Hum. Percept. Perform. 27, 229–240. doi: 10.1037/0096-1523.27.1.229
Fredrickson, B. L. (2004). The broaden-and-build theory of positive emotions. Philos. Trans. R. Soc. Lond. B Biol. Sci. 359, 1367–1378. doi: 10.1098/rstb.2004.1512
Gable, P. A., and Harmon-Jones, E. (2008). Approach-motivated positive affect reduces breadth of attention. Psychol. Sci. 19, 476–482. doi: 10.1111/j.1467-9280.2008.02112.x
Gendolla, G. H. E., Wright, R. A., and Richter, M. (2011). “Effort intensity: some insights from the cardiovascular system,” in The Oxford Handbook of Motivation, ed R. Ryan (New York, NY: Oxford University Press), 420–440.
Hazy, T. E., Frank, M. J., and O'Reilly, R. C. (2007). Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system. Philos. Trans. R. Soc. B Biol. Sci. 362, 1601–1613. doi: 10.1098/rstb.2007.2055
Heitz, R. P., Schrock, J. C., Payne, T. W., and Engle, R. W. (2008). Effects of incentive on working memory capacity: behavioral and pupillometric data. Psychophysiology 45, 119–129.
Hommel, B. (1996). The cognitive representation of action: automatic integration of perceived action effects. Psychol. Res. 59, 176–186. doi: 10.1007/BF00425832
Hommel, B. (2013). “Ideomotor action control: on the perceptual grounding of voluntary actions and agents,” in Action Science: Foundations of an Emerging Discipline, eds W. Prinz, M. Beisert, and A. Herwig (Cambridge, MA: MIT Press), 113–136.
Hommel, B., Müsseler, J., Aschersleben, G., and Prinz, W. (2001). The theory of event coding (TEC): a framework for perception and action planning. Behav. Brain Sci. 24, 849–937. doi: 10.1017/S0140525X01000103
Köpetz, C., Faber, T., Fishbach, A., and Kruglanski, A. W. (2011). The multifinality constraints effect: how goal multiplicity narrows the means set to a focal end. J. Pers. Soc. Psychol. 100, 810–826. doi: 10.1037/a0022980
Kray, J., Eenshuistra, R., Kerstner, H., Weidema, M., and Hommel, B. (2006). Language and action control: the acquisition of action goals in early childhood. Psychol. Sci. 17, 737–741. doi: 10.1111/j.1467-9280.2006.01774.x
Kruglanski, A. W., Shah, J. Y., Fishbach, A., Friedman, R., Chun, W. Y., and Sleeth-Keppler, D. (2002). “A theory of goal-systems,” in Advances in Experimental Social Psychology, Vol. 34, ed M. P. Zanna (New York, NY: Academic Press), 331–378.
Kunde, W., Hoffmann, J., and Zellmann, P. (2002). The impact of anticipated action effects on action planning. Acta Psychol. (Amst) 109, 137–155. doi: 10.1016/S0001-6918(01)00053-1
Lindemann, O., Stenneken, P., Van Schie, H. T., and Bekkering, H. (2006). Semantic activation in action planning. J. Exp. Psychol. Hum. Percept. Perform. 32, 633–643. doi: 10.1037/0096-1523.32.3.633
Locke, E. A., and Latham, G. P. (1990). A Theory of Goal Setting and Task Performance. Englewoods Cliffs, NJ: Prentice Hall.
Locke, H. S., and Braver, T. S. (2010). “Motivational influences on cognitive control: a cognitive neuroscience perspective,” in Self control in Society, Mind, and Brain, eds R. R. Hassin, K. N. Ochsener, and Y. Trope (Oxford: Oxford University Press), 114–140. doi: 10.1093/acprof:oso/9780195391381.003.0007
Marien, H., Aarts, H., and Custers, R. (2012). Being flexible or rigid in goal-directed behavior: when positive affect implicitly motivates the pursuit of goals or means. J. Exp. Soc. Psychol. 48, 277–283. doi: 10.1016/j.jesp.2011.08.013
Miller, E. K., and Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167–202. doi: 10.1146/annurev.neuro.24.1.167
Miyake, A., and Shah, P. (1999). Models of Working Memory: Mechanisms of Active Maintenance and Executive Control. New York, NY: Cambridge University Press. doi: 10.1017/CBO9781139174909
Morsella, E., Bargh, J. A., and Gollwitzer, P. M. (2009). Oxford Handbook of Human Action, Vol. 2. New York, NY: Oxford University Press.
Muhle-Karbe, P. S., and Krebs, R. M. (2012). On the influence of reward on action-effect binding. Front. Psychol. 3:450. doi: 10.3389/fpsyg.2012.00450
Muller, J., Dreisbach, G., Goschke, T., Hensch, T., Lesch, K. P., and Brocke, B. (2007). Dopamine and cognitive control: the prospect of monetary gains influences the balance between flexibility and stability in a set-shifting paradigm. Eur. J. Neurosci. 26, 3661–3668. doi: 10.1111/j.1460-9568.2007.05949.x
Pessoa, L. (2009). How do emotion and motivation direct executive control? Trends Cogn. Sci. 13, 160–166. doi: 10.1016/j.tics.2009.01.006
Postle, B. R. (2006). Working memory as an emergent property of the mind and brain. Neuroscience 139, 23–38. doi: 10.1016/j.neuroscience.2005.06.005
Pulvermüller, F. (2005). Brain mechanisms linking language and action. Nat. Rev. Neurosci. 6, 576–582. doi: 10.1038/nrn1706
Shin, Y. K., Proctor, R. W., and Capaldi, E. J. (2010). A review of contemporary ideomotor theory. Psychol. Bull. 136, 943–974. doi: 10.1037/a0020541
Shizgal, P. (1999). “On the neural computation of utility: implications from studies of brain stimulation reward,” in Well-being: The Foundations of Hedonic Psychology, eds D. Kahneman, E. Diener, and N. Schwarz (New York, NY: Russell Sage Foundation), 500–524.
Shizgal, P., and Conover, K. (1996). On the neural computation of utility. Curr. Dir. Psychol. Sci. 5, 37–43. doi: 10.1111/1467-8721.ep10772715
Silvestrini, N., and Gendolla, G. H. (2013). Automatic effort mobilization and the principle of resource conservation: one can only prime the possible and justified. J. Pers. Soc. Psychol. 104, 803–816. doi: 10.1037/a0031995
Smith, E. E., and Jonides, J. (1999). Storage and executive processes in the frontal lobes. Science 283, 1657–1661. doi: 10.1126/science.283.5408.1657
Suddendorf, T., and Corballis, M. C. (2007). The evolution of foresight: what is mental time travel, and is it unique to humans? Behav. Brain Sci. 30, 299–312. doi: 10.1017/S0140525X07001975
Turatto, M., Benso, F., Galfano, G., and Umiltà, C. (2002). Nonspatial attentional shifts between audition and vision. J. Exp. Psychol. Hum. Percept. Perform. 28, 628–639. doi: 10.1037/0096-1523.28.3.628
Vallacher, R. R., and Wegner, D. M. (1989). Levels of personal agency: individual variation in action identification. J. Pers. Soc. Psychol. 57, 660–671. doi: 10.1037/0022-3514.57.4.660
van Wouwe, N. C., Band, G. P., and Ridderinkhof, K. R. (2011). Positive affect modulates flexibility and evaluative control. J. Cogn. Neurosci. 23, 524–539. doi: 10.1162/jocn.2009.21380
Veling, H., and Aarts, H. (2010). Cueing task goals and earning money: relatively high monetary rewards reduce failures to act on goals in a Stroop task. Motiv. Emot. 34, 184–190. doi: 10.1007/s11031-010-9160-2
Veltkamp, M., Custers, R., and Aarts, H. (2011). Motivating consumer behavior by subliminal conditioning in the absence of basic needs: striking even while the iron is cold. J. Consum. Psychol. 21, 49–56. doi: 10.1016/j.jcps.2010.09.011
Keywords: goal-directed action, motivation, adaptive control, outcome representation, reward signal
Citation: Marien H, Aarts H and Custers R (2013) Adaptive control of human action: the role of outcome representations and reward signals. Front. Psychol. 4:602. doi: 10.3389/fpsyg.2013.00602
Received: 18 July 2013; Paper pending published: 21 July 2013;
Accepted: 19 August 2013; Published online: 09 September 2013.
Edited by:
Ezequiel Morsella, San Francisco State University and University of California, USAReviewed by:
Ezequiel Morsella, San Francisco State University and University of California, USAT. Andrew Poehlman, Southern Methodist University, USA
Copyright © 2013 Marien, Aarts and Custers. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Hans Marien, Department of Psychology, Utrecht University, Heidelberglaan 1, 3584CS Utrecht, PO Box 80140, 3508 TC Utrecht, Netherlands e-mail: h.marien@uu.nl