Punishment sensitivity modulates the processing of negative feedback but not error-induced learning
- Department of Psychology, Development of Language, Learning, and Action, Saarland University, Saarbruecken, Germany
Accumulating evidence suggests that individual differences in punishment and reward sensitivity are associated with functional alterations in neural systems underlying error and feedback processing. In particular, individuals highly sensitive to punishment have been found to be characterized by larger mediofrontal error signals as reflected in the error negativity/error-related negativity (Ne/ERN) and the feedback-related negativity (FRN). By contrast, reward sensitivity has been shown to relate to the error positivity (Pe). Given that Ne/ERN, FRN, and Pe have been functionally linked to flexible behavioral adaptation, the aim of the present research was to examine how these electrophysiological reflections of error and feedback processing vary as a function of punishment and reward sensitivity during reinforcement learning. We applied a probabilistic learning task that involved three different conditions of feedback validity (100%, 80%, and 50%). In contrast to prior studies using response competition tasks, we did not find reliable correlations between punishment sensitivity and the Ne/ERN. Instead, higher punishment sensitivity predicted larger FRN amplitudes, irrespective of feedback validity. Moreover, higher reward sensitivity was associated with a larger Pe. However, only reward sensitivity was related to better overall learning performance and higher post-error accuracy, whereas highly punishment sensitive participants showed impaired learning performance, suggesting that larger negative feedback-related error signals were not beneficial for learning or even reflected maladaptive information processing in these individuals. Thus, although our findings indicate that individual differences in reward and punishment sensitivity are related to electrophysiological correlates of error and feedback processing, we found less evidence for influences of these personality characteristics on the relation between performance monitoring and feedback-based learning.
Introduction
Learning from reward and punishment is a prerequisite for flexible behavioral adaptation to changing environmental conditions. There is, however, considerable evidence to suggest that individuals vary in their responsiveness to rewarding and punishing stimuli (Depue and Collins, 1999; Pickering and Gray, 2001; Corr, 2004). According to a prominent neurophysiologically oriented theory of personality, three systems underlie interindividual differences in reward and punishment processing (Gray, 1982; Gray and McNaughton, 2000; McNaughton and Corr, 2004). The behavioral activation system (BAS) is thought to be activated by appetitive stimuli and to promote reward-directed approach behavior. In contrast, the fight-flight-freeze system (FFFS) is presumed to be activated by aversive cues and to mediate defensive avoidance. Activation of the behavioral inhibition system (BIS) has been linked to the detection of conflict between competing goals (e.g., approach-avoidance conflict), resulting in increased arousal, focused attention, and enhanced information processing. The BIS is assumed to inhibit prepotent response tendencies and to arbitrate between conflicting BAS- and FFFS-controlled behaviors by promoting risk-assessment along with a negative processing bias. While reward sensitivity has primarily been related to BAS-functioning, punishment sensitivity has been-related to combined FFFS/BIS-functioning (Corr, 2004).
Recent findings indicate that BAS-reactivity is associated with dopamine-dependent activity cortex (e.g., Beaver et al., 2006; Hahn et al., 2009; Simon et al., 2010). BIS/FFFS-reactivity has been linked to functional variations in a distributed network of neural structures including septo-hippocampal system and amygdala, possibly mediated by serotonergic and noradrenergic mechanisms (Gray and McNaughton, 2000; Smillie, 2008). Moreover, a number of event-related potential (ERP) studies point to a link between self-reported punishment sensitivity and functioning of the medial prefrontal cortex (mPFC), specifically the anterior cingulate cortex (ACC) (e.g., Boksem et al., 2006; Amodio et al., 2008; Balconi and Crivelli, 2010). The ACC has been shown to be involved in the processing of motivationally salient events such as errors, conflict, and punishment cues, and more generally, in integrating action selection with motivational and affective processes (Devinsky et al., 1995; Shackman et al., 2011).
The error negativity (Ne; Falkenstein et al., 1990), or error-related negativity (ERN; Gehring et al., 1993) and the feedback-related negativity (FRN; Miltner et al., 1997) are ERP correlates of error or conflict monitoring and feedback processing that are thought to reflect the evaluative functions subserved by the mPFC/ACC (Ridderinkhof et al., 2004; Taylor et al., 2007). The Ne/ERN is a fronto-centrally distributed negative deflection that peaks within 100 ms after an individual's erroneous response. A morphologically similar component, the FRN, is elicited ~250–300 ms following the presentation of performance feedback. The FRN is more pronounced after negative compared to positive feedback, indicating that it is sensitive to the valence of an outcome (e.g., Gehring and Willoughby, 2002; Yeung and Sanfey, 2004). Subjects scoring high on measures of negative affectivity and punishment sensitivity appear to be characterized by a larger Ne/ERN (Hajcak et al., 2003, 2004; Boksem et al., 2006, 2008; Amodio et al., 2008; Dennis and Chen, 2009) and FRN (Sato et al., 2005; Balconi and Crivelli, 2010; De Pascalis et al., 2010; Santesso et al., 2011a,b), presumably reflecting enhanced reactivity of the medial prefrontal action monitoring system to outcomes signaling potential threat. In line with this notion, Boksem and colleagues (2008) found that high punishment sensitivity was associated with larger Ne/ERN amplitudes when participants tried to prevent monetary loss but not when they aimed to maximize monetary gain.
Interestingly, Boksem and colleagues (2006, 2008) also reported a positive correlation between reward sensitivity and the error positivity (Pe; Falkenstein et al., 1990), a slow positive-going deflection with a maximum amplitude between 200 and 400 ms after an erroneous response. The Pe shows a centro-parietal scalp distribution and has been mapped to distinct neural generators in the (rostral) ACC and the parietal cortex (Van Veen and Carter, 2002; O'Connell et al., 2007). There is some evidence that the Pe reflects salience or motivational significance of an error and thus may be functionally related to the P300 (Overbeek et al., 2005; Ridderinkhof et al., 2009). In addition, the Pe has been linked to the conscious recognition of an error (Falkenstein et al., 1990; Leuthold and Sommer, 1999; Nieuwenhuis et al., 2001; Endrass et al., 2007). According to Boksem and colleagues (2006, 2008), higher Pe amplitudes in subjects highly sensitive to reward might indicate proactive engagement in the service of maximizing future rewards.
Although the error-related ERP components have been proposed to reflect processes that support flexible behavioral adaptation (Falkenstein et al., 1990; Gehring et al., 1993; Holroyd and Coles, 2002; Yeung et al., 2004; Frank, 2005; Frank et al., 2007a), it remains largely unclear whether variations in Ne/ERN, FRN, and Pe amplitude as a function of punishment and reward sensitivity are accompanied by behavioral alterations. On the one hand, a central implication following from the conceptualization of BIS/FFFS and BAS is that highly punishment sensitive individuals should learn more efficiently from negative action outcomes than less punishment sensitive individuals, whereas high reward sensitivity should be associated with better learning under positive reinforcement (Pickering and Gray, 2001; Corr, 2004). On the other hand, previous studies using reinforcement learning paradigms indicate that Ne/ERN and FRN are neural manifestations of negative reward prediction errors, possibly coded by phasic activity of the midbrain dopamine system (Holroyd and Coles, 2002; Frank et al., 2005). These error signals are assumed to be used by the mPFC to guide adaptive action selection. In support of this view, it has been demonstrated that larger Ne/ERN and FRN amplitudes are associated with a stronger tendency to subsequently avoid the same maladaptive response (Frank et al., 2005; van der Helden et al., 2010; Unger et al., 2012).
So far, most studies reporting a relationship between punishment/reward sensitivity and ERP correlates of error and feedback processing have used response conflict and gambling tasks (Boksem et al., 2006, 2008; Amodio et al., 2008; Santesso et al., 2011b). To our knowledge, only one study has investigated the influence of individual differences in punishment and reward sensitivity on feedback processing in a Go-NoGo learning task (De Pascalis et al., 2010). Although this study failed to obtain a significant correlation between punishment sensitivity and the FRN, individuals with higher trait sensitivity to punishment showed larger FRN amplitudes on NoGo trials than less punishment sensitive individuals when the groups were defined by median split. The main goal of the present research was to further investigate the influence of individual differences in punishment and reward sensitivity on error and feedback processing as reflected in the Ne/ERN, FRN, and Pe. Specifically, we aimed to determine whether the effects of punishment sensitivity on the Ne/ERN and FRN are associated with changes in error-induced behavioral adjustments during reinforcement learning.
To address these issues, we applied a reinforcement learning task that has been used by a number of previous studies to examine learning-related changes in the Ne/ERN and FRN (e.g., Holroyd and Coles, 2002; Eppinger et al., 2008). Since the neural mechanisms of error processing have been shown to be sensitive to the uncertainty of stimulus-response (S-R) mappings inherent in a probabilistic learning task (e.g., Eppinger et al., 2008; Gründler et al., 2009), we manipulated the validity of feedback information by including a deterministic learning condition (100% valid), a probabilistic learning condition (80% valid), and a chance condition (50%). In addition, we administered the Carver and White (1994) BIS/BAS Scales to measure punishment and reward sensitivity. It should be noted, that Ne/ERN, FRN, and Pe have not consistently been found to vary as a function of punishment and reward sensitivity (e.g., Cavanagh and Allen, 2008; Van den Berg et al., 2011). These inconsistencies might partly result from the fact that some of the relevant studies used relatively small samples (<30), limiting the generalizability of the corresponding findings. The present study therefore included a comparatively large sample of 105 participants.
At the behavioral level, we expected higher punishment sensitivity to be associated more efficient error-related behavioral adjustments, i.e., higher post-error accuracy. We also expected to find a positive, albeit weaker, relationship between overall accuracy and both punishment and reward sensitivity, as punishment and reward learning can contribute to better overall performance on the task employed in this study. However, given that BIS and BAS are thought to interact in that the activation of one system inhibits the other, effects of punishment and reward sensitivity on overall learning performance might be hard to detect in tasks involving both reward and punishment cues (Pickering et al., 1997; Corr, 2002).
Regarding the relationship between reward and punishment sensitivity and the ERP correlates of error and feedback processing we expected to replicate previous findings that (i) punishment sensitivity correlates positively with the magnitude of Ne/ERN and FRN and (ii) reward sensitivity correlates positively with the Pe.
Moreover, we examined whether the relations between punishment/reward sensitivity and the error-related ERP components vary over the course of learning. Specifically, we expected the effects of punishment and reward sensitivity on the Ne/ERN and Pe to be larger toward the end compared to the beginning of learning as well as in the deterministic compared to the probabilistic condition, reflecting the participants' ability to represent the correctness of their responses (Holroyd and Coles, 2002). Importantly, previous findings indicate that larger Ne/ERN and FRN amplitudes are associated with more efficient error-related behavioral adjustments (van der Helden et al., 2010; Unger et al., 2012). Moreover, it has recently been shown that hyperresponsivity to punishment cues might be reflected in a strengthening of the coupling between error-related neural responses and behavioral adaptation rather than in performance differences per se (Cavanagh et al., 2011a,b). On the basis of these findings, we hypothesized that the relationship between the error-related ERP components and accuracy measures might vary as a function of punishment (and reward) sensitivity.
Materials and Methods
Participants
One-hundred and five participants (71 female, 34 male) were recruited from the student population of Saarland University. All had normal or corrected-to-normal vision, no known neurological or psychiatric diseases and were free from psychoactive medication or drug use. Mean age was 22 years (range = 18–33 years). Participants gave informed written consent in accordance with the protocols approved by the local ethics committee and received course credit or payment at a rate of € 8/h as well as a small monetary bonus. A further 20 students participated in the study but had to be excluded from analyses because of (i) fewer than 15 electroencephalogram (EEG) epochs in one or more conditions (2), (ii) poor performance on the learning task (less than 55% correct in the deterministic learning condition) (9), (iii) excessive noise in the EEG data (3), and (iv) technical problems during EEG recordings (6).
Learning Task
The stimuli used were colored images of objects (Snodgrass and Vanderwart, 1980). The learning task required participants to make a forced-choice decision (left vs. right button press) upon presentation of the target stimulus. Following the response, either the word “RICHTIG” (“correct”), or “FALSCH” (“incorrect”) was shown. If a response exceeded the adaptive deadline (see below), the feedback “ZU LANGSAM” (“too slow”) was presented. Participants had to learn the response mappings by trial and error. The task involved three conditions of feedback validity (100%, 80%, and 50%). In the deterministic learning condition, feedback was always valid (100%). In the probabilistic learning condition, feedback was valid on 80% of the trials only. That is, if a stimulus was assigned to the right response key, participants received “Correct” feedback in 80% and “Incorrect” feedback in 20% of right button presses and vice versa for left button presses. In the chance condition, “Correct” and “Incorrect” feedback was delivered at random (see Figure 1 for an illustration of the experimental paradigm). Four stimuli were associated with each condition, resulting in a total of 12 stimuli presented throughout the task. Within each learning condition, two stimuli were mapped to the left and right response key, respectively. Each stimulus was presented 50 times in pseudo-randomized order, resulting in a total of 600 trials. Participants first completed 60 practice trials. During the experiment, they took self-paced breaks every 30 trials.
Figure 1. Schematic illustration of the learning paradigm.
Trials began with a variable fixation period of 250–500 ms, which was followed by the presentation of the imperative stimulus for 500 ms. Stimuli were presented on a light gray background. The screen was then blanked for a variable interval of 600–1500 ms, depending on the variable response time window. In order to obtain a sufficient number of error trials, we applied an adaptive response deadline. Based on the proportion of time-out trials, the response window was individually adjusted in steps of 100 ms within an overall range of 400–1000 ms (for a similar procedure see (Eppinger et al., 2008)). The time window between the response and the feedback was fixed to 500 ms and feedback was displayed for 500 ms. The next trial started after a randomly jittered 1250–2000 ms interval. Participants were informed that they would gain a point for each correct response and lose a point for each incorrect or too slow response and that they could earn a monetary bonus up to 10 Euro depending on the total sum of points obtained. During the breaks, a feedback screen indicated the sum of points they had collected.
Psychometric Tests and Questionnaires
A German version of the Carver and White (1994) behavioral inhibition scale/behavioral activation scale (BIS/BAS) Scale was used to measure trait-level punishment (seven items; Cronbachs α = 0.73; e.g., “I worry about making mistakes.”) and reward sensitivity (13 items; Cronbachs α = 0.59; e.g., “When good things happen to me, it affects me strongly.”). Note that the “BIS” scale contains both FFFS and BIS items. The BIS/BAS scores were in normal ranges for healthy young adults (Strobel et al., 2001). Table 1 shows means and standard deviations, for the total sample and separately for male and female participants. Consistent with previous reports (Leone et al., 2001), females were characterized by higher punishment sensitivity. We additionally administered the questionnaires assessing positive and negative affectivity (PANAS, Watson et al., 1988) as well as state vs. action orientation (ACS-90, Kuhl, 1994). However, none of these variables was related to learning performance or ERP measures.
Table 1. BIS/BAS scores (Means, Standard deviations, and t-test results).
Procedure
After a brief description of the experiment, participants filled out a consent form and a short demographic questionnaire. Prior to the electrophysiological recordings, they completed the two psychometric tests and the BIS/BAS scales. The learning task was run in an electrically shielded, dimly lit, sound-attenuated chamber. Stimuli were presented on a CTX 17” monitor and participants responded by pressing the keys C or M on a standard computer keyboard with the left and right index finger, respectively. The entire experiment took approximately 1.5 h.
Electrophysiological Recording
The continuous EEG was recorded from 58 Ag/AgCl electrodes arranged according to the extended 10–20 system and referenced to the left mastoid, using Brain Amp DC Recorder (BrainVision recorder acquisition software). EEG signals were sampled in DC mode with a low-pass filter at 70 Hz and digitized at a sampling rate of 500 Hz. Impedances were kept below 5 kΩ. Electro-oculographic activity (EOG) was recorded from electrodes placed on the outer canthi of the two eyes (horizontal EOG) and on the infra- and supra-orbital ridges of the left eye (vertical EOG). The data were re-referenced offline to the linked mastoids and band-pass filtered from 0.1 to 30 Hz. The impact of blinks and eye movements was corrected using an independent component analysis algorithm embedded in the BrainVision Analyzer Software Package (Brain products, Gilching, Germany). Trials containing EEG activity exceeding ±100 μV, changing more than 50 μV between samples or containing DC drifts were removed by means of a semiautomatic artifact inspection procedure.
Data Analysis
Behavioral data analyses
Responses faster than 240 ms (<2 SD) or exceeding the adaptive response deadline were excluded from further analyses (on average, the mean number of responses exceeding the individually adjusted deadline was 4%). To examine the course of learning, the behavioral data were averaged into six bins of 100 trials each, i.e., Bin 1 contained Trials 1–100, Bin 2 contained Trials 101–200, and so on. Within each bin, mean reaction times (RTs) and mean accuracy rates were computed separately for the three learning conditions. Only valid trials were included for the probabilistic learning condition. To analyze trial-to-trial behavioral adjustments, we additionally determined post-error accuracy by calculating mean accuracy rates for the next presentation of a given stimulus after an erroneous response, separately for each learning condition.
ERP analyses
Artifact-free EEG data were segmented relative to response and feedback onset to extract response-related and feedback-related ERPs. The response-locked and feedback-locked epochs were baseline corrected with respect to the average voltage during a −200 to −50 ms pre-response interval and a 100 ms pre-stimulus interval, respectively.
As in previous studies employing reinforcement learning tasks (e.g., Frank et al., 2005; Eppinger et al., 2008), we defined the Ne/ERN after 15 Hz low-pass filtering at electrode site FCz as peak-to-peak difference in voltage between the most negative peak in a time window between −50 and 100 ms around the response and the most positive peak within the preceding 100 ms time window. Peak-to-peak voltage was measured to determine baseline-independent amplitudes and to minimize distortions due to the positivity on which the Ne/ERN is superimposed. However, to keep the present results compatible with those from previous studies, we additionally created difference waveforms by subtracting the activity correct trials from the activity on error trials (ΔNe). The ΔNe was defined as the mean amplitude in a 0–100 ms post-response time window covering the peak of the difference wave in each learning condition. Similarly, the FRN was quantified as peak-to-peak voltage difference between the most negative peak in a 200–400 ms time window after feedback onset and the preceding positive peak in a 150–300 ms post-feedback interval at electrode FCz (see Yeung and Sanfey, 2004; Frank et al., 2005). In a further step, ΔFRN amplitude was determined by subtracting the activity after correct feedback from the activity after negative feedback. The ΔFRN was defined as mean amplitude in a 50 ms time window centered on the individual peaks of the difference waves in a 200 and 400 ms post-feedback interval. Note that larger negative values correspond to larger Ne/ERN and FRN amplitudes. We selected FCz for analyses based on visual inspection of the waveforms and the corresponding scalp topographies, which showed a fronto-central maximum of Ne and FRN (see Figure 3). Following previous studies (Hajcak et al., 2004; Wiswede et al., 2009), the Pe was measured as the mean amplitude between 200 and 400 ms after the response at electrode Pz.
To examine learning-related changes in Ne/ERN, FRN, and Pe (Holroyd and Coles, 2002), EEG epochs were averaged separately for each incentive condition across the first (Bin 1) and the second half of trials within each block (Bin 2). This analysis, however, included only a subsample of 68 participants (46 female, 22 male), which committed at least 15 errors in each learning condition in both halves of the learning task.
Statistical analyses
Pearson's correlations were calculated to examine the relationships between personality measures, accuracy, and ERP components. Differences between correlation coefficients were tested using the Hotelling–William test (Steiger, 1980). Learning-related effects in the behavioral and ERP data were analyzed using repeated measures analyses of covariance (ANCOVAs) with BIS/BAS as continuous between-subjects factors. The covariates were mean-centered before entering the analysis (Delaney and Maxwell, 1981). To test for differential effects of personality measures in males and females, we performed the same set of analyses with gender as a further between-subjects factor. These analyses did not yield evidence for moderator effects of gender and thus are not reported in the following. In order to test for moderator effects of punishment and reward sensitivity on the relationship between the ERP components and behavioral adjustments, we used multiple regression analyses. The regression models included BIS/BAS, ERP amplitude (Ne/ERN vs. FRN vs. Pe), and the corresponding cross-product terms as predictors and overall accuracy vs. post-error accuracy as criterion. Separate models were tested for the deterministic and probabilistic learning condition. Similar to the ANCOVAs, the independent variables were mean-centered.
Whenever necessary, the Geisser-Greenhouse correction was applied (Geisser and Greenhouse, 1958) and corrected p-values are reported together with the uncorrected degrees of freedom and the epsilon-values (ε). Planned comparisons were performed to decompose significant high-level interactions.
Results
The results section is structures into three parts. In the first part, we will report the analyses of the behavioral data (RT, accuracy, post-error accuracy). The second part involves the correlation analyses of behavioral, personality, and ERP measures. In the third part, we will present the findings on the effects of personality on learning-related modulations of the ERP components.
Reaction Times
Mean RTs for correct and incorrect trials were 443 ms (SD = 33 ms) and 434 ms (SD = 52 ms) in the deterministic learning condition, 451 ms (SD = 36 ms) and 450 ms (SD = 45 ms) in the probabilistic learning condition, and 454 ms (SD = 41 ms) and 454 ms (SD = 42 ms) in the chance condition. Response latencies were subjected to an ANCOVA with the within-subjects factors learning condition (deterministic, probabilistic, and chance condition, bin (Bins 1–6), and correctness (correct vs. incorrect) and the continuous between-subjects factors BIS and BAS. The analysis yielded significant main effects of learning condition [F(2, 204) = 74.80, p < 0.001, η2 = 0.43] and bin [F(5, 510) = 211.34, p < 0.001, η2 = 0.68, ε = 0.38]. Contrasts revealed shorter RTs in the deterministic and probabilistic learning condition compared to the chance condition as well as in the deterministic compared to the probabilistic learning condition (ps < 0.001). Response latencies decreased from Bin 1 to Bin 6, reflected in a significant linear trend across the bins (p < 0.001). There were also a main effect of correctness [F(1, 102) = 72.95, p < 0.001, η2 = 0.42], an interaction between bin and correctness [F(5, 510) = 15.99, p < 0.001, η2 = 0.14, ε = 0.76], and an interaction between learning condition, bin, and correctness [F(10, 1020) = 11.52, p < 0.001, η2 = 0.10, ε = 0.71]. Decomposing the Three-Way interaction yielded significant bin × correctness interactions for the deterministic and probabilistic learning condition (ps < 0.001, η2s > 0.06), indicating that RTs decreased more strongly for erroneous compared to correct responses with increasing time on task in these two conditions. Thus, toward the end of the learning task, RTs were faster on erroneous compared to correct trials, particularly in the deterministic learning condition. No main effect or interaction involving BIS/BAS approached significance.
Accuracy
Overall accuracy
An ANCOVA with the within-subjects factors learning condition and bin and the continuous between-subjects factors BIS and BAS yielded a reliable main effect of learning condition [F(2, 204) = 533.17, p < 0.001, η2 = 0.84], indicating that accuracy was higher in the deterministic and probabilistic learning condition compared to the chance condition as well as in the deterministic compared to the probabilistic learning condition (ps < 0.001, η2s > 0.53) (see Figure 2). Moreover, we found a reliable main effect of bin [F(5, 510) = 56.92, p < 0.001, η2 = 0.36, ε = 0.79] that was qualified by an interaction between learning condition and bin [F(5, 510) = 14.28, p < 0.001, η2 = 0.12, ε = 0.81]. Contrasts revealed a significant interaction when comparing the linear increase of accuracy across bins for deterministic and probabilistic learning condition to the linear increase in the chance condition [F(1, 102) = 47.72, p < 0.001, η2 = 0.32], but not for the deterministic compared to the probabilistic learning condition (p = 0.64). As can be seen from Figure 2, these findings indicate that accuracy increased across bins in the deterministic and probabilistic learning condition but not in the chance condition. Furthermore, we obtained significant quadratic and cubic interactions between learning condition and bin (p < 0.001 and 0.01, η2 = 0.34 and 0.09, respectively), reflecting that accuracy increased only from Bin 1 to Bin 3 and reached asymptote thereafter.
Figure 2. (A) Learning curves (mean accuracy) for the three learning conditions. (B) Mean post-error accuracy rates (collapsed across bins) for the three learning conditions. Error bars indicate standard error.
As was indicated by a significant main effect of BIS [F(1, 102) = 5.24, p < 0.05, η2 = 0.05], higher punishment sensitivity predicted lower overall accuracy (partial r = −0.20, p < 0.05). By contrast, a reliable main effect of BAS [F(1, 102) = 4.88, p < 0.05, η2 = 0.05] and an interaction between BAS and learning condition [F(2, 210) = 3.52, p < 0.05, η2 = 0.03], showed that higher reward sensitivity was associated with higher overall accuracy in the deterministic learning condition (partial r = 0.25, p < 0.05) but not in probabilistic learning or chance condition (ps > 0.10).
Post-error accuracy
Mean post-error accuracy rates (see Figure 2) were subjected to an ANCOVA with the within-subject factor learning condition and the continuous between-subjects factors BIS and BAS. The analysis revealed significant main effects of learning condition [F(2, 210) = 132.09, p < 0.001, η2 = 0.57]. Contrasts revealed post-error accuracy to be higher in the deterministic compared to the probabilistic learning condition as well as for the two learning conditions compared to the chance condition (ps < 0.001, η2s > 0.22).
Moreover, we found a main effect of BAS [F(1, 102) = 5.44, p < 0.05, η2 = 0.05] and an interaction between BAS and learning condition [F(2, 210) = 3.79, p < 0.05, η2 = 0.04]. Similar to the findings for overall accuracy, higher reward sensitivity was associated with higher post-error accuracy in the deterministic learning condition only (partial r = 0.26, p < 0.01). In contrast to overall accuracy, post-error accuracy did not relate to punishment sensitivity.
Correlations between Personality, Behavior, and ERP Components
Figure 3 displays the response- and feedback-locked ERPs on correct and incorrect trials, separately for the three learning conditions. The Ne/ERN and the FRN were evident as negative going deflections over fronto-central scalp regions, whereas the Pe was evident as a centro-parietally distributed positive slow wave. Bivariate correlations between Ne/ERN, FRN, Pe, personality measures, and behavior are shown in Table 2 and Table 3, separately for the deterministic and probabilistic learning condition, respectively.
Figure 3. Response- and feedback-locked ERPs on correct (dashed lines) and incorrect (solid lines) trials and corresponding topographical maps, displayed separately for the three learning conditions.
Table 2. Pearson's correlations between personality measures, behavioral measures, and ERP components in the deterministic learning condition.
Table 3. Pearson's correlations between personality measures, behavioral measures, and ERP components in the probabilistic learning condition.
Ne/ERN
Contrary to our predictions, we did not observe a significant relationship between BIS score and Ne/ERN measures in either learning condition (|rs| < 0.08, ps > 0.42). Instead, higher BAS scores were related to larger (i.e., more negative) ΔNe/ERN amplitude in the deterministic learning condition (r = −0.25, p < 0.05). However, this latter correlation failed to reach significance after partialling out the influence of overall accuracy and post-error accuracy (p = 0.37). As illustrated in Figure 4, larger Ne/ERN amplitudes were also associated with higher overall accuracy and post-error accuracy in both the deterministic and probabilistic learning condition (rs < −0.33, ps < 0.001). Considering that the negative correlation between BIS and overall performance may have disguised a relationship between BIS and Ne/ERN, we conducted partial correlations controlling for overall accuracy. Nonetheless, the correlation between punishment sensitivity and Ne/ERN remained non-significant (rs < 0.09, ps > 0.39).
Figure 4. Scatterplots showing the relationships between the ERP components (Ne/ERN, FRN, Pe) and learning performance (overall accuracy, post-error accuracy), and personality (BIS, BAS). The first row shows the correlation between the Ne/ERN (measured peak-to-peak) and learning performance (left) and BIS (right). The second row displays the correlation between the FRN (measured peak-to-peak) and learning performance (left) and BIS (right). The first row shows the correlation between the ΔPe and learning performance (left) and BAS (right).
FRN
As expected, higher BIS scores were related to larger FRN amplitudes in the deterministic (r = −0.29, p < 0.01), probabilistic (r = −0.26, p < 0.05), and chance condition (r = −0.27, p < 0.01) (see Figure 4). Similarly, self-reported punishment sensitivity correlated with the ΔFRN in the probabilistic learning and chance condition (rs < −0.19, ps < 0.05). In contrast to the Ne/ERN, however, the FRN was largely unrelated to learning performance and post-error accuracy. Only in the deterministic learning condition, ΔFRN correlated with overall accuracy (r = 0.31, p < 0.01). Since previous studies reported an association between punishment sensitivity and larger FRN amplitudes to positive feedback (Balconi and Crivelli, 2010; Santesso et al., 2011b), we additionally tested the correlation between BIS/BAS scores and the FRN on correct trials. The analyses only revealed a marginally significant correlation between punishment sensitivity and FRN amplitude in the chance condition (r = −0.18, p = 0.07; deterministic and probabilistic learning condition: ps > 0.15). Furthermore, we probed the relationship between punishment sensitivity and the FRN to invalid negative feedback in the probabilistic learning condition. The correlation coefficient was highly similar to that observed for valid negative feedback (r = −0.25, p < 0.05), suggesting that the relationship between BIS and FRN was not modulated by the degree of expectancy violation.
Pe
Subjects scoring higher on BAS showed greater (i.e., more positive) Pe/ΔPe amplitudes in the deterministic learning condition (rs > 0.24, ps < 0.05) (see Figure 4) but not in the probabilistic learning condition (rs < 0.16, ps > 0.12). However, only for the ΔPe, there was a marginally significant difference between the two correlation coefficients [t(102) = 1.34, p < 0.10]. In addition, as displayed in Figure 4, larger ΔPe amplitudes were associated with higher overall accuracy in both learning conditions (rs > 0.22, ps < 0.05), whereas only in the deterministic learning condition, ΔPe was significantly related to post-error accuracy (r = 0.39, p < 0.001). To examine whether BAS and ΔPe contributed independently to learning performance in the deterministic learning condition, we included them as predictors in multiple regression analyses with overall and post-error accuracy as criterion. Higher overall accuracy was related to larger ΔPe amplitudes (β = 0.28, t = 2.87, p < 0.01), whereas the relationship with BAS was only marginally significant (β = 0.16, t = 1.69, p = 0.09). Similarly, higher post-error accuracy was significantly associated with larger ΔPe amplitudes (β = 0.34, t = 3.61, p < 0.001), but not with higher BAS scores (β = 0.17, t = 1.81, p = 0.06). These findings suggest that the positive relationship between reward sensitivity and learning performance was partly mediated by shared variance with the Pe. We also regressed the two accuracy measures as a function of ΔPe and Ne/ERN. These analyses revealed that higher overall accuracy in the deterministic learning condition was associated with both greater ΔPe (β = 0.28, t = 3.18, p < 0.01) and Ne/ERN amplitudes (β = −0.36, t = 4.10, p < 0.001). Likewise, the two components made independent contributions to post-error accuracy (|βs| > 0.33, |ts| > 4.08, ps < 0.001). Finally, in contrast to the Ne/ERN, ΔPe correlated negatively with error RT (rs < −0.20, ps < 0.05), reflecting that faster responses on error trials were associated with smaller ΔPe amplitudes.
Moderator Effects of BIS and BAS on the Relationship between ERP Components and Behavior
Previous research suggested that affect-related modulations in neuroelectric responses to errors may be associated with a stronger impact of these error signals on learning-related behavioral adaptation (Cavanagh et al., 2011a,b). Therefore, in a further step, we tested whether the relation between the ERP components and behavioral adjustments varies as a function of punishment/reward sensitivity. Separate moderated multiple regression models for the deterministic and probabilistic learning condition included BIS, BAS, ERP amplitude (Ne/ERN vs. FRN vs. Pe), and the corresponding interaction terms (i.e., Ne/ERN × BIS, Ne/ERN × BAS vs. FRN × BIS, FRN × BAS vs. Pe × BIS, Pe × BAS) as predictors and overall accuracy vs. post-error accuracy as criterion.
The interaction terms were non-significant in all analyses (|βs| < 0.18, |ts| < 1.60, ps > 0.10). Thus, we did not find evidence for a moderating effect of punishment sensitivity or reward sensitivity on the relationship between the ERP components (Ne/ERN, FRN, Pe) and learning performance in terms of overall accuracy or post-error accuracy.
Influence of Personality on Learning-Related Modulations in the ERP Components
For a subsample of 68 participants who committed enough errors in both halves of the learning task to obtain reliable measures of the ERP components, Ne/ERN, FRN, and Pe amplitudes were subjected to separate ANCOVAs with the within-subject factors learning condition (deterministic, probabilistic, and chance condition) and bin (Bin 1 vs. 2) and the continuous between-subjects factors BIS and BAS. For reasons of parsimony, we will only report analyses of the peak-to-peak measures of Ne/ERN and FRN as well as analyses of ΔPe amplitudes.
Ne/ERN
The ANCOVA yielded a significant main effect of learning condition [F(2, 130) = 47.28, p < 0.001, η2 = 0.42, ε = 0.82]. Contrasts revealed the Ne/ERN to be larger in the deterministic compared to the probabilistic learning condition and in the two learning conditions compared to the chance condition (ps < 0.01, η2s > 0.12) (see Figures 3 and 5). As was indicated by an interaction of learning condition and bin [F(2, 130) = 6.97, p < 0.01, η2 = 0.10], the Ne/ERN was differentially modulated over the course of learning in the three conditions. Follow-up comparisons showed a significant increase in Ne/ERN amplitude for the deterministic learning condition only [t(67) = 1.90, p < 0.05, one-tailed]. While the Ne/ERN did not reliably change from Bin 1 to Bin 2 in the probabilistic learning condition (p = 0.33), it decreased in the chance condition [t(67) = −3.31, p < 0.01, two-tailed].
Figure 5. Bar graphs show the amplitude of the Ne/ERN at FCz in Bin 1 and Bin 2 for (A) the total sample (error and correct trials) and (B) high vs. low BIS subjects (only error trials).
Furthermore, the analysis revealed a significant interaction between BIS, learning condition, and bin [F(2, 130) = 3.80, p < 0.05, η2 = 0.06]. As can be seen from Figure 5, this interaction reflects that only highly punishment sensitive individuals showed a learning-related increase of the Ne/ERN in the deterministic learning condition, whereas the Ne/ERN did not change from Bin 1 to Bin 2 for less punishment sensitive individuals (defined by median split). Follow-up correlation analyses yielded a marginally significant relation between BIS and learning-related changes in Ne/ERN amplitude (Ne2—Ne1) in the deterministic learning condition (partial r = −0.24, p = 0.06). The correlation between punishment sensitivity and Ne/ERN, however, was non-significant both in Bin 1 and Bin 2 (partial rs < 0.17, ps > 0.18).
FRN
The analysis yielded a significant main effect of BIS only [F(1, 65) = 11.88, p < 0.01, η2 = 0.15], reflecting that higher punishment sensitivity predicted larger FRN amplitudes (partial r = −0.38, p < 0.01; FRN collapsed across bins and learning conditions). Figure 6 illustrates that the FRN did not reliably change over the course of learning in either the deterministic or probabilistic learning condition.
Figure 6. Bar graphs show the amplitude of the FRN at FCz in Bin 1 and Bin 2 for “correct” and “incorrect” feedback.
Pe
The ANCOVA revealed a reliable main effect of learning condition [F(2, 130) = 42.84, p < 0.001, η2 = 0.40]. As can be seen from Figure 7 (see also Figure 3), the ΔPe was larger for the deterministic compared to the probabilistic learning conditions as well as for the two learning conditions compared to the chance condition (ps < 0.01, η2s > 0.12). Furthermore, we found a significant main effect of bin [F(1, 65) = 43.11, p < 0.001, η2 = 0.40] and an interaction between learning condition and bin [F(2, 130) = 11.13, p < 0.001, η2 = 0.15]. Contrasts revealed that the learning-related changes in the ΔPe were larger for the deterministic and probabilistic learning condition compared to the chance condition (p < 0.001, η2 = 0.25), but did not differ between the two learning conditions (p = 0.79). Follow-up comparisons confirmed that the ΔPe increased with learning in the deterministic and probabilistic learning condition only (ps < 0.001).
Figure 7. Bar graphs show the amplitude of the ΔPe at Pz in Bin 1 and Bin 2 for (A) the total sample and (B) high vs. low BAS subjects.
In addition, the analysis revealed a significant interaction between BAS, learning condition, and bin [F(2, 130) = 3.61, p < 0.05, η2 = 0.05]. Contrasts showed that the interaction between bin and BAS differed for the deterministic compared to the probabilistic learning condition (p < 0.05, η2 = 0.07) but not for the two learning conditions compared to the chance condition (p = 0.10). Figure 7 illustrates that highly reward sensitive individuals showed a more pronounced learning-related increase in ΔPe amplitude in the deterministic learning condition than subjects with lower BAS scores. In line with this, correlation analyses yielded a significant relationship between reward sensitivity and the increase of ΔPe from Bin 1 to Bin 2 (ΔPe2—ΔPe1) for the deterministic learning condition (partial r = 0.29, p < 0.05). Notably, we found a significant correlation between BAS and ΔPe in Bin 2 (partial r = 0.29, p < 0.05) but not in Bin 1 (p = 0.81).
Summary of Main Findings
Analyses of accuracy data showed that higher reward sensitivity was associated with better overall learning performance and higher post-error accuracy in the deterministic learning condition. Conversely, and contrary to our predictions, higher punishment sensitivity was associated with impaired performance both in the deterministic and probabilistic learning condition, but was not related to post-error accuracy in either of the two conditions.
Critically, correlation analyses did not reveal a significant relationship between punishment sensitivity and Ne/ERN. However, as expected, larger Ne/ERN amplitudes were associated with better learning performance and higher post-error accuracy. Moreover, punishment sensitivity modulated learning-related changes of the Ne/ERN. Only for highly punishment sensitive individuals, we found an increase of the Ne/ERN over the course of learning in the deterministic learning condition.
In line with prior studies, higher punishment sensitivity was associated with enhanced FRN amplitudes. Interestingly, this relationship appeared to be insensitive to feedback validity. In contrast to the Ne/ERN, the FRN was not clearly related to learning performance.
Furthermore, the present results replicate prior findings that higher reward sensitivity relates to larger Pe amplitudes, but this was only the case toward the end of learning in the deterministic learning condition. Moreover, participants highly sensitive to reward showed a more pronounced learning-related increase of the Pe in the deterministic learning condition. Similar to the Ne/ERN, greater Pe amplitudes were associated higher overall and post-error accuracy.
Finally, we found no evidence that individual differences in punishment or reward sensitivity modulate the relationship between error- and feedback-processing—as reflected in the Ne/ERN, FRN, and Pe—and learning-related behavioral adjustments.
Discussion
Numerous reports have suggested that individual differences in punishment (BIS/FFFS) and reward sensitivity (BAS) are reflected in neurocognitive mechanisms of error and feedback processing. The main goal of the present investigation was to further examine the impact of these interactions between affect-related traits and action monitoring on the ability to use error signals for behavioral adaptation during reinforcement learning. In contrast to previous studies employing simple motor tasks, such as the Flankers and Go/No-Go task (Boksem et al., 2006, 2008; Amodio et al., 2008), we found no relation between punishment sensitivity and the Ne/ERN. However, consistent with past research, higher punishment sensitivity was related to larger FRN amplitudes (Balconi and Crivelli, 2010; De Pascalis et al., 2010; Santesso et al., 2011b). These results indicate that highly punishment sensitive individuals were characterized by an enhanced responsivity to external rather than internal error cues. Furthermore, higher reward sensitivity was associated with increased neural responses during later stages of error processing as reflected in the Pe, replicating prior findings (Boksem et al., 2006, 2008). Although both FRN and Pe are thought to play a functional role in post-error adaptation, only reward sensitivity was related to better overall learning performance and higher post-error accuracy. By contrast, participants with higher trait sensitivity to punishment showed impaired learning performance.
The negative correlation between punishment sensitivity and overall accuracy was somewhat surprising, as higher BIS-reactivity has been claimed to trigger enhanced attention and information processing (Gray and McNaughton, 2000; Smillie, 2008). Still, BIS-activation has also been linked to anxious rumination and worry, which might interfere with task-related processing such as updating of S-R mappings. Moreover, as was pointed out by Pickering and colleagues (1997), learning tasks involving both rewards and punishments can cause mutually inhibitory interactions between BIS/FFFS and BAS. One should note that learning was accompanied by an increasing proportion of positive feedback, perhaps shifting the balance between the two systems toward a relative dominance of the BAS. Thus, relatively stronger reward reactivity may have contributed to better overall performance in less punishment sensitive individuals by facilitating appetitive learning or proactive engagement (Corr, 2004; Braver et al., 2007).
Given the comparatively large sample size, the lack of BIS/FFFS-related variations in Ne/ERN amplitude was unlikely to reflect insufficient statistical power, at least if the effect size is assumed to be small to moderate. One might argue that the negative correlation between punishment sensitivity and overall accuracy on the one hand, and the positive correlation between overall accuracy and Ne/ERN magnitude on the other hand, have neutralized the relationship between punishment sensitivity and Ne/ERN. Partial correlation analysis controlling for overall learning performance suggested that this was not the case. There is also no indication that the correlation coefficient was deflated due to restricted variability of BIS scores. However, the Ne/ERN was relatively small as is typically the case when using probabilistic learning tasks, in which participants are less certain about the correctness of their responses. It is thus possible that reduced variability of the Ne/ERN has decreased the probability of obtaining a significant correlation with punishment sensitivity.
Otherwise, it has been suggested that the delivery of trial-to-trial performance feedback leads participants to rely more strongly on external than internal error cues (Nieuwenhuis et al., 2005). This might be especially true for individuals highly sensitive to punishment as they appear to be characterized by low-level personal agency, which means that their actions are controlled by environmental cues rather than internal standards (Balconi and Crivelli, 2010). The unique association between punishment sensitivity and FRN found in the present study is consistent with this view. Interestingly, the relationship did not vary as a function of feedback validity or learning, suggesting that highly punishment sensitive individuals were generally more vigilant to negative feedback cues, irrespective of whether they were unexpected or not. Moreover, we found no clear evidence for a relation between punishment sensitivity and the FRN to positive feedback, consistent with what has been reported for individuals high in trait negative affect as well as moderately depressed subjects (Tucker et al., 2003; Sato et al., 2005; Santesso et al., 2011a). Thus, while punishment sensitivity has also been shown to be associated with an increased FRN elicited by unexpected (large) rewards (Santesso et al., 2011b), our findings indicate that highly punishment sensitive individuals are particularly characterized by enhanced mPFC responses to environmental cues signaling punishment. However, future studies should determine under what circumstances positive feedback elicits increased FRN amplitudes in highly punishment sensitive and whether these modulations reflect blunted responses to reward or higher vigilance to both positive and negative performance feedback.
Although high trait-level sensitivity to punishment was not associated with an overall enhancement of Ne/ERN amplitudes, self-reported BIS/FFFS-reactivity modulated learning-related changes of this component. The Ne/ERN increased with learning of the S-R mappings only for highly punishment sensitive individuals in deterministic learning condition, whereas no learning-related changes in Ne/ERN amplitude were observed for less punishment sensitive individuals or in the probabilistic learning condition. An explanation of this finding could be that highly punishment sensitive individuals were less prone to motivational disengagement. Punishment sensitivity has been linked to higher persistence, reflected in a relatively smaller decrease in behavioral performance and Ne/ERN amplitude with increasing time on task (Boksem et al., 2006; Tops and Boksem, 2010). Thus, disengagement could have attenuated a learning-related enhancement of the Ne/ERN more clearly for individuals with low compared to high BIS scores. This explanation, however, leaves open the question of why higher punishment sensitivity was related to worse overall performance. Further studies are necessary to clarify whether this might reflect differences ability to use positive feedback for behavioral adaptation.
Previous ERP studies have shown that BIS/FFFS-related differences in mPFC functioning are more pronounced in aversive compared to appetitive motivational contexts and in response to intense negative events (Boksem et al., 2008; Santesso et al., 2011a). The motivational context could be an important determinant of whether or not punishment sensitivity is also reflected in higher responsivity to internal indicators of response errors, even if continuous external performance feedback is provided. Indeed, we recently found that highly punishment sensitive participants showed a larger Ne/ERN to errors resulting in loss or gain omission during a learning task involving trial-to-trial manipulation of incentive value (Unger and Kray, in preparation). By contrast, consistent with the present results, punishment sensitivity did not relate to Ne/ERN amplitude on neutral trials. Interestingly, the association between punishment sensitivity and Ne/ERN was stronger at the beginning than at the end of learning, arguing against the view that undetermined S-R mappings per se account for the present null-finding. Under threatening conditions, activity of the medial prefrontal performance monitoring system appears to be more sensitive to individual differences in self-reported BIS/FFFS-reactivity when the optimal course of action is uncertain and cognitive control demands are high.
According to a recent proposal, the ACC integrates punishment-related information from multiple sources in order to support instrumental behaviors, particularly in unstable and threatening environments (Shackman et al., 2011). From this perspective, the relation between punishment sensitivity and FRN might reflect that affect-related traits bias cognitive processing and regulate action selection in accordance with an individual's overarching goals and beliefs (Huys and Dayan, 2009; Cavanagh et al., 2011a). Even so despite the proposed link between FRN and future behavioral adaptation (Holroyd and Coles, 2002; Frank et al., 2005), accuracy data suggest that larger error signals to negative feedback in highly punishment sensitive individuals were not beneficial for learning or may even reflect dysfunctional processing. One interpretation of this finding could be that the FRN enhancement is primarily related to the regulation of negative emotions (Pizzagalli, 2011; Santesso et al., 2011a,b). The ACC has been assigned an important role in controlling amygdala responsivity to fear-related stimuli. Dysregulated interactions between ACC and amygdala may be associated with a negative processing bias that is reflected in enhanced attentional capture by potential threat cues, anxious rumination, and inability to disengage from negative events and have been linked to anxiety and depression (Bishop, 2007; Pizzagalli, 2011). Moreover, it may be important to consider that rapid trial-to-trial adjustments as assessed in the current investigation are thought to primarily reflect explicit/declarative learning (Frank et al., 2007b). Previous research, however, suggests that individual differences in punishment sensitivity rather affect implicit/habitual learning. In particular, Cavanagh and colleagues (2011a,b) showed that increased mPFC responses to negative feedback in punishment hypersensitive participants were specifically associated with alterations in slow integrative avoidance learning, presumably mediated by phylogenetically old non-declarative learning systems.
The second set of findings from our study concerns the relationship between reward sensitivity and Pe. In line with previous reports (Boksem et al., 2006, 2008), self-reported reward sensitivity correlated positively with the magnitude of the Pe. However, this relationship was only significant during later stages of learning in the deterministic learning condition, indicating that it depended on the participants' ability to internally represent the correct response. Further corroborating this notion, higher BAS scores were related to a more pronounced learning-related increase in Pe amplitude in the deterministic learning condition.
Drawing on the proposal that there is a link between approach motivation and a bias toward proactive control (Braver et al., 2007), Boksem and colleagues (2006, 2008) suggested that larger Pe amplitudes in highly reward sensitive individuals are functionally related to subsequent engagement in proactive behaviors. Our finding that greater Pe amplitudes were associated with higher overall accuracy and post-error accuracy seems consistent with the proposed link. Although strictly speaking, for action control to be implemented proactively, predictive contextual cues have to be present prior to the imperative stimulus (Braver et al., 2007). This is typically not the case during reinforcement learning, presumably limiting the utility of proactive strategies in a narrow sense. Nonetheless, it is conceivable that highly reward sensitive individuals tend to respond to errors with positive approach behaviors such as reactivation of the potentially disrupted representation of the correct S-R mappings. The idea that BAS-related modulations of the Pe reflect active updating of task-set representations in working memory corresponds to previous reports stressing the morphological and functional similarity between the Pe and the stimulus-evoked P300 (Leuthold and Sommer, 1999; Davies et al., 2001; Overbeek et al., 2005). In this regard, it seems noteworthy that high reward sensitivity has also been found to be associated with enhanced P300 amplitudes to negative feedback (Balconi and Crivelli, 2010).
Although the neurobiological basis of the BAS has been described in terms of dopaminergic mechanisms (Gray and McNaughton, 2000; Smillie, 2008), the Pe and the P300 have primarily been linked to noradrenergic neurotransmission (Nieuwenhuis et al., 2005; Frank et al., 2007a). Moreover, the Pe has previously been found to be affected by functional polymorphisms of the serotonin transporter gene, possibly mediated by its regulatory influence on the amygdala (Althaus et al., 2009; but see Beste et al., 2010). Despite the pivotal role that dopamine is assumed to play in the generation of the FRN (Holroyd and Coles, 2002), serotonergic functioning is also likely to be involved in the observed relationship between punishment sensitivity and FRN amplitude. Several reports showed that genetic and pharmacological variations in serotonergic neurotransmission are accompanied by changes in mPFC responses to errors and conflict as well as amygdala/hippocampus reactivity to aversive and threatening stimuli (Canli et al., 2005; Cools et al., 2005; Chamberlain et al., 2006; Harmer et al., 2006; Finger et al., 2007). In addition, variations in serotonin transmission have been associated with individual differences in anxiety and depression-related traits (Sen et al., 2004). It has been proposed that the modulatory influence of serotonin on the prefrontal dopamine system may constitute the neurophysiological basis of altered action monitoring functions in individuals high in negative affectivity, including anxiety and depression (Beste et al., 2010). Clearly, more research is needed to determine whether opponency between the serotonergic and dopaminergic system underlies cognitive-affective interactions in learning and decision making (Cools et al., 2008; Jocham and Ullsperger, 2009).
Some limitations of the present study should be noted. First, the observed effects of personality measures on ERP correlates of error and feedback processing were rather small-sized (r ≤ 0.30), particularly when compared to the relationship between accuracy measures and ERP components. Although larger correlation coefficients have been reported in the literature, these were typically derived from small samples and hence likely to be inflated (Ioannidis, 2008). Note that the strength of the relations is already constrained by the internal reliability of the BIS/BAS measures (Cronbachs α = 0.73/0.59). Second, the current investigation included a very homogeneous sample of under-graduate university students. It is possible that higher correlations will be found in more heterogeneous samples such as clinical populations or different age groups. Finally, the present study reported only correlational data, leaving unspecified the direction of the observed effects.
To summarize and conclude, the present study shows that individual differences in punishment sensitivity are associated with larger FRN amplitudes, indicating an increased mPFC responsivity to negative performance feedback. However, the negative correlation between punishment sensitivity and overall accuracy suggests that the alterations in mPFC functioning are not beneficial for learning-related behavioral adaptation and may reflect non-adaptive forms of emotion regulation. Future research is needed to determine whether the negative processing bias specifically affects incremental habitual learning mechanisms rather than rapid trial-to-trial adjustments as assessed in the current task. Furthermore, higher reward sensitivity was related to larger Pe amplitudes and better learning performance, suggesting that self-reported BAS-reactivity is associated with an enhanced use of deliberate proactive strategies to support future performance. Importantly, the Pe and the Ne/ERN appeared to make independent contributions to overall learning performance and error-related behavioral adjustments, consistent with the notion that the two components reflect activity of separable action monitoring systems, which may mediate automatic vs. more controlled forms of post-error adaptation (cf. Ridderinkhof et al., 2009). In line with previous studies, the present findings indicate that individual differences in reward and punishment sensitivity are associated with unique functional alterations of these systems.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
This work was funded by the German Research Foundation (Deutsche Forschungsgesellschaft; grant IRTG 1457). We gratefully thank Michael Herbert, Anna Orth, Svenja Schieren, Verena Schnitzler, and Jenny Sinzig for help during data acquisition.
References
Althaus, M., Groen, Y., Wijers, A. A., Mulder, L. J., Minderaa, R. B., Kema, I. P., Dijck, J. D., Hartman, C. A., and Hoekstra, P. J. (2009). Differential effects of 5-HTTLPR and DRD2/ANKK1 polymorphisms on electrocortical measures of error and feedback processing in children. Clin. Neurophysiol. 120, 93–107.
Amodio, D. M., Master, S. L., Yee, C. M., and Taylor, S. E. (2008). Neurocognitive components of the behavioral psychophysiology inhibition and activation systems: implications for theories of self-regulation. Psychophysiology 45, 11–19.
Balconi, M., and Crivelli, D. (2010). FRN and P300 ERP effect modulation in response to feedback sensitivity: the contribution of punishment-reward system (BIS/BAS) and behaviour identification of action. Neurosci. Res. 66, 162–172.
Beaver, J. D., Lawrence, A. D., van Ditzhuijzen, J., Davis, M. H., Woods, A., and Calder, A. J. (2006). Individual differences in reward drive predict neural responses to images of food. J. Neurosci. 26, 5160–5166.
Beste, C., Domschke, K., Kolev, V., Yordanova, J., Baffa, A., Falkenstein, M., and Konrad, C. (2010). Functional 5-HT1a receptor polymorphism selectively modulates error-specific subprocesses of performance monitoring. Hum. Brain Mapp. 31, 621–630.
Bishop, S. J. (2007). Neurocognitive mechanisms of anxiety: an integrative account. Trends Cogn. Sci. 11, 307–316.
Boksem, M. A. S., Meijman, T. F., and Lorist, M. M. (2006). Mental fatigue, motivation and action monitoring. Biol. Psychol. 72, 123–132.
Boksem, M. A. S., Tops, M., Kostermans, E., and De Cremer, D. (2008). Sensitivity to punishment and reward omission: evidence from error-related ERP components. Biol. Psychol. 79, 185–192.
Braver, T. S., Gray, J. R., and Burgess, G. C. (2007). “Explaining the many varieties of working memory variation: dual mechanisms of cognitive control,” in Variation in Working Memory, eds A. R. A. Conway, C. Jarrold, M. Kane, A. Miyake, and J. N. Towse (Oxford: Oxford University Press), 76–106.
Canli, T., Omura, K., Haas, B. W., Fallgatter, A., Constable, R. T., and Lesch, K. P. (2005). Beyond affect: a role for genetic variation of the serotonin transporter in neural activation during a cognitive attention task. Proc. Natl. Acad. Sci. U.S.A. 102, 12224–12229.
Carver, C. S., and White, T. L. (1994). Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: the BIS/BAS scales. J. Pers. Soc. Psychol. 67, 319–333.
Cavanagh, J. F., and Allen, J. J. B. (2008). Multiple aspects of the stress response under social evaluative threat: an electrophysiological investigation. Psychoneuroendocrinology 33, 41–53.
Cavanagh, J. F., Frank, M. J., and Allen, J. J. B. (2011a). Social stress reactivity alters reward and punishment learning. Soc. Cogn. Affect. Neurosci. 6, 311–320.
Cavanagh, J. F., Bismark, A. J., Frank, M. J., and Allen, J. B. (2011b). Larger error signals in major depression are associated with better avoidance learning. Front. Psychol. 2:331. doi: 10.3389/fpsyg.2011.00331
Chamberlain, S. R., Muller, U., Blackwell, A. D., Clark, L., Robbins, T. W., and Sahakian, B. J. (2006). Neurochemical modulation of response inhibition and probabilistic learning in humans. Science 311, 861–863.
Cools, R., Calder, A. J., Lawrence, A. D., Clark, L., Bullmore, E., and Robbins, T. W. (2005). Individual differences in threat sensitivity predict serotonergic modulation of amygdala response to fearful faces. Psychopharmacology 180, 670–679.
Cools, R., Roberts, A. C., and Robbins, T. W. (2008). Serotoninergic regulation of emotional and behavioural control processes. Trends Cogn. Sci. 12, 31–40.
Corr, P. J. (2002). Gray's reinforcement sensitivity theory: tests of the joint subsystems hypothesis of anxiety and impulsivity. Pers. Individ. Dif. 33, 511–532.
Corr, P. J. (2004). Reinforcement sensitivity theory and personality. Neurosci. Biobehav. Rev. 28, 317–332.
Davies, P. L., Segalowitz, S. J., Dywan, J., and Pailing, P. E. (2001). Error-negativity and positivity as they relate to other ERP indices of attentional control and stimulus processing. Biol. Psychol. 56, 191–206.
Delaney, H. D., and Maxwell, S. E. (1981). On using analysis of covariance in repeated measures designs. Multivariate Behav. Res. 16, 105–123.
Dennis, T. A., and Chen, C. C. (2009). Trait anxiety and conflict monitoring following threat: an ERP study. Psychophysiology 46, 122–131.
De Pascalis, V., Varriale, V., and D'Antuono, L. (2010). Event-related components of the punishment and reward sensitivity. Clin. Neurophysiol. 121, 60–76.
Depue, R. A., and Collins, P. F. (1999). Neurobiology of the structure of personality: dopamine, facilitation of incentive motivation, and extraversion. Behav. Brain Sci. 22, 491–517.
Devinsky, O., Morrell, M. J., and Vogt, B. A. (1995). Contributions of anteror cingulate cortex to behaviour. Brain 118, 279–306.
Endrass, T., Reuter, B., and Kathmann, N. (2007). ERP correlates of conscious error recognition: aware and unaware errors in an antisaccade task. Eur. J. Neurosci. 26, 1714–1720.
Eppinger, B., Kray, J., Mock, B., and Mecklinger, A. (2008). Better or worse than expected? Aging, learning, and the Ne/ERN. Neuropsychologia 46, 521–539.
Falkenstein, M., Hohnsbein, J., Hoormann, J., and Blanke, L. (1990). “Effects of errors in choice reaction tasks on the ERP under focused and divided attention,” in Psychophysiological Brain Research, eds C. H. M. Brunia, A. W. K. Gaillard, and A. Kok (Tilburg: TilburgUniversity Press), 192–195.
Finger, E. C., Marsh, A. A., Buzas, B., Kamel, N., Rhodes, R., Vythilingham, M., Pine, D. S., Goldman, D., and Blair, J. R. (2007). The impact of tryptophan depletion and 5-HTTLPR genotype on passive avoidance and response reversal instrumental learning tasks. Neuropsychopharmacology 32, 206–215.
Frank, M. J. (2005). Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J. Cogn. Neurosci. 17, 51–72.
Frank, M. J., D'Lauro, C., and Curran, T. (2007a). Cross-task individual differences in error processing: neural, electrophysiological, and genetic components. Cogn. Affect. Behav. Neurosci. 7, 297–308.
Frank, M. J., Moustafa, A. A., Haughey, H. M., Curran, T., and Hutchison, K. E. (2007b). Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc. Natl. Acad. Sci. U.S.A. 104, 16311–16316.
Frank, M. J., Woroch, B. S., and Curran, T. (2005). Error-related negativity predicts reinforcement learning and conflict biases. Neuron 47, 495–501.
Gehring, J. W., Goss, B., Coles, M. G., Meyer, D. E., and Donchin, E. (1993). A neural system for error detection and compensation. Psychol. Sci. 4, 385–390.
Gehring, W. J., and Willoughby, A. R. (2002). The medial frontal cortex and the rapid processing of monetary gains and losses. Science 295, 2279–2282.
Geisser, S., and Greenhouse, S. W. (1958). An extension of box's results on the use of the F-distribution in multivariate analysis. Ann. Math. Stat. 29, 885–891.
Gray, J. A. (1982). The Neuropsychology of Anxiety: An Enquiry into the Functions of the Septo-Hippocampal System. Oxford: Oxford University Press.
Gray, J. A., and McNaughton, N. (2000). The Neuropsychology of Anxiety: An Enquiry into the Functions of the Septo-Hippocampal System. Oxford: Oxford University Press.
Gründler, T. O. J., Cavanagh, J. F., Figueroa, C. M., Frank, M. J., and Allen, J. J. B. (2009). Task-related dissociation in ERN amplitude as a function of obsessive-compulsive symptoms. Neuropsychologia 47, 1978–1987.
Hahn, T., Dresler, T., Ehlis, A. C., Plichta, M. M., Heinzel, S., Polak, T., Lesch, K. P., Breuer, F., Jakob, P. M., and Fallgatter, A. J. (2009). Neural response to reward anticipation is modulated by Gray's impulsivity. Neuroimage 46, 1148–1153.
Hajcak, G., McDonald, N., and Simons, R. F. (2003). Anxiety and error-related brain activity. Biol. Psychol. 64, 77–90.
Hajcak, G., McDonald, N., and Simons, R. F. (2004). Error-related psychophysiology and negative affect. Brain Cogn. 56, 189–197.
Harmer, C. J., Mackay, C. E., Reid, C. B., Cowen, P. J., and Goodwin, G. M. (2006). Antidepressant drug treatment modifies the neural processing of nonconscious threat cues. Biol. Psychiatry 59, 816–820.
Holroyd, C. B., and Coles, M. G. (2002). The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev. 109, 679–709.
Huys, Q. J., and Dayan, P. (2009). A Bayesian formulation of behavioral control. Cognition 113, 314–328.
Ioannidis, J. P. (2008). Why most discovered true associations are inflated. Epidemiology 19, 640–648.
Jocham, G., and Ullsperger, M. (2009). Neuropharmacology of performance monitoring. Neurosci. Biobehav. Rev. 33, 48–60.
Kuhl, J. (1994). “Action and state orientation: psychometric properties of the action control scales (ACS-90),” in Volition and Personality: Action versus State Orientation, eds J. Kuhl and J. Beckmann (Göttingen: Hogrefe), 47–59.
Leone, L., Perugini, M., Bagozzi, R. P., Pierro, A., and Mannetti, L. (2001). Construct validity and generalizability of the carver-white behavioural inhibition system/behavioural activation system scales. Eur. J. Pers. 15, 373–390.
Leuthold, H., and Sommer, W. (1999). ERP correlates of error processing in spatial S-R compatibility tasks. Clin. Neurophysiol. 110, 342–357.
McNaughton, N., and Corr, P. J. (2004). A two-dimensional neuropsychology of defense: fear/anxiety and defensive distance. Neurosci. Biobehav. Rev. 28, 285–305.
Miltner, W. H. R., Braun, C. H., and Coles, M. G. H. (1997). Event-related brain potentials following incorrect feedback in a time estimation task: evidence for a “generic” neural system for error detection. J. Cogn. Neurosci. 9, 788–798.
Nieuwenhuis, S., Nielen, M. M., Mol, N., Hajcak, G., and Veltman, D. J. (2005). Performance monitoring in obsessive–compulsive disorder. Psychiatry Res. 134, 111–122.
Nieuwenhuis, S., Ridderinkhof, K. R., Blom, J., Band, G. P. H., and Kok, A. (2001). Error-related brain potentials are differentially related to awareness of response errors: evidence from an antisaccade task. Psychophysiology 38, 752–760.
O'Connell, R. G., Dockree, P. M., Bellgrove, M. A., Kelly, S. P., Hester, R., Garavan, H., and Foxe, J. J. (2007). The role of cingulate cortex in the detection of errors with and without awareness: a high-density electrical mapping study. Eur. J. Neurosci. 25, 2571–2579.
Overbeek, T. J. M., Nieuwenhuis, S., and Ridderinkhof, K. R. (2005). Dissociable components of error processing-on the functional significance of the Pe vis-à-vis the ERN/Ne. J. Psychophysiol. 19, 319–329.
Pickering, A. D., Corr, P. J., Powell, J. H., Kumari, V., Thornton, J. C., and Gray, J. A. (1997). “Individual differences in reactions to reinforcing stimuli are neither black nor white: to what extent are they Gray?” in The Scientific Study of Human Nature: Tribute to Hans J. Eysenck At Eighty, ed H. Nyborg (London: Elsevier Sciences), 36–67.
Pickering, A. D., and Gray, J. A. (2001). “Dopamine, appetitive reinforcement, and the neuropsychology of human learning: an individual differences approach,” in Advances in Individual Differences Research, eds A. Eliaszand and A. Angleitner (Lengerich, Germany: PABST Science Publishers), 113–149.
Pizzagalli, D. A. (2011). Frontocingulate dysfunction in depression: towards biomarkers of treatment response. Neuropsychopharmacol. Rev. 36, 183–206.
Ridderinkhof, K. R., Ramautar, J. R., and Wijnen, J. G. (2009). To Pe or not to Pe: a P3-like ERP component reflecting the processing of response errors. Psychophysiology 46, 531–538.
Ridderinkhof, K. R., Ullsperger, M., Crone, E. A., and Nieuwenhuis, S. (2004). The role of the medial frontal cortex in cognitive control. Science 306, 443–446.
Santesso, D. L., Bogdan, R., Birk, J. L., Goetz, E. L., Holmes, A. J., and Pizzagalli, D. A. (2011a). Neural responses to negative feedback are related to negative emotionality in healthy adults. Soc. Cogn. Affect. Neurosci. doi: 10.1093/scan/nsr054. [Epub ahead of print].
Santesso, D. L., Dzyundzyak, A., and Segalowitz, S. J. (2011b). Age, sex and individual differences in punishment sensitivity: factors influencing the feedback-related negativity. Psychophysiology 48, 1481–1489.
Sato, A., Yasuda, A., Ohira, H., Miyawaki, K., Nishikawa, M., Kumano, H., and Kuboki, T. (2005). Effects of value and reward magnitude on feedback negativity and P300. Neuroreport 16, 407–411.
Sen, S., Burmeister, M., and Ghosh, D. (2004). Meta-analysis of the association between a serotonin transporter promoter polymorphism (5-HTTLPR) and anxiety-related personality traits. Am. J. Med. Genet. 127B, 85–89.
Shackman, A. J., Salomons, T. V., Slagter, H. A., Fox, A. S., Winter, J. J., and Davidson, R. J. (2011). The integration of negative affect, pain and cognitive control in the cingulated cortex. Nat. Rev. Neurosci. 12, 154–167.
Simon, J. J., Walther, S., Fiebach, C. J., Friederich, H. C., Stippich, C., Weisbrod, M., and Kaiser, S. (2010). Neural reward processing is modulated by approach- and avoidance-related personality traits. Neuroimage 49, 1868–1874.
Smillie, L. D. (2008). What is reinforcement sensitivity? Neuroscience paradigms for approach-avoidance process theories of personality. Eur. J. Pers. 22, 359–384.
Snodgrass, J. G., and Vanderwart, M. (1980). A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. J. Exp. Psychol. Hum. Learn. Mem. 6, 174–215.
Steiger, J. H. (1980). Tests for comparing elements of a correlation matrix. Psychol. Bull. 87, 245–251.
Strobel, A., Beauducel, A., Debener, S., and Brocke, B. (2001). Psychometrische und strukturelle Merkmale einer deutschsprachigen Version des BIS/BAS Fragebogens von Carver und White [Psychometric and structural features of a German version of the BIS/BAS scales of Carver and White]. Zeitschrift für Differentielle und Diagnostische Psychologie 22, 216–227.
Taylor, S. F., Stern, E. R., and Gehring, W. J. (2007). Neural systems for error monitoring: recent findings and theoretical perspectives. Neuroscientist 13, 160–172.
Tops, M., and Boksem, M. A. S. (2010). Absorbed in the task. Personality measures predict engagement during task performance as tracked by error negativity and asymmetrical frontal activity. Cogn. Affect. Behav. Neurosci. 20, 441–453.
Tucker, D. M., Luu, P., Frishkoff, G., Quiring, J., and Poulsen, C. (2003). Frontolimbic response to negative feedback in clinical depression. J. Abnorm. Psychol. 112, 667–678.
Unger, K., Kray, J., and Mecklinger, A. (2012). Worse than feared? Failure induction modulates the electrophysiolgical signature of error monitoring during subsequent learning. Cogn. Affect. Behav. Neurosci. 12, 34–51.
Van Veen, V., and Carter, C. S. (2002). The timing of action-monitoring processes in the anterior cingulate cortex. J. Cogn. Neurosci. 14, 593–602.
Van den Berg, I., Franken, I. H., and Muris, P. (2011). Individual differences in sensitivity to reward: association with electrophysiological responses to monetary gains and losses. J. Psychophysiol. 25, 81–86.
van der Helden, J., Boksem, M. A. S., and Blom, J. H. G. (2010). The importance of failure: feedback related negativity predicts motor learning efficiency. Cereb. Cortex 20, 1596–1603.
Watson, D., Clark, L. A., and Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: the PANAS scales. J. Pers. Soc. Psychol. 54, 1063–1070.
Wiswede, D., Münte, T. F., and Rüsseler, J. (2009). Negative affect induced by derogatory verbal feedback modulates the neural signature of error detection. Soc. Cogn. Affect. Neurosci. 4, 227–237.
Yeung, N., Botvinick, M. M., and Cohen, J. D. (2004). The neural basis of error detection: conflict monitoring and the error-related negativity. Psychol. Rev. 111, 931–959.
Keywords: reinforcement learning, BIS, BAS, punishment sensitivity, reward sensitivity, error-related negativity (ERN), feedback-related negativity (FRN), error positivity (Pe)
Citation: Unger K, Heintz S and Kray J (2012) Punishment sensitivity modulates the processing of negative feedback but not error-induced learning. Front. Hum. Neurosci. 6:186. doi: 10.3389/fnhum.2012.00186
Received: 16 March 2012; Paper pending published: 28 March 2012;
Accepted: 04 June 2012; Published online: 27 June 2012.
Edited by:
Patrizia Thoma, Ruhr-University Bochum, GermanyReviewed by:
Josep Marco-Pallares, University of Barcelona, SpainMichael Falkenstein, Leibniz Research Centre für Working Environment and Human Factors, Germany
Copyright: © 2012 Unger, Heintz and Kray. This is an open-access article distributed under the terms of the Creative Commons Attribution Non Commercial License, which permits non-commercial use, distribution, and reproduction in other forums, provided the original authors and source are credited.
*Correspondence: Kerstin Unger, Department of Psychology, Development of Language, Learning, and Action, Saarland University, P. O. Box 15 11 50, D-66041 Saarbruecken, Germany. e-mail: k.unger@mx.uni-saarland.de