- 1 Laboratory of Experimental Neuropsychology, Neuropsychology Unit, Neurology Clinic, Geneva University Hospital, Geneva, Switzerland
- 2 Faculty of Psychology and Educational Science, University of Geneva, Geneva, Switzerland
- 3 Geneva Neuroscience Centre, University of Geneva, Geneva, Switzerland
- 4 Faculty of Education, Department of Learning Disabilities, Edmond J. Safra Brain Research Center for the Study of Learning Disabilities, University of Haifa, Haifa, Israel
A number of investigations have reported that emotional faces can be processed subliminally, and that they give rise to specific patterns of brain activation in the absence of awareness. Recent event-related potential (ERP) studies have suggested that electrophysiological differences occur early in time (<200 ms) in response to backward-masked emotional faces. These findings have been taken as evidence of a rapid non-conscious pathway, which would allow threatening stimuli to be processed rapidly and subsequently allow appropriate avoidance action to be taken. However, for this to be the case, subliminal processing should arise even if the threatening stimulus is not attended. This point has in fact not yet been clearly established. In this ERP study, we investigated whether subliminal processing of fearful faces occurs outside the focus of attention. Fourteen healthy participants performed a line judgment task while fearful and non-fearful (happy or neutral) faces were presented both subliminally and supraliminally. ERPs were compared across the four experimental conditions (i.e., subliminal and supraliminal; fearful and non-fearful). The earliest differences between fearful and non-fearful faces appeared as an enhanced posterior negativity for the former at 170 ms (the N170 component) over right temporo-occipital electrodes. This difference was observed for both subliminal (p < 0.05) and supraliminal presentations (p < 0.01). Our results confirm that subliminal processing of fearful faces occurs early in the course of visual processing, and more importantly, that this arises even when the subject’s attention is engaged in an incidental task.
Introduction
Over the last decade, an increasing amount of investigations have been performed showing that emotionally expressive faces can be processed without awareness. Firstly, a number of case reports have described patients who have lost their primary visual cortex and yet who maintain the ability to process emotional expressions presented in the visual modality (de Gelder et al., 1999; Pegna et al., 2005) even though they cannot consciously report the stimuli. Evidence from these patients suggests that the amygdala, most probably the right, processes these stimuli through a direct colliculo-pulvino-amygdalar route (Morris et al., 2001), although other pathways involving projections from subcortical regions to the extrastriate cortex may also possible (e.g., Gonzalez-Andino et al., 2009; see also Tamietto and de Gelder, 2010 for a recent review).
In order to investigate non-conscious processing in healthy controls, backward masking studies have been used in which emotional faces are presented very briefly and are followed by a visual mask. These procedures have been used in functional imaging experiments and have suggested that the amygdala responds to emotional expressions even when they are not consciously detected (Morris et al., 1998, 1999; Whalen et al., 1998; Liddell et al., 2005; Williams et al., 2006), although it has also been hypothesized that this may be due to the stimuli being insufficiently masked (Pessoa et al., 2006).
With respect to the timing of the phenomenon, electroencephalography (EEG) and event-related potential (ERP) procedures have also investigated the brain responses to subliminal emotional faces. Initially, the earliest difference between fearful and neutral faces was reported at the N2 component (Liddell et al., 2004). However, two recent studies have observed earlier differences, in particular within 140–180 ms time window, one over anterior electrodes (Kiss and Eimer, 2008), and the second over temporal electrodes, in particular on the N170 component (Pegna et al., 2008b). These results point to an early processing of non-conscious emotional faces, and are consistent with the view that a crude but rapid pathway, such as the hypothesized colliculo-pulvino-amygdalar route (LeDoux, 1998), may allow emotional stimuli to be analyzed before awareness arises.
It should be emphasized however, that the ERP studies of subliminal face processing required subjects to participate actively in the task and to try and detect the target expressions. An open question remains unanswered, namely whether such differences might be found when attention is not focused on detecting the target face. Indeed, the explanation subtending subliminal emotional processing is that the fast, unconscious pathway exists to allow relevant stimuli to be brought rapidly to attention, bringing about the necessary behavioral response. If this is true, subliminal processing should occur even when attention is not engaged in detecting the target face. This point remains to be established.
Consequently, we carried out a study in which masked fearful and non-fearful [(NF); i.e., happy or neutral] faces were presented subliminally or supraliminally, while subjects were engaged in a task that did not involve the emotional stimuli. Namely, the task consisted in comparing the lengths of two vertical flanker bars presented on either side of the faces. The procedure was carried out while recording EEG and ERP responses to the unattended, subliminal and supraliminal, fearful and NF faces. We hypothesized that ERP differences would be observed for subliminal faces even when attention was engaged elsewhere. Furthermore, in line with our previous findings (Pegna et al., 2008b), we expected to find these differences on the N170 component.
Materials and Methods
Subjects
Twenty students (10 females) from the University of Geneva took part in this study after giving their informed consent in accordance with the requirements of the local Ethics Committee. Average age was 25.9 years (SD: 3.7). Participants were paid for their participation. All had normal or corrected-to-normal vision, were medication-free and had no history of psychiatric disorder. All participants were right-handed based on the Oldfield-Edinburgh questionnaire (mean laterality quotient 80.2, SD = 24.2; Oldfield, 1971).
Procedure and Stimuli
Neutral, happy or fearful faces, flanked by two vertical bars were presented to the participants who were asked to decide whether the two bars were of equal length or not (see Figure 1). The two vertical target bars were white lines placed 4.8° to the left and right of the center of fixation. Two different lengths were used: short (1.6°) or long (3.3°) target bars. Targets were presented in all possible combinations (i.e., long-long, long-short, short-long, and short-short respectively on the left and right). Each combination occurred in 25% of the trials.
Figure 1. Illustration of the experimental procedure. Target bars (flanking the fearful and NF face) appeared for 300 ms after a fixation cross of variable duration and were of the same or different lengths. Faces were presented centrally, onsetting at the same time as the target bars, and were followed by a mask composed of a scrambled face for a total duration of 300 ms. Faces were either subliminal (16 ms) or supraliminal (166 ms) and represented either fearful or NF expressions. After a blank 300 ms screen, subjects were prompted to respond via a keypress whether the two bars were of equal length. The paradigm for the behavioral detection task was identical but the flanker bars were removed. In this case, the response prompt indicated “Fear?” whereupon subjects were asked to respond yes/no by pressing a key.
The faces presented at the center of the screen were greyscale photographs of neutral, happy and fearful faces of six male and six female stimuli from the Ekman-Friesen series (Ekman and Friesen, 1975). In 50% of the trials, a fearful expression was presented, while the remaining trials were composed of NF expressions (25% happy; 25% neutral). The border of the faces was removed by keeping only the central oval area whose diameters measured 17 by 11.5 cm along the vertical and horizontal axes respectively. Stimuli subtended a visual angle of 8.1° (vertically) by 5.5° (horizontally) at the viewing distance used in this study. Twelve masks were created by scrambling the neutral faces with Adobe Photoshop v6. This was obtained by selecting a 13 cm × 8 cm area centered on the middle of every stimulus and dividing it into 8 × 4 squares that were then shuffled randomly (see example in Figure 1).
Stimuli were presented on a black background at the center of a 22” CRT monitor situated 120 cm from the participant. Each trial began with a fixation cross that was presented randomly for durations of 800, 1000, or 1200 ms. This was followed by the face stimulus and the two flankers. Faces were either fearful (F) or NF. Presentation times were 16 ms (subliminal – Sub–) or 166 ms (supraliminal – Supra–) and were followed by a mask that appeared immediately at the offset of the stimulus for durations of respectively 284 or 134 ms. In this manner, the visual stimulation time always totalled 300 ms. There were therefore 2 × 2 conditions: fearful faces presented subliminally (FSub), non-fearful faces presented subliminally (NFSub), fearful faces presented supraliminally (FSupra) and non-fearful faces presented supraliminally (NFSupra).
The flankers appeared at the onset of the stimulus and lasted 300 ms, thus disappearing with the offset of the mask. Small and long bars appeared the same number of times on each side and there were an equal number of “same” and “different” trials. At the offset of the stimuli, a blank screen was presented for another 300 ms, after which the response prompt appeared asking participants to respond whether the two flanker bars were equal in length. Participants were explicitly instructed to maintain their gaze on the central fixation cross throughout the experiment, to minimize eye blinks and eye movements, and to focus on evaluating the lengths of the two vertical bars. The response prompt remained visible until the subject pressed a key, at which time the following trial was initiated. Same/different responses were given by pressing one of two keys on a response box with the index and middle finger of the right hand. Response fingers were counterbalanced across subjects.
The experiment was run in two blocks of 192 trials using E-prime (v.1.1; www.pstnet.com/eprime) for a total of 96 trials in each of the four conditions (FSub, NFSub, FSupra, NFSupra).
Behavioral face-detection task
A behavioral face-detection task was also performed in order to confirm that the face stimuli were in fact subliminal for all subjects at 16 ms. A second procedure was run that was identical on all points to the line comparison task described above, except that this time the flanker bars did not appear. In this case, subjects were instructed to try to detect the presence of a fearful face. After the presentation of the face-mask pair, a response prompt asked the subject to state (or guess) if the facial expression was one of fear. The same four conditions were maintained (FSub, NFSub, FSupra, NFSupra) with 24 trials in each one.
“Yes/No” responses were given by pressing one of two keys on the response box with the index and middle finger of the right hand and response fingers were counterbalanced across subjects.
Behavioral analysis was performed by computing the number of hits for each participant. The probability of the result appearing by chance (50%) was then determined with reference to a binomial distribution. This was carried out in order to remove participants who would score above chance in the subliminal presentation, as well as those who might respond randomly in the supraliminal condition. Detection was additionally ascertained using another measure, d′, derived from signal detection theory (Green and Swets, 1966; Macmillan and Creelman, 1991), since the number of hits can be biased by the fact that participants might favor a particular response over another when they are unsure or unable to detect the target. The d′ value allows a participant’s detection level to be established while circumventing this bias, arising when a particular response is favored over another. In SDT, both the target and the background noise (from which the former must be extracted) are represented as two normal distributions. When the two distributions are distinct, the signals are easily detected, whereas with large overlaps, lower d′ values are observed. In practical terms, the d′ value is measured as the distance between the means of the two distributions, with greater values of d′ indicating a good discrimination, and a zero value indicating a complete overlap of signal and noise distributions. Since d′ is an open-ended scale, values will vary from 0 for undetectable signals, to values typically of 2 and above for highly detectable stimuli.
ERP acquisition and analysis. Experiments were performed in a quiet, electrically shielded room with low lighting. Continuous EEG was acquired at 500 Hz using a Geodesics system (Electrical Geodesics, Inc., USA) with 256 scalp electrodes referenced to the vertex. Impedances were kept below 50 kΩ. The EEG was filtered offline from 1 to 30 Hz and recalculated against the average reference (Lehmann and Skrandies, 1980). ERP epochs of 50 ms pre- to 450 ms post-stimulus onset were averaged separately for target durations of 16 ms and 166 ms, for fearful and NF expressions separately. The 50 ms pre-stimulus epoch served to establish baseline. Trials on which the participant’s behavioral response was incorrect, as well as those containing blinks, eye movements, or other artifacts (EEG sweeps with amplitude exceeding ± 100 μV criterion) were excluded from the averaging procedure. Two participants were discarded due to numerous artifacts and one subject was removed due to a noise-contaminated EEG recording.
The first ERP analysis involved a paired point-wise t-test comparing fearful and NF faces. This provides t-values (and their corresponding p-values) for every electrode at every time point of the ERP. In this manner, the electrodes and periods of significant differences can be observed without a priori selection of electrodes or components. Here, the threshold of significance was set at 0.05 with the two following constraints: firstly the 0.05 threshold had to be maintained for at least 10 consecutive milliseconds (five time frames) in any electrode, and secondly, the threshold had to be reached in at least five spatially adjacent electrodes. Isolated electrodes, i.e., electrodes that did not have at least four immediate neighbors also reaching the threshold, were not considered significant. Two separate t-tests were performed, one for supraliminal and one for subliminal presentations.
A second set of analyses was also performed, this time using ANOVAs for repeated measures on the mean amplitudes in specific groups of electrodes in P1, N170, and P2 time windows. This was done to compare conditions of presentation, differences in laterality, as well as any interactions between expression, laterality, and presentation. For this purpose, mean values were computed within a 10 ms time window centered on the peak of the component across a group of electrodes that were chosen to include the electrode producing the maximum amplitude for the given component.
Results
Behavior
Face-detection task
The hit rates in the subliminal and supraliminal conditions were determined for each subject separately and were then compared to chance level (50%) using a binomial distribution. Three subjects obtained a score at 16 ms which differed from chance (p < 0.01) and were therefore excluded from further analysis. The mean hit rate for the remaining 14 subjects was 54% in the subliminal condition and 88% in the supraliminal condition. In addition, the d′ values computed for the 14 subjects, confirmed that detection was impossible at subliminal levels (mean d′ = 0.32; SD = ± 0.32), but was good at supraliminal levels (mean d′ = 2.2; SD = ± 0.51).
Line judgment task
In the ERP experiment proper, the rate of correct responses was high with 95% for FSub, 94% for NFSub, 95% for FSupra, and 95% for NFSupra. A non-parametric ANOVA computed on the raw values showed that the differences between conditions were not significant (Friedman ANOVA chi-square(3) = 4.64, p < 0.1).
ERP Analysis
The ERPs in the four conditions showed distinct P1, N170, and P2 components over posterior electrodes. Figure 2A illustrates the ERPs of five electrodes (including T6 which yielded the maximum negativity at the N170) in the supraliminal conditions and Figure 3A displays the same electrodes in the subliminal condition (fearful expressions are shown in black and NF expressions in red).
Figure 2. (A) Supraliminal grand average ERPs at electrodes FCz, T5, T6, O1, and O2. The vertical bar in the ERP indicates the time of stimulus presentation. Fear produced a greater negativity than NF faces over the right electrodes (maximum negativity was observed over for T6). (B) Point-wise t-test comparing fear and non-fear presentations over time (x axis) for all electrodes (y axis). The upper half of the y axis illustrates the electrodes of the anterior scalp, which are represented in a clockwise order from left to right. The lower half of the y axis represent the electrodes of the posterior scalp, displayed from left to right progressing in an anti-clockwise manner. Red areas indicate periods of significance at p < 0.05 (see text for details). The first window of significance began on the negative deflection of the N170. The time frame of maximum difference is indicated by an arrow. The t-map (top inset) at this time frame is shown with the scalp viewed from above (top = nasion, left ear on the left) and a color code ranging from blue (t = −4.5) to red (t = 4.7). Inset below shows all 204 electrodes (indicated as colored balls) with significant electrodes in red (p < 0.05) and non-significant ones in green.
Figure 3. (A) Subliminal grand average ERPs at electrodes FCz, T5, T6, O1, and O2. The vertical bar in the ERP indicates the time of stimulus presentation. Fear produced a greater negativity than NF faces over the right electrodes (maximum negativity was again observed over for T6). (B) Point-wise t-test comparing fear and non-fear presentations over time (x axis) for all electrodes (y axis). The y axis represents anterior electrodes (top) and posterior electrodes (bottom) with left leads on top and right leads below, as in Figure 2. Red areas indicate periods of significance at p < 0.05 (see text for details). As for supraliminal presentations, the first window of significance began on the negative-going slope of the N170 but on a smaller group of 12 right temporal electrodes. The time frame of maximum difference in this window is indicated by an arrow. The t-map (top inset) at this time frame is shown with the scalp viewed from above (top = nasion, left ear on the left) and a color code ranging from blue (t = −3.1) to red (t = 2.2). Inset below shows all 204 electrodes (indicated as colored balls) with significant electrodes in red (p < 0.05) and non-significant ones in green. A later period of significant differences can be seen from around 200–250 ms over posterior and (left) fronto-central electrodes.
The results of the point-wise t-test comparing supraliminal fearful and NF expressions are illustrated in Figure 2B, those for the subliminal condition are shown in Figure 3B.
For supraliminal presentations, the major earliest difference appeared during the time window of the N170 component (Figure 2B). These differences involved the right posterior, as well as the left anterior and lateral leads (see Figure 2A). Afterward, fear and non-fear remained statistically indistinct until 380 ms. In the subliminal conditions, the point-wise t-test (Figure 3B) comparing fearful and NF expressions again showed that a group of right posterior electrodes differed significantly during the N170 component (see examples in Figure 3A), although in this case the effect was less extended than in the supraliminal condition. This was followed by a difference on the P2 component that was observed over posterior occipital leads bilaterally, as well as in a group of fronto-central leads, appearing as a negativity that was lateralised slightly to the left.
The earliest differences across expressions for both modes of presentation thus began at the onset of the N170, with additional differences at the P2 for subliminal but not supraliminal conditions.
The latencies of the first three peaks P1, N170, and P2 in the four conditions were submitted to standard ANOVAs, along with the mean amplitudes of the peaks that were computed over electrodes and time windows of interest in order to verify their significance when the mode of presentation was taken into account.
P1
Mean P1 latencies (±SEM) were present at 125 ms (±4), 126 ms (±4), 125 ms (±4) and 127 ms (±5) for FSub, NFSub, FSupra, and NFSupra respectively. These differences were not statistically significant (all p’s > 0.1 in a 2 (presentation) × 2 (expression) ANOVA). The P1 amplitudes computed over the 18 (nine right and nine left) occipital leads shown in the inset in Figure 4B. Left leads: FSub = 0.8 μV (1.1); NFSub = 1.0 (0.9); FSupra = 0.8 (1.1); NFSupra = 0.5 (1.0); Right leads: FSub = 2.4 (1.2); NFSub = 2.5 (1.3); FSupra = 2.5 (0.9); NFSupra = 2.5 (1.0). The amplitudes were compared on a 2 × 2 × 2 ANOVA using side (left vs. right), presentation (subliminal vs. supraliminal) and expression (fear vs. non-fear) as repeated measures. Only the factor side showed a significant difference due to the fact that the P1 produced a greater positivity on the right than on the left in all conditions [F(1, 13) = 14.13, p < 0.005]. Presentation and emotional expression did not differ significantly [F(1, 13) = 0.47, p > 0.1 and F(1, 13) = 0.008, p > 0.1 for the two factors respectively] and none of the interactions were below p = 0.1.
Figure 4. (A) N170 amplitudes. The mean N170 amplitudes averaged over the region of interest (electrodes depicted in inset) are plotted for each experimental condition. (B) P2 amplitudes. The mean P2 amplitudes averaged over the region of interest (18 electrodes depicted in inset), are plotted for each experimental condition. Amplitudes and SE in microvolts (y axis) are given for subliminal and supraliminal presentations (x axis), and for fearful (circles and solid lines) and non-fearful faces (squares and dashed lines). The left panel indicates the values for the left leads and the right panel shows the values for the right leads.
N170
Average latencies for the N170 were similar across conditions (mean ± sem: FSub = 174ms ± 3; NFSub = 175ms ± 3; FSupra = 177ms ± 3; NFSupra = 177ms ± 4). In a 2 (presentation) × 2 (expression) ANOVA for repeated measures, neither the interaction, nor the main effects were significant at the 0.05 level.
Mean amplitudes were computed over 18 (nine right and nine left) temporal and temporo-occipital electrodes which included T5 and T6 (which showed the maximum negativity at the peak), along with their eight closest neighbors (see inset in Figure 4A).
Figure 4A plots the amplitude of the N170 peak for the four conditions computed on these 18 channels. An ANOVA was performed using side (left vs. right), presentation (subliminal vs. supraliminal) and expression (fear vs. non-fear) as repeated measures. This revealed a significant interaction between expression and side [F(1, 13) = 24.2, p < 0.001]. A post hoc comparison using the Tukey Honest Significant Difference test showed that this was due to the fact that fear and non-fear differed significantly over the right (p < 0.001) but not over the left region (p > 0.1). The effects were subsequently explored over the right leads alone, using a 2 (presentation) × 2 (expression) ANOVA. This showed a highly significant main effect of emotional expression [F(1, 13) = 21.6, p < 0.001] and a marginal effect of presentation [F(1, 13) = 4.4, p = 0.06], while the interaction between the two factors was not significant [F(1, 13) = 0.14, p > 0.1]. The main effect of emotion was due to fearful expressions giving rise globally to a more negative amplitude, an effect that tended to be more marked in supraliminal than subliminal conditions. In order to confirm the validity of the effect in the subliminal mode of presentation, a separate ANOVA was performed on the group of right electrodes, comparing fear, and non-fear in this condition. This yielded a significant difference [F(1, 13) = 7.7, p < 0.05] as expected in view of the point-wise t-test.
Finally, the statistical validity of the difference between fearful and NF faces was also verified in the supraliminal condition by comparing the mean amplitude of these right posterior leads in response to fear and non-fear. The enhanced negativity for fearful faces was confirmed in this condition too, as shown also by the point-wise t-test described above [F(1, 13) = 16.0, p < 0.01].
P2
P2 latencies did not differ significantly across conditions with peaks at 257 ms (±7), 257 ms (±7), 254 ms (±7), and 257 ms (±6) for FSub, NFSub, FSupra, and NFSupra respectively (all p’s > 0.1 in a 2 (presentation) × 2 (expression) ANOVA).
The amplitudes obtained from the group of 18 electrodes are plotted in Figure 4B (inset shows electrodes used for computation of the mean).
The 2 × 2 × 2 ANOVA showed an interaction between presentation and side [F(1, 13) = 6.0, p < 0.05]. A post hoc Tukey test showed that this interaction was due to differences between subliminal and supraliminal presentations over the right leads (p < 0.05) but not the left (p > 0.1), with amplitudes being globally less positive for supraliminal than subliminal presentations. In addition, a main effect of emotional expression was found [F(1, 13) = 13.3, p < 0.005] with fearful faces showing more positive amplitudes. In accordance with the running t-tests, the 2 × 2 (emotion × side) ANOVAs on the subliminal P2 values showed that effects of both emotional expression and side were significant (respectively F(1, 13) = 14.0, p < 0.005 and F(1, 13) = 9. 9, p < 0.01) with more positive amplitudes for fearful than NF faces and more positive values for right compared to left leads. However these two factors did not interact [F(1, 13) = 0.23, p > 0.1].
As observed in the point-wise t-test, a fronto-central negativity occurred concomitantly with the posterior positivity in the subliminal condition. A two-way ANOVA comparing fear and non-fear in the subliminal condition was performed on the mean amplitude computed over four central electrodes (FCz, Cz, and their two closest common left and right neighbors) during this same time window. This confirmed the significant difference over anterior electrodes [F(1, 13) = 10.7, p < 0.01] in which fear produced a more negative amplitude (−2.3 μV ± 0.9) than non-fear (−1.9 μV ± 0.8).
The ANOVA comparing the P2 amplitudes at the 18 posterior electrodes (Figure 4B) in response to fearful and NF faces in the supraliminal presentations showed no significant effects either for side [F(1, 13) = 3.1, p > 0.05] or for emotion [F(1, 13) = .68, p > 0.1] as expected from the running t-test.
Discussion
The results of our study show that subliminal presentations of fearful faces produce an N170 modulation, even when attention is engaged in an incidental task, and provide further evidence that emotional face processing begins within 200 ms of visual presentation.
Previous studies have investigated subliminal processing of emotional faces, generally using the emotional faces as targets and thus requiring that attention be focused on detection of the face. In a previous investigation, we had used a similar paradigm, but with attention focused on the faces (Pegna et al., 2008b). When subjects attempted to detect the target face the same results were observed, namely that right posterior leads were more negative for fearful faces (whether consciously reported or not). As noted above, Kiss and Eimer (2008) also observed a difference for subliminal fearful compared to neutral faces within the same time window (140–180 ms), although their findings pointed to a greater positivity over fronto-central leads, a result which might partly be explained by the use of different references in the two studies. Our present finding thus confirms that subliminal fearful faces do give rise to early ERP effects, and are consistent with the hypothesis of rapid non-conscious processing for these stimuli, further extending this observation to task-irrelevant stimuli. In addition, we also found an enhanced posterior positivity and fronto-central negativity for fearful vs. NF faces at around 200–250 ms in the subliminal condition, a difference that had not been observed in our preceding study (Pegna et al., 2008b). In a previous report, Liddell et al. (2004) investigated the ERP responses to passive viewing of subliminal fearful and neutral faces and also noted an increased negativity for the former stimuli over midline electrodes (particularly evident over Fz and Cz). They interpreted this enhanced N2 as the sign of a rapid, non-conscious attention-orienting response for fearful faces. In point of fact, the negativity observed over fronto-central leads around 200–250 ms in our study is similar and an interpretation in terms of attention-orienting remains quite plausible. However, we would surmise on the basis of our results, that this second period of significant differences appears only after an initial processing of the relevant stimulus which occurs around 170 ms, and that this effect might be specific to unattended stimuli.
The fact that non-masked faces may be processed when unattended is not new. Behavioral studies have shown that emotional faces can influence participants’ performance even when they are incidental to the task at hand (e.g., Fox, 2002; Eastwood et al., 2003) and studies of patients with brain damage producing unilateral spatial neglect or simultanagnosia have shown that emotional faces can be processed despite the attentional deficit (Vuilleumier and Schwartz, 2001; Fox, 2002; Williams and Mattingley, 2004; Pegna et al., 2008a). Furthermore, brain imaging investigations have supported the view that emotional faces are processed even if they are placed outside of the focus of attention, both in healthy participants and in unilateral spatial neglect (Vuilleumier and Schwartz, 2001; Vuilleumier et al., 2002). ERP procedures have also investigated the effect of non-masked, unattended faces and shown an “attention-grabbing” effect of emotional faces. For example, Pourtois et al. (2004) carried out a modified dot-probe task in which fearful and neutral faces were used as cues. Probes appearing in a location previously held by a fearful face were found to produce a greater P1 response, compared to when it appeared at the location of a neutral face. In another ERP investigation, Holmes et al. (2003) attempted to replicate a previous fMRI observation (Vuilleumier et al., 2001) that had shown that emotional faces were processed even when they were unattended and irrelevant to the task at hand. Stimulus arrays which included two houses and two emotional (fearful or neutral) faces were presented to participants who were asked to focus their attention on one of the two categories that appeared at specific locations. When attended, fearful faces produced an increased positivity over anterior electrodes that began at around 100 ms, but in contrast to the fMRI study, unattended fearful faces showed no difference. In a subsequent study, Holmes et al. (2006) presented faces foveally with two vertical bars placed laterally to the left and right of the face, as in our present report. When the participants were asked to attend the face, the investigators found an enhanced positivity for fearful faces from 160 ms onward. When subjects were asked to attend the lines (thus ignoring the faces), a difference was still observable from 160 ms to 220 ms, although they were no longer seen after 220 ms (by opposition with the attended condition).
Thus, evidence appears to favor the view that (non-masked) emotional faces are processed even when they are not the focus of attention, although it has been pointed out that such findings may be due to residual attention stemming from low attentional loads of the incidental task (Pessoa et al., 2002).
One question that remains insufficiently addressed is the fate of stimuli that are unattended and subliminal. Mogg and Bradley (1999) first investigated this in three experiments using a dot-probe paradigm in which neutral or emotional faces were presented subliminally in the left and right visual fields. The authors found that reaction times were faster in response to probes presented at the location previously held by an angry face, rather than neutral or happy one. This effect was apparent when the stimuli and probes were presented in the left visual field (experiments 1 and 3). The findings suggested that attention is oriented toward the location of threatening stimuli even when the duration of presentation precludes any conscious detection.
Carlson and Reinke (2008) recently replicated this finding in two behavioral dot-probe experiments that controlled for the perceptual inconsistencies that might have been produced by the use of neutral faces as masks. Their results confirmed that subliminal emotional faces (this time fearful expressions) attracted attention. In a subsequent fMRI study investigating the neural basis of this phenomenon (Carlson et al., 2009), the left amygdala, as well as the left anterior cingulate cortex, right superior temporal sulcus, and right lingual gyrus were found to be involved in directing attention to masked fearful faces presented in the left (but not the right) visual field.
The only electrophysiological evidence currently available regarding the possible time course of attentional modulation by subliminal emotional faces is a recent study by Carlson and Reinke (2010). In this investigation, participants again performed a dot-probe task in which the cues were backward-masked, fearful, or neutral faces that were presented subliminally for 33 ms in the left or right visual field. These were followed by the target to which the subjects responded. The ERP results revealed an enhanced contralateral N170 for masked fearful faces presented again to the left but not the right visual field. Their N170 enhancement for left visual field presentations thus mirrors our findings for central presentations. One caveat however was the durations of 33 ms used for subliminal processing. Previous observations have shown that many subjects may in fact reliably detect emotional faces at this duration (Pessoa et al., 2005). It could thus be argued that the N170 enhancement observed by Carlson and Reinke (2010) might have been linked to insufficiently subliminal presentations, however in view of our findings, such an explanation that cannot be upheld.
One potential confound could be put forward concerning the attentional demands of the line judgment task, which may have been too low to preclude any residual attention (Pessoa et al., 2002). Indeed, the participants’ performance on the line judgment task was excellent and did not differ significantly across conditions. This possibility cannot be entirely ruled out and future studies could address this by increasing task difficulty and investigating this effect on the N170 enhancement. Nevertheless, even in the behavioral task where attention was entirely mobilized in an effort to detect the stimulus, conscious report was at chance level at 16 ms. It is therefore unlikely that any form of conscious detection could have occurred under the condition of subliminality and diverted attention, accounting for the results obtained here.
The enhanced N170 for emotional faces in our experiment might come as a surprise since several studies have claimed that it was not sensitive to emotional expressions (Munte et al., 1998; Bobes et al., 2000; Krolak-Salmon et al., 2001; Eimer and Holmes, 2002; Herrmann et al., 2002). This view has in fact been contradicted by a number of more recent observations showing that emotional faces, in particular fearful expressions, actually do produce larger N170 components (Batty and Taylor, 2003; Stekelenburg and de Gelder, 2004; Blau et al., 2007; Pegna et al., 2008a). A probable interpretation for this enhanced response is that recurrent feedback from the amygdala to the visual extrastriate region may increase extrastriate responsiveness in these conditions. Amygdala activation is known to increase in response to fearful faces and has been found to occur in conjunction with increased extrastriate activation, with some studies showing a right lateralisation (e.g., Breiter et al., 1996; Morris et al., 1999; Vuilleumier et al., 2001b; Ishai et al., 2004). Furthermore, as noted above, the characteristic amygdala activity is even observed when the faces are not consciously perceived due to masking or brain damage (Morris et al., 1998, 1999, 2001; Whalen et al., 2001; Pegna et al., 2005).
A likely explanation appears to be that direct and/or indirect projections to the amygdala might provide the feedback which leads to the enhanced response in visual areas (Vuilleumier et al., 2001a). Evidence corroborating this view has been obtained in ERP and fMRI studies of epileptic patients, which showed that damage to the amygdala hinders the expected increase in fusiform and occipital activity (Vuilleumier et al., 2004), as well as the early (100–150 ms) ERP enhancement associated with fearful expressions (Rotshtein et al., 2010).
Our findings could thus be accounted for by an increase in extrastriate activation resulting from enhancement by the amygdala at around 170 ms, further corroborating the idea that this structure boosts visual processing for fearful faces even when the stimuli are not consciously seen, and more importantly, even when they are irrelevant to the task at hand.
Conclusion
Our results show that fearful faces are processed even when participants are involved in an incidental task, and that this occurs whether the stimuli are presented supraliminally or subliminally (i.e., whether they are consciously detected or not). The earliest differences were found at the N170 component, suggesting that non-conscious processing of task-irrelevant fearful faces takes place within the first 200 ms following stimulus presentation.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
This investigation was supported by the Swiss National Science Foundation for Scientific Research (Grant no. #320030-125196). The Cartool software (http://brainmapping.unige.ch/Cartool.htm) used for analysis in this study was programmed by Denis Brunet from the Functional Brain Mapping Laboratory, Geneva, Switzerland, and is supported by the Center for Biomedical Imaging of Geneva and Lausanne.
References
Batty, M., and Taylor, M. J. (2003). Early processing of the six basic facial emotional expressions. Cog. Brain Res. 17, 613–620.
Blau, V. C., Maurer, U., Tottenham, N., and McCandliss, B. D. (2007). The face-specific N170 component is modulated by emotional facial expression. Behav. Brain Funct. 3, 7.
Bobes, M. A., Martin, M., Olivares, E., and Valdes-Sosa, M. (2000). Different scalp topography of brain potentials related to expression and identity matching of faces. Brain Res. Cogn. Brain Res. 9, 249–260.
Breiter, H. C., Etcoff, N. L., Whalen, P. J., Kennedy, W. A., Rauch, S. L., Buckner, R. L., Strauss, M. M., Hyman, S. E., and Rosen, and B. R. (1996). Response and habituation of the human amygdala during visual processing of facial expression. Neuron 17, 875–887.
Carlson, J. M., and Reinke, K. S. (2008). Masked fearful faces modulate the orienting of covert spatial attention. Emotion 8, 522–529.
Carlson, J. M., and Reinke, K. S. (2010). Spatial attention-related modulation of the N170 by backward masked fearful faces. Brain Cogn. 73, 20–27.
Carlson, J. M., Reinke, K. S., and Habib, R. (2009). A left amygdala mediated network for rapid orienting to masked fearful faces. Neuropsychologia 47, 1386–1389.
de Gelder, B., Vroomen, J., Pourtois, G., and Weiskrantz, L. (1999). Non-conscious recognition of affect in the absence of striate cortex. Neuroreport 10, 3759–3763.
Eastwood, J. D., Smilek, D., and Merikle, P. M. (2003). Negative facial expression captures attention and disrupts performance. Percept. Psychophys. 65, 352–358.
Eimer, M., and Holmes, A. (2002). An ERP study on the time course of emotional face processing. Neuroreport 13, 427–431.
Ekman, P., and Friesen, W. V. (1975). Pictures of Facial Affect. Palo Alto, CA: Consulting Psychologists Press.
Fox, E. (2002). Processing emotional facial expressions: the role of anxiety and awareness. Cogn. Affect. Behav. Neurosci. 2, 52–63.
Gonzalez-Andino, S. L., Menendez, R. G., Khateb, A., Landis, T., and Pegna, A. J. (2009). Electrophysiological correlates of affective blindsight. Neuroimage 44, 581–589.
Herrmann, M. J., Aranda, D., Ellgring, H., Mueller, T. J., Strik, W. K., Heidrich, A., and Fallgatter, A. J. (2002). Face-specific event-related potential in humans is independent from facial expression. Int. J. Psychophysiol. 45, 241–244.
Holmes, A., Kiss, M., and Eimer, M. (2006). Attention modulates the processing of emotional expression triggered by foveal faces. Neurosci. Lett. 394, 48–52.
Holmes, A., Vuilleumier, P., and Eimer, M. (2003). The processing of emotional facial expression is gated by spatial attention: evidence from event-related brain potentials. Brain Res. Cogn. Brain Res. 16, 174–184.
Ishai, A., Pessoa, L., Bikle, P. C., and Ungerleider, L. G. (2004). Repetition suppression of faces is modulated by emotion. Proc. Natl. Acad. Sci. U.S.A. 101, 9827–9832.
Kiss, M., and Eimer, M. (2008). ERPs reveal subliminal processing of fearful faces. Psychophysiology 45, 318–326.
Krolak-Salmon, P., Fischer, C., Vighetto, A., and Mauguiere, F. (2001). Processing of facial emotional expression: spatio-temporal data as assessed by scalp event-related potentials. Eur. J. Neurosci. 13, 987–994.
Lehmann, D., and Skrandies, W. (1980). Reference-free identification of components of checkerboard-evoked multichannel potential fields. Electroencephal. Clin. Neurophysiol. 48, 609–621.
Liddell, B. J., Brown, K. J., Kemp, A. H., Barton, M. J., Das, P., Peduto, A., Gordon, E., and Williams, L. M. (2005). A direct brainstem-amygdala-cortical ‘alarm’ system for subliminal signals of fear. Neuroimage 24, 235–243.
Liddell, B. J., Williams, L. M., Rathjen, J., Shevrin, H., and Gordon, E. (2004). A temporal dissociation of subliminal versus supraliminal fear perception: an event-related potential study. J. Cogn. Neurosci. 16, 479–486.
Macmillan, N. A., and Creelman, C. D. (1991). Detection Theory: A User’s Guide. Cambridge: Cambridge University Press.
Mogg, K., and Bradley, B. P. (1999). Orienting of attention to threatening facial expressions presented under conditions of restricted awareness. Cogn. Emot. 13, 713–740.
Morris, J. S., DeGelder, B., Weiskrantz, L., and Dolan, R. J. (2001). Differential extrageniculostriate and amygdala responses to presentation of emotional faces in a cortically blind field. Brain 124(Pt 6), 1241–1252.
Morris, J. S., Ohman, A., and Dolan, R. J. (1998). Conscious and unconscious emotional learning in the human amygdala. Nature 393, 467–470.
Morris, J. S., Ohman, A., and Dolan, R. J. (1999). A subcortical pathway to the right amygdala mediating “unseen” fear. Proc. Natl. Acad. Sci. U.S.A 96, 1680–1685.
Munte, T. F., Brack, M., Grootheer, O., Wieringa, B. M., Matzke, M., and Johannes, S. (1998). Brain potentials reveal the timing of face identity and expression judgments. Neurosci. Res. 30, 25–34.
Oldfield, R. C. (1971). The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9, 97–113.
Pegna, A. J., Caldara-Schnetzer, A.-S., and Khateb, A. (2008a). Visual search for facial expressions of emotion is less affected in simultanagnosia. Cortex 44, 46–53.
Pegna, A. J., Landis, T., and Khateb, A. (2008b). Electrophysiological evidence for early non-conscious processing of fearful facial expressions. Int. J. Psychophysiol. 70, 127–136.
Pegna, A. J., Khateb, A., Lazeyras, F., and Seghier, M. L. (2005). Discriminating emotional faces without primary visual cortices involves the right amygdala. Nat. Neurosci. 8, 24–25.
Pessoa, L., Japee, S., Sturman, D., and Ungerleider, L. G. (2006). Target visibility and visual awareness modulate amygdala responses to fearful faces. Cereb. Cortex 16, 366–375.
Pessoa, L., Japee, S., and Ungerleider, L. (2005). Visual awareness and detection of fearful faces. Emotion 5, 243–247.
Pessoa, L., McKenna, M., Gutierrez, E., and Ungerleider, L. G. (2002). Neural processing of emotional faces requires attention. Proc. Natl. Acad. Sci. U.S.A. 99, 11458–11463.
Pourtois, G., Grandjean, D., Sander, D., and Vuilleumier, P. (2004). Electrophysiological correlates of rapid spatial orienting towards fearful faces. Cereb. Cortex 14, 619–633.
Rotshtein, P., Richardson, M. P., Winston, J. S., Kiebel, S. J., Vuilleumier, P., Eimer, M., Driver, J., and Dolan, R. J. (2010). Amygdala damage affects event-related potentials for fearful faces at specific time windows. Hum. Brain Mapp. 31, 1089–1105.
Stekelenburg, J. J., and de Gelder, B. (2004). The neural correlates of perceiving human bodies: an ERP study on the body-inversion effect. Neuroreport 15, 777–780.
Tamietto, M., and de Gelder, B. (2010). Neural bases of the non-conscious perception of emotional signals. Nat. Rev. Neurosci. 11, 697–709.
Vuilleumier, P., Armony, J. L., Clarke, K., Husain, M., Driver, J., and Dolan, R. J. (2002). Neural response to emotional faces with and without awareness: event-related fMRI in a parietal patient with visual extinction and spatial neglect. Neuropsychologia 40, 2156–2166.
Vuilleumier, P., Armony, J. L., Driver, J., and Dolan, R. J. (2001a). Effects of attention and emotion on face processing in the human brain: an event-related fMRI study. Neuron 30, 829–841.
Vuilleumier, P., Sagiv, N., Hazeltine, E., Poldrack, R. A., Swick, D., Rafal, R. D., and Gabrieli, J. D. (2001b). Neural fate of seen and unseen faces in visuospatial neglect: a combined event-related functional MRI and event-related potential study. Proc. Natl. Acad. Sci. U.S.A. 98, 3495–3500.
Vuilleumier, P., Richardson, M. P., Armony, J. L., Driver, J., and Dolan, R. J. (2004). Distant influences of amygdala lesion on visual cortical activation during emotional face processing. Nat. Neurosci. 7, 1271–1278.
Vuilleumier, P., and Schwartz, S. (2001). Emotional facial expressions capture attention. Neurology 56, 153–158.
Whalen, P. J., Rauch, S. L., Etcoff, N. L., McInerney, S. C., Lee, M. B., and Jenike, M. A. (1998). Masked presentations of emotional facial expressions modulate amygdala activity without explicit knowledge. J. Neurosci. 18, 411–418.
Whalen, P. J., Shin, L. M., McInerney, S. C., Fischer, H., Wright, C. I., and Rauch, S. L. (2001). A functional MRI study of human amygdala responses to facial expressions of fear versus anger. Emotion 1, 70–83.
Williams, L. M., Das, P., Liddell, B. J., Kemp, A. H., Rennie, C. J., and Gordon, E. (2006). Mode of functional connectivity in amygdala pathways dissociates level of awareness for signals of fear. J. Neurosci. 26, 9264–9271.
Keywords: ERP/EEG, awareness, subliminal, face, N170
Citation: Pegna AJ, Darque A, Berrut C and Khateb A (2011) Early ERP modulation for task-irrelevant subliminal faces. Front. Psychology 2:88. doi: 10.3389/fpsyg.2011.00088
Received: 04 January 2011; Accepted: 26 April 2011;
Published online: 06 May 2011.
Edited by:
Paul Whalen, Darmouth College, USAReviewed by:
Erno Hermans, RadBoud University Nijmegen, NetherlandsBarak Morgan, University of Cape Town, South Africa
Copyright: © 2011 Pegna, Darque, Berrut and Khateb. This is an open-access article subject to a non-exclusive license between the authors and Frontiers Media SA, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and other Frontiers conditions are complied with.
*Correspondence: Alan J. Pegna, Laboratory of Experimental Neuropsychology, Neuropsychology Unit, Neurology Clinic, Geneva University Hospital, 4 Rue Gabrielle-Perret-Gentil, 1211 Geneva 14, Switzerland. e-mail: alan.pegna@hcuge.ch