Original Research ARTICLE
The neural substrates of natural reading: a comparison of normal and nonword text using eyetracking and fMRI
- Department of Psychology, Institute for Mind and Brain, University of South Carolina, Columbia, SC, USA
Most previous studies investigating the neural correlates of reading have presented text using serial visual presentation (SVP), which may not fully reflect the underlying processes of natural reading. In the present study, eye movements and BOLD data were collected while subjects either read normal paragraphs naturally or moved their eyes through “paragraphs” of pseudo-text (pronounceable pseudowords or consonant letter strings) in two pseudo-reading conditions. Eye movement data established that subjects were reading and scanning the stimuli normally. A conjunction fMRI analysis across natural- and pseudo-reading showed that a common eye-movement network including frontal eye fields (FEF), supplementary eye fields (SEF), and intraparietal sulci was activated, consistent with previous studies using simpler eye movement tasks. In addition, natural reading versus pseudo-reading showed different patterns of brain activation: normal reading produced activation in a well-established language network that included superior temporal gyrus/sulcus, middle temporal gyrus (MTG), angular gyrus (AG), inferior frontal gyrus, and middle frontal gyrus, whereas pseudo-reading produced activation in an attentional network that included anterior/posterior cingulate and parietal cortex. These results are consistent with results found in previous single-saccade eye movement tasks and SVP reading studies, suggesting that component processes of eye-movement control and language processing observed in past fMRI research generalize to natural reading. The results also suggest that combining eyetracking and fMRI is a suitable method for investigating the component processes of natural reading in fMRI research.
Understanding the neural architecture of reading is one of the central issues in cognitive neuroscience (Reichle et al., 2011). Although a variety of neural aspects of reading have been understood via functional neuroimaging, most of these findings have been obtained from paradigms in which single words are presented to readers with a secondary meta-linguistic task for each word, such as lexical decision, semantic categorization, and covert or overt naming (for a comprehensive review, see Price, 2012). Even in those cases involving sentence or paragraph reading rather than single-word reading, the words have typically been presented one at a time in serial visual presentation (SVP, e.g., Martin-Loeches et al., 2008; Fedorenko et al., 2011; Pallier et al., 2011), with participants often asked to do a secondary task such as probe matching.
In contrast to single-word and SVP reading, during natural reading the eyes move through text in a series of rapid movements (saccades) and brief static periods (fixations), with mean fixation durations of 200–250 ms and mean saccade lengths of 7–9 characters (for reviews, see Rayner, 1998, 2009). The pattern of eye movements during reading is substantially correlated with linguistic factors, implying that readers' eye movements during reading reflect online cognitive processes (Rayner, 1998, 2009; Henderson, 2013). It has previously been shown that both behavioral and imaging data related to sentence processing differ for SVP and whole-sentence reading (Lee and Newman, 2010). The importance of eye movements in natural reading was recently highlighted by an eyetracking study reported by Schotter et al. (2014), showing that sentence comprehension is negatively affected when regressive eye movements are not available during reading.
In the majority of fMRI studies that have presented an entire sentence or passage at once (e.g., Ferstl and von Cramon, 2001; Bohrn et al., 2013; Altmann et al., 2014; Hsu et al., 2015), eye movements have not been monitored, so it has not been possible to investigate questions concerning the integration of language comprehension and eye movement control during natural reading. In a recent study, Hillen et al. (2013) examined how neural activation is elicited by eye movements in text by asking subjects to move their eyes through sentences in an fMRI study. The authors compared fMRI activation for normal sentences, scrambled sentences, nonword sentences, and pseudo-text made up of Landolt rings (circle-like shapes). Hillen et al. found activation of a common gaze network across these conditions that included bilateral frontal eye fields (FEF), supplementary eye field (SEF), and right intraparietal sulcus (rIPS), the same areas reported in other fMRI studies using simple eye movement tasks like the pro- and anti-saccade tasks (Paus et al., 1993; Sweeney et al., 1996; Everling and Munoz, 2000; Ford et al., 2005; Ettinger et al., 2008, for a recent review, see Jamadar et al., 2013). Although the Hillen et al. study suggests that sequential reading-like tasks activate the same eye movement network that has been observed in simpler saccadic oculomotor tasks, subjects were not actually engaged in natural reading. Instead, subjects were asked to detect characters that looked like a “left-opened o” (Landolt C) that were embedded in the real and pseudoword texts. These instructions might cause subjects to use relatively different (e.g., more controlled) scanning strategies than those used in natural reading. In a related study, Richlan et al. (2014) compared reading materials (words or pronounceable nonwords) to non-reading materials (lines or Hebrew characters) using a fixation-related fMRI analysis method. However, in that study, the items were presented in arrays and did not comprise connected text, so the task was quite different than natural reading.
Given the importance of eye movements in natural reading, it is surprising that the neurocognitive basis of natural reading in which readers actively move their eyes through units of text (e.g., sentences or paragraphs) is largely unknown. In the current study, we pursued three goals. The first two were related to the nature of the eye movement control network during reading. First, we investigated the general characteristics of the eye movement network when participants sequentially move their eyes through text and text-like stimuli. In their study, Hillen et al. (2013) reported evidence for a gaze control network that was common across reading and pseudo-reading conditions, though subjects were engaged in a search task. In the current study, we examined the nature of the eye-movement network common to natural reading and sequential pseudo-reading tasks that did not include a search or other secondary task. Our second goal was to investigate how activation in the eye movement network differs in natural reading and pseudo-reading by directly comparing these conditions. Our third goal was to investigate the language processing network in normal reading. Previous studies have shown that SVP reading produces a distinct pattern of neural activation compared with pseudo-reading involving nonword strings or false fonts (Noppeney and Price, 2004; Fedorenko et al., 2011, 2012; Hillen et al., 2013). It is not currently known whether this pattern generalizes to natural reading in which subjects actively control their eye movements and therefore control the timing and order of text encoding and analysis.
In sum, the present study was designed to investigate the nature of the networks activated when participants read naturally via eye movements, compared to pseudo-reading controls. We simultaneously recorded eye movements and BOLD activity while subjects read passages of text or moved their eyes through similarly arranged pseudo-text made up of pronounceable pseudowords or consonant strings. Because we were interested in natural reading, in the reading condition subjects simply read naturally with no secondary task. To facilitate natural reading, we presented full paragraphs rather than sentences, and the paragraphs were connected across trials in coherent passages.
Thirty-three subjects (12 male) participated in this study. Two of them did not finish the experiment. Therefore, 31 participants' data were included in the analysis. They were all right-handed native speakers of English, aged 18–35 years (Mean Age: 21.48). Thirty subjects were students from the University of South Carolina and three were recruited from the community in Columbia, South Carolina. All subjects gave informed consent and were screened for MRI safety, following the ethics protocol approved by the Institutional Review Board of the University of South Carolina. All subjects reported normal or corrected-to-normal vision and were given $10 per hour for participation in the study.
The experiment consisted of three conditions: Normal Text (NT), Pseudoword Text (PW), and Consonant String Text (CS). In the NT condition, 22 paragraphs were selected from two sources, The Emperor's New Clothes by Hans Christian Andersen (11 paragraphs), and a Nelson-Denny Practice Test (11 paragraphs). Paragraphs consisted of 49 to 66 words. In the PW condition, 22 paragraphs were created with pseudowords that were generated from the ARC Nonword Database (available at http://www.psy.uwa.edu.au/MRCDataBase/uwa_mrc.htm). The pseudowords were in accordance with the phonotactic rules of English so that they were pronounceable. The CS condition included 22 paragraphs with consonant-string nonwords that were created using randomly chosen consonants. Text was presented in Courier New font (monospaced) with 4.3 characters subtending 1° of visual angle. All nonword stimuli were matched to the words used in the NT condition with respect to the number of lines, the number of words, word length, and the position of punctuation.
Stimuli were presented using an Avotec Silent Vision 6011 projector in its native resolution (1024 × 768) and a refresh rate of 60 Hz. Eye-movements were monitored via a SR Research Eyelink 1000 long-range MRI eyetracker with a sampling rate of 1000 Hz. Viewing was binocular and eye-movements were recorded from the right eye.
In the scanner, a thirteen-point calibration procedure was administrated before each of the two functional runs to correctly map eye position to screen coordinates. Eye movements were recorded throughout the runs to ensure that natural reading eye-movements were executed during in the NT condition and that scanning eye movements were executed in the PW and CS conditions.
Each functional run consisted of 11 normal text paragraphs (the NT condition), 11 pseudoword paragraphs (the PW condition), and 11 consonant string paragraphs (the CS condition), as well as 11 filler trials containing pictures not relevant to the current study. Each trial was presented for 12 s preceded by a fixation cross for 6 s. Within each run, normal texts, pseudo texts and filler trials were presented in a random order for each participant. Participants therefore saw 22 trials in each condition over the two runs. Each functional run lasted about 14 min. Participants were asked to read paragraphs silently as if they were reading a novel when a text paragraph was presented, and to move their eyes “as if they were reading” in the PW and CS conditions.
MRI Data Acquisition
MR data were collected on a Siemens Medical Systems 3T Trio. A 3D T1-weighted “MPRAGE” RF-spoiled rapid flash scan in the sagittal plane, and a T2/PD-weighted multi-slice axial 2D dual Fast Turbo spin-echo scan in the axial plane was used. The multi-echo whole brain T1 scans had 1 mm isotropic voxel size and sufficient field of view to cover from the top of the head to the neck with the following protocol parameters: TR = 2530 ms, TE1 = 1.74 ms, TE2 = 3.6 ms, TE3 = 5.46 ms, TE4 = 7.32 ms, flip angle = 7°. All functional runs were acquired using gradient echo, echo-planar images with the following protocol parameters: TR = 1850 ms, TE = 30 ms, flip angle = 75°. Volumes consisted of thirty-four 3 mm slices with transversal orientation. Each volume covered the whole brain with FOV = 208 mm and 64 × 64 matrix, resulting in 3.3 × 3.3 × 3 mm voxel size.
The AFNI software package (Cox, 1996) was used for image analysis. Within-subject analysis involved slice timing correction, spatial co-registration (Cox and Jesmanowicz, 1999) and registration of functional images to the anatomy (Saad et al., 2009). Voxel-wise multiple linear regression was performed with the program 3dREMLfit, using reference functions representing each condition convolved with a standard hemodynamic response function. Reference functions representing the six motion parameters were included as covariates of no interest. In addition, the signal extracted from CSF and white matter was also included as noise covariates of no interest. General linear tests were conducted to obtain contrasts between conditions of interest.
The individual statistical maps and the anatomical scans were projected into standard stereotaxic space (Talairach and Tournoux, 1988) and smoothed with a Gaussian filter of 5 mm FWHM. In a random effects analysis, group maps were created by comparing activations against a constant value of 0. The group maps were thresholded at voxelwise p < 0.01 and corrected for multiple comparisons by removing clusters with below-threshold size to achieve a mapwise corrected p < 0.05. Using the 3dClustSim program with 10000 iterations, the cluster threshold was determined through Monte Carlo simulations that estimate the chance probability of spatially contiguous voxels exceeding the voxelwise p threshold, i.e., of false positive noise clusters. The smoothness of the data was estimated with the AFNI program 3dFWHMx using regression residuals as input. The analysis was restricted to a mask that excluded areas outside the brain, as well as deep white matter areas and the ventricles.
Table 1 shows basic eye-movement data for each condition. Data during track losses were eliminated and fixations meeting the following criteria were excluded from this analysis: A fixation made before or after a blink and fixation durations less than 50 ms or greater than 1500 ms. In total, 12.5% of fixations (11.3% for the NT condition, 13.5% for the PW condition, and 13% for the CS condition) were excluded from analysis. As seen in Table 1, mean fixation duration was statistically different across the three conditions, F(2, 62) = 56.67, p < 0.001, in that the NT condition had shorter fixation durations than the average of the PW and CS conditions, F(1, 31) = 62.58, p < 0.001. There was no difference in mean fixation duration between the PW and the CS condition, F(1, 31) = 0.004, p = 0.948. The pattern of results for mean fixation duration was also found in the first-pass reading time measures [first fixation duration (FFD), single fixation duration (SFD), and gaze duration (GD)], [F(2, 62) = 42.92, p < 0.0001 for FFD; F(2, 62) = 39.17, p < 0.0001 for SFD; F(2, 62) = 33.79, p < 0.0001 for GD], with those for the NT condition shorter relative to the average of the PW and the CS conditions, [F(1, 31) = 49.3, p < 0.0001 for FFD, F(1, 31) = 47.16, p < 0.0001 for SFD, F(1, 31) = 39.72, p < 0.0001 for GD], and no difference between the PW and the CS conditions, Fs < 1, ns. Saccadic amplitude also differed across the three conditions, F(2, 62) = 19.91, p < 0.001, with the NT condition producing greater saccadic amplitude than the average of the two nonword text conditions, F(1, 31) = 19.92, p < 0.001, and the PW condition producing greater saccadic amplitude compared with the CS condition, F(1, 31) = 19.63, p < 0.0001. The proportion of regressions (RegProp) also differed across the three conditions, F(2, 62) = 6.92, p < 0.005, with the NT condition producing more regressions than the average of the PW and the CS conditions, F(1, 31) = 8.01, p < 0.01, and a marginal difference between the PW and CS conditions, F(1, 31) = 4.0, p = 0.054. This general pattern of results in eye movements was similar to that reported in an analogous fMRI study (Henderson et al., 2014) and those obtained outside the scanner comparing natural and false-font texts (e.g., Henderson and Luke, 2012, 2014; Luke and Henderson, 2013).
Table 2. Talairach coordinates, volume of the cluster (μl), maximum z-score, and the label of anatomical structure for the normal text (NT) condition > fixation analysis, L, left hemisphere; R, right hemisphere.
Areas activated during natural reading of normal text are shown in Figure 1A and Table 2. These included cortical and subcortical areas associated with the eye-movement control network: bilateral FEF, SEF, bilateral IPS, bilateral superior colliculus (SC), and bilateral thalamus. Areas related to language processing were also strongly activated including bilateral middle temporal gyrus (MTG), bilateral superior temporal gyrus/sulcus (STG/STS), bilateral inferior frontal gyrus (IFG), and angular gyrus (AG). In addition, areas involved in visual processing were activated: bilateral cuneus, bilateral lingual gyrus, bilateral occipital pole, and left fusiform gyrus (FG).
Figure 1. Areas of significant activation in a whole-brain analysis of natural reading of normal text [NT condition, (A)], and “reading” pseudowords [PW condition, (B)] and consonant strings [CS condition, (C)]. L, Left Hemisphere; R, Right Hemisphere.
Areas activated during pseudo-reading of pseudowords are shown in Figure 1B and Table 3. Activation was seen in bilateral FEF, SEF, and IPS, as well as left STS and bilateral IFG. Visual activation was also observed in bilateral cuneus and left occipital cortex. Subcortical activation was seen in caudate, putamen, and pallidum.
Table 3. Talairach coordinates, volume of the cluster (μl), maximum z-score, and the label of anatomical structure for the pseudoword text (PW) condition > fixation analysis, L, left hemisphere; R, right hemisphere.
Areas activated during pseudo-reading of consonant strings are shown in Figure 1C and Table 4. Activation was seen in bilateral FEF, SEF, and IPS, left posterior STS (LpSTS), bilateral IFG, middle frontal gyrus (MFG), bilateral orbital gyrus, occipital cortex, and FG.
Table 4. Talairach coordinates, volume of the cluster (μl), maximum z-score, and the label of anatomical structure for the consonant string text (CS) condition > fixation analysis, L, left hemisphere; R, right hemisphere.
Conjunction of normal text, pseudowords, and consonant strings
A conjunction analysis of the NT, PW, and CS conditions was conducted to examine the common eye-movement network. Activation in this contrast would also be expected for areas involved in processing character strings (e.g., orthographic and potentially phonological processing) related to the presence of alphabetic characters. Figure 2 shows the results. Activation was observed in bilateral FEF, SEF, and IPS. Activation was also observed in LpSTS, left IFG, left precentral gyrus, and left MFG (premotor area, BA6).
Figure 2. Conjunction results for the normal text (NT), pseudowords (PW), and consonant string (CS) conditions (NT – fixation n PW− fixation n CS− fixation). L, Left Hemisphere; R, Right Hemisphere.
Normal text vs. average of pseudowords and consonant strings
The NT condition was compared to the two nonword conditions to examine normal reading versus pseudo-reading. The PW and CS conditions produced similar patterns of activation against fixation baseline (see Figures 1B,C), so these conditions were averaged for this comparison. Activated areas are shown in Figure 3 and Table 5. Areas producing greater activation in the NT condition were left MFG including lateral FEF, bilateral SFG including pre-SMA, bilateral STS and anterior STG, bilateral MTG, bilateral AG, bilateral IFG (pars triangularis), bilateral cuneus, and bilateral precuneus. Subcortical activation was seen in caudate, thalamus, and ventral diencephalon.
Table 5. Talairach coordinates, volume of the cluster (μl), maximum z-score, and the label of anatomical structure for the normal text vs. the nonword texts analysis, L, left hemisphere; R, right hemisphere.
Figure 3. Areas activated in a whole-brain analysis of the contrast of natural text reading (NT) versus the average of pseudowords and consonant strings (mean of PW and CS). Hot regions represent more activation for text whereas cool regions reflect more activation for nonwords. L, Left Hemisphere; R, Right Hemisphere.
Greater activation was produced in the average of the two nonword conditions in bilateral SFG, bilateral anterior and posterior cingulate, bilateral supramarginal gyrus, bilateral transverse temporal gyrus/sulcus, bilateral MFG/MFS, bilateral postcentral gyrus/sulcus, bilateral hippocampus, bilateral parieto-occipital sulcus, bilateral insula, bilateral IFG (pars opercularis), bilateral superior parietal lobule and IPS, right subcentral gyrus/sulcus, right superior occipital gyrus, right middle occipital gyrus, right orbital sulcus, and right precentral sulcus.
Pseudowords vs. consonant strings
Figure 4 and Table 6 show the results of a PW versus CS contrast. Activation was greater in the PW condition in left IPS, right inferior temporal gyrus (ITG), right FG, and right caudate. No regions produced more activation in the CS condition.
Figure 4. Areas activated in a whole-brain analysis of the contrast of the pseudowords versus consonant string conditions. Hot regions represent more activation in the pseudowords condition than in the consonant string condition. L, Left Hemisphere; R, Right Hemisphere.
Table 6. Talairach coordinates, volume of the cluster (μl), maximum z-score, and the label of anatomical structure for the pseudo text vs. the consonant string text analysis, L, left hemisphere; R, right hemisphere.
This study was designed to investigate the neural correlates of natural reading. Subjects read passages of text presented in paragraph form while both eye movements and the BOLD signal were recorded. The natural reading condition was compared to two pseudo-reading conditions in which words were replaced by either pronounceable pseudowords or consonant strings. In the latter two conditions subjects were asked to move their eyes through the text “as if they were reading” (Vitu et al., 1995; Rayner and Fischer, 1996; Nuthmann and Engbert, 2009; Reichle et al., 2010; Henderson and Luke, 2012, 2014; Luke and Henderson, 2013). Three specific questions were addressed: the nature of the common eye-movement control network in these sequential scanning tasks, the nature of the eye movement network in natural reading, and the nature of the language network in natural reading.
Common Eye Movement Control Network
A common eye movement control network was revealed across the three conditions. As can be seen in Figure 2, bilateral FEF, bilateral IPS, and SEF were activated when sequential and complex eye movements were executed during these task. The observed areas are consistent with those that have been proposed as the core eye movement control network, as observed in single-saccade eye movement tasks (Pierrot-Deseilligny et al., 2004; McDowell et al., 2008; Müri and Nyffeler, 2008; Jamadar et al., 2013). These results are also consistent with those of Hillen et al. (2013) in which subjects searched text and pseudo-text for Landolt C targets. The results suggest that this core eye movement control network is functional in single-saccade eye-movements tasks, sequential search tasks, and in natural reading.
The activated regions reported here were very similar to those reported by Hillen et al. (2013) in their conjunction analysis across five conditions to identify the common gaze control network. However, the network reported by Hillen et al. did not include the left IPS, whereas the current results showed bilateral IPS activation. Although both Hillen et al. and the present study measured BOLD responses with sequential oculomotor paradigms rather than the SVP paradigm, Hillen et al. employed a secondary search task, which was not used in the present study. This difference between the two studies might explain why slightly different patterns of activation were observed in the eye movement control network. Note that simple eye-movement tasks usually generate activation in bilateral IPS (see Jamadar et al., 2013), implying that the IPS in both hemispheres is involved in saccade control and/or attentional processes. Hillen et al. did not discuss why activation in the left IPS was not obtained in the conjunction analysis in their paper, but did discuss the right IPS activation with respect to attentional processes (Culham et al., 2006) and planning of movements (Barthélémy and Boulinguez, 2002). Activation of the IPS in eye movement control might be associated either with saccadic movement, attentional processes, or saccade-related attentional processes. As Pierrot-Deseilligny et al. (2004) pointed out, it is relatively difficult to separate activation by saccades from activation by attentional processes, because they typically co-occur. Indeed, covert attention and eye movement control are tightly linked in reading (Henderson and Ferreira, 1990), and this link forms the basis of several current computational models of reading, (e.g., Reichle et al., 1998; Engbert et al., 2005). Further research will be needed to differentiate the neurobiology of these processes in natural reading.
Eye Movement Control Network for Reading
The contrast analysis between normal reading and pseudo-reading showed that lateral FEF was more activated during normal reading compared to pseudo-reading. McDowell et al. (2008) proposed that lateral FEF is more associated with visually guided saccadic eye movements. In addition, Jamadar et al. (2013), in their meta-analysis, found more lateral FEF activation in pro-saccades relative to a fixation control, supporting the idea that lateral FEF is more involved in visually guided automatic eye movements. Although eye movements during natural reading are not as simple and reflexive as those in the pro-saccade task, they are highly automatized and guided by visual information from the upcoming word in the parafovea. This similarity might account for the greater activation that was observed in lateral FEF in the normal reading condition relative to the pseudo-reading conditions. At the same time, the pseudo-reading conditions showed more activation in the eye movement control network including bilateral IPS, as well as ACC, relative to the normal reading condition. These structures have been reported to be associated with both attentional processes and eye movement control (Pierrot-Deseilligny et al., 2004). Although, as indicated above, it is very difficult to functionally differentiate attentional control from eye movement control, this result may suggest that pseudo-reading requires more attentional control and/or more fine-grained eye movement coordination compared to normal reading, consistent with the idea that eye movement control is more automatized in natural reading.
Other Regions of the Common Network
In addition to the eye-movement network, activation across the three conditions was also observed in left posterior MFG and premotor area, left posterior IFG, and posterior STS. These regions are not commonly thought to be related to eye movement control, and could be associated with task-dependent processing. For example, premotor and posterior IFG activation could be related to covert articulation (McGuire et al., 1996; Ghosh et al., 2008; Peeva et al., 2010; Rogalsky and Hickok, 2011). In the two nonword reading conditions, subjects may have been able to pronounce the nonwords in both the PW and CS conditions, activating phonological representations and phonological working memory. Because we matched strings in length across conditions, many of the consonant strings may have been pronounceable because they were one- to three-letters long. For example, a two-letter consonant string like sp can be pronounced.
The conjunction analysis also showed activation of the LpSTS which has been suggested to be a region of multi-functionality (Hein and Knight, 2008; Liebenthal et al., 2014). Here, the likely role of LpSTS is also in phonological processing as part of a dorsal/posterior pathway that represents transient phonological representations and maps them to articulatory codes in premotor and posterior inferior frontal regions (Wise et al., 2001; Hickok and Poeppel, 2007; Desai et al., 2008; Obleser and Eisner, 2009). Note that the normal reading condition activated these areas (left IFG/MFG and LpSTS) to a greater extent than the pseudo-reading conditions in the contrast analysis (see Figure 3), suggesting greater and more automatic phonological processing in normal reading compared to pseudo reading.
Linguistic and Related Cognitive Processes
Reading paragraphs for meaning requires many levels of language representation and processing including orthographic/phonological processing, lexical access, retrieval of lexico-semantic information, syntactic computation, and semantic interpretation. The language network observed in previous studies, including STG, STS, MTG, IFG, MFG, AG, and precuneus were also more activated during normal paragraph reading compared to the pseudo-reading conditions in the present study. Regions in the lateral temporal lobe, AG, and precuneus form the core of a semantic network observed in numerous studies that typically use word or sentence stimuli (Binder et al., 2009). This network was activated strongly for natural reading of text relative to the nonword conditions in the present study, extending these past findings to natural text reading.
At the same time, the nonword conditions showed more activation in medial frontal/ACC, posterior cingulate cortex, and bilateral IPS, areas associated with attentional brain networks (for a recent review, see Petersen and Posner, 2012). The greater attentional network activation observed here for nonwords suggests that “reading” paragraphs with nonword stimuli requires substantial attentional resources, extending findings from several single-word studies that show a similar pattern (e.g., Price et al., 1996; Hagoort et al., 1999; Mechelli et al., 2000; Paulesu et al., 2000; Binder et al., 2005; Vigneau et al., 2006). In our nonword reading conditions, readers were asked to imitate normal reading patterns with eye movements, which is a relatively unpracticed task that requires encoding nonwords, inhibiting neighbor words, and coordinating eye movement control. These processes likely require more attentional resources than natural reading. Hillen et al. (2013) also reported that a condition in which text was replaced by Landolt rings showed more activation in the right inferior parietal lobule relative to the conditions that used alphabetic characters, suggesting that “Landolt reading” similarly requires more attentional resources compared to the other conditions. In addition to the neural data, the behavioral eye movement data in the current experiment support this idea in that fixation durations in nonword reading were longer than those for normal reading, indicating that more effort is necessary for nonword reading than for normal reading (for similar findings, see Henderson and Luke, 2012, 2014; Henderson et al., 2014). Another way to state this is that natural reading is highly automatized and therefore requires less attentional control than does consciously executing similar sequences of eye movements.
The text vs. nonword comparison also showed activation in bilateral IFG, both for text and for nonwords, consistent with previous single word and sentence processing studies. The IFG has a well-established role in language, including possible semantic, syntactic, phonological, articulatory, and executive functions (e.g., Hagoort, 2005; Grodzinsky and Santi, 2008; Friederici, 2009; Duncan, 2010; Rogalsky and Hickok, 2011). Regions of the anterior IFG overlapping BA 47 and 45, activated more for text, likely reflect semantic retrieval processes. The posterior IFG, activated to a greater extent for nonwords, likely reflects more effortful covet articulation and phonological processing. Additionally, as argued by Duncan (2010), posterior IFG is part of a multiple-demand network that includes posterior IFS, anterior insula/frontal operculum, pre-SMA/ACC, and IPS, and is associated with cognitive control. In the current study, nonword reading requires more effortful processing to perform complex saccadic coordination relative to normal reading because no linguistic information is provided in foveal or parafoveal vision, whereas sequential saccadic movements in normal reading can be guided by linguistic information both at the fovea and parafovea. Accordingly, readers are likely to use a less efficient control mechanism for eye movements in nonword reading relative to normal reading, as indicated by the greater activation in frontoparietal areas including PO, anterior insula, and IPS.
In summary, the present study showed that (1) activation of a core eye-movement control network was observed when participants naturally read text paragraphs or moved their eyes through nonword text, (2) differences in activation of the eye movement control network in natural reading and pseudo-reading suggest that readers use automatized saccadic coordination during natural reading whereas they require more complex attentional and control processes during pseudo-reading, and (3) normal reading produced distinct patterns of neural activation in a language-related network, extending previous findings with word and sentence stimuli. These results indicate that presenting entire paragraphs (or sentences) in fMRI during natural reading can provide important data with respect to the neural understanding of language processing and eye-movement control in reading.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
This research was supported by grants from the National Science Foundation (BCS-1151358) and the National Institutes of Health (R01 DC010783). The authors declare no competing financial interests.
Altmann, U., Bohrn, I. C., Lubrich, O., Menninghaus, W., and Jacobs, A. M. (2014). Fact vs fiction-how paratextual information shapes our reading processes. Social Cognitive and Affective Neuroscience, 9, 22–29. doi: 10.1093/scan/nss098
Barthélémy, S., and Boulinguez, P. (2002). Manual asymmetries in the directional coding of reaching: further evidence for hemispatial effects and right hemisphere dominance for movement planning. Experimental Brain Research, 147, 305–312. doi: 10.1007/s00221-002-1247-x
Binder, J. R., Desai, R. H., Graves, W. W., and Conant, L. L. (2009). Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. Cerebral Cortex, 19, 2767–2796. doi: 10.1093/cercor/bhp055
Binder, J. R., Medler, D. A., Desai, R., Conant, L. L., and Liebenthal, E. (2005). Some neurophysiological constraints on models of word naming. Neuroimage, 27, 677–693. doi: 10.1016/j.neuroimage.2005.04.029
Bohrn, I. C., Altmann, U., Lubrich, O., Menninghaus, W., and Jacobs, A. M. (2013). When we like what we know – a parametric fMRI analysis of beauty and familiarity. Brain and Language, 124, 1–8. doi: 10.1016/j.bandl.2012.10.003
Culham, J. C., Cavina-Pratesi, C., and Singhal, A. (2006). The role of parietal cortex in visuomotor control: what have we learned from neuroimaging? Neuropsychologia, 44, 2668–2684. doi: 10.1016/j.neuropsychologia.2005.11.003
Desai, R., Liebenthal, E., Waldron, E., and Binder, J. R. (2008). Left posterior temporal regions are sensitive to auditory categorization. Journal of Cognitive Neuroscience 20, 1174–1188. doi: 10.1162/jocn.2008.20081
Ettinger, U., Ffytche, D. H., Kumari, V., Kathmann, N., Reuter, B., Zelaya, F., et al. (2008). Decomposing the neural correlates of antisaccade eye movements using event-related fMRI. Cerebral Cortex 18, 1148–1159. doi: 10.1093/cercor/bhm147
Fedorenko, E., Behr, M., and Kanwisher, N. (2011). Functional specificity for high-level linguistic processing in the human brain. Proceedings of National Academy of Sciences, 108, 16428–16433. doi: 10.1073/pnas.1112937108
Ferstl, E. C., and von Cramon, D. Y. (2001). The role of coherence and cohesion in text comprehension: an event-related fMRI study. Cognitive Brain Research, 11, 325–340. doi: 10.1016/S0926-6410(01)00007-6
Ford, K. A., Goltz, H. C., Brown, M. R. G., and Everling, S. (2005). Neural processes associated with antisaccade task performance investigated with event-related fMRI. Journal of Neurophysiology. 94, 429–440. doi: 10.1152/jn.00471.2004
Ghosh, S. S., Tourville, J. A., and Guenther, F. H. (2008). A neuroimaging study of premotor lateralization and cerebellar involvement in the production of phonemes and syllables. Journal of Speech, Language, and Hearing Research, 51, 1183–1202. doi: 10.1044/1092-4388(2008/07-0119)
Hagoort, P., Indefrey, P., Brown, C., Herzog, H., Steinmetz, H., and Seitz, R. (1999). The neural circuitry involved in the reading of German words and pseudowords: a PET study. Journal of Cognitive Neuroscience, 11, 383–398. doi: 10.1162/089892999563490
Henderson, J. M., Choi, W., and Luke, S. G. (2014). Morphology of primary visual cortex predicts individual differences in fixation duration during text reading. Journal of Cognitive Neuroscience, 26, 2880–2888. doi: 10.1162/jocn_a_00668
Henderson, J. M., and Ferreira, F. (1990). The effects of foveal difficulty on the perceptual span in reading: implications for attention and eye movement control. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 417–429. doi: 10.1037/0278-73220.127.116.117
Henderson, J. M., and Luke, S. G. (2014). Stable individual differences in saccadic eye movements during reading, pseudo-reading, scene viewing, and scene search. Journal of Experimental Psychology: Human Perception and Performance, 40, 1390–1400. doi: 10.1037/a0036330
Hillen, R., Günther, T., Kohlen, C., Eckers, C., van Ermingen-Marbach, M., Sass, K., et al. (2013). Identifying brain systems for gaze orienting during reading: fMRI investigation of the Landolt paradigm. Front. Hum. Neurosci. 7:384. doi: 10.3389/fnhum.2013.00384
Hsu, C. T., Jacobs, A. M., and Conrad, M. (2015). Can Harry Potter still put a spell on us in a second language? An fMRI study on reading emotion-laden literature in late bilinguals. Cortex 63, 282–295. doi: 10.1016/j.cortex.2014.09.002
Jamadar, S. D., Fielding, J., and Egan, G. F. (2013). Quantitative meta-analysis of fMRI and PET studies reveals consistent activation in fronto-striatal-parietal regions and cerebellum during antisaccades and prosaccades. Front. Psychol. 4:749. doi: 10.3389/fpsyg.2013.00749
Liebenthal, E., Desai, R., Humphries, C., and Sabri, M. (2014). The functional organization of the left STS for speech: a large scale meta-analysis of PET and fMRI studies of healthy adults. Front. Neurosci. 8:289. doi: 10.3389/fnins.2014.00289
Luke, S. G., and Henderson, J. M. (2013). Oculomotor and cognitive control of eye movements in reading: evidence from mindless reading. Attent. Percept. Psychophys. 75, 1230–1242. doi: 10.3758/s13414-013-0482-5
Martin-Loeches, M., Casado, P., Hernandez-Tamames, J. A., and Alvarez-Linera, J. (2008). Brain activation in discourse comprehension: a 3T fMRI study. Neuroimage 41, 614–622. doi: 10.1016/j.neuroimage.2008.02.047
McDowell, J. E., Dyckman, K. A., Austin, B. P., and Clementz, B. A. (2008). Neurophysiology and neuroanatomy of reflexive and volitional saccades: evidence from studies of humans. Brain Cogn. 68, 255–270. doi: 10.1016/j.bandc.2008.08.016
McGuire, P. K., Silbersweig, D. A., Murray, R. M., David, A. S., Frackowiak, R. S., and Frith, C. D. (1996). Functional anatomy of inner speech and auditory verbal imagery. Psychol. Med. 26, 29–38. doi: 10.1017/S0033291700033699
Mechelli, A., Friston, K. J., and Price, C. J. (2000). The effects of presentation rate during word and pseudoword reading: a comparison of PET and fMRI. J. Cogn. Neurosci. 12(Suppl. 2), 145–156. doi: 10.1162/089892900564000
Müri, R. M., and Nyffeler, T. (2008). Neurophysiology and neuroanatomy of reflexive and volitional saccades as revealed by lesion studies with neurological patients and transcranial magnetic stimulation (TMS). Brain Cogn. 68, 284–292. doi: 10.1016/j.bandc.2008.08.018
Paus, T., Petrides, M., Evans, A. C., and Meyer, E. (1993). Role of the human anterior cingulate cortex in control of oculomotor, manual and speech responses: a positron emission tomography study. J. Neurophysiol. 70, 453–469.
Peeva, M. G., Guenther, F. H., Tourville, J. A., Nieto-Castanon, A., Anton, J. L., Nazarian, B., et al. (2010). Distinct representations of phonemes, syllables, and supra-syllabic sequences in the speech production network. Neuroimage 50, 626–638. doi: 10.1016/j.neuroimage.2009.12.065
Rayner, K. (2009). The Thirty Fifth Sir Frederick Bartlett Lecture: eye movements and attention in reading, scene perception, and visual search. Q. J. Exp. Psychol. 62, 1457–1506. doi: 10.1080/17470210902816461
Reichle, E. D., Tokowicz, N., Liu, Y., and Perfetti, C. A. (2011). Testing an assumption of the E-Z Reader model of eye-movement control during reading: using event-related potentials to examine the familiarity check. Psychophysiology 48, 993–1003. doi: 10.1111/j.1469-8986.2011.01169.x
Richlan, F., Gagl, B., Hawelka, S., Braun, M., Schurz, M., Kronbichler, M., et al. (2014). Fixation-related fMRI analysis in the domain of reading research: Using self-paced eye movements as markers for hemodynamic brain responses during visual letter string processing. Cereb. Cortex 24, 2647–2656. doi: 10.1093/cercor/bht117
Saad, Z. S., Glen, D. R., Chen, G., Beauchamp, M. S., Desai, R., and Cox, R. W. (2009). A new method for improving functional-to-structural MRI alignment using local Pearson correlation. Neuroimage 44, 839–848. doi: 10.1016/j.neuroimage.2008.09.037
Schotter, E. R., Tran, R., and Rayner, K. (2014). Don't believe what you read (only once): Comprehension is supported by regressions during reading. Psychol. Sci. 25, 1218–1226. doi: 10.1177/0956797614531148
Sweeney, J. A., Mintun, M. A., Kwee, S., Wiseman, M. B., Brown, D. L., Rosenberg, D. R., et al. (1996). Positron emission tomography study of voluntary saccadic eye movements and spatial working memory. J. Neurophysiol. 75, 454–468.
Van Essen, D. C., Drury, H. A., Dickson, J., Harwell, J., Hanlon, D., and Anderson, C. H. (2001). An integrated software suite for surface-based analyses of cerebral cortex. J. Am. Med. Inform. Assoc. 8, 443–459. doi: 10.1136/jamia.2001.0080443
Vigneau, M., Beaucousin, V., Herve, P. Y., Duffau, H., Crivello, F., Houde, O., et al. (2006). Meta-analyzing left hemisphere language areas: phonology, semantics, and sentence processing. Neuroimage 30, 1414–1432. doi: 10.1016/j.neuroimage.2005.11.002
Vitu, F., O'Regan, J. K., Inhoff, A. W., and Topolski, R. (1995). Mindless reading: Eye-movement characteristics are similar in scanning letter strings and reading texts. Percept. Psychophys. 57, 352–364.
Keywords: reading, eye movements, fMRI, pseudo-reading, attention
Citation: Choi W, Desai RH and Henderson JM (2014) The neural substrates of natural reading: a comparison of normal and nonword text using eyetracking and fMRI. Front. Hum. Neurosci. 8:1024. doi: 10.3389/fnhum.2014.01024
Received: 30 October 2014; Paper pending published: 23 November 2014;
Accepted: 04 December 2014; Published online: 23 December 2014.
Edited by:Arthur M. Jacobs, Freie Universität Berlin, Germany
Reviewed by:Roel M. Willems, Donders Institute for Brain, Cognition and Behaviour, Netherlands
Mario Braun, University of Salzburg, Austria
Copyright © 2014 Choi, Desai and Henderson. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: John M. Henderson, Institute for Mind and Brain, University of South Carolina, 1800 Gervais Street, Columbia, SC 29208, USA e-mail: email@example.com