Eye movement-related brain activity during perceptual and cognitive processing
Andrey R. Nikolaev
Sebastian Pannasch
Junji Ito
Artem V. Belopolsky- 1Research Group Experimental Psychology, KU Leuven, Leuven, Belgium
- 2Department of Psychology, Technische Universität Dresden, Dresden, Germany
- 3Research Center Juelich, Institute of Neuroscience and Medicine (INM-6), Juelich, Germany
- 4Department of Cognitive Psychology, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
For several decades researchers have been recording electrical brain activity associated with eye movements in attempt to understand their neural mechanisms. However, recent advances in eye-tracking technology have allowed researchers to use eye movements as the means of segmenting the ongoing brain activity into episodes relevant to cognitive processes in scene perception, reading, and visual search. This opened doors to uncovering the active and dynamic neural mechanisms underlying perception, attention and memory in naturalistic conditions. The present eBook contains a representative collection of studies from various fields of visual neuroscience that use this cutting edge approach of combining eye movements and neural activity.
The majority of the articles in the eBook combine the measurement of eye movements with the recording of the electroencephalogram (EEG) in human subjects performing various psychological tasks. The most common methodological approach is examination of the EEG activity time-aligned to certain eye movement events, such as the onset of a fixation or a start of a saccadic eye movement (Fischer et al., 2013; Frey et al., 2013; Henderson et al., 2013; Hutzler et al., 2013; Nikolaev et al., 2013; Richards, 2013; Simola et al., 2013). Several works employ the time-frequency and synchrony analysis (Fischer et al., 2013; Hoffman et al., 2013; Ito et al., 2013; Nakatani and Van Leeuwen, 2013; Nakatani et al., 2013).
The advantage of simultaneous EEG and eye movement recording is most evident in investigation of perceptual and cognitive processes during free eye movement behavior. Saccades in free viewing are guided by the bottom-up and top-down attentional mechanisms. To study interactions between these mechanisms Fischer et al. (2013) explored the eye fixation-related potentials (EFRP) and EEG power during extended picture viewing. The difference between the mechanisms was reflected in the EFRP components and in the power of the frontal beta- and theta activity. Nakatani and Van Leeuwen (2013) recorded EEG and eye movements during free viewing of the Necker cube. They found that saccades and blinks facilitate perceptual switches. Moreover, the amplitude of alpha activity preceding these eye events predicted whether a blink or a saccade results in the switch. Nikolaev et al. (2013) examined the pre-saccadic EEG activity during free visual exploration of a natural scene in anticipation of a memory test. Their findings illustrate how pre-saccadic activity differentiates encoding of visual information and selection of a target for the next fixation. Simola et al. (2013) investigated attention and emotion processes by analyzing EFRPs in free viewing. They found that emotional processing depends on the overt attentional resources. Ito et al. (2013) observed interaction between low and high frequency components in the local field potentials (LFP) recorded in the visual cortex of monkeys performing voluntary saccades during natural scene viewing. They concluded that the cross-frequency interaction is a manifestation of the mechanism which coordinates oculomotor behavior and sensory processing. Hoffman et al. (2013) recorded the fixation-related neural activity in the human and macaque hippocampus during unrestricted visual search. They found in both species that the fixation-related phase alignment of the hippocampal low-frequency oscillations depends on the visual task.
Not only EEG or LFP recordings, but fMRI can be also related to eye movements in free viewing of scenes (Marsman et al., 2013). The authors investigated the neural correlates of ambient and focal processing using fixation-based event-related (FIBER) fMRI in combination with independent component analysis. They reported the eye-movement related activity in the ventromedial and ventrolateral visual cortices.
As shown in this eBook, reading studies also benefit from the combination of EEG and eye movement recordings. Henderson et al. (2013) devised an advanced procedure to correct EFRP for eye movement artifacts. After correction the early EFRP components were different between reading and pseudo-reading (control) condition. To investigate parafoveal pre-processing in reading Hutzler et al. (2013) applied the X-mask in the task of new/old word judgment. The time course of the EFRP indicated dynamical interference of the parafoveal mask with foveal word recognition. Frey et al. (2013) studied decision making during reading using EFRP. They compared EFRP when participants fixated the words related and unrelated to the predefined decision conditions. The late EFRP components appeared to be indicative of the semantic decision.
Furthermore, two studies investigated brain activity during performance of particular saccade tasks. Nakatani et al. (2013) measured cross-frequency phase coupling in peri-fixation brain activity during semantic judgment in the controlled saccade conditions. They concluded that the cross-frequency phase synchrony constitutes a plausible mechanism for tagging of fixation information. Richards (2013) used prosaccade and antisaccade tasks to examine the cortical sources of the related EEG activity. He demonstrated the value of this approach to differentiate the brain activities associated with general preparatory processes and with eye movement execution.
The methodological aspects of co-registration of brain activity with eye movements are addressed in the work which introduces a video projection system for high-speed, gaze-contingent stimulus presentation (Richlan et al., 2013).
Besides the chapters which describe the brain activity related to saccadic eye movements, the eBook also features a review on cognitive processes involved in smooth pursuit eye movements in humans and animals (Fukushima et al., 2013). The authors discuss involvement of cognitive processes in the smooth pursuit including neural substrates, behavioral evidence and clinical correlations.
Taken together, the present eBook demonstrates that relating brain activity and eye movements is a fruitful way of studying a wide range of psychological processes without imposing an artificial task structure. This approach is particularly useful to demonstrate how brain dynamics underlying perceptual and cognitive processes unfolds over time in naturalistic conditions.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Fischer, T., Graupner, S. T., Velichkovsky, B. M., and Pannasch, S. (2013). Attentional dynamics during free picture viewing: evidence from oculomotor behavior and electrocortical activity. Front. Syst. Neurosci. 7:17. doi: 10.3389/fnsys.2013.00017
Frey, A., Ionescu, G., Lemaire, B., Lopez-Orozco, F., Baccino, T., and Guerin-Dugue, A. (2013). Decision-making in information seeking on texts: an eye-fixation-related potentials investigation. Front. Syst. Neurosci. 7:39. doi: 10.3389/fnsys.2013.00039
Fukushima, K., Fukushima, J., Warabi, T., and Barnes, G. R. (2013). Cognitive processes involved in smooth pursuit eye movements: behavioral evidence, neural substrate and clinical correlation. Front. Syst. Neurosci. 7:4. doi: 10.3389/fnsys.2013.00004
Henderson, J. M., Luke, S. G., Schmidt, J., and Richards, J. E. (2013). Co-registration of eye movements and event-related potentials in connected-text paragraph reading. Front. Syst. Neurosci. 7:28. doi: 10.3389/fnsys.2013.00028
Hoffman, K. L., Dragan, M. C., Leonard, T. K., Micheli, C., Montefusco-Siegmund, R., and Valiante, T. A. (2013). Saccades during visual exploration align hippocampal 3-8 Hz rhythms in human and non-human primates. Front. Syst. Neurosci. 7:43. doi: 10.3389/fnsys.2013.00043
Hutzler, F., Fuchs, I., Gagl, B., Schuster, S., Richlan, F., Braun, M., et al. (2013). Parafoveal X-masks interfere with foveal word recognition: evidence from fixation-related brain potentials. Front. Syst. Neurosci. 7:33. doi: 10.3389/fnsys.2013.00033
Ito, J., Maldonado, P., and Grun, S. (2013). Cross-frequency interaction of the eye-movement related LFP signals in V1 of freely viewing monkeys. Front. Syst. Neurosci. 7:1. doi: 10.3389/fnsys.2013.00001
Marsman, J. B., Renken, R., Haak, K. V., and Cornelissen, F. W. (2013). Linking cortical visual processing to viewing behavior using fMRI. Front. Syst. Neurosci. 7:109. doi: 10.3389/fnsys.2013.00109
Nakatani, C., Chehelcheraghi, M., Jarrahi, B., Nakatani, H., and Van Leeuwen, C. (2013). Cross-frequency phase synchrony around the saccade period as a correlate of perceiver's internal state. Front. Syst. Neurosci. 7:18. doi: 10.3389/fnsys.2013.00018
Nakatani, H., and Van Leeuwen, C. (2013). Antecedent occipital alpha band activity predicts the impact of oculomotor events in perceptual switching. Front. Syst. Neurosci. 7:19. doi: 10.3389/fnsys.2013.00019
Nikolaev, A. R., Jurica, P., Nakatani, C., Plomp, G., and Van Leeuwen, C. (2013). Visual encoding and fixation target selection in free viewing: presaccadic brain potentials. Front. Syst. Neurosci. 7:26. doi: 10.3389/fnsys.2013.00026
Richards, J. E. (2013). Cortical sources of ERP in prosaccade and antisaccade eye movements using realistic source models. Front. Syst. Neurosci. 7:27. doi: 10.3389/fnsys.2013.00027
Richlan, F., Gagl, B., Schuster, S., Hawelka, S., Humenberger, J., and Hutzler, F. (2013). A new high-speed visual stimulation method for gaze-contingent eye movement and brain activity studies. Front. Syst. Neurosci. 7:24. doi: 10.3389/fnsys.2013.00024
Keywords: eye movements, saccade, smooth pursuit, eye tracking, free viewing, EEG, local field potentials, fMRI
Citation: Nikolaev AR, Pannasch S, Ito J and Belopolsky AV (2014) Eye movement-related brain activity during perceptual and cognitive processing. Front. Syst. Neurosci. 8:62. doi: 10.3389/fnsys.2014.00062
Received: 08 February 2014; Accepted: 04 April 2014;
Published online: 24 April 2014.
Edited and reviewed by: Maria V. Sanchez-Vives, ICREA-IDIBAPS, Spain
Copyright © 2014 Nikolaev, Pannasch, Ito and Belopolsky. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: andrey.nikolaev@ppw.kuleuven.be