# HIGH-LEVEL ADAPTATION AND AFTEREFFECTS

EDITED BY: Rocco Palumbo, Stefania D'Ascenzo and Luca Tommasi PUBLISHED IN: Frontiers in Psychology and Frontiers in Neuroscience

#### *Frontiers Copyright Statement*

*© Copyright 2007-2017 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.*

*The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.*

*Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.*

*Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.*

*As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.*

> *All copyright, and all rights therein, are protected by national and international copyright laws.*

*The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.*

ISSN 1664-8714 ISBN 978-2-88945-147-0 DOI 10.3389/978-2-88945-147-0

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# **HIGH-LEVEL ADAPTATION AND AFTEREFFECTS**

Topic Editors: **Rocco Palumbo,** Harvard Medical School, USA **Stefania D'Ascenzo,** University of Bologna, Italy **Luca Tommasi,** "G. d'Annunzio" University, Italy

Aftereffects generally occur after a prolonged exposure (adaptation) to a first stimulus possessing one given property followed by presentation of a stimulus bearing a neutral value of that property. The aftereffect consists in a change in appearance of the neutral stimulus following the adapter, compared to the appearance of the neutral stimulus when it is perceived without any previous exposure to the adapter. The transient phenomena of perceptual aftereffects are believed to depend on the activation of neuron populations that respond selectively to a given property of the stimuli. Studying how adaptation occurs (which stimulus properties are sensitive to it, which timings are necessary, whether individual differences modulate its occurrence) has thus become an indirect way to probe the plasticity of sensory functions in the nervous system, recently extending to more cognitive and representational aspects of neural coding.

In the last two decades, indeed, it has been demonstrated that aftereffects occur not only for low-level properties of stimuli (such as motion, color, or orientation) but also for high-level properties. Many studies have proven that high-level proprieties of the stimuli, e.g. gender, identity, ethnicity, or age of a face or a voice, are sensitive to this phenomenon. It has been shown, for example, that the prolonged exposure to a female or male face produces a gender misperception in the opposite direction when an androgynous face is shown after the adapter. Furthermore, recent studies have also shown that aftereffects are not strictly contingent upon the physical features that make up stimuli, but they seem to run across the high-level proprieties subjects are adapted to. These evidences are supported by cross-category adaptation studies, which underlie how aftereffects occur even across stimuli that do not share physical features (e.g. bodies and faces) but that instead, share common higher-level properties, such as gender.

Given the growing body of research focused on adaptation and aftereffects in high-level perception at the boundaries with perceptual learning, attention and cognition, the purpose of this topic is to provide a picture of the state of the art relative to the specific phenomena of adaptation in high-level perceptual processing.

**Citation:** Palumbo, R., D'Ascenzo, S., Tommasi, L., eds. (2017). High-Level Adaptation and Aftereffects. Lausanne: Frontiers Media. doi: 10.3389/978-2-88945-147-0

# Table of Contents


### **Aftereffects and emotions**

*67 Emotional Actions Are Coded via Two Mechanisms: With and without Identity Representation*

Joanna Wincenciak, Jennie Ingham, Tjeerd Jellema and Nick E. Barraclough

*80 Adaptation to Complex Pictures: Exposure to Emotional Valence Induces Assimilative Aftereffects*

Rocco Palumbo, Stefania D'Ascenzo, Angelica Quercia and Luca Tommasi

### **What's Next**


Carine Michel

# Editorial: High-Level Adaptation and Aftereffects

#### Rocco Palumbo1, 2 \*, Stefania D'Ascenzo<sup>3</sup> and Luca Tommasi <sup>2</sup>

*<sup>1</sup> Schepens Eye Research Institute, Harvard Medical School, Boston, MA, USA, <sup>2</sup> Department of Psychological Science, Humanities and Territory, "G. d'Annunzio" University, Chieti, Italy, <sup>3</sup> Department of Philosophy and Communication, University of Bologna, Bologna, Italy*

Keywords: aftereffects, adaptation, high-level adaptation, visual cognition, perception

**Editorial on the Research Topic**

#### **High-Level Adaptation and Aftereffects**

Adaptation, in sensory and perceptual science, refers to the action of a prolonged exposure of a receiver (at the cellular level a sensory receptor, at the organismic level an animal endowed with sensory organs) to an environmental stimulus. While adaptation of sensory receptors is strictly equivalent to the physiological phenomenon of a reduced electrical response by a receptor following its sustained stimulation, in this Frontiers Research Topic "adaptation" will be conceived at the organismic level, in which a wide spectrum of perceptual effects, mostly known as "aftereffects," has been observed for more than two thousand years (Aristotle, ca 350 B.C.<sup>1</sup> ). Aftereffects refer to those changes in the way a stimulus (the test) is perceived following a prolonged exposure to a previous stimulus (the adaptor). Undoubtly, for a change to be deemed relevant, a comparison must be made between the perception of the test following the presentation of the adaptor vs. the perception of the same test presented in isolation.

Aftereffects following adaptation have been reported for all sensory modalities, and for a growing variety of features in each modality (see Hollins and Favorov, 1994; Schweinberger et al., 2008), although the most investigated domain is certainly vision. Aftereffects have indeed been documented for brightness, color, motion, orientation, size, shape and for various combinations of these visual features (for a review, see Webster, 2015). A general interpretation of aftereffects following adaptation of low-level vision is related to the phenomenon of neural adaptation, which is an extension of sensory receptor adaptation, in that it entails the weakening of electrochemical activity of neural units responsible for processing and communicating the signals corresponding to low-level visual information beyond the level of sensory receptors, in the visual brain (Barlow and Hill, 1963). In other words, the sensory and neural adaptation following sustained stimulation may well provide the basis to explain the change in perceptual appearance that is typical of aftereffects in many low-level visual adaptation phenomena. The fact that there is a strict contingency between sensory and neural change and perceptual outcomes has even referred to perceptual adaptation as the "psychophysicist's microelectrode" (Frisby, 1979), because the behavioral quantification of its nature and dynamics is an important behavioral window on the underlying processes of the visual brain.

Besides the classical aftereffects related to low-level vision, in recent decades perception science has intriguingly escaladed toward high-level adaptation aftereffects, thus paralleling two crucial discoveries in the neurosciences: (i) the perceptual brain is organized hierarchically, with low-level units (and regions) feeding high-level units (and regions) that represent objects (Webster, 2011),

<sup>1</sup>Aristotle (ca 350 B.C.). Parva Naturalia.

Edited and reviewed by: *Rufin VanRullen, Paul Sabatier University, France*

> \*Correspondence: *Rocco Palumbo roccopalumbo@gmail.com*

#### Specialty section:

*This article was submitted to Perception Science, a section of the journal Frontiers in Psychology*

Received: *24 January 2017* Accepted: *06 February 2017* Published: *17 February 2017*

#### Citation:

*Palumbo R, D'Ascenzo S and Tommasi L (2017) Editorial: High-Level Adaptation and Aftereffects. Front. Psychol. 8:217. doi: 10.3389/fpsyg.2017.00217* and (ii) among units and regions representing objects, some of them are specialized to process objects that have a very important adaptive value for the organism, such as faces (e.g., Webster et al., 2004), bodies (e.g., Winkler and Rhodes, 2005; Palumbo et al., 2013), emotions(e.g., Fox and Barton, 2007), and other complex environmental and social entities (Clifford and Rhodes, 2005; Greene and Oliva, 2010).

Studies focusing on high-level adaptation have involved more and more complex objects of perception, but have also moved in the direction of testing the boundaries of these time-sensitive perceptual (and neural) phenomena on many fronts: (i) does the duration of stimulus exposure influence the ensuing effects in systematic ways? (ii) are the effects contrastive (the test is perceived in the direction opposite to the adaptor) or assimilative (the test is perceived in the same direction as the adaptor)? (iii) are the effects of adaptation confined to a given modality or do they extend crossomodaly? (iv) are the effects merely perceptual, or are they contingent and/or permeable to other processing levels (attention, learning, imagery, language)? (v) does the behavioral evidence fit with data obtained from the neurosciences, and does it provide us with a tenable benchmark to make predictions on and build models of the brain? We believe that replying to all of these questions is both important and far-reaching. It is important because the answers seem to strongly reaffirm the role of perceptual adaptation in the cognitive neurosciences broadly conceived, possibly updating its symbolic status to that of a "psychophysicist's microelectrode and scanner," to move with the current times (Cziraki et al., 2010). It is also far-reaching because adaptation offers a great window of opportunity to put diverse aspects into contact. Being a timeand-order-sensitive phenomenon, adaptation is akin to many other neural and psychological (even merely methodological) effects that span across multiple research traditions (sensitization, peak shift, priming, serial order effects, anchoring, carry-over effects, etc.), and that still do not have a unifying framework. The usefulness of adaptation to test aftereffects occurring across modalities (e.g., Skuk and Schweinberger, 2013) is an added value to the field of crossmodal perception research. Additionally, even more promising is the understanding of aftereffects occurring across perceptual and conceptual categories and the role of other levels of processing in adaptation (Ghuman et al., 2010; Palumbo et al., 2015), to the point that adaptation stands as a key tool not only for the psychophysicist, but for the cognitive scientist interested in mapping the boundaries between perception and cognition, and in the interplay among representations and processes (Leopold et al., 2001; Storrs, 2015).

We hope that some glimpses of all of these opportunities will be conveyed by this Frontiers Research Topic, which starts with some papers dealing with face aftereffects, a domain of adaptation studies that has grown at a fast pace in the last two decades (see Strobach and Carbon, 2013; Hills, 2013). The first paper, by Zimmer et al. is focused on the effects of variation of the temporal factor (adaptation duration) on the manifestation of facial identity aftereffects and specific time-sensitive components recorded by means of ERPs. The authors highlight the role of exposure duration in evoking the electrophysiological components of face processing, showing that longer adaptation induces enhanced components and a more articulated segregation of the stages of category-, image- and identity-related processing. The paper by Kloth et al. investigates the potential interplay between sex and gaze direction in a gaze adaptation paradigm, concluding in favor of the nature of gaze direction as a natural adaptable category, independent of sex contingencies, and showing how a thorough manipulation of stimulus dimensions in the experimental design of an adaptation paradigm can shed light on the important issue of hierarchy and functional independence in perceptual processing. In the paper by Davidenko et al., the role of attention is carefully examined in an adaptation paradigm that is focused on either the ethnicity or the sex of faces, showing that attention plays no role in the direction of aftereffects, though it has an effect in modulating the speed of response of participants. Finally, the paper by Ross and Palmeri is an overview of the requirements (and the pitfalls) of a biologically- and behaviorally-inspired approach to modeling face processing, that capitalizes on data from face adaptation phenomena, providing valuable directions and theoretical discussion for computational modeling in general.

The following two papers revolve around aftereffects observed for another important high-level entity, the human body. The article by Brooks et al. investigates the differential role of adaptating to one's own or another person's body size (along the dimension of fatness/thinness), on perceived body size aftereffects, concluding (similarly to Kloth et al.'s paper) that body size is a visual feature that can produce adaptation effects per se, independently in this case, of any contingency with self vs. other identity of the observed body. In the second paper, Stephen et al., propose a novel methodological paradigm useful to assess body size aftereffects (again, along the dimension of fatness/thinness), factoring in the role of explicit vs. implicit attentional set, and concluding (similarly to Davidenko et al.'s paper) that attention does not play a major role in visual adaptation to body size.

Two further papers delve more into aspects concerning adaptation to emotional features. In the paper by Wincenciak et al., which could also be listed alongside the two previous papers, the authors make use of adaptation to human bodies, although they do so to assess whether the emotional information conveyed by whole-body actions is contingent upon the identity of the actors performing those actions. They demonstrate that identity appears to act as a determinant in the modulation of emotional action aftereffects, albeit in a graceful fashion, with stronger and longer-lasting aftereffects in the case of same identities, as compared to the case of different identities, which evoke aftereffects in a weaker and short-lived fashion. The paper by Palumbo et al. moves far from the domain of strictly biological entities, in the attempt to evaluate the effect of adaptation to the overall emotional valence of complex images. Carefully excluding faces from the set of images employed, the authors show that aftereffects can be evoked following adaptation to complex scenes, but have an assimilative rather than a contrastive nature, suggesting that visually perceived emotional valence is grounded on brain mechanisms eluding the phenomenon of neural adaptation.

The final two papers in the Research Topic are also based on less common adaptation paradigms, and demonstrate both the extent of the field and the many potential bridges to other fields. Miyoshi et al., provide evidence that a long adaptation to pictures reduces the strength of priming across disparate domains (images and words), suggesting the action of conceptual factors in mediating the effects, and the major role of exposure duration in explaining experience-driven phenomena such as priming and adaptation. Finally, the paper by Michel offers a brief review of the literature on prism adaptation, a century-old method consisting in visually shifting the entire visual field by wearing prismatic lenses. Although the effects of prism adaptation are primarily sensorimotor (thus constituting a valuable tool for rehabilitation of neglect patients) a growing body of research has investigated cognitive aftereffects following this treatment, and the paper offers important insights on the continuity (or better the discontinuity) between perception, action and higher cognitive processes, and on the ties between adaptation and plasticity.

### REFERENCES


To conclude, the papers in this Research Topic offer a wide cross-section of the research being carried out in the field of high-level adaptation, a complex territory that has now a well-established body, but also branches to other territories, demonstrating a great potential for further developments. Aftereffects, once a phenomenon confined to perception science, have indeed stood the test of time as a tool for studying perception, but have also assumed a wider relevance, as they now allow us to better understand the relationships between perception and other levels of representation (action, imagery, cognition, emotion, language), to behaviorally gauge the workings of the brain, and to build more robust computational models.

### AUTHOR CONTRIBUTIONS

All authors listed, have made substantial, direct and intellectual contribution to the work, and approved it for publication.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Palumbo, D'Ascenzo and Tommasi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

## Adaptation Duration Dissociates Category-, Image-, and Person-Specific Processes on Face-Evoked Event-Related Potentials

#### Márta Zimmer <sup>1</sup> \*, Adriana Zban¸t <sup>2</sup> , Kornél Németh<sup>1</sup> and Gyula Kovács 3, 4 \*

<sup>1</sup> Department of Cognitive Science, Budapest University of Technology and Economics, Budapest, Hungary, <sup>2</sup> Faculty of Philosophy and Education, University of Vienna, Vienna, Austria, <sup>3</sup> Department of Cognitive Neuroscience, Institute of Psychology, Friedrich Schiller University Jena, Jena, Germany, <sup>4</sup> DFG Research Unit Person Perception, Friedrich Schiller University Jena, Jena, Germany

#### Edited by:

Stefania D'Ascenzo, University of Modena and Reggio Emilia, Italy

#### Reviewed by:

David Melcher, The Center for Mind/Brain Sciences (CIMeC), Italy Maximilian Bruchmann, University of Münster, Germany

#### \*Correspondence:

Márta Zimmer mzimmer@cogsci.bme.hu; Gyula Kovács gyula.kovacs@uni-jena.de

#### Specialty section:

This article was submitted to Perception Science, a section of the journal Frontiers in Psychology

Received: 17 September 2015 Accepted: 03 December 2015 Published: 22 December 2015

#### Citation:

Zimmer M, Zban¸t A, Németh K and Kovács G (2015) Adaptation Duration Dissociates Category-, Image-, and Person-Specific Processes on Face-Evoked Event-Related Potentials. Front. Psychol. 6:1945. doi: 10.3389/fpsyg.2015.01945 Several studies demonstrated that face perception is biased by the prior presentation of another face, a phenomenon termed as face-related after-effect (FAE). FAE is linked to a neural signal-reduction at occipito-temporal areas and it can be observed in the amplitude modulation of the early event-related potential (ERP) components. Recently, macaque single-cell recording studies suggested that manipulating the duration of the adaptor makes the selective adaptation of different visual motion processing steps possible. To date, however, only a few studies tested the effects of adaptor duration on the electrophysiological correlates of human face processing directly. The goal of the current study was to test the effect of adaptor duration on the image-, identity-, and generic category-specific face processing steps. To this end, in a two-alternative forced choice familiarity decision task we used five adaptor durations (ranging from 200–5000 ms) and four adaptor categories: adaptor and test were identical images—Repetition Suppression (RS); adaptor and test were different images of the Same Identity (SameID); adaptor and test images depicted Different Identities (DiffID); the adaptor was a Fourier phase-randomized image (No). Behaviorally, a strong priming effect was observed in both accuracy and response times for RS compared with both DiffID and No. The electrophysiological results suggest that rapid adaptation leads to a category-specific modulation of P100, N170, and N250. In addition, both identity and image-specific processes affected the N250 component during rapid adaptation. On the other hand, prolonged (5000 ms) adaptation enhanced, and extended category-specific adaptation processes over all tested ERP components. Additionally, prolonged adaptation led to the emergence of image-, and identity-specific modulations on the N170 and P2 components as well. In other words, there was a clear dissociation among category, identity-, and image-specific processing steps in the case of longer (3500 and 5000 ms) but not for shorter durations (<3500 ms), reflected in the gradual reduction of N170 and enhancement of P2 in the No, DiffID, SameID, and RS conditions. Our findings imply that by manipulating adaptation duration one can dissociate the various steps of human face processing, reflected in the ERP response.

Keywords: neural adaptation, adaptation duration, face processing, event-related potentials, face after-effect

### INTRODUCTION

The ability of the visual system to rapidly adjust to changing environmental conditions is one of its key characteristics (Mesik et al., 2013). Following the prolonged exposure of a visual stimulus, a subsequently viewed stimulus sometimes appears conspicuously distorted. This phenomenon, termed as adaptation-related after-effect, can affect the perception of various stimulus attributes (Müller et al., 2009). Moreover, adaptation has served as a powerful psychophysical tool in the past for demonstrating selective neural sensitivities to different stimulus dimensions from low-level stimulus features (such as contrast, orientation, spatial frequency, texture, or motion) to high-level object and face properties (Fang et al., 2006). Regarding faces, several studies have demonstrated that the way we perceive human faces can be systematically biased by a previously presented face. At the behavioral level, such face-adaptation-related after-effects are present for various dimensions or aspects of face processing—such as identity (Leopold et al., 2001), distortion (Webster and MacLin, 1999), race, expression, gender (Webster et al., 2004; Kovács et al., 2005, 2006, 2007), attractiveness (Rhodes et al., 2003), or eye-gaze direction (Schweinberger et al., 2007a). It is worth noting, however, that in some cases, repeated presentations of a given stimulus (or subsequent presentations of two stimuli from the same category) do not lead to perceptual biases (as in the case of adaptation-related after-effects) but rather to behavioral facilitations. The latter phenomenon is usually referred to as priming, typically associated with faster and/or more accurate responses. Although these two phenomena share some properties, such as their size-, viewpoint-, or positioninvariances, they are clearly distinguishable regarding their behavioral consequences (Zhao and Chubb, 2001; Brooks et al., 2002; Zimmer and Kovács, 2011a). Since there are both differences and similarities at the behavioral level, the degree to which the two phenomena share the same neuronal mechanisms is under debate. It is well-known that repetition of a particular stimulus usually lowers the firing rate of the responsive neurons, reduces the BOLD responses and the MEG/EEG signals (for reviews see Grill-Spector et al., 2006; Krekelberg et al., 2006). In the case of faces, various cortical areas and structures showed decreased activation in an fMRI adaptation paradigm (fMRI-A; Grill-Spector and Malach, 2001), such as the fusiform face area (FFA, Kanwisher et al., 1997), the occipital face area (OFA, Gauthier et al., 2000), the posterior superior temporal sulcus (pSTS, Puce et al., 1998), or the lateral occipital complex (LO or LOC, Malach et al., 1995). In electrophysiological studies, neural adaptation [or its clearest form: the so-called repetition suppression (RS)] paradigms provide a powerful technique to determine the functional properties of face-evoked ERP components and their relation to underlying neural processing modules (Eimer et al., 2011).

Regarding human face perception, electrophysiological studies have described a large, early positive (P1 or P100) and a negative (N170) wave over the occipito-temporal areas (Bentin et al., 1996; Eimer, 2000; Itier and Taylor, 2004a). Although several researchers noted that P100 also responds to faces when compared to other non-face objects (Itier and Taylor, 2004b; Herrmann et al., 2005a), most studies linked P100 rather to early pictorial encoding (Desjardins and Segalowitz, 2009). The N170 is typically regarded as a marker for the structural encoding of faces (Schweinberger, 2011). This component is larger in amplitude and shorter in latency to face stimuli than to non-face objects (Bentin et al., 1996; Rossion and Jacques, 2008) and has clear right hemisphere dominance (Bentin et al., 1996; Allison et al., 1999). The subsequent component of the ERP, the P2, is characterized by a positive-going deflection over the lateral occipito-temporal areas peaking at about 200–220 ms after stimulus onset. This component has been linked to the processing of facial relations between facial features in individual faces (Latinus and Taylor, 2006) and generally larger P2s were observed in tasks needing expertise (Stahl et al., 2008; Wiese et al., 2009). Altogether, this component might be involved in the deeper and more advanced analysis of faces when compared to earlier face-evoked components (Németh et al., 2014). In contrast to the N170, the second negative peak, the so-called N250 (or N250r), has consistently emerged as an electrophysiological correlate of face recognition (Schweinberger, 2011). This later negative component reaches its maximum between 230 and 330 ms after stimulus onset and is generated by the FFA (Eger et al., 2005).

The electrophysiological correlates of facial after-effects have been reported as early as 140–200 ms post-stimulus onset (in the time window of the N170 component). Following the initial study of Kovács et al. (2006), further works showed reduction of the N170 (or the M170 in magnetoencephalographic studies) when the adaptor image was a face when compared with noise stimuli or non-face objects (Harris and Nakayama, 2007, 2008; Kloth et al., 2010; Nemrodov and Itier, 2012). It is worth noting, however, that in most studies, this N170 adaptation to identitycongruent adaptors seems small or absent (Amihai et al., 2011). Even though the authors of previous face adaptation studies did not concentrate directly on the positive ERP components (P100 and P2), the increase of their amplitude after adaptation is evident in the grand average ERP figures of these studies (Kovács et al., 2007; Zimmer and Kovács, 2011b; Feuerriegel et al., 2015). These findings show that there are different modulating effects on the positive components (signal enhancement) when compared with the N170 as a consequence of adaptation, suggesting separate mechanisms that elicit these components and different roles that they play in face perception. It can also be hypothesized that the effect reflected on the positive components is linked to priming rather than to adaptation. The face-evoked ERP component that is more closely associated with face repetition priming is the N250 (N250r) component (Schweinberger et al., 2007b). Although a clear N250 amplitude increase was observed for repetitions of familiar faces across different images, this increase was larger for repetitions using the pixel-by-pixel same image when compared with different images of the same identity (Schweinberger et al., 2002a).

To the best of our knowledge, so far the only study testing priming and adaptation simultaneously within a single paradigm is a study by Walther et al. (2013). The authors presented pairs of stimuli, where the first stimulus was either a famous face (from three different identities) or a 50/50% morph between two famous faces and a Fourier phase-randomized image that served as control. The second, test stimulus was always a face from the morph continuum. In a 2AFC identity discrimination task the authors dissociated priming and adaptation effects based on the ambiguity of the test stimuli. They concluded that their results show that face-related adaptation and priming are both present in the same paradigm, but are never observed simultaneously for a given test stimulus. This indicates that exclusive mechanisms may underlie both adaptation and priming.

One of the important parameters affecting the mechanisms of adaptation is its timescale and this can be tested by either the manipulation of the exposition time of the adaptor or by using different adaptor-test image inter-stimulus intervals (ISIs). Determining how adaptation duration influences changes in neuronal response properties is central to the understanding whether the sensory system employs different strategies for adjusting its sensitivity on the different timescales (Patterson et al., 2013). From single cell recording results, we know that brief and prolonged adaptation can lead to qualitatively different changes in neural tuning. For example, adaptation takes place in motion direction selective neurons of macaque area MT after short but not after long exposure to their preferred motion direction (Priebe et al., 2002; Kohn and Movshon, 2003). In the case of the orientation selective neurons of V1, Patterson et al. (2013) found that adaptation with small gratings reduced responsivity and caused tuning shifts away from the adaptor grating. This effect became more pronounced with more prolonged adaptation durations. Brief and prolonged adaptation produced, however, indistinguishable effects on responsivity, but caused opposite shifts in tuning preferences in the case of large gratings. Regarding human face perception based on psychophysical data, it was shown that perceptual aftereffects (namely the face identity after-effect) for simple visual attributes processed early in the cortical hierarchy increase logarithmically with adapting duration and decay exponentially with test duration (Leopold et al., 2005; Rhodes et al., 2007). In human electrophysiological [or magnetoencephalographic (MEG)] experiments where the adaptor and test stimuli are presented in rapid succession, strong N/M170 amplitude reductions are observed for every face (or face-part) adapted condition (Harris and Nakayama, 2007, 2008; Eimer et al., 2010; Nemrodov and Itier, 2011, 2012). In one of our previous studies, we manipulated the effect of adaptation duration and stimulus position together (Kovács et al., 2007). We have found that that for shorter adaptor durations (500 ms) the adaptation effect was entirely position independent, whereas for longer durations (5000 ms), the effect was partly position invariant but partly position dependent. As it has already been mentioned, other studies varied the ISI to test the timescale of adaptation. Daelli et al. (2010) presented prototypical adaptors (S1), followed by ambiguous target objects (S2), and found aftereffects when the ISI was short (50 ms) but priming effects when the ISI was long (3100 ms). An MEG study by Harris and Nakayama (2007) also investigated the effects elicited by different ISIs (100, 200, 300, and 600 ms) and found that the M170 response to the second face decreased in a linear manner with decreasing ISI. This shows that adaptation depends on the ISI, even though this effect is shortlived as suggested by the fact that for an ISI of 800 ms there was no difference between adapted and non-adapted conditions. In a recent work, Feuerriegel et al. (2015) measured the categoryspecificity of adaptation over a limited range of adaptor durations (200, 500, and 1000 ms) and for two short ISIs (200 and 500 ms). Their results indicated that, not surprisingly, at the level of N170 there was no category adaptation for faces. Face adaptors led to the smallest N170 amplitudes for both target faces and chairs after 500 ms adaptor duration.

In the present study, we systematically manipulated the duration of adaptation in the commonly applied range of 200– 5000 ms with different facial adaptation conditions. In a 2AFC familiarity decision task we used three different face adaptors: different identity (DiffID), different image of the same identity (SameID), and identical image of the same person (repetition suppression, RS). A Fourier phase-randomized noise image served as control (No). By using these conditions we intended to separate three major steps of face processing. The GENERIC, face category specific processes were isolated by the comparison of noise adaptation (No) with the adaptation to a face that is unrelated to the test stimulus identity (DiffID). IDENTITY-SPECIFIC effects were tested by the DiffID vs. SameID comparison while IMAGE-SPECIFIC effects were measured by the SameID vs. RS comparison. We hypothesized that by varying the duration of adaptation time, we might dissociate generic-, identity-, and image-based information processing, reflected on the face-evoked ERP components.

### MATERIALS AND METHODS

### Participants

Sixteen naïve healthy-volunteers (15 right-handed and one ambidextrous, six females, mean age = 22.69 years, ± 4.83 years SD) participated in the study. They received partial course credits for their participation and gave signed, informed consent in accordance with the Ethical Committee of the Budapest University of Technology and Economics prior to testing. All participants had normal or corrected-to-normal visual acuity, no previous history of any neurological or ophthalmological diseases and were not under any medication. One additional participant was excluded from the behavioral and electrophysiological analysis (right-handed 22 years old male) due to an insufficient number of recognized familiar faces and a high number of recognized unfamiliar faces, which was evident in the post-testing questionnaire.

#### Stimuli

Grayscale images of familiar and unfamiliar faces were used, with sixteen different identities (eight female) for each category. Familiar faces were digital images of famous persons, ranging from Hollywood celebrities to politicians that Hungarian students were likely to recognize, whereas unfamiliar faces were digital images of less-known persons that were unlikely to be recognized by the participants (such as Finnish politicians or actresses from Iceland). Two different images portraying the same person were used for each identity, corresponding to a number of 32 images in total. All images were downloaded from freely available websites and converted into grayscale (8 bit) using Adobe Photoshop CS3 Extended 10.0 (Adobe Systems Inc.). Images were then cropped to reveal only the contour of the face, including hair. Stimuli subtended 3.6 × 5 ◦ of visual angle. Since previous studies have shown that early ERP components, such as P100, are sensitive to luminance (Johannes et al., 1995) and that neural processes are sensitive to luminance histogram skewness (Olman et al., 2008), stimuli were equated in luminance and their histograms were matched using the lummatch and histmatch functions of MATLAB (Mathworks, Natick Massachusetts, USA) SHINE toolbox (Willenbockel et al., 2010).

Three different images were used as adaptors: an image that was identical to the test stimulus [repetition suppression (RS) condition], a different image depicting the same identity as the test stimulus [same identity (SameID) condition], and an image depicting a same gender different identity than the test stimulus [different identity (DiffID) condition]. For the control condition, sixteen Fourier phase-randomized images were created from the face images and used as adaptors [non-adapted (No) condition].

Stimuli were presented centrally on a uniform gray background. An LG Flatron W2600 HP monitor (resolution: 1920 × 1200 pixels, refresh rate: 60 Hz) was used for stimulus presentation, while viewing distance (57 cm) was maintained constant using a chinrest. Stimulus presentation was controlled using MATLAB 2008a Psychtoolbox 3.0.9 (Brainard, 1997; Pelli, 1997) and custom-made scripts.

#### Procedure

Subjects were instructed to perform a two-alternative forcedchoice (2AFC) familiarity task for faces by pressing the key labeled "7" on a keyboard when the second, target face was perceived as familiar and the key labeled "8" when it was perceived as unfamiliar. A yellow fixation cross presented in the center of the screen indicated the presence of the target stimulus to the subject. Prior to the presentation of the target, a gray screen was presented for a random period of time between 500 and 700 ms, followed by the adaptor image. The duration of the adaptor image varied randomly between one of five values: 200, 1200, 2000, 3500, and 5000 ms. The adaptor was followed by a 500 ms gray screen after which the target stimulus appeared for 200 ms (**Figure 1**). To ensure that the subjects were also focusing on the adaptor image, a blue fixation cross directed their attention to the center of the screen during the presentation of the adaptor. Subjects were also instructed to refrain from movements during the experiment and from blinking during the presentation of the target stimulus. Each subject completed a total of 640 trials [2 (familiarity: familiar (F) vs. unfamiliar (UF)) × 4 (adaptor category: No, RS, SameID, DiffID) × 5 (adaptor duration: 200, 1200, 2000, 3500, 5000 ms) × 16 (identity)] with five breaks in between testing blocks. Adaptor categories and durations were intermixed and presented in random order. An experimental session lasted approximately 50 min.

At the end of the session participants had to answer a questionnaire that verified their familiarity with the identities used in the experiment. The questionnaire consisted of presenting each previously presented person and asking the

participants to identify the familiar ones by naming them (or at least giving some correct information about them, such as occupation and nationality). In case the subject did not recognize one or more of the faces belonging to the familiar category or recognized faces belonging to the unfamiliar category, the trials containing those identities were removed from the statistical analysis for both behavioral and electrophysiological data. From the 32 faces on average 2.56 ± 2.25 faces were excluded from further analyses across participants.

#### Behavioral Data Analysis

Accuracy and response times (RTs) were recorded during the experiment. In the case of accuracy, we calculated d′ -values that were analyzed with a 4 × 5 repeated measures ANOVA with adaptor category (4; No, RS, SameID, DiffID) and adaptor duration (5; 200, 1200, 2000, 3500, 5000) as within-subject factors. RTs were analyzed with a 2 × 4 × 5 repeated measures ANOVA with familiarity (2; familiarity: F vs. UF), adaptor category (4; No, RS, SameID, DiffID) and adaptor duration (5; 200, 1200, 2000, 3500, 5000 ms) as within-subject factors. All analyses involved Greenhouse—Geisser adjusted degrees of freedom for correction for non-sphericity. Post-hoc-t-statistics were performed by Fisher's LSD tests.

### Electrophysiological Recording and Analysis

#### EEG Acquisition and Processing

Electroencephalography (EEG) data was recorded using a Brain-Amp (BrainProducts GmbH, Munich, Germany) amplifier from 60 Ag/AgCl scalp electrodes placed according to the international 10/10 electrode system (Chatrian et al., 1985) and mounted on an EC80 Easy Cap (Easycap, HerrschingBreitbrunn, Germany). Eye movements were recorded using two electrodes placed on the outer canthi of the eyes and one electrode placed on the infraorbital ridge of the right eye. All channels were referenced online to an average of the activity recorded at the two reference electrodes placed on the left and right earlobe and digitally transformed to a common averaged reference offline. The ground electrode was placed on the forehead and all input impedances were kept below 8 k. EEG was digitized at 1000 Hz sampling rate (with an analog bandpass filter of 0.016–1000 Hz).

#### ERP Data Analysis

EEG data were analyzed using Brain Vision Analyzer 1.05.0002 (BrainProducts GmbH., Munich, Germany). After correcting ocular movement artifacts and digitally re-referencing to a common average, the EEG was segmented offline into 700 ms long epochs starting 100 ms prior to target stimulus onset and ending 600 ms after target stimulus onset. DC trend correction was applied, and a semi-automatic artifact rejection was implemented. Segments containing blinks, movement artifacts, and baseline drifts were rejected on the basis of visual inspection. After cleaning the EEG data, ∼87% of the trials remained available for further analysis. It is worth noting, however, that the reason of removing a given segment in most cases (8% from the 13% removed trials) was that the familiarity of the given person was misjudged by the subject. EEG epochs were averaged separately for each condition and participant. Averages were band-pass filtered (Butterworth zero phase filter; 0.1– 30 Hz; slope: 12 dB/oct) and baseline corrected using a 100 ms pre-stimulus baseline. The peak amplitude and latency of the individually averaged ERPs were extracted using a semiautomatic detection algorithm that identified the global maxima separately for each selected channels in a specific time window as follows. P100 was defined as a main positive deflection in the 80–130 ms time window, whereas the N170 was defined as a negative component in the 135–190 ms time interval. P2 was the second positive component in the 195–250 ms time window, while N250 was defined as the second negative component at around 230– 330 ms.

P100 amplitude was measured over O1 (left hemisphere, LH) and O2 (right hemisphere, RH) electrode positions (Herrmann et al., 2005a,b). For the N170, the standard posterior-occipitotemporal sites that correspond to the P7, P9, and PO9 (LH) and P8, P10, and PO10 (RH) channels were used (Wong et al., 2009). P2 amplitude was measured over O1, PO3, and PO7 (LH) and O2, PO4, and PO8 (RH) (Wang et al., 2014), while the amplitude of the N250 component was measured over P7, P9, PO9, and TP9 (LH) and P8, P10, PO10, and TP10 (RH) (Schweinberger et al., 2002b). A Five-way repeated-measures ANOVA was conducted for the amplitude values of the pooled values of the relevant electrodes with familiarity (2; F vs. UF), adaptor category (4; No, RS, SameID, DiffID), adaptation duration (5; 200, 1200, 2000, 3500, 5000 ms), hemisphere (2; LH vs. RH), and electrode (3 for N170 and P2 or 4 for N250) as within-subject factors separately for each component. (It is worth noting that for the P100 component a Four-way repeatedmeasures ANOVA was applied since we used only two electrodes, one for the LH and RH recordings.) The Greenhouse—Geisser correction was applied to correct for violations of sphericity, while Post-hoc-t-statistics were computed using Fisher's LSD tests.

In addition, we defined three different types of effects: ADAGENERIC as referred to the No vs. DiffID comparison; ADAIDENTITY as referred to the SameID vs. DiffID comparison; and ADAIMAGE as referred to the SameID vs. RS comparison. The magnitudes of electrophysiological adaptation effects were defined as the absolute value of the differences in amplitude that were calculated by subtracting the P100, N170, P2, and N250 amplitudes obtained during the relevant two adapted conditions. After all, we tested whether the different types of adaptations modify the ERP components differently. Therefore, a Four-way repeated-measures ANOVA was conducted on the electrophysiological adaptation indices with adaptation effect (3; ADAGENERIC, ADAIDENTITY, ADAIMAGE), ERP component (4; P100, N170, P2, N250), hemisphere (2; LH vs. RH), and duration (5; 200, 1200, 2000, 3500, 5000 ms) as within-subject factors. A Greenhouse—Geisser correction was applied to correct for violations of sphericity, while Post-hoc-t-statistics were computed using Fisher's LSD tests.

### RESULTS

### Behavioral Results

#### Accuracy

We observed a main effect of adaptation CONDITION on the d ′ -values of the behavioral performance [F(1.68, 25.22) = 7.56, p = 0.0003, η 2 <sup>p</sup> <sup>=</sup> 0.34; **Figure 2A**]. Post-hoc comparisons suggested that it was due to a significantly worse performance (or according to the interpretation of the d′ analysis due to a significant lower sensitivity) in the case of DiffID when compared with other conditions (all ps < 0.009). Neither a main effect of DURATION [F(4, 60) = 1.47, p = 0.22, n.s.] nor a significant CONDITION × DURATION interaction [F(5.92, 88.74) = 1.53, p = 0.12, n.s.] were found.

#### Response Times

A main effect of CONDITION was observed [F(3, 45) = 61.54, p < 0.0001, η 2 <sup>p</sup> <sup>=</sup> 0.8] in a step-by-step manner: the fastest decisions were observed for RS (all ps < 0.0001) followed by SameID (all ps < 0.0001), however, no significant difference was found between No and DiffID conditions (p = 0.3). No main effect of DURATION was found [F(2.03, 30.5) = 1.19, p = 0.32], while adaptation duration modified the main effect of condition [significant CONDITION × DURATION interaction: F(12, 180) = 2.76, p = 0.002, η 2 <sup>p</sup> <sup>=</sup> 0.16]. We have observed longer RT-values in the case of 5000 ms adaptation duration when compared with 200 ms (p = 0.0033) and with 3500 ms (p = 0.019) in the RS condition. In the case of the SameID condition, on the one hand, faster decisions were observed for the shortest adaptation duration (200 ms) when compared with the other durations (all ps < 0.017) except for 2000 ms duration (p = 0.31). On the other hand, smaller RT-values were observed for 2000 ms duration when compared with longer durations (p < 0.0001 for both 2000 vs. 3500 and 2000 vs. 5000 comparisons). In the case of the DiffID condition,

longer RTs were measured for 1200 and 2000 ms durations when compared with the shortest (200 ms) duration (p = 0.047 and p = 0.022, respectively). Faster decisions were observed for familiar faces when compared with unfamiliar ones, suggesting a significant priming effect [main effect of FAMILIARITY: F(1, 15) = 5.27, p = 0.037, η 2 <sup>p</sup> <sup>=</sup> 0.26; **Figure 2B**]. Although we have found that FAMILIARITY modifies the main effect of CONDITION [significant FAMILIARITY × CONDITION interaction: F(2.19, 32.84) = 12.09, p < 0.0001, η 2 <sup>p</sup> <sup>=</sup> 0.45], the only relevant difference was faster decisions for RS (p < 0.0001), SameID (p = p < 0.0001), and DiffID p = 0.043) but not for No (p = 0.21) for familiar faces when compared with unfamiliar ones.

symbols indicate p < 0.01, and three symbols indicate p < 0.001.

### Electrophysiological Results

In the subsequent ERP Results Section, we only focus on the three above-mentioned adaptation effects (GENERIC, IDENTITY-SPECIF, and IMAGE-SPECIFIC EFFECTS) and therefore we only discuss the main effects of CONDITION and DURATION, and the significant CONDITION × DURATION interactions in details. Any other significant main effects and interactions are presented in **Tables 1, 2**, respectively.

#### P100

Significant GENERIC EFFECT was observed suggested by the larger P100 amplitude values in the DiffID condition when compared with the control (No) (main effect of CONDITION: F(3, 45) = 9.83, p < 0.0001, η 2 <sup>p</sup> <sup>=</sup> 0.4, post-hoc p <sup>&</sup>lt; 0.0001; **Figures 3A,B**). Neither IDENTITY-SPECIFIC nor IMAGE-SPECIFIC EFFECTs were found (DiffID vs. SameID p = 0.49, SameID vs. RS p = 0.94). In the case of shorter adaptation durations (200 and 1200 ms), smaller P100s were measured when compared with longer adaptation durations [main effect of DURATION: F(4, 60) = 12.09, p < 0.0001, η 2 <sup>p</sup> <sup>=</sup> 0.45, post-hoc comparisons show all ps < 0.0028]. No other effects of duration were observed (all other ps > 0.71). Moreover, the main effect of condition was not modified by adaptation duration [nonsignificant CONDITION × DURATION interaction: F(12, 180) = 1.34, p = 0.2]. Altogether, these results suggest that the P100 reflects only generic, face category specific adaptation processes and that this effect is independent of the adaptation time.

#### N170

The N170 showed a significant main effect of CONDITION [F(3, 45) = 47.12, p < 0.0001, η 2 <sup>p</sup> <sup>=</sup> 0.76] in a step-by-step manner (**Figures 4A,B**). The largest N170s were found in the case of the control (No) condition (all ps < 0.0001) followed by DiffID (all ps < 0.0035), however, no significant difference was observed between SameID and RS (p = 0.68) suggesting that at the level of N170 component, both GENERIC (p < 0.0001) and IDENTITY-SPECIFIC EFFECTs (p = 0.0035) can be observed but no IMAGE-SPECIFIC EFFECT (p = 0.68) is reflected on the component. Adaptation duration modified the main effect of condition [significant CONDITION × DURATION interaction: F(6.27, 93.99) = 3.33, p = 0.0002, η 2 <sup>p</sup> <sup>=</sup> 0.18]. In detail, GENERIC EFFECT was observed for all durations (all ps < 0.018), IDENTITY-SPECIFIC effect was found in the cases of 1200 and 3500 ms durations (p = 0.015 and 0.003, respectively), while IMAGE-SPECIFIC EFFECT was only observed in the case of the longest exposure time of the adaptor (5000 ms; p = 0.038) but not for the shorter ones (all ps > 0.09).

These results suggest that the N170 adaptation effects strongly depend on the adaptation duration: while for short adaptation durations the N170 reflects only generic, category specific effects, prolonging the adaptation duration leads to the emergence of identity and image specific adaptation effects as well.

#### P2

Although there was a strong tendency for both GENERIC EFFECT (namely larger P2s in the case of DiffID when compared with No, p = 0.054) and IMAGE-SPECIFIC EFFECT (larger P2s

#### TABLE 1 | Other significant main effects.


In case of negative components larger (>) means more negative.

#### TABLE 2 | Other significant interactions.


In case of negative components larger (>) means more negative.

in the case of RS when compared with SameID, p = 0.053) and a somewhat weaker tendency for IDENTITY-SPECIFIC EFFECT (namely smaller P2s in the case of DiffID when compared with SameID, p = 0.065), neither of the comparisons reached the level of significance even though a significant main effect of CONDITION was observed [F(2.19, 32.8) = 12.63, p < 0.0001, η 2 <sup>p</sup> <sup>=</sup> 0.46] (**Figures 5A,B**). Another strong tendency was observed in the case of adaptation duration [main effect of DURATION: F(4, 60) = 2.43, p = 0.057, η 2 <sup>p</sup> <sup>=</sup> 0.14], suggesting slightly larger P2s in both 1200 and 2000 ms durations when compared with the shortest (200 ms) duration. The main effect of condition, however, was strongly modified by adaptation duration [significant CONDITION × DURATION interaction: F(6.37, 95.58) = 2.4, p = 0.007, η 2 <sup>p</sup> <sup>=</sup> 0.14]. In the case of the shortest adaptation duration, there were no significant effects reflected on the amplitude of the component (all ps > 0.3). In the case of longer durations (either 3500 or 5000 ms long exposures) all three effects were observed (GENERIC EFFECT: ps < 0.035, IDENTITY-SPECIFIC EFFECT ps < 0.009, and IMAGE-SPECIFIC EFFECT ps < 0.019).

#### N250

Supporting prior studies of N250r (Schweinberger et al., 2002a, 2004; Kaufmann et al., 2009), the most negative N250s were observed in the RS condition when compared with all other conditions [main effect of CONDITION: F(1.89, 28.33) = 11.94, p < 0.0001, η 2 <sup>p</sup> <sup>=</sup> 0.44, all <sup>p</sup><sup>s</sup> <sup>&</sup>lt; 0.0041]. We also observed less negative N250s for DiffID than for any other conditions (all ps <

0.012). N250s evoked by target faces were larger for the longest adaptation duration (5000 ms) when compared with the shorter durations [main effect of DURATION: F(2.44, 36.66) = 3.02, p = 0.025, η 2 <sup>p</sup> <sup>=</sup> 0.17, post-hoc comparisons: all <sup>p</sup><sup>s</sup> <sup>&</sup>lt; 0.035]. Although the statistical investigation of the CONDITION × DURATION interaction did not reach the level of significance [F(12, 180) = 1.63, p = 0.09], significant three-way CONDITION × DURATION × HEMISPHERE interaction [F(4.67, 70.1) = 2.15, p = 0.016, η 2 <sup>p</sup> <sup>=</sup> 0.13] was found, suggesting that in the case of 1200 ms duration neither GENERIC (p = 0.11) nor IMAGE-SPECIFIC EFFECTs (p = 0.44) were observed (**Figures 6A,B**) over the right hemisphere. It was also the case for the two longest durations (3500 and 5000 ms) in the right hemisphere in the case of the IDENTITY-SPECIFIC EFFECT (all p = 0.18 for 3500 ms and p = 0.35 for 5000 ms). Since every other relevant comparisons were significantly different (all ps < 0.04) we can further emphasize the role of N250 in image and identity specific stimulus encoding.

By running an analysis based on different types of adaptation effects, we suggest that altogether larger adaptation effects

were observed for GENERIC when compared with IDENTITY-SPECIFIC adaptation [main effect of ADAPTATION EFFECT: F(2, 30) = 3.97, p = 0.03, η 2 <sup>p</sup> <sup>=</sup> 0.21, post-hoc comparison: p = 0.01]. There was a weak tendency that ADAIMAGE was slightly larger than ADAIDENTITY (p = 0.066). Although no main effect of DURATION was observed [F(4, 60) = 0.83, p = 0.51], the significant ADAPTATION EFFECT × DURATION interaction suggested that the ADAGENERIC effects were the most pronounced for the 2000 and 3500 ms adaptation durations [F(8, 120) = 2.21, p = 0.03, η 2 <sup>p</sup> <sup>=</sup> 0.13, post-hoc comparison: all ps < 0.042]. Interestingly, ADAGENERIC was the strongest in the case of 2000 ms duration, while at this duration the ADAIDENTITY was the smallest one.

#### DISCUSSION

The goal of the present work was to study the behavioral and electrophysiological effects of systematically varying adaptation

duration and whether this variation leads to a differentiation between the various stages of face processing, such as the generic category coding vs. processing of identity and image specific information. By using a large range of adaptation times we expected to find a differentiation between generic-, identity-, and image-specific processes.

The behavioral results indicated a better performance and faster decisions for the SameID and RS conditions when compared with the DiffID condition, corresponding to a strong priming effect in these cases (Ellis et al., 1987; Roberts and Bruce, 1989; Johnston and Barry, 2001). It is worth noting that the best performance and fastest decisions were observed in the case of RS when compared with the control (No) condition when the exposition time of the adaptor reached 1200 ms. In a recent study, Walther et al. (2013) found that the ambiguity of the stimuli regarding their classification plays an important role in differentiating between priming and adaptation-related aftereffects with priming effects mostly found for unambiguous ones. This priming effect [namely in cases where the face adaptor image belonged to the same identity (SameID or RS) vs. to another identity (DiffID)] was to be expected, since the stimuli used in our experiment were unambiguous to the expected decision of the participants (as we used original photos without manipulating the familiarity information within the given face by morphing or other techniques that would have made the task more difficult). In the case of the speed of decision, we have found faster RTs for familiar faces, which is in line with earlier findings (Tong and Nakayama, 1999) that showed that familiar faces are processed faster than unfamiliar ones.

In the case of the P100 component, larger amplitudes were observed for all face-adapted conditions when compared with control (No) and the main effect of condition suggests a generic adaptation effect (ADAGENERIC) already at this level. This general signal enhancement is in line with our previous findings (Kovács et al., 2005, 2006, 2007; Zimmer and Kovács, 2011b), suggesting that there can be different modulating effects

on the early P100 when compared to the later N170 as a consequence of adaptation, emphasizing separate mechanisms eliciting these components as well as their different roles in face perception.

All face adapted conditions caused a signal reduction of the N170 component when compared with noise adaptation (No). This general effect might be related to face detection and can be observed already after the shortest adaptation duration (200 ms). Recently, Feuerriegel et al. (2015) varied the exposition time of the adaptor stimulus systematically from 200 to 1000 ms and found no generic adaptation effect on the N170. These conflicting results can be explained by the fact that Feuerriegel et al. (2015) compared faces against another object category (chairs) while in the present study a uniform noise pattern was used as control condition. From the results of rapid adaptation studies where the authors tested the categoryspecificity of the adaptation effect (comparing ERPs evoked by face-adapted faces vs. non-face object adapted faces), we know that 200 ms duration is not enough to produce categoryspecific adaptation effects within the time-window of the N170 component (Nemrodov and Itier, 2012). It is worth noting, however, that Tian et al. (2015) have emphasized that the length of the ISI plays also an important role in the strength of the adaptation effect. In a rapid adaptation paradigm the authors have found both within-category (face adapted faces) and crosscategory (house adapted faces) adaptation effects but only in the case of relatively shorter (∼450 ms) ISI while in the case of longer ISI (∼850 ms) only face-sensitive adaptation effects were observed. Therefore, it is possible that (at least at the shortest adaptation times) the N170 is also sensitive to the presentation of an object shape or Gestalt, a hypothesis requiring further tests. Moreover, since both SameID and RS adaptors are related to the test stimulus, it is possible that the bilateral reduction of the N170 component is also related to the processing of identity, a conclusion also supported by earlier findings (Jacques and Rossion, 2006; Jacques et al., 2007; Caharel et al., 2009). It is worth noting, however, that this additional signal reduction when compared to the generic effect is present only for the longest adaptation duration (5000 ms). This result also supports the claims of Nemrodov and Itier (2012) regarding the validity of rapid adaptation paradigms in testing category specific processes.

Regarding the P2 component, in the case of the shortest duration, there was no significant difference between conditions. However, as adaptation duration reached 1200 ms a step-bystep differentiation was found among No, DiffID, SameID, and RS conditions in an ascending manner for both hemispheres. This means that for adaptation durations longer than ∼1000 ms the P2 component reflected generic, identity-specific and imagespecific effects as well. Whereas the P100 and N170 components are clearly linked to special and well-defined stages of face processing, this does not seem to hold for P2. Some studies linked this component to task difficulty (Philiastides et al., 2006) while others emphasized the role of P2 in face-related tasks for which we have expertise (such as in case of own race effect, Stahl et al., 2008). P2 enhancement was also found recently investigating the effect of added noise (Bankó et al., 2011, 2013; Németh et al., 2014). Even though the LO/LOC is believed to be an object-selective area, some studies have found that its caudal part is also responsive to faces. Indeed, Nagy et al. (2012) and Hermann et al. (2015) have found bidirectional connections between the LO and the face-selective OFA-FFA complex, with faces modulating the LO-FFA connection and objects modulating the LO-OFA connection. Moreover, it is known that the neural generator of both P100 and P2 is the LOC (for review, see Schendan and Lucia, 2010). Therefore, one would expect a similar behavior of these two components—namely a large and pronounced enhancement for repeated stimulus presentations (RS). Since the largest alteration was measured between RS and control (No) and considering that LOC is involved in the generation of this component one could interpret this signal enlargement as a re-entrant loop from the OFA-FFA complex to the LOC. It is worth noting, however, that there are studies emphasizing other brain regions that can be involved in the generation of the early visual components, such as P100. Studies using less complex visual stimuli (isoluminant checkerboard stimuli) differentiated an early and a late phase of the P100 component that were localized to sources in dorsal extrastriate cortex of the middle occipital gyrus, and to sources in the ventral extrastriate cortex of the fusiform gyrus, respectively (Di Russo et al., 2001, 2005). Other source localization studies that used human face stimuli also have mentioned the fusiform gyrus as the neural generator of the P100 component (Herrmann et al., 2005b).

Regarding N250, not surprisingly, familiar faces evoked larger N250s, supporting prior results (Begleiter et al., 1995; Schweinberger et al., 1995; Pfüzte et al., 2002). At the level of this component generic-, identity-, and image-specific effects were found equally for the shortest and longest durations. These findings show that all observed adaptation effects (generic, identity-specific, and image-specific) happen in parallel and not independent from one another in this time-window.

Our results show that on the one hand, the observed face-evoked ERP components reflect generic, identity-specific, and image-specific adaptation effects that are modulated by adaptation duration in a way that in the case of the shortest duration (commonly applied in rapid adaptation paradigms) only the earlier P100 and N170 components reflect generic adaptation effects (for summary see our model on **Figure 7**). On the other hand generic, identity-, and image-specific effects can equally be observed on the amplitude modulation of the P2 and N250 components. In the case of longer durations however, generic effects have already been measurable from the earliest face-evoked responses onwards while identity-, and image-specific effects are present beyond the N170 component.

In conclusion, the current study is the first to investigate the relationship between systematically varying the adaptor duration in a broad range and the behavioral and electrophysiological responses elicited by the test stimuli after adapting different processing stages. Our results suggest that after long-term adaptation generic, identity-specific, and image-specific effects are reflected equally and earlier on the ERP components when compared to short-term adaptation conditions. In the case

of rapid adaptation, these effects are delayed and can be observed only on the N250 component. Altogether we can summarize the consequences of our findings as follows: (1) While prior results rather suggested a sequential processing of faces, starting with a generic face categorization at about 150 ms (within the time-window of the N170 component), and identification/recognition being associated with three later components at around 250 ms (within the time-window of the N250 component), ours is the first study that clearly shows that with long-term adaptation these processes can all be observed from early components onwards and they are parallel to each other. (2) Our results suggest that in order to see the identity-specific processing steps completely, one needs to apply several seconds-long adaptation. (3) Our results also support those single-cell recording studies that suggest that short- and long-term adaptation have entirely different neural mechanisms (Priebe et al., 2002; Kohn and Movshon, 2003; Patterson et al., 2013).

### REFERENCES


### AUTHOR CONTRIBUTIONS

Designed the experiment: MZ, KN, GK; data acquisition: MZ, AZ, KN; data analyses: MZ, AZ, GK; interpretation of the data: MZ, AZ, GK; provided materials: MZ, AZ, KN, GK; wrote the article: MZ, AZ, GK; proofed/revised the article: MZ, AZ, KN, GK.

### FUNDING

This work was supported by a Deutsche Forschungsgemeinschaft Grant (KO 3918/1-2; 2-1).

### ACKNOWLEDGMENTS

We thank Dorottya Bencze for her help in running the experiments.


unfamiliar faces. Electroencephalogr. Clin. Neurophysiol. 94, 41–49. doi: 10.1016/0013-4694(94)00240-L


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Zimmer, Zban¸t, Németh and Kovács. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# **Absence of Sex-Contingent Gaze Direction Aftereffects Suggests a Limit to Contingencies in Face Aftereffects**

*Nadine Kloth 1,2 \*, Gillian Rhodes 1,2 and Stefan R. Schweinberger 2,3*

*<sup>1</sup> Australian Research Council Centre of Excellence in Cognition and its Disorders, School of Psychology, The University of Western Australia, Perth, WA, Australia, <sup>2</sup> DFG Research Unit Person Perception, Friedrich Schiller University of Jena, Jena, Germany, <sup>3</sup> Department of Psychology, Friedrich Schiller University of Jena, Jena, Germany*

Face aftereffects (e.g., expression aftereffects) can be simultaneously induced in opposite directions for different face categories (e.g., male and female faces). Such aftereffects are typically interpreted as indicating that distinct neural populations code the categories on which adaptation is contingent, e.g., male and female faces. Moreover, they suggest that these distinct populations selectively respond to variations in the secondary stimulus dimension, e.g., emotional expression. However, contingent aftereffects have now been reported for so many different combinations of face characteristics, that one might question this interpretation. Instead, the selectivity might be generated during the adaptation procedure, for instance as a result of associative learning, and not indicate pre-existing response selectivity in the face perception system. To alleviate this concern, one would need to demonstrate some limit to contingent aftereffects. Here, we report a clear limit, showing that gaze direction aftereffects are not contingent on face sex. We tested 36 young Caucasian adults in a gaze adaptation paradigm. We initially established their ability to discriminate the gaze direction of male and female test faces in a preadaptation phase. Afterwards, half of the participants adapted to female faces looking left and male faces looking right, and half adapted to the reverse pairing. We established the effects of this adaptation on the perception of gaze direction in subsequently presented male and female test faces. We found that adaptation induced pronounced gaze direction aftereffects, i.e., participants were biased to perceive small gaze deviations to both the left and right as direct. Importantly, however, aftereffects were identical for male and female test faces, showing that the contingency of face sex and gaze direction participants experienced during the adaptation procedure had no effect.

**Keywords: perceptual adaptation, gaze direction aftereffect, face adaptation, simultaneous opposite aftereffects, contingent aftereffects, face sex**

## **INTRODUCTION**

The human face provides us with cues about a person's identity, age, sex, ethnicity, emotional expression, and current focus of attention (Calder et al., 2011). Our ability to quickly and accurately perceive this information helps us to successfully navigate social interactions.

The efficiency with which our visual system processes the various facial cues is partly owed to *perceptual adaptation* (for a review, see Webster and MacLeod, 2011). The response properties of

*Edited by:*

*Rocco Palumbo, Harvard Medical School, USA*

#### *Reviewed by:*

*Hiroshi Ashida, Kyoto University, Japan Valentina Cazzato, University of Bradford, UK*

*\*Correspondence: Nadine Kloth nadine.kloth@uwa.edu.au*

#### *Specialty section:*

*This article was submitted to Perception Science, a section of the journal Frontiers in Psychology*

*Received: 24 September 2015 Accepted: 11 November 2015 Published: 01 December 2015*

#### *Citation:*

*Kloth N, Rhodes G and Schweinberger SR (2015) Absence of Sex-Contingent Gaze Direction Aftereffects Suggests a Limit to Contingencies in Face Aftereffects. Front. Psychol. 6:1829. doi: 10.3389/fpsyg.2015.01829* face-sensitive neurons in the visual system constantly adjust to the specific characteristics of the faces surrounding us. Neural responses to frequently occurring stimulus characteristics are down-regulated, which frees up capacity to respond to novel stimuli and equips us with highly sensitive face discrimination abilities. Behaviorally, the consequences of adaptation are revealed in *aftereffects*, in which perception is systematically biased away from an adapted characteristic. For instance, adaptation to faces with expanded features will bias participants to perceive a subsequently presented undistorted face as slightly compressed. Conversely, after adaptation to compressed faces they will be biased to perceive the same undistorted face as expanded (face distortion aftereffect, Webster and MacLin, 1999).

Aftereffects have been referred to as the "psychologist's microelectrode" (Frisby, 1980) because they can provide insight in the neuronal organization of the visual system through noninvasive behavioral experiments. Adaptation paradigms have therefore been enthusiastically used to study the face perception system. This research has revealed that the system adapts to variations in practically any facial signal, including identity (Leopold et al., 2001; Rhodes and Jeffery, 2006), sex (Webster et al., 2004; Kovacs et al., 2006; Pond et al., 2013), age (Schweinberger et al., 2010; O'Neil et al., 2014), ethnicity (Webster et al., 2004), emotional expression (Webster et al., 2004), attractiveness (Rhodes et al., 2003; Hayn-Leichsenring et al., 2013), and gaze direction (Jenkins et al., 2006; Seyama and Nagayama, 2006), suggesting that the face perception system contains neural entities that preferentially code, and selectively adapt to, each of these characteristics.

In *contingent face aftereffects* adaptation to opposite characteristics occurs simultaneously for different face categories. This was first described by Rhodes et al. (2004) who used a contingent adaptation paradigm to investigate whether distinct mechanisms code upright and inverted faces. They presented participants with a sequence of alternating faces that were distorted in opposite directions depending on their orientation, for instance expanded upright faces and contracted inverted faces, and studied the aftereffects for subsequently presented undistorted test faces presented in both orientations. The rationale was that if the same neural populations coded upright and inverted faces, the effects of the opposite adaptation distortions, expansion and contraction, should cancel out, and no aftereffect should be observed. However, if distinct populations coded upright and inverted faces, the adaptation procedure should induce separate opposite aftereffects for faces in both orientations. This is indeed what was found. Adaptation to upright expanded and inverted contracted faces induced aftereffects of perceived contraction in upright faces and perceived expansion in inverted faces, indicating that distinct neural populations code upright and inverted faces, and that both of these populations are sensitive toward structural changes on a compressed-expanded dimension (for related findings, see Watson and Clifford, 2006).

Since this initial report, various other studies have used contingent aftereffect paradigms to explore the neural organization of the face perception system. For instance, Little et al. (2005) investigated whether simultaneous opposite aftereffects could also be induced for male and female faces. In separate experiments, they presented participants with male and female faces that systematically differed with respect to their eye spacing, identity, or sexual dimorphism. Sexcontingent opposite aftereffects were found for all of these characteristics. Moreover, other studies have now demonstrated simultaneous opposite distortion aftereffects (Jaquet and Rhodes, 2008), expression aftereffects (Bestelmeyer et al., 2010), and age aftereffects (Schweinberger et al., 2010) for male and female faces. In combination, these studies provide strong evidence that separate neural populations code male and female faces, and that each of these sex-specific populations is sensitive to variations in eye spacing, identity, sexual dimorphism, configuration, emotional expression, and age.

Simultaneous opposite aftereffects have also been induced for European and African faces (ethnicity-contingent eye-spacing aftereffects, Little et al., 2008), for European and Chinese faces (ethnicity-contingent face distortion aftereffects, Jaquet et al., 2008), and for East Asian and African faces (ethnicity-contingent emotional expression aftereffects, Bestelmeyer et al., 2008). Moreover, contingent aftereffects have been described for human and monkey faces (species-contingent eye-spacing aftereffects, Little et al., 2008), for children's and adult faces (age-contingent eye-spacing aftereffects, Little et al., 2008), and for different familiar face identities (identity-contingent distortion aftereffects, Rooney et al., 2012).

In fact, simultaneous opposite aftereffects have now been reported for so many different combinations of face characteristics that it seems important to pause and question whether they can really provide the proposed level of insight into the structure of the face perception system. Ultimately, it does not seem particularly parsimonious to assume that the face perception system is subdivided to such an extreme extent, i.e., with every possible face category basically possessing its own little face perception subsystem. Is it indeed realistic that information such as the eye spacing, identity, sexual dimorphism, configuration, emotional expression, and age is coded separately for male and female faces? Is it plausible that we have subsets of face-sensitive neurons that almost exclusively respond to Caucasian, East Asian, and African faces and that each of these subsets also selectively codes the eye spacing, configuration, and emotional expression of these faces? Is the eye spacing of children's and adult faces really coded in distinct channels? In short, can contingent aftereffects indeed be meaningfully interpreted as indicating pre-existing response selectivity toward secondary dimensions in an adapted channel?

A possible alternative explanation is that simultaneous opposite aftereffects reflect a "selectivity" that is only generated during the adaptation procedure, for instance through associative or probabilistic learning. It has been suggested earlier that associative learning might play a role in simultaneous opposite aftereffects patterns (e.g., Murch, 1976). So far, such learning accounts have mostly been discussed for contingent aftereffects in the perception of simple stimulus attributes, such as the McCollough effect, however, they might also explain contingent face aftereffects. In the McCollough effect adaptation to alternating grids of black-and-green vertical bars and blackand-red horizontal bars induces orientation-contingent color aftereffects in the perception of black-and-white grids. Specifically, participants experience the white bars in vertical grids as red, and the white bars in horizontal grids as green. Just like contingent face aftereffects, the McCollough effect was initially suggested to indicate that cells that selectively code for local orientation are also sensitive to color (McCollough, 1965). However, other accounts have suggested that there might be an associative basis to the effect. Specifically, the orientation of the lined grid might act as a conditioned stimulus whereas color represents an unconditioned stimulus and the color aftereffect an unconditioned response. In such a scenario, pairing the conditioned and unconditioned stimulus might induce a conditioned response, a color aftereffect that depends on the orientation of the grid (Murch, 1976; Siegel et al., 1992; Allan and Siegel, 1997).

It is generally conceivable that such associative mechanisms also underlie simultaneous opposite aftereffect patterns in contingent face adaptation paradigms, at least under certain conditions. Prior research has shown that face aftereffects cannot be made contingent on just any physically dissociable categorical stimulus, such as color (Yamashita et al., 2005; Little et al., 2011), unless participants associate these with socially meaningful categorical labels (such as introvert vs. extravert, see Little et al., 2011). This finding might suggests that only visually derivable meaningful social categories can potentially serve as conditioned stimuli (see also, Bestelmeyer et al., 2008). For instance, in sex-contingent emotional expression aftereffects (Bestelmeyer et al., 2010) the sex of the adaptor face might act as a conditioned stimulus, whereas its emotional expression represents an unconditioned stimulus and the expression aftereffect an unconditioned response. Importantly, if contingent face aftereffects were indeed caused by such associative learning mechanisms, it should be possible to make practically any established face aftereffect contingent on any other categorical face characteristic that can serve as a conditioned stimulus. For instance, considering the number and variety of face aftereffects that have been found to be contingent on face sex, one might claim that face sex is a particularly efficient conditioned stimulus and that any face aftereffect might be made "contingent" on it. However, in the present paper we show that this is not the case, and present clear evidence that, unlike so many other face characteristics, gaze direction aftereffects are not contingent on face sex.

Simple gaze direction aftereffects have been well established and often replicated (Jenkins et al., 2006; Seyama and Nagayama, 2006; Schweinberger et al., 2007; Calder et al., 2008; Kloth and Schweinberger, 2008, 2010; Rhodes et al., 2015). Adaptation to faces consistently looking in one direction, for instance 25° to the right, induces strong aftereffects in the perception of gaze in subsequently presented test faces. Specifically, faces with smaller gaze deviations in the adapted direction, e.g., 5° or 10° to the right, are typically falsely perceived to be looking straight at the observer (Jenkins et al., 2006; Seyama and Nagayama, 2006; Schweinberger et al., 2007; Kloth and Schweinberger, 2008, 2010).

Here, we studied whether gaze direction aftereffects can be made contingent on face sex. We tested two groups of participants, the first of which adapted to male faces with leftward gaze and female faces with rightward gaze. The second group adapted to male faces with rightward gaze and female faces with leftward gaze. Sex-contingent gaze direction aftereffects would be indicated by different gaze direction aftereffect patterns for male and female test faces between the two groups of participants. The first group would be expected to falsely categorize male faces with left gaze and female faces with right gaze as looking straight ahead. For the second group, however, the opposite pattern would be predicted, resulting in incorrect perceptions of direct gaze from male faces with right gaze and female faces with left gaze (**Figure 1A**).

If gaze direction aftereffects are not contingent on face sex, alternating adaptation to male and female faces with leftward and rightward gaze should reveal a similar pattern of results for male and female faces in both groups of participants. However, unlike for face distortion aftereffects (Rhodes et al., 2004), the absence of sex-contingency would *not* be indicated by an overall absence of significant gaze direction aftereffects. This is because gaze direction is neurally coded in a non-opponent multichannel system that consists of at least three channels, one primarily responsive to leftward gaze, one primarily responsive to rightward gaze, and a third one primarily sensitive to direct gaze (Calder et al., 2008). This organization makes gaze aftereffects ideally suited to be studied in a contingent adaptation paradigm. Even in the absence of sex contingency, alternating left and right adaptation should lead to a bias in gaze direction perception, characterized by an increase in incorrect classifications of small gaze deviations to the left and right as direct for test faces of both sexes (**Figure 1B**).

## **MATERIALS AND METHODS**

### **Participants**

Thirty-six Caucasian participants (six men, 18–31 years, *M* = 22, SD = 3) contributed data. All participants reported normal or corrected-to-normal vision and were naïve to the purposes of the study. This study was carried out in accordance with the ethical guidelines of the Declaration of Helsinki and was approved by the ethics committee of the University of Jena. All participants gave written informed consent in accordance with the Declaration of Helsinki and were debriefed after completing the study.

## **Stimuli**

Test stimuli were color photographs of six male and six female young Caucasian adults with neutral emotional expression taken from earlier research (Jenkins et al., 2006). Each model posed at gaze angles of 10° left (L10), 5° left (L05), direct (S00), 5° right (R05), and 10° right (R10; all directions from the observer's point of view). Photos of the same 12 models gazing 25° left (L25) and 25° right (R25) were used as adaptation stimuli. All faces were presented in a black elliptical mask. Test stimuli measured 13 cm *×* 7.5 cm and adaptation stimuli measured 19 cm *×* 11 cm. A constant viewing distance of *∼*87 cm was ensured by using a chin rest.

aftereffects. Irrespective of the adaptation condition and the sex of the test face, there is a general increase in "direct" responses to all test faces.

### **Procedure**

The experiment consisted of five consecutive phases (**Figure 2**). The *baseline phase* was identical for all participants and served to establish their general ability to determine the gaze direction of the test faces without prior adaptation. Sixty test faces (12 identities *×* 5 gaze directions) were presented twice in random order. Participants indicated for each face whether it showed left, direct or right gaze direction by pressing one of three labeled response keys. In each trial, a question mark was first presented (800 ms), was then replaced by the test face (400 ms), and followed by a blank screen (2000 ms) during which participants responded. After half of the trials, participants were given a self-paced break. The baseline phase took 6.5 min to complete.

After the baseline phase, participants underwent an adaptation phase and a post-adaptation phase, followed by a second adaptation phase and a second post-adaptation phase. Participants were randomly assigned to one of two adaptation conditions, which were characterized by different adaptor sex/gaze direction contingencies. Half of the participants (*N* = 18, three men, 19–31 years, *M* = 22 *±* 3.0) always adapted to male faces with leftward gaze and female faces with rightward

gaze, the other half of the participants (*N* = 18, three men, 18–31 years, *M* = 22 *±* 3.8) always adapted to male faces with rightward gaze and female faces with leftward gaze. In the *first adaptation phase* participants were presented with three consecutive runs of twelve adaptation stimuli each, presented in pseudo-randomized order. Half of the adaptation stimuli displayed left gaze direction, the other half displayed right gaze direction. Male and female stimuli were presented alternatingly and faces of the same sex always had the same gaze direction. Depending on the adaptation condition participants had been assigned to, all male faces displayed gaze averted 25° to the left and all female faces displayed gaze averted 25° to the right, or *vice versa*. For half of the participants in each adaptation condition, the first adaptation phase started with a male face and ended with a female face, for the other half of participants the first adaptation phase started with a female face and ended with a male face. Exposure duration was 3500 ms for each adaptation stimulus, with an inter-stimulus interval of 200 ms. The adaptation block had a total duration of about 2 min. There was no task associated with this phase, participants were simply asked to passively view the adaptors.

This first adaptation phase was immediately followed by the *first post-adaptation phase* during which participants were again asked to determine the gaze direction of test stimuli. In general, the procedure of the post-adaptation phase was largely equivalent to the baseline phase. The critical difference was that each test stimulus was preceded by four consecutive top-up adaptors (3500 ms each) presented before the question mark (800 ms) and the test face (400 ms) to ensure consistently high levels of adaptation throughout the entire post-adaptation phase. Top-up stimuli also alternated in sex and began with the same sex as the adaptation sequence in the first adaptation phase. Neither of the top-up adaptation stimuli carried the same identity as the following test face. There were 60 trials in the first post-adaptation block and participants were given self-paced breaks after every 20 trials. The completion of the first post-adaptation phase took about 16 min.

To avoid inducing systematic biases toward the gaze direction (and sex) of the top-up face that was presented immediately before the test face, the first post-adaptation phase was followed by another adaptation sequence. This *second adaptation phase* differed from the first one only with respect to the order in which adaptors of the different sexes were presented. Participants who had adapted to a sequence of faces beginning with a male face in the first adaptation phase, were presented with a sequence of faces that started with a female face in the second adaptation phase. In the following *second post-adaptation phase*, the order of male and female top-up stimuli was adjusted accordingly. Importantly, the contingency of adaptor sex and adaptor gaze direction in the second adaptation and post-adaptation phase was identical to the one participants had experienced in the first adaptation and postadaptation phase. The reversal of the order in which male and female adaptors were presented simply ensured that across the whole experiment, test faces were equally likely to be immediately preceded by a male or a female adaptor. Overall, the experiment took 45–50 min to complete.

### **RESULTS**

Gaze direction adaptation typically leads to an increased tendency to falsely classify gaze in the adapted direction as direct (Jenkins et al., 2006; Schweinberger et al., 2007; Calder et al., 2008; Kloth and Schweinberger, 2008). Gender-contingent gaze adaptation would therefore be revealed by a pattern of increased "direct" classifications that differed systematically between male and female test faces. Specifically, there should be a selective increase in "direct" responses only for those test faces that look in the same direction as the sex-congruent adaptors. For instance, participants who adapted to male stimuli with left gaze and female stimuli with right gaze would be expected to show a selective increase in "direct" responses only to male test faces with left gaze and female test faces with right gaze, but not to male faces with right gaze and female faces with left gaze (see **Figure 1A**). The empirical data do not suggest such a pattern (**Figure 3**). Instead, participants seem to generally show aftereffects in the perception of both left and right gaze direction, indicated by an overall increase in "direct" classifications, irrespective of the sex of the test face (cf., **Figure 1B**).

For statistical analysis, we obtained a measure for the size of the gaze direction aftereffect by subtracting the percentage of "direct" classifications in the pre-adaptation baseline from the percentage of "direct" classifications after adaptation (collapsed across the first and second post-adaptation phase) for each participant and each condition. Positive scores indicate an increase in direct classifications after adaptation while negative scores indicate a decreased in direct classifications compared to baseline (**Figure 4**). These aftereffect scores were entered into an analysis of variance (ANOVA) with Sex of test face (male, female) and Gaze direction of test face (L10, L05, S00, R05, R10) as within-participants factors and Adaptation Condition (Male:left, Female:right; Male:right, Female:left) as a between-participants factor.

There was a main effect of Gaze Direction, *F*(4,136) = 10.86, *p <* 0.001, η 2 *<sup>p</sup>* = 0.24. Single-sample *t*-tests indicated that aftereffects were significantly larger than 0 for test faces with 5° leftward gaze, *t*(35) = 4.48, *p <* 0.001, *d* = 1.49, 5° rightward gaze, *t*(35) = 5.29, *p <* 0.001, *d* = 1.76, and 10° rightward gaze, *t*(35) = 2.89, *p* = 0.007, *d* = 0.98. Aftereffects for faces with 10° leftward gaze and direct gaze were not significant, *t*(35) = 1.74, *p* = 0.09, *d* = 0.59 and *t*(35) = 0.80, *p* = 0.43, *d* = 0.27, respectively. Aftereffects were larger for test stimuli with 5° gaze deviation than with 10° gaze deviation, *t*(35) = 3.94, *p <* 0.001, *d* = 0.66, and *t*(35) = 4.10, *p <* 0.001, *d* = 0.68, for the comparison of aftereffects for L05 vs. L10 and R05 vs. R10 test stimuli, respectively. Additionally, significantly larger aftereffects were observed for R10 than L10 stimuli, *t*(35) = 3.04, *p <* 0.01, *d* = 0.51.

Importantly, while the present paradigm clearly induced significant gaze direction aftereffects, these did not differ significantly for male and female faces in the two different adaptation conditions, *F*(4,136) = 1.49, *p* = 0.21, η 2 *<sup>p</sup>* = 0.04, for the interaction of Sex of test face, Gaze Direction of test face, and Adaptation Condition. The main effect of Adaptation Condition was not significant, *F*(1,34) = 2.58, *p* = 0.12, η 2 *<sup>p</sup>* = 0.07. There were no other significant effects (all *F*s *<* 1, all *p*s *>* 0.40).

Critically, the absence of a significant three-way interaction in the above analysis cannot be taken as conclusive evidence against contingency. In a final step, therefore, we calculated contingency scores for test stimuli with averted gaze, separately for participants in the two adaptation conditions based on the predicted contingency pattern. For participants who had adapted to male faces with leftward gaze and female faces with rightward gaze, one would predict a selective increase in "direct" classifications relative to baseline only for male faces with 5° and 10° leftward gaze, but not for female faces with 5° and 10° leftward gaze. Conversely, one would predict a selective increase in "direct" classifications only for female faces with 5° and 10° rightward gaze, but not for male faces with 5° and 10° rightward gaze. For participants in this adaptation condition, contingency scores for the leftward gaze conditions were therefore calculated by subtracting the increase in direct responses made to female faces with leftward gaze after adaptation relative to baseline (i.e., the unpredicted

aftereffect scores for female faces) from the increase in direct responses made to male faces with leftward gaze after adaptation relative to baseline (i.e., the predicted aftereffect scores for male faces). Conversely, contingency scores for the rightward gaze conditions were calculated by subtracting the unpredicted increase in direct classifications made to male faces with rightward gaze from the predicted increase in direct classifications made to female faces with rightward gaze. For participants who had adapted to male faces with rightward gaze and female faces with leftward gaze, contingency scores were calculated accordingly. Positive contingency scores therefore indicate a contingent aftereffect pattern, with larger aftereffects for test

Contingency scores for participants who adapted to male faces with right gaze and female faces with left gaze. Error bars indicate 95% confidence intervals.

faces of the same sex as the adaptors looking in the same direction compared to opposite-sex faces, contingency scores around 0 would indicate the absence of such contingencies, i.e., similar aftereffects for test faces of both sexes. Importantly, and confirming the above analyses, contingency scores were close to 0, and 95% confidence intervals included 0 in all conditions (**Figure 5**). One-sample *t*-tests confirmed that contingency scores were not significantly different from 0 for any gaze direction in either adaptation condition (all *t*s *<* 1.5, all *p*s *>* 0.18).

### **DISCUSSION**

We have shown that adaptation to alternating male and female faces with opposite gaze directions induces significant gaze direction aftereffects that are not contingent on face sex. Instead, test faces of either sex and with either leftward or rightward gaze were more likely to be perceived as looking straight ahead after adaptation than during the baseline phase (cf., Calder et al., 2008). These data indicate that gaze direction aftereffects are not contingent on face sex. Moreover, they suggest that the same neural populations process the gaze direction of male and female faces.

Importantly, the neural coding principles underlying gaze direction perception allow us to interpret this negative finding without having to worry that the absence of sexcontingent gaze direction aftereffects might simply indicate an inefficient adaptation procedure. Gaze direction perception is multichannel coded and therefore alternating left and right adaptation does not cancel out, but induces aftereffects in the perception of both gaze directions (Calder et al., 2008). We replicate this finding here, which demonstrates that our adaptation procedure was effective and produced significant gaze direction aftereffects. Importantly though, these aftereffects are indistinguishable for male and female faces, indicating that they are completely unaffected by the contingency between gaze direction and face sex present in the adaptation sequence.

Our findings suggest that contingency is not a general property of face aftereffects, despite having been observed for various different combinations of face dimensions (e.g., Rhodes et al., 2004; Little et al., 2005, 2008; Fox and Barton, 2007; Bestelmeyer et al., 2008; Schweinberger et al., 2010). Moreover, our data also suggest that the many sex-contingent opposite face aftereffects reported previously are unlikely to simply have resulted from associative learning during the contingent adaptation sequence. An associative learning account of contingent adaptation (Murch, 1976; Siegel et al., 1992; Allan and Siegel, 1997) would predict that once a stimulus category (e.g., face sex) has been demonstrated to have the potential to work as a conditioned stimulus for a face aftereffect (e.g., emotion aftereffect, eye-spacing aftereffect, age aftereffect), any other established face aftereffect (e.g., gaze direction aftereffect) should be easily made contingent on that category as well. Here we show, that this is not the case, providing some indirect evidence for the traditional assumption that contingent aftereffects indicate pre-existing response selectivity toward a second dimension in an adapted channel.

When considered in combination with earlier studies, our findings suggest that the perception of only some face characteristics is contingent on a second category, whereas the perception of other characteristics seems to be more general and independent of other categories, as indicated by an absence of contingent aftereffects. The variety of published reports on contingent face aftereffects (Rhodes et al., 2004; Little et al., 2005, 2008; Fox and Barton, 2007; Bestelmeyer et al., 2008; Schweinberger et al., 2010) and the lack of studies reporting an absence of contingent aftereffects, might suggest that more face signals are processed interactively rather than independently. However, the lack of studies reporting an absence of contingent aftereffects might also reflect a publication bias due to the relative difficulty to publish negative findings.

Interestingly, earlier research has shown that *eye spacing* aftereffects are contingent on face sex (Little et al., 2005), suggesting that configural changes in the eye region of male and female faces are processed by different neural populations. This finding may be related to evidence that face identity is coded in a norm-based manner, which is largely specific to the sex of the face (Jaquet and Rhodes, 2008). In other words, separate sexspecific prototypes seem to be used for the identification of male and female faces, and these can be simultaneously adapted in opposite directions. The fact that different norms exist for identity processing of male and female faces, but that there is no evidence for similarly sex-specific gaze processing mechanisms confirms the idea that, despite evidence for integrative processing of various signals in the face, at least some characteristics are processed independently (Bruce and Young, 1986; Haxby et al., 2000).

Earlier research studying the interdependence of the processing of different facial signals provides converging evidence that some of them might be processed more interdependently than others. For instance, using a selective attention paradigm to study the interdependency of identity and expression processing, Schweinberger and Soukup (1998) reported that variations in identity slowed down classifications of emotional expression, but that variations in emotional expressions did not affect classifications of identity. Interestingly, more recent research has replicated parts of this asymmetrical interference in a contingent aftereffect paradigm: In line with the observations of Schweinberger and Soukup (1998), emotional expression aftereffects have been found to be contingent on face identity (Fox and Barton, 2007). Moreover, while we are not aware of any research that would have investigated whether identity aftereffects are also contingent on emotional expressions, there is at least evidence that identity aftereffect completely transfer across different emotional expression (Fox et al., 2008).

Our data suggest that gaze direction perception is largely independent of face sex. Interestingly, there is substantial evidence from various other studies that gaze direction perception *can* generally be affected by other facial signals. For instance, the emotional expression of a face (Ganel et al., 2005; Graham and LaBar, 2007; Ewbank et al., 2009) as well as its attractiveness (Kloth et al., 2011) have both been found to affect the perceived gaze direction of a face. There is also evidence that information from the eye region can be diagnostic about a face's sex and that certain gaze directions can enhance or decrease ratings of masculinity and femininity in faces (Campbell et al., 1996). The present results might be taken to suggest that this influence likely originates from post-perceptual processes such as top-down strategies, for instance self-referential positivity biases (Lobmaier et al., 2008), rather than early perceptual integration (see also, Kloth et al., 2011).

Gaze direction might be rather unique compared to other facial signals, in that it might generally be more independent of the overall structure of the face than any other face signal (cf., Rhodes et al., 2015). There is evidence that local shape and luminance information play an important role in the perception of gaze direction, suggesting that gaze direction perception relies more on relatively simple cues processed on low-levels of the visual system (Anstis et al., 1969; Ando, 2002, 2004; Jenkins, 2007) than other face information. Having said that, it is important to keep in mind that gaze direction aftereffects clearly do also involve higher levels of the visual system (Jenkins et al., 2006), suggesting that their perception is not solely based on low-level visual processing.

A potential limitation of our research is that we do not have explicit evidence that the male and female adaptor faces used in the present study are generally able to induce sexcontingent aftereffects on other face attributes than gaze direction. However, visual inspection of the stimuli suggests that the faces have perfectly normal sexual dimorphism. Therefore, there is no immediate reason to assume that they would not be able to induce sex-contingent gaze direction aftereffects, if separate gazesensitive neural channels existed for male and female faces. We also note that our participant sample was predominantly female, but there is no reason to think that participant sex would affect the potential for sex-contingent aftereffects.

In summary, we used a sex-contingent gaze adaptation paradigm to explore whether, like so many other face aftereffects, gaze direction aftereffects are contingent on face sex. We found significant gaze direction aftereffects in all experimental conditions, however, these were completely unaffected by the contingency of adaptor sex and gaze direction presented during the adaptation sequence. These data suggest that it is rather unlikely that the large variety of sex-contingent face aftereffects reported in earlier work is due solely to associative learning mechanisms during the contingent adaptation sequence. Instead, separate neural populations seem to selectively respond to male and female faces, and separately code signals such as the emotional expression, eye spacing, sexual dimorphism and age of these faces. In contrast to these signals, gaze direction appears to be coded more generally, and independently of the sex of a face.

## **AUTHOR CONTRIBUTIONS**

Designed the experiment: NK, SS; data acquisition: NK; data analyses: NK, GR, SS; interpretation of the data: NK, GR, SS; provided materials: NK, SS; wrote the article: NK, GR, SS; proofed/revised the article: NK, GR, SS.

### **ACKNOWLEDGMENTS**

This research was supported by the Australian Research Council Centre of Excellence in Cognition and its Disorders (project number CE110001021), an ARC Professorial Fellowship to Rhodes (DP0877379), an ARC Discovery Outstanding Researcher Award to Rhodes (DP130102300), a grant from the Friedrich Schiller University in Jena to NK, and a grant from the Deutsche Forschungsgemeinschaft to SS (DFG grant FOR 1097, Person Perception). We thank Andrea Kowallik, Maria Rohmann, and Sina Schneider for their help with data collection. We are indebted to Andy Calder for helpful discussions about the data, and for the permission to use the stimuli from Jenkins et al. (2006).

### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2015 Kloth, Rhodes and Schweinberger. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Attending to Race (or Gender) Does Not Increase Race (or Gender) Aftereffects

#### Nicolas Davidenko\*, Chan Q. Vu, Nathan H. Heller and John M. Collins

Department of Psychology, University of California, Santa Cruz, Santa Cruz, CA, USA

Recent research has shown that attention can influence the strength of face aftereffects. For example, attending to changes in facial features increases the strength of identity and figural aftereffects relative to passive viewing (Rhodes et al., 2011). Here, we ask whether attending to a specific social dimension of a face (such as race or gender) influences the strength of face aftereffects along that dimension. Across three experiments, participants completed many single-shot face adaptation trials. In each trial, participants observed a computer-generated adapting face for 5 s while instructed to focus on either the race or gender of that adapting face. Adapting faces were either Asian and female or Caucasian and male. In Experiment 1, all trials included an intermediate question (IQ) following each adaptation period, soliciting a rating of the adapting face on the attended dimension (e.g., race). In Experiment 2, only half of the trials included this IQ, and in Experiment 3 only a quarter of the trials did. In all three experiments, participants were subsequently presented with a race- and gender-neutral face and asked to rate it on either the attended dimension (e.g., race, attention-congruent trials) or the unattended dimension (e.g., gender, attentionincongruent trials) using a seven-point scale. Overall, participants showed significant aftereffects in all conditions, manifesting as (i) higher Asian ratings of the neutral faces following Caucasian vs. Asian adapting faces and (ii) higher female ratings of neutral faces following male vs. female adapting faces. Intriguingly, although reaction times were shorter during attention-congruent vs. attention-incongruent trials, aftereffects were not stronger along attention-congruent than attention-incongruent dimensions. Our results suggest that attending to a facial dimension such as race or gender does not result in increased adaptation to that dimension.

#### Keywords: face adaptation, attention, race, gender, aftereffects

### INTRODUCTION

For almost 20 years, understanding face adaptation has been a central focus of face perception research. As with motion and color adaptation, face adaptation has been shown to shift perceptual ratings for certain facial characteristics or dimensions, and it is thought to function by retuning populations of neurons that code for those features (Yang et al., 2011). Adapting to a face for several seconds or minutes results in systematic shifts in the perception of subsequently viewed faces, known as figural aftereffects (AEs; Webster and Maclin, 1999; Webster and MacLeod, 2011). These high-level AEs depend on the characteristics of the adapting face; for instance, adapting to a face with male characteristics leads participants to rate a neutral face as appearing more female. A wide

#### Edited by:

Rocco Palumbo, Schepens Eye Research Institute, USA

#### Reviewed by:

Michael Webster, University of Nevada, Reno, USA Justin Michael Gaetano, Southern Cross University, Australia

> \*Correspondence: Nicolas Davidenko ndaviden@ucsc.edu

#### Specialty section:

This article was submitted to Perception Science, a section of the journal Frontiers in Psychology

Received: 10 January 2016 Accepted: 01 June 2016 Published: 17 June 2016

#### Citation:

Davidenko N, Vu CQ, Heller NH and Collins JM (2016) Attending to Race (or Gender) Does Not Increase Race (or Gender) Aftereffects. Front. Psychol. 7:909. doi: 10.3389/fpsyg.2016.00909

variety of facial characteristics can be adapted to, including identity, gender, race, age, expression, eye gaze, and figural distortions (Hsu and Young, 2004; Webster et al., 2004; Rhodes and Jeffery, 2006; Seyama and Nagayama, 2006; Davidenko et al., 2008; Little et al., 2008). The current study investigates whether attention can influence which facial characteristics become most adapted to. We begin more generally by reviewing how visual attention and face perception interact.

Palermo and Rhodes (2007) composed a comprehensive review of studies examining interactions between face perception and attention, classifying them into two broad categories: studies showing that certain automatic processes in face perception, such as detection, occur in the absence of attention; and studies examining the role of selective attention in modulating more detailed face processing tasks, such as preferences for expressive faces. In the former category, Driver et al. (1999) demonstrated that simply observing an image of a face that is gazing either left or right influences the reaction time to targets appearing next to the face, with faster reactions to targets that were congruent with the gaze. Even when participants were told that the targets were four times more likely to appear on the side opposite the gaze direction, participants were still driven to respond congruently with the gaze. These findings strongly support the claim that eye gaze is processed automatically and leads to attentional shifts, even when participants actively attempt to ignore it. However, a subsequent study by Ristic and Kingstone (2005) showed that these gaze cueing effects only take place when a person consciously recognizes the stimulus as a face. By using an ambiguous stimulus that could be seen as either the front of a car or as gazing eyes, the authors showed that it was only when participants were explicitly told that the cue was a pair of eyes that they were reflexively driven by gaze direction. These studies show that certain facial characteristics can be processed automatically, but rely on face detection having already taken place. Thus, aspects of face perception can be both immune to and beholden to attention.

As Palermo and Rhodes (2002) point out in their review, the relationship between attention and face perception is complex and depends on (i) the aspect of face perception being considered, (ii) the facial features being processed, and (iii) the type of attentional task being employed. For instance, a study conducted by the same authors (Palermo and Rhodes, 2002) tested how holistic face processing might be modulated by attention. The holistic processing account posits that faces are perceived in their entirety, rather than as an aggregate of their parts (Tanaka and Farah, 1993). Palermo and Rhodes (2002) used the part-whole task (Tanaka and Farah, 1993), where participants had to identify parts of faces that they have seen before, either in isolation or incorporated into a whole-face image. In addition, flanker faces appeared on either side of the target stimuli. Half of participants were told they would later have to match the identity of the flanker faces, and the other half were not. The authors found that holistic face processing (manifesting as better performance in whole-face trials) was intact for participants who could ignore the flankers, but was impaired for participants who attended to the flankers, showing that attending to face identity coopted attentional resources and suppressed holistic processing. However, a recent study by Norman and Tokarev (2014) also investigated the relationship between attention and holistic processing using different methods and found different results. In their study, the authors measured whether the composite face effect (Young et al., 1987) was modulated by the presentation of an exogenous cue. On half of the trials, participants' attention was drawn away from the test face and on the other half attention was made to align with the test face. The authors found no differences in holistic processing under these conditions, and thus concluded that spatial attention does not impact the attentional resources needed for holistic face processing. Therefore, even the same aspect of face perception (holistic processing) can interact differently with attention depending on the attentional task itself.

Similar complexities emerge regarding how attention modulates perception of different facial characteristics, such as gender, race, expression, and attractiveness. Reddy et al. (2004) carried out a study where they pitted a demanding letter discrimination task against a gender discrimination task and found that their participants could accurately tell the gender of a face even while attending to the letters. However, Murray et al. (2011) challenged the view that this sort of feature discrimination is entirely independent of attention. In their study, they varied the perceptual load of a word search task, requiring participants to devote more or less attention to it, and then asked them to discriminate the race or gender of a task-irrelevant face that was presented peripherally. They showed that under low attentional load, the race or gender of the peripheral face was indeed processed automatically, but as the perceptual load increased, the race or gender of a face was effectively ignored.

Relatively less is known regarding the effects of attention on face adaptation. One recent study carried out by Murray et al. (2012) found that figural face AEs occurred even when participants attended to a letter/color discrimination task and adapting faces were presented peripherally. Their study demonstrated that like other aspects of face perception, figural face AEs can take place even when adapting faces are not being attended to. Nevertheless, an earlier study conducted by Moradi et al. (2005) established that identity AEs are only observed when observers are explicitly aware of the adapting face. Using a binocular rivalry technique, the authors showed that when an adapting face is successfully suppressed for the whole trial, adaptation does not take place. Thus similarly to the gaze cueing effect, face adaptation is in some cases automatic and occurs without explicit attention, and yet it is dependent on some degree of conscious perception.

Rhodes et al. (2011) were the first to demonstrate a positive effect that attention can have on face AEs. The authors conducted two experiments, each with a different adapting category and a different attentional task. In the first experiment, they examined identity AEs while participants detected whether or not the lips or eyes of the adapting face became lighter in shade; in the second experiment, they examined figural AEs while participants performed a one-back task with the adapting faces. Each experiment also included a passive-viewing condition. In both experiments, the authors found that AEs were greater during the attention condition than during the passive-viewing condition, demonstrating that attending to specific features or to

the identity of a face increases the degree of adaptation relative to passive viewing.

In the present study, we ask the following question: does attending to one facial characteristic of a face influence how another facial characteristic is adapted to? Whereas previous studies considered how the presence or absence of attention modulates adaptation more generally, our study addresses how the specific dimension of attention impacts that specific dimension of adaptation. We conducted three experiments that compared how attending to race or gender impacts adaptation to race or gender. In each experiment, we asked participants to focus on either the race or gender of an adapting face, and then measured subsequent AEs in the race and gender dimensions.

### MATERIALS AND METHODS

### Participants

Participants were 199 University of California, Santa Cruz undergraduates (ages 18–23; 141 female) who gave informed consent and participated for course credit. The University of California, Santa Cruz IRB approved the experimental paradigm. There were 54 participants in Experiment 1, 91 in Experiment 2, and 54 in Experiment 3.

#### Stimuli

We used FaceGen Modeller 3.1<sup>1</sup> to generate faces varying across dimensions of race and gender. To construct each face, we started with a random initial face and adjusted the race and gender scales to generate 12 Asian female and 12 Caucasian male faces. For Experiments 1 and 2, we similarly generated neutral faces by starting with 12 random initial faces, adjusting the gender scale to a perceptually neutral level, and matching the contribution of Caucasian and Asian races while minimizing the contribution of other races. Example stimuli are shown in **Figure 1**. We found that faces generated on the midpoint of FaceGen's gender scale appeared more in male than female. Therefore to generate a set of faces that were as neutral as possible on both the race and gender dimensions, we conducted a short calibration study soliciting ratings from 12 research assistants in our lab who were naive to the hypothesis of our study. In the calibration study, participants were presented with a superset of 48 potential neutral faces with instructions to rate each one on a seven-point race or gender scale. Based on these results, we selected the set of 12 neutral faces that elicited the most balanced ratings of race and gender.

For Experiment 3, we constructed neutral faces in a different way, by creating morphs between pairs of adapting faces. Because 50% morphs between arbitrary pairs of adapting faces did not typically result in neutral faces on these dimensions, we adjusted the morph level for each face until it appeared as neutral as possible (see **Figure 5**). To validate the neutral faces, we conducted a short calibration study on 19 new participants recruited from University of California, Santa Cruz. In the calibration study, participants were presented with a randomly

<sup>1</sup>http://www.facegen.com/

chosen adapting face (either Asian female or Caucasian male) for 5 s, with instructions to focus on the face. After each adapting face, a neutral face was displayed for 250 ms and participants were asked to rate it on either race or gender, using a seven-point scale. Based on the results, the initial set of 12 neutral faces was parsed down to 8 faces that elicited the most balanced ratings of race and gender across all adapting faces.

In all three experiments, the face images were converted to gray scale to eliminate the potential use of color information for judgments of race and gender, and presented on a black background.

#### Procedure

#### General Procedure

In Experiments 1, 2, and 3, participants sat approximately 18<sup>00</sup> away from a 21-inch LCD screen, where face images subtended approximately 7◦×7 ◦ of visual angle. The experiment presentation and data collection scripts were written in Matlab. Before each experiment began, a research assistant read the instructions out loud, asking participants to pay close attention to faces that would be displayed and to follow the focusing instructions. Participants were not instructed to provide responses quickly. In each trial, participants observed a randomly chosen adapting face for 5 s with overlaid instructions to focus on either the race or the gender of the face (these instructions appeared 700 ms prior to the presentation of the face Experiments 2 and 3 and remained present throughout the subsequent 5 s). After a 700 ms black screen, participants were then presented with an IQ asking them to rate that adapting face on a sevenpoint scale. For attend-race trials, the rating scale ranged from 1 (extremely Asian) to 7 (extremely Caucasian). For attend-gender trials, the rating scale ranged from 1 (extremely female) to 7 (extremely male). Following the participant's response, a black screen appeared for 300 ms, followed by a presentation of a randomly chosen neutral face for 300 ms, followed by another black screen prompting participants to rate that neutral face on either race or gender, using the corresponding seven-point scale. Half of the trials were attention-congruent (e.g., participants attended to the race of the adapting face and then rated the race of the neutral face) and half were attention-incongruent (e.g., participants attended to the gender of the adapting face but rated the race of the neutral face).

#### **Experiment 1 Procedure**

In Experiment 1, 54 participants completed 96 trials where they adapted to a face and rated a subsequent face by typing a number from 1 to 7 on a keyboard. On race trials, 1 corresponded to "extremely Asian" and 7 to "extremely Caucasian"; on gender trials, 1 corresponded to "extremely female" and 7 to "extremely male". The IQ regarding the race or gender of the adapting face was presented in every trial.

#### **Experiment 2 Procedure**

In Experiment 2, 91 participants completed 160 trials similar to those in Experiment 1, except that they entered their ratings by clicking on a seven-point scale on the screen. To more clearly dissociate the race and gender scales, we presented the race

FIGURE 1 | Example stimuli from Experiments 1 and 2. Left panel: Asian female adapting faces; center panel: race- and gender-neutral test faces; right panel: Caucasian male adapting faces.

FIGURE 2 | Results of Experiment 1. (A) Mean race ratings of adapting faces on a seven-point scale where 1 = extremely Asian and 7 = extremely Caucasian. (B) Mean gender ratings of adapting faces on a seven-point scale where 1 = extremely female and 7 = extremely male. (C) Mean race ratings of neutral faces as a function of the adapting face. (D) Mean gender ratings of neutral face as a function of the adapting face. "AE" refers to the aftereffect-consistent difference between ratings as a function of the adapting face. Error bars denote standard error of mean across participants.

scale horizontally (with "extremely Asian" on the far left and "extremely Caucasian" on the far right) and the gender scale vertically (with "extremely female" on the top of the scale and "extremely male" on the bottom). In this experiment, the IQ was presented on only 50% of trials (in the other 50%, the neutral test face was presented immediately following the black screen after the adapting face).

#### **Experiment 3 Procedure**

In Experiment 3, 54 participants completed 160 trials identical to those in Experiment 2, except that the neutral faces were constructed differently (see Stimuli and **Figure 4**) and the IQ was presented on only 25% of trials.

### RESULTS

In the three experiments, we present results averaged across participants with error bars showing standard error of the mean. For reaction time data, we computed each participant's median reaction time in each type of trial, and we present the mean across participants along with the standard error of the mean. To quantify AEs for race and gender, we subtracted ratings of neutral faces following the two types of adapting faces. For example, to measure gender AEs, we subtracted gender ratings of neutral faces following Asian female adapting faces from those following Caucasian male adapting faces. The aftereffectconsistent difference measures (AE) for race and gender are

therefore defined as follows:

AERace = RNeutral adapt Asian − RNeutral adapt Caucasian

dimension during adaptation. Error bars denote standard error of the mean across 54 participants.

AEGender = GNeutral adapt female − GNeutral (adapt male)

where RNeutral is the average race ratings of neutral faces, and GNeutral is the average gender ratings of neutral faces.

#### Experiment 1 Results

Based on responses to the IQ, we confirmed that the adapting faces were unambiguous on the race and gender dimensions. On average, Asian female faces were rated as 2.07 ± 0.09 on the seven-point race scale and as 2.77 ± 0.11 on the seven-point gender scale; Caucasian male faces were rated as 6.42 on the race scale and 6.69 on the gender scale (see **Figures 2A,B**). We did observe a significant male bias in these ratings, wherein male adapting faces were rated as significantly more male than female adapting faces were rated as female (**Figure 2B**). We noted that similar male biases have been reported in the rating human faces (e.g., Davidenko, 2007, in the perception of face silhouettes; Gaetano et al., 2016, in the perception of faces and hands under a variety of formats), and Bayesian models have been proposed to explain these biases (Clifford et al., 2015). Nevertheless, the variable of interest here was the difference in ratings between the different adapting categories, regardless of any general biases. Overall, the difference in race ratings between adapting faces (4.35 ± 0.17) was slightly larger than the difference in gender ratings between adapting faces (3.92 ± 0.14; t), although this difference was small. Based on previous work that shows a monotonic relationship between the extremeness of adapting faces and the strength of aftereffects (e.g., Jeffery et al., 2011), we predicted that aftereffects for race and gender would be comparable (or if anything, race aftereffects would be slightly stronger than gender aftereffects).

To quantify race and gender aftereffects (AEs), we compared ratings of neutral faces following the two types of adaptors. Mean race ratings of neutral faces were 4.30 ± 0.11 following Asian female adaptors, compared to 3.88 ± 0.10 following Caucasian male adaptors (AERace: 0.43 ± 0.09). Mean gender ratings of neutral faces were 4.73 ± 0.12 following Asian female adaptors, compared to 3.89 ± 0.14 following Caucasian male adaptors (AEGender: 0.84 ± 0.12; see **Figures 2C,D**). Overall, AEs were significantly stronger for gender than for race AEs (t<sup>53</sup> = 4.28, p < 0.0001), a difference we did not predict and that we address in Experiment 3.

The question of interest was whether the attention manipulation (i.e. "focus on the race" or "focus on the gender") influenced adaptation. To validate the effectiveness of the attention manipulation itself, we compared reaction times to rate the neutral face during attention-congruent vs. attention-incongruent trials. For race rating trials, mean RTs to rate the neutral face were significantly shorter when participants had been asked to focus on the race (2.16 s ± 0.08) vs. the gender (2.30 s ± 0.09) of the adapting face (mean difference: 0.14 s ± 0.06; t<sup>53</sup> = 2.41, p < 0.05). Similarly, reaction times during gender rating trials were significantly shorter when attending to the gender (1.96 s ± 0.07) vs. the race (2.17 s ± 0.08) of the adapting face (mean difference: 0.20 s ± 0.04; t<sup>53</sup> = 5.36, p < 0.00001; see **Figure 3A**). Collapsing across race and gender trials, participants were significantly faster to respond during attention-congruent trials (2.06 s ± 0.08) than during attention-incongruent trials (2.23 ± 0.08; t = 4.83, p < 0.0001).

Although the attention manipulation strongly influenced reaction times, it had no influence on the strength of AEs. Race AEs were no larger when participants had attended to the race (AERace\_attend\_race: 0.36 ± 0.10) vs. the gender (AERace\_attend\_gender: 0.49 ± 0.10) of the adapting face (mean difference: 0.13 +0.10; t<sup>53</sup> = –1.33, p > 0.1). Likewise,

function of attended dimension during adaptation. Error bars denote standard error of the mean across 91 participants.

gender AEs during attend-gender trials (AEGender\_attend\_gender 0.78 ± 0.14) were not larger than those during attend-race trials (AEGender\_attend\_race 0.88 ± 0.11; difference: 0.10 +0.11; t<sup>53</sup> = –0.90, p > 0.1; see **Figure 3B**). In fact, there was a non-significant negative trend when collapsing across race and gender trials, wherein attention-incongruent trials elicited slightly stronger AEs (0.68 ± 0.09) than attention-congruent trials (0.57 ± 0.11; t<sup>53</sup> = –1.58, p = 0.12).

To account for this unexpected trend, we considered the possibility that the IQ itself may have led to a response repetition bias, which would counteract the adaptation AEs during attention-congruent trials. For example, a participant responding to the IQ about the race of an adapting Caucasian face might accidentally repeat that response when asked about the race of the neutral face. In contrast, during attention-incongruent trials, the second question would be about the gender of the neutral face, eliminating the possibility of a response repetition bias. Because such a mechanism would weaken the apparent strength of AEs specifically during attention-congruent trials, it may conceal any attention-related enhancement in AEs. For this reason, in Experiments 2 and 3 we interspersed trials with and without the IQ to examine its possible role in the adaptation process and whether this ironic trend persists in the absence of an IQ. In addition, to further disambiguate responses along the two dimensions, participants in Experiments 2 and 3 entered their responses by clicking on one of two perpendicular scales on the screen (a horizontal scale for race and a vertical scale for gender), rather than by inputting numbers on the keyboard. This served to more clearly distinguish the two scales for participants and to provide a more sensitive response measure for each rating.

#### Experiment 2 Results

We present results separately for IQ trials (those that included the IQ regarding the adapting face) and No-IQ trials (those with no IQ). In IQ trials, the attention instructions strongly influenced

reaction times, just as in Experiment 1 (see **Figure 4A**). Ratings of neutral faces took significantly less time during attention-congruent trials (1.75 s ± 0.06) than during attentionincongruent trials (2.01 s ± 0.06; t<sup>90</sup> = 8.06, p < 0.00001), serving to validate the attention manipulation. However, AEs were not stronger during attention-congruent trials (0.30 ± 0.06) than during attention-incongruent trials (0.44 ± 0.04). In fact, overall AEs were weaker in attention-congruent trials (t<sup>90</sup> = –2.19, p = 0.03; see **Figure 4B**), corroborating the ironic trend we had observed in Experiment 1.

In No-IQ trials, the attention manipulation had a smaller but still significant effect on reaction times to neutral face,

with a mean of 1.96 s ± 0.06 for attention-congruent trials and 2.13 s ± 0.06 for attention-incongruent trials (t<sup>90</sup> = 4.54, p < 0.0001; **Figure 4C**), validating the effectiveness of the attention manipulation even in the absence of an IQ. Unlike reaction times, the magnitude of AEs was not influenced by the attention instructions, with overall AEs of 0.67 ± 0.06 for attention-congruent trials and 0.69 ± 0.06 for attention-incongruent trials (non-significant difference, p > 0.5; **Figure 4D**). Importantly, when the IQ was not present, there was no ironic effect of attention on AEs.

By considering both IQ and No-IQ trials in the same experiment, we were able to characterize the effect of the IQ on adaptation process. Including the IQ led to significantly weaker overall AEs (0.37 ± 0.04) compared to not including the IQ (0.68 ± 0.04; t<sup>90</sup> = 5.86, p < 0.00001). This difference may be attributed to the additional delay and/or perceptual disruption imposed by the IQ itself, or as we suggested above, to a possible response-repetition bias elicited during attention-congruent IQ trials. Given the attenuating effect of the IQ on the strength of AEs, and its possible role in obscuring attention-related effects, we designed Experiment 3 to focus primarily on No-IQ trials by including 75% No-IQ trials and only 25% IQ trials. The reason to include any IQ trials is that they provide an incentive for participants to follow the focusing instructions.

Finally, we found in Experiment 2 just as in Experiment 1 that overall gender AEs (0.67 ± 0.05) were significantly stronger than race AEs (0.37 ± 0.04; t<sup>90</sup> = 5.5, p < 0.00001). This difference cannot be attributed to differences in the unambiguity of the adaptors with respect to race and gender, which if anything should have led to slightly stronger race AEs (see **Figures 2A,B**). Instead, this bias may be due to the method by which the neutral faces were constructed. Due to the interdependence of facial dimensions in the face construction software, neutral faces had nonzero contributions of other races besides Asian and Caucasian. Therefore these neutral stimuli might not have served as ideal neutral test points to measure race AEs. We address this in Experiment 3 by constructing a new set of neutral faces that involved creating perceptually neutral morphs between pairs of adapting faces (see Methods/Stimuli; **Figure 5**).

#### Experiment 3 Results

We present results for No-IQ trials, which constituted 75% of all trials. As we anticipated, the new method of generating neutral faces eliminated the discrepancy between race and gender AEs we had observed in Experiments 1 and 2. Specifically, we found that the overall magnitude of race AEs (0.81 ± 0.09) did not differ significantly from the overall magnitude of gender AEs (0.74 ± 0.10; t<sup>53</sup> = 0.66, p > 0.5).

As in Experiments 1 and 2, reaction times in Experiment 3 were significantly influenced by the attention manipulation, wherein attention-congruent trials elicited shorter RTs (1.76 s ± 0.06) than attention-incongruent trials (1.83 s ± 0.07; t<sup>53</sup> = 2.23, p < 0.01; see **Figure 6A**). Nevertheless, the strength of AEs once again was not influenced by attention (**Figure 6B**). Collapsing across race and gender rating trials, AEs on attention-congruent trials (0.81 ± 0.08) were similar to AEs on attention-incongruent trials (0.74 ± 0.09; t<sup>53</sup> = 1.05, p > 0.2). Together with the results of Experiments 1 and 2, our data confirm that attending to the race or gender of a face does not influence the extent of adaptation to race or gender.

### DISCUSSION

Our results show that race and gender AEs can be elicited in a single-shot adaptation paradigm, where both dimensions are adapted to simultaneously. Based on the results of Experiments 1 and 2, it seemed that gender AEs somehow dominated race AEs; however, a careful reconstruction of neutral stimuli in Experiment 3 based on directly morphing pairs of adapting faces

eliminated this discrepancy and showed that race and gender AEs are comparable in magnitude.

Our data also show that soliciting a rating of the adapting face via an IQ can attenuate AEs due to a possible responserepetition bias that can occur only for attention-congruent trials. For example, a participant who responds "male" to an adapting face may inadvertently respond "male" again when asked about the gender of the neutral face, reducing the apparent magnitude of the AE. This repetition bias would not occur on attentionincongruent trials, where the participant is asked a question about race at test.

Across the three experiments, we showed when participants are asked to focus on the race or gender of an adapting face, their reaction times to rate a neutral face are influenced by the congruence of the two questions. This was the case in the presence or absence of an IQ, although it was stronger when the IQ reaffirmed the attended dimension. Despite this reaction time effect of attention, we found no evidence in any of our conditions that attending to race or gender increased the magnitude of AEs along that dimension. Across experiments, AEs during attentionincongruent trials were just as strong as (if not stronger than) AEs during attention-congruent trials.

It is possible that our attention manipulation did not provide enough incentive to participants to strongly focus on the race or gender of the faces. The validation measure for our attention manipulation was based on comparing reaction times to rate neutral faces during attention-congruent vs. attentionincongruent trials, with a greater reaction time during attentionincongruent trials indicating that the attention manipulation was successful. Even when the attention manipulation was successful in influencing reaction times, it is possible that the mechanism involved something other than encoding differences during adaptation. For example, participants may have been preparing to answer a question about race (or gender) rather than consciously attending to that attribute of the adapting face. While our data cannot rule out such an interpretation, the robustness of the attentional effect on response times across both IQ and NIQ trials suggests that participants were actually attending to the dimension as instructed. We note that the response time effect in Experiment 2 (where the IQ appeared on 50% of trials, and the mean reaction time difference was 165 ms ± 47 ms) trended to be stronger in Experiment 3 (where the IQ appeared on only 25% of trials, and the mean reaction time difference was 71 ms ± 32 ms, two-sample t<sup>143</sup> = 1.77, p = 0.08). It is possible then that the presence of the IQ served as both an incentive to follow the attention instructions and a dampener of the adaptation effect. Therefore a future study may devise a stronger attention manipulation that can provide reliable incentives for participants to attend to relevant facial dimensions, without biasing or dampening the adaptation process itself.

Another possible reason we found no enhancement in attention-congruent AEs is that race and gender perception both involve similar and overlapping holistic processes (Tanaka et al., 2004; Zhao and Hayward, 2010). Because of the overlap in features that cue race and features that cue gender, perhaps observers cannot selectively attend to race cues while ignoring gender cues, or vice versa. For example, Ito and Urland (2003) conducted an event-related potential (ERP) study where participants were instructed to attend either to the race or gender of a sequence of faces. Although race-related activity was found to be greater and occur earlier than gender-related activity, both race and gender information were processed during the task, regardless of the attention condition. Furthermore, there is a general overlap between the facial areas used to make gender and race distinctions. Dupuis-Roy et al. (2009) showed that the eyebrow area contained the most gender information, while Blais et al. (2008) found that Western Caucasian observers fixated on the eye and mouth regions when determining the race of a face, though they also found a significant cultural difference, with East Asian observers focusing more centrally when determining race. Indeed, even among participants of European origin, there were differences in the eye movements elicited by same- and other-race face (Brielmann et al., 2014). In our studies, participants were free to inspect the faces freely during the 5-s adaptation phases, so a future eye tracking study may illuminate whether attending to race vs. gender results in different scan paths, and in turn, different adaptation mechanisms for some observers.

There is evidence in other visual domains that attending differentially to the same stimulus can lead to different AEs. For example, Lankheet and Verstraten (1995) showed that the direction of motion AEs can be modulated by attention. Participants were asked to view a moving random dot pattern that contained both a leftward and rightward motion signal, each of which could be selectively attended to. The authors found that if one or the other direction was attended to during adaptation, a corresponding motion AE was observed. In a later study, Spivey and Spirn (2000) investigated the tilt AE in a similar fashion. Normally, adapting to tilted bars in a particular direction of tilt causes a subsequent vertical bar to appear tilted in the opposite direction. In Spivey and Spirn's (2000) study, the adapting stimuli contained overlapping tilted bars rotated clockwise and counterclockwise. The authors found when participants were instructed to attend to the clockwise tilted bars, the direction of the tilt AE was in the counterclockwise direction (and vice versa). Both of these studies demonstrate that in lower level visual processes, attending to a specific dimension of a stimulus can drive differential AEs with respect to that dimension. It remains to be shown why such attention-dependent AEs are not observed in the perception of race and gender.

### AUTHOR CONTRIBUTIONS

ND was primarily responsible for the conception of the project, development of the adaptation paradigm, analysis of data, and writing the methods, results, and discussion of the manuscript. CV was critically involved in the construction and validation of the adapting and neutral face stimuli and coding experimental scripts. NH wrote the first draft of the introduction and contributed along with CV to interpreting results and validating the face stimuli. JC helped in stimulus design by testing the efficacy of different formats of face images in eliciting aftereffects using web surveys. All authors contributed to the editing of the final draft.

### REFERENCES

fpsyg-07-00909 June 15, 2016 Time: 13:46 # 10


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Davidenko, Vu, Heller and Collins. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# The Importance of Formalizing Computational Models of Face Adaptation Aftereffects

#### David A. Ross<sup>1</sup> \* and Thomas J. Palmeri<sup>2</sup>

<sup>1</sup> Department of Psychological and Brain Sciences, University of Massachusetts, Amherst, MA, USA, <sup>2</sup> Department of Psychology, Vanderbilt University, Nashville, TN, USA

Face adaptation is widely used as a means to probe the neural representations that support face recognition. While the theories that relate face adaptation to behavioral aftereffects may seem conceptually simple, our work has shown that testing computational instantiations of these theories can lead to unexpected results. Instantiating a model of face adaptation not only requires specifying how faces are represented and how adaptation shapes those representations but also specifying how decisions are made, translating hidden representational states into observed responses. Considering the high-dimensionality of face representations, the parallel activation of multiple representations, and the non-linearity of activation functions and decision mechanisms, intuitions alone are unlikely to succeed. If the goal is to understand mechanism, not simply to examine the boundaries of a behavioral phenomenon or correlate behavior with brain activity, then formal computational modeling must be a component of theory testing. To illustrate, we highlight our recent computational modeling of face adaptation aftereffects and discuss how models can be used to understand the mechanisms by which faces are recognized.

#### Edited by:

Stefania D'Ascenzo, University of Bologna, Italy

#### Reviewed by:

K. Suzanne Scherf, Pennsylvania State University, USA Aaron Michael Clarke, Bilkent University, Turkey

> \*Correspondence: David A. Ross david.a.ross2409@gmail.com

#### Specialty section:

This article was submitted to Perception Science, a section of the journal Frontiers in Psychology

Received: 23 January 2016 Accepted: 17 May 2016 Published: 13 June 2016

#### Citation:

Ross DA and Palmeri TJ (2016) The Importance of Formalizing Computational Models of Face Adaptation Aftereffects. Front. Psychol. 7:815. doi: 10.3389/fpsyg.2016.00815 Keywords: face recognition, adaptation, exemplar, norm, computational modeling

### INTRODUCTION

Adaptation aftereffects have been used as a tool for studying low-level vision (e.g., Gibson and Radner, 1937; Blakemore and Sutton, 1969; Frisby, 1979; Wade, 1994; Anstis et al., 1998; Kohn, 2007; Thompson and Burr, 2009; Webster, 2015) and there has been of growing interest in their potential for informing high-level vision. Like low-level aftereffects (color and tilt) just a few seconds of adaptation are needed to bias the perception of high-level stimuli (faces, bodies, and objects; Leopold et al., 2005; Rhodes et al., 2007; Javadi and Wee, 2012; Rhodes et al., 2013). For example, brief adaptation to a face with a wide jaw and narrow eye separation will cause an aftereffect, biasing perception toward a psychologically opposite stimulus, such that a subsequently presented face may appear to have a narrower jaw and wider eye separation (Leopold et al., 2001; Rhodes and Jeffery, 2006; Robbins et al., 2007; Tangen et al., 2011).

The face recognition literature has used adaptation to address debates about the nature of face representations. Unlike low-level visual features, such as color or orientation, faces are complex with high dimensionality. The concept of face space has pervaded theorizing about face recognition, providing a framework for understanding how such multidimensional stimuli are represented in the brain (Valentine, 1991). Faces are represented as points along a collection

Ross and Palmeri Modeling Face Adaptation

of psychological dimensions, with the distance between points reflecting the similarity between faces. Each dimension of face space is assumed to represent some component of variation across the population of know faces; dimensions could correspond to physical differences, such as the distances between the eyes, or more holistic sources (O'Toole et al., 1993; Burton et al., 1999; Dailey and Cottrell, 1999; Wilson et al., 2002).

Debate has centered on the nature of the face representations that reside within face space (see Valentine, 2001; Rhodes et al., 2005; Rhodes and Leopold, 2011; Valentine et al., 2015). Norm models propose that faces are represented with respect to their deviation from a prototypical or average face, the norm (Valentine, 1991; Byatt and Rhodes, 1998; Loffler et al., 2005; Rhodes and Jeffery, 2006), whereas exemplar models propose that faces are represented with respect to their similarity to individually encoded faces, exemplars (Valentine, 1991; Lewis, 2004).

Although representations assumed by norm and exemplar models are fundamentally different, both turn out to make similar predictions about key aspects of face recognition, such as effects of typicality and distinctiveness (e.g., Valentine and Bruce, 1986; Valentine, 1991) and effects of caricature (e.g., Lewis and Johnston, 1998, 1999; Lewis, 2004). One seemingly promising approach to differentiating predictions of norm and exemplar models relies on face adaptation (e.g., Leopold et al., 2001; Rhodes and Jeffery, 2006; Robbins et al., 2007).

### FACE ADAPTATION AFTEREFFECTS

In a now classic paper, Leopold et al. (2001) had participants first learn to identify four novel target faces (Adam, Jim, John, and Henry) and then tested their ability to identify test faces as one of the four studied targets. Test faces were positioned along morph lines that projected from each of the target identities, through the norm, to the opposite side of face space (**Figure 1**). On some trials, participants were first briefly shown an adaptor face for a few seconds and then identified a test face. Without adaptation, the average face at the center of the space was equally likely to be identified as one of the four targets. But with exposure to an adaptor located on the opposite side of the norm from a target, call it "anti-Adam," the average face was more likely identified as "Adam." Relative to baseline without adaptation, the psychometric function for identification as a function of the distance of a test face from the average (identity strength), is significantly shifted to the left by adaptation to a matching antiface (adaptation to anti-Adam, testing on morphs of Adam); this contrasts with the psychometric function for non-matching antifaces (adaptation to anti-Jim, testing on morphs of Adam), which is slightly shifted to the right.

Rhodes and Jeffery (2006) extended this paradigm with the inclusion of a critical control condition. In addition to contrasting adaptation along opposite morph trajectories that passed through the norm (Leopold et al., 2001), they tested adaptation along nonopposite morph trajectories that did not (**Figure 1**). Adaptation affected face identification with respect to the norm, such that post-adaptation identification thresholds were significantly lower for faces along opposite trajectories, compared to those along non-opposite trajectories. Rhodes and Jeffery (2006) reasoned that since the magnitude of adaptation was dependent on whether the morph trajectory passed through the norm (opposite trajectories) or not (non-opposite trajectories), the psychological representation of faces must make some reference to a norm face.

### INTUITIONS ABOUT FACE ADAPTATION

Finding that adaptation biases face recognition with respect to the norm rather than simply biasing recognition away from the adaptor has been widely interpreted as evidence for norm-based coding (Rhodes et al., 2005; Rhodes and Jeffery, 2006; Robbins et al., 2007; Susilo et al., 2010; Rhodes and Leopold, 2011; Rhodes and Calder, 2014; Short et al., 2014; Walsh et al., 2015, but see Zhao et al., 2011; Storrs and Arnold, 2012). At its most simplest, finding that face adaptation is sensitive to the norm intuitively suggests that face representations are constructed with respect to a norm.

Intuitions are also often supported by considering illustrations of one-dimensional two-pool (norm) and multichannel (exemplar) models (**Figure 2**). In a two-pool model, face representation are assumed to be broadly tuned, responding maximally to a particular extreme within face space. For example, in **Figure 2**, the two pools encode variations in eye height, with one pool preferring faces with extremely high eyes and the other pool preferring faces with extremely low eyes. The location of a face along a given dimension is encoded by the proportional activity of the two opposing pools. Because the pools are explicitly specified to intersect at the location of the norm face, two-pool models are generally considered instantiations of norm models. By contrast, in a multichannel model, face representations are assumed to be narrowly tuned, preferring faces at a particular location along some dimension. These are generally considered analogous to exemplar models because faces are encoded only with respect to other faces, rather than making reference to any norm face.

For both two-pool and multichannel models, face representations are assumed to adapt in proportion to their level of activation to the adaptor. In the case of the two-pool norm model, adapting one of the pools more strongly than the other will cause a subsequent bias in the relative activation of the two pools, with the effect that after adaptation, perception will be biased toward the opposite extreme. By contrast, for the multichannel exemplar models, adaptation will only affect representations at the location of the adaptor, causing subsequent perception to be biased away from the adaptor, not specifically toward the opposite extreme.

A significant limitation of this past work is that nearly all of the assumptions about how norm and exemplar models might respond to face adaptation have been based on intuitions and pictorial illustrations of one- or two-dimensions. Rarely are these intuitions supported by explicit simulations, where predictions are formally generated, compared, and evaluated. The mechanisms involving face recognition undoubtedly involves combinations of high-dimensionality, non-linear activation,

and parallelism – all properties of the human brain. Making predictions based on intuition or illustration alone must be the subject of scrutiny (e.g., see Hintzman, 1990; Burton and Vokey, 1998; Lewis, 2004; Palmeri and Cottrell, 2009), as we recently illustrated (Ross et al., 2014).

### MODELING FACE ADAPTATION AFTEREFFECTS

Part of our motivation for a close examination of formal predictions of norm and exemplar models comes from lessons learned in the categorization literature. Posner and Keele (1968, 1970) had people learn to classify novel dot pattern stimuli into different categories, each defined by a unique prototype. Even though they never saw the prototypes during learning, being trained only on distortions of prototypes, when tested after learning, classification of unseen prototypes was better than classification of distortions. The canonical interpretation of prototype enhancement (e.g., Reed, 1972; Rosch, 1975; Homa, 1984) was that category learning involves some form of prototype abstraction. How else could an unseen prototype be classified so well than it had been abstracted as a product of learning? These intuitions were challenged when prototype and exemplar models were formalized in mathematics and simulations and their predictions explicitly generated and compared to data. Models that assume memory for category exemplars, without any abstraction, account quite well for prototype enhancement effects as well as phenomena that pure prototype abstraction models cannot (e.g., see Busemeyer et al., 1984; Hintzman, 1986; Nosofsky, 1992; Palmeri and Nosofsky, 2001). Could a similar disconnect between intuition and formal prediction be the case for norm versus exemplar debate in face recognition?

Ross et al. (2014), we instantiated an exemplar model of face recognition, which bore similarities to exemplar models of categorization (e.g., Nosofsky, 1986; Kruschke, 1992) and descriptions of multichannel models (e.g., Robbins et al., 2007). We also instantiated two versions of a norm-based model. One version, which we referred to as a traditional norm-based model, had been formalized previously by Giese and Leopold (2005); Ophthalmology (©ARVO).

fpsyg-07-00815 June 9, 2016 Time: 14:36 # 4

the other, a two-pool model, was adapted from descriptions of norm-based coding (e.g., Rhodes and Jeffery, 2006). In the following descriptions, we omit many of the mathematical and computational details, focusing instead on what we did and why, and recommend the reader to refer to Ross et al. (2014).

Our core modeling framework is no different from dozens, if not hundreds, of other models of perception and cognition. There is an input layer that models the perceptual representation of the face, an intermediate layer that models alternative assumptions about face space (exemplar, traditional norm, and two-pool), and an output layer that generates a decision about the face (Adam, Jim, John, or Henry). Because we were interested in directly comparing three alternative theories of face space, we assumed the same input representation and output mechanism for every model. All that varied was the internal face-space representation.

To briefly outline, when a test face, say a particular face along the Anti-Adam/Adam morph continuum, is presented to the model for identification, a multidimensional perceptual representation is created by the visual system. Instead of attempting to model the complete processing hierarchy of the ventral stream visual system that creates this perceptual representation (e.g., Jiang et al., 2006; Serre et al., 2007), we considered two simpler possibilities. The first made no specific assumptions about how the perceptual representation of a face is created from its 2D retinal image, and simply assumed that a randomly sampled face is represented by a random sample from a multivariate Gaussian (normal) distribution (see also Lewis, 2004). The second used an actual 2D face image as input and created a multidimensional input representation of it via principal components analysis (PCA), much like many other models (e.g., O'Toole et al., 1993; Burton et al., 1999; Dailey and Cottrell, 1999; Giese and Leopold, 2005; Richler et al., 2007).

In both cases the perceptual representation is simply a vector specifying the location of a given test face along each dimension of face space. This face representation then activates exemplars, norms, or pools in the face-space layer according to the rules for that particular model of face space. In each case, we assumed that pre-experimental experience populated face space with a collection of exemplars, norms, or pools, depending on the model. Like other exemplar-based face-space model (e.g., Lewis, 2004; Giese and Leopold, 2005), activation of a given exemplar in face space is a non-linearly decreasing function of the distance of that exemplar to the test face. For the traditional normbased model (Giese and Leopold, 2005), norm representations are activated by a test face as a function of both the difference in angular distance with respect to the norm as well as their relative distance from the norm. And for the two-pool model (e.g., Rhodes and Jeffery, 2006), competing pools of units on opposite side of the norm are activated as a function of the relative similarity of a test face to members of each pair in the pool. Simple assumptions were made to implement adaptation within the three models as a temporary rescaling of maximal activation according to the similarity of each exemplar, norm, or pool to the adaptor.

Finally, the distributed pattern of activity across these exemplars, norms, or pools is associated with output nodes for Adam, Jim, John, or Henry, with the relationship between a given pattern of face-space activations and a particular identity learned using a standard delta-rule learning algorithm. This step mirrors the initial learning of face identities by participants in the face adaptation experiments. Activation of the identity nodes is readily translated into identification probabilities (following, for example, Kruschke, 1992). While the actual identification of the learned face is a critical component of face adaptation experiments, most predictions described for different face-space

models focus entirely on the face-space representation, largely ignoring how the activation of those representations in face space might be combined to generate the learned face identification response that is the key measure in the task.

We tested the three models on how well they could qualitatively predict the observed data from Leopold et al. (2001) and Rhodes and Jeffery (2006), outlined earlier, as well as a related data from Leopold and Bondar (2005). Each of the variants has a small collection of free parameters, including such things as the number of dimensions in the input perceptual representations, the width of the tuning of exemplars, norms, or pools, and scaling parameters that define things like the strength of adaptation and how output activity is mapped onto response probabilities. We explored a wide range of model parameterizations, and not only looked for parameter combinations that provided a good quantitative account of the observed data in each experiment, but also evaluated whether models made parameter-free qualitative predictions irrespective of particular parameterizations. Simply put, outside of subsets of parameters that produced no significant effect of adaptation at all – for example, having the scaling on adaptation too low or the width of the tuning function too small – in all cases, the quantitative predictions illustrated below map onto the qualitative predictions across parameters sets.

All three models are able to account for the data from Leopold et al. (2001) and only the exemplar model and the two-pool model are able to account for the data from Rhodes and Jeffery (2006), with the traditional norm-based model failing on that score (**Figure 1**). Despite prevalent intuitions that face adaptation aftereffects clearly support norm models over exemplar models, those intuitions were not borne out when those models were explicitly implemented and simulated and their predictions compared to observed data. In fact, our variant of the traditional norm-based model could not explain key aspects of the observed face adaptation data.

### WHY INTUITIONS FAIL

Why do common intuitions about norm and exemplar face adaptation fail? To begin with, most illustrations depict one or two dimensions but face space likely has 10, 20, 100, or more dimensions. Human intuition beyond two or three dimensions often fails miserably (e.g., DiCarlo and Cox, 2007). Combine this high dimensionality with the non-linearities in activation functions and decision rules, and intuition is bound to fail.

Consider also the claim that representations in a multichannel model must be narrowly tuned (e.g., Robbins et al., 2007). On a single dimension perhaps this corresponds to a representation having at most significant activation across one quarter of the span of the dimension (**Figure 2**, right). But, consider instead a representation in two dimensions; we now have a circle that occupies significantly less than one quarter (one quadrant) of the total space. While in one dimensions it seems that each exemplar must be so broadly tuned that it would convey little useful information to discriminate different faces, in multiple dimensions this need not be the case.

Illustrations of exemplar models (**Figure 2**, right) also tend to impose a complete tiling across face space. To begin with, while a complete tiling might be feasible for one or even two dimensions, the sheer number of nodes necessary to fully tile a 100-dimension face space is far more than the number of neurons in the brain (and likely more than the number of atoms in the known universe). More importantly, exemplar models assume that face space will be populated based on specific experience with faces, and do not assume any a priori covering map across space. The combination of a finite number of face representations, together with non-uniform exemplar distribution, along with exemplar tuning widths required in high dimensions, creates a situation in which face adaptation is dependent on the adaptor faces location relative to the entire population of exemplars. Like other examples from the face recognition and category learning literatures (e.g., Lewis, 2004), the ability of the exemplar model to make predictions that appear to require a norm is driven by the fact that the average is implicitly represented in the statistical distribution of faces, not any explicit norm representation.

## DISCUSSION

Face adaptation aftereffects are used to draw mechanistic conclusions about how faces are represented. Our work makes clear that mechanistic conclusions really must be supported by formalized computational models that delineate specific testable assumptions about how faces are represented, how face knowledge is represented and used, how adaptation works, and how face identities are determined. Our assumptions about any of these components may be wrong, or there may be alternatives that we did not consider. But that is a strength of formal modeling, not a weakness, especially when compared to mechanistic predictions derived from intuitions or simplified illustrations, as has been the case for much theoretical work concerning face adaptation. In nearly every area of perception and cognition where a computational modeling has been deployed, there are examples of empirical phenomena that intuitively point to one particular mechanistic explanation but in fact can be explained as well or better by other explanations when those alternatives are formally evaluated (e.g., Farrell and Lewandowsky, 2010).

### AUTHOR CONTRIBUTIONS

Both DR and TP contributed equally to the conception, design and drafting of this manuscript and are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

### ACKNOWLEDGMENT

This research was supported in part by NSF grants SBE-1257098 and SMA-1041755 (Temporal Dynamics of Learning Center) and by a Discovery Grant from Vanderbilt University.

### REFERENCES

fpsyg-07-00815 June 9, 2016 Time: 14:36 # 6



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Ross and Palmeri. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Body Image Distortion and Exposure to Extreme Body Types: Contingent Adaptation and Cross Adaptation for Self and Other

Kevin R. Brooks 1, 2 \*, Jonathan M. Mond1, <sup>3</sup> , Richard J. Stevenson1, 2 and Ian D. Stephen1, 2, 4

*<sup>1</sup> Department of Psychology, Macquarie University, Sydney, NSW, Australia, <sup>2</sup> Faculty of Human Sciences, Perception in Action Research Centre, Macquarie University, Sydney, NSW, Australia, <sup>3</sup> Department of Psychology, Centre for Emotional Health, Macquarie University, Sydney, NSW, Australia, <sup>4</sup> ARC Centre of Excellence in Cognition and its Disorders, Macquarie University, Sydney, NSW, Australia*

Body size misperception is common amongst the general public and is a core component of eating disorders and related conditions. While perennial media exposure to the "thin ideal" has been blamed for this misperception, relatively little research has examined visual adaptation as a potential mechanism. We examined the extent to which the bodies of "self" and "other" are processed by common or separate mechanisms in young women. Using a contingent adaptation paradigm, experiment 1 gave participants prolonged exposure to images both of the self and of another female that had been distorted in opposite directions (e.g., expanded other/contracted self), and assessed the aftereffects using test images both of the self and other. The directions of the resulting perceptual biases were contingent on the test stimulus, establishing at least some separation between the mechanisms encoding these body types. Experiment 2 used a cross adaptation paradigm to further investigate the extent to which these mechanisms are independent. Participants were adapted either to expanded or to contracted images of their own body or that of another female. While adaptation effects were largest when adapting and testing with the same body type, confirming the separation of mechanisms reported in experiment 1, substantial misperceptions were also demonstrated for cross adaptation conditions, demonstrating a degree of overlap in the encoding of self and other. In addition, the evidence of misperception of one's own body following exposure to "thin" and to "fat" others demonstrates the viability of visual adaptation as a model of body image disturbance both for those who underestimate and those who overestimate their own size.

Keywords: adaptation, body image, psychophysics, perception, eating disorders, aftereffects, neural representation

## INTRODUCTION

The misperception of one's own body as wider or narrower than its veridical size is a growing health concern, affecting a substantial minority of women and, increasingly, men (Mond et al., 2013; Quick et al., 2014; Griffiths et al., 2016). Those experiencing body size misperception often view themselves as much fatter than they really are. This is thought to be a risk factor for eating-disordered behavior,

#### Edited by:

*Rocco Palumbo, Harvard Medical School, USA*

#### Reviewed by:

*Maria Olkkonen, Durham University, UK Ko Sakai, University of Tsukuba, Japan*

\*Correspondence: *Kevin R. Brooks kevin.brooks@mq.edu.au*

#### Specialty section:

*This article was submitted to Perception Science, a section of the journal Frontiers in Neuroscience*

Received: *12 April 2016* Accepted: *01 July 2016* Published: *15 July 2016*

#### Citation:

*Brooks KR, Mond JM, Stevenson RJ and Stephen ID (2016) Body Image Distortion and Exposure to Extreme Body Types: Contingent Adaptation and Cross Adaptation for Self and Other. Front. Neurosci. 10:334. doi: 10.3389/fnins.2016.00334* negative affect, and poor mental health more generally (Stice, 2002; Mond et al., 2013). On the other hand, many overweight and obese individuals misperceive their body as being thinner than it really is (Truesdale and Stevens, 2008; Wetmore and Mokdad, 2012), which has serious implications for physical health (Powell et al., 2010; Yaemsiri et al., 2011).

Along with social pressure to be thin (Stice et al., 2003), body size misperception is often attributed to exposure to unrealistic body ideals as portrayed in the popular media, such as magazines, cinema, and TV. Explanations for the relationship between the media and body image distortions have predominantly focused on sociocognitive processes such as the role of social comparison (Wheeler and Miyake, 1992; Harrison, 2000; Barlett et al., 2008). Such exposure can have negative effects on cognitive schemas concerning one's own body (Stice et al., 1994; Groesz et al., 2001), which is thought to produce the observed changes in body satisfaction (Barlett et al., 2008; Harper and Tiggemann, 2008), perceptions of ideal body size (Stephen and Perera, 2014), and actual judgments of body size and shape itself. Meanwhile, it has been suggested that body size underestimation by larger individuals may result from a form of normalization due to exposure to a population in which obesity is ever more prevalent (Burke et al., 2010; Yaemsiri et al., 2011). However, little is known about the perceptual mechanisms underlying these phenomena.

To students of perception, these observations are reminiscent of a phenomenon known as visual adaptation that has been documented for hundreds or arguably thousands of years (Verstraten, 1996). Adaptation involves prolonged exposure to an "adaptation" stimulus, causing an aftereffect of distorted perception for similar "test" stimuli. Although the original examples involved extended viewing of motion in a particular direction, leading stationary stimuli to appear to drift in the opposite direction (Addams, 1834; Wohlgemuth, 1911), equivalent effects have been documented for other simple stimulus attributes such as orientation (Gibson and Radner, 1937), color (Helmholtz, 1924), depth (Köhler and Emery, 1947), and spatial scale (Blakemore and Sutton, 1969). Such adaptation effects can be brief, resulting from relatively short exposures, or, when the initial exposure is prolonged or when frequent reexposure is involved, they can be far longer lasting (Webster, 2015).

While these aftereffects provide a degree of amusement, their scientific appeal lies in their ability to reveal the underlying neural mechanisms of perception. The extended period of exposure is known to affect the response properties of neurons in the relevant visual areas of the brain, leading to a temporary imbalance of activity, and a consequent perceptual bias (Barlow and Hill, 1963; Mollon, 1974; Ibbotson, 2005; Krekelberg et al., 2006; Thompson and Burr, 2009). Changes of stimulus attributes can cause observable variations in aftereffect magnitude, allowing us to infer the less easily observable neural processes that underpin these aspects of visual perception, such as the stimulus selectivity of cell populations. For example, by presenting adaptation stimuli in one eye and test stimuli in the other, several researchers have investigated the extent to which stimuli in each eye are processed by common mechanisms (e.g., Wade, 1976). While an absence of aftereffect indicates an independent monocular mechanism for each eye, a substantial "cross adaptation" effect (i.e., evidence of the aftereffect when the adaption and test stimuli appear in different eyes) indicates a binocular processing mechanism. In addition, contingent adaptation effects can provide evidence of a separation of mechanisms for different stimulus types. The original example (McCollough, 1965) involved concurrent adaptation to vertical orange stripes and to horizontal blue stripes, following which vertical test patterns showed a blue-green color aftereffect while horizontal patterns appeared orange. That the aftereffect is contingent on the orientation of the test stimulus suggests that color is processed separately in visual channels that are selective for orientation.

More recently, aftereffects have been demonstrated for more complex stimulus attributes such as the structural properties of a face (Webster and MacLin, 1999; Gwinn and Brooks, 2013). For example, participants who are exposed for a period of time to contracted faces (wherein the features have been digitally altered to be spaced closer together than usual) subsequently see unaltered faces as expanded. Alongside their implications for neural processing, in psychological terms these "high-level" effects are thought to reveal updates to perceptual norms, i.e., changes in the properties of faces that are considered to be average (Rhodes et al., 2005). In addition to face aftereffects, a small number of studies have demonstrated similar phenomena in human bodies. For example, adaptation to a male (female) body (Palumbo et al., 2013) or face (Palumbo et al., 2015) can cause an androgynous body to appear feminine (masculine). Further, participants exposed to images of bodies that have been digitally edited to be thinner than usual subsequently see unaltered bodies as wider than they really are, with reductions of perceived size for those exposed to artificially widened bodies (Winkler and Rhodes, 2005; Glauert et al., 2009; Hummel et al., 2012a,b, 2013; Brooks et al., in press; Mohr et al., 2016). These effects appear to be robust to the specific method of stimulus manipulation, with early studies employing simple changes of aspect ratio (Winkler and Rhodes, 2005), while more recent reports have used synthetic/computer generated images (Glauert et al., 2009), computer-manipulated photographs of human torsos (Hummel et al., 2012a,b, 2013; Mohr et al., 2016), or actual photographs of full human bodies (Robinson and Kirkham, 2014). Further, a recent report has shown independent aftereffects of fat and muscle using morphed stimuli designed to isolate these independent dimensions of body composition (Brooks et al., in press).

While the visual aftereffect paradigm has proven effective for these stimuli, as yet few studies have used this technique to probe the details of the representation of human bodies. In the current study, we present two experiments aimed at probing the extent to which the size of one's own, or of other people's bodies is processed by common, independent, or partially overlapping neural mechanisms. In addition to its theoretical interest in terms of underlying neural structures and perceptual processes, this issue has implications for attempts to account for body image distortion amongst the general public and in clinical populations using visual adaptation processes. For visual adaptation to be considered to be a viable experimental model of body image distortion, some degree of overlap of the representation of self and other would be essential to allow the transfer of aftereffects from media images to the perception of one's own body.

### EXPERIMENT 1: CONTINGENT ADAPTATION

The contingent adaptation paradigm has previously been used to infer that the perception of facial structure is mediated by separate neural substrates for faces that differ in terms of race, sex, gender, and even species (Little et al., 2005; Jaquet and Rhodes, 2008; Gwinn and Brooks, 2013, 2015a,b). In the current experiment, we employ the same technique, simultaneously adapting participants to full body images of the self and other that have been manipulated in opposite directions to appear either fatter or thinner than normal. If the perception of body size is mediated by a single neural population for all identities, we should expect both of the opposing adaptation stimuli to affect this mechanism, with their combined effects canceling out. This should result in little or no observable aftereffect for either test stimulus type, and similar results for both conditions. On the other hand, if the processes underlying the perception of body size are to some degree separate for self and other, then the two adaptation stimulus types should affect these separate processes independently.

### Methods Participants

Twenty-four female Caucasian students enrolled in an undergraduate Psychology course at Macquarie University received course credit for their participation. Participants' ages ranged from 18 to 29 years (M = 19.5, SD = 2.5). All participants had normal or corrected-to-normal vision and gave written informed consent before agreeing to participate. All experiments were conducted in accordance with the declaration of Helsinki and were approved by the Macquarie University Human Research Ethics Committee.

#### Design

The experiment employed a 2 × 2 mixed factorial design, with one within-subjects and one between-subjects independent variable. The within-subjects variable was test identity, with two levels: self and other. The between-subjects variable was adaptation condition with two levels: expanded self/contracted other (Self+/Other−) and contracted self/expanded other (Self−/Other+). The dependent variable was the point of subjective normality (PSN: the stimulus size that appeared normal), which was measured twice—once before (baseline) and once after adaptation. The difference between these two scores (PSN change) was calculated for each individual and the means across participants were subjected to analysis in a factorial ANOVA. Specifically, after adaptation to expanded images of the self and contracted images of the other body, a larger image of the self would be expected to be subjectively normal, whereas a smaller image of others will appear normal. Conversely, for participants adapted to contracted images of the self but expanded images of the other body, the point of subjective normality should be decreased for the self, but increased for others. This pattern of results corresponds to a cross-over interaction in statistical terms.

#### Stimuli

Full body images were created from digital photographs of each participant. Participants maintained a neutral expression and posed in standard anatomical position (standing upright, feet positioned approximately at shoulder width, arms straight at the side of the body, hands ∼20 cm from the body with palms facing forward, with neutral facial expressions: see **Figure 1**). All wore

FIGURE 1 | Examples of test stimuli. Seven examples, at −30, −20, −10, 0, +10, +20, and +30% size manipulation levels are shown. Although faces have been omitted to preserve subject anonymity, they were fully visible in all parts of each experiment.

standard tight-fitting gray cycling shorts and a gray singlet so that the body shape of each individual was visible. Long hair was pulled back and held in place by a hair band and all jewelry and make-up was removed. Photography took place within a booth painted in Munsell N5 neutral gray located in a room devoid of other lighting. The booth was illuminated with 15 high accuracy d65 fluorescent Philips tubes mounted in high frequency fixtures to reduce the effects of flicker. The camera (Canon EOS 70D) settings (exposure, white balance, ISO, zoom, etc.) were held constant. The photograph displayed the participants' whole body including the head, to ensure recognition of their own images. Weight and height were recorded to obtain a measurement of Body Mass Index (mean BMI = 23.0, SD = 4.2). Participants were randomly assigned one unfamiliar "other" body from an existing database of images collected under identical conditions, based on a match of similar BMI and age.

To simulate variations of body size, several versions of each participant's body photograph were created in Adobe Photoshop CS6. Participants' whole bodies from neck to ankles were either horizontally contracted or expanded using the "spherize" function. A feathered marquee allowed these distortions to be blended smoothly into the unmanipulated areas of the image to avoid image discontinuities. Importantly, the size of participants' heads remained un-manipulated to preclude face adaptation effects. Thirteen test images were created, involving spherize manipulations of between −30 and +30% in 5% steps, including the original image (0%). Test images were formatted to a standard height of 720 pixels and a width of 320 pixels. Examples of 7 of these stimuli (omitting −25, −15, −5, +5, +15, and +25% spherize levels due to space constraints) can be seen in **Figure 1**. Expanded and contracted adaptation images were more extreme, at −50% (**Figure 2A**) and +50% (**Figure 2B**) respectively. These were formatted to a standard height of 960 pixels and width of 426 pixels—33% larger than test images—so as to minimize the potential contribution of low-level retinotopic adaptation. All images were presented using Matlab <sup>R</sup> version 7, operating Psychophysics Toolbox extensions (Kleiner et al., 2007) on a Dell P1130, 21′′ color monitor, viewed from a distance of 110 cm.

#### Procedure

The psychophysical task was conducted 2–7 days after stimulus collection. The experiment included image familiarization and a short practice phase followed by a baseline block prior to the adaptation block, and took ∼20 min to complete.

Before familiarization, participants were presented with a photograph of the face of a potential "other" partner matched closely for BMI and age, and were asked to indicate whether this person was familiar to them. If participants indicated that they were familiar with the person, another face was shown until an unfamiliar "other" could be established. Participants were then instructed to familiarize themselves with the bodies shown in unmanipulated 13 × 18 cm photographs of the self and their partnered "other" for 1 min. Participants were given an additional min to observe these images between the baseline and adaptation phases.

For baseline data collection, a "yes-no" psychophysical procedure was used. In each trial a body stimulus appeared on the screen for 1 s, followed by a gray screen and a brief tone to

FIGURE 2 | Examples of adaptation stimuli. (A) Expansion condition at +50%. (B) Contraction condition at −50%.

indicate that a response was required. Participants used a twobutton mouse to indicate whether they thought the image seemed larger or smaller than the unmanipulated images to which they had been familiarized. Participants were encouraged to respond as quickly and as accurately as possible, following which the next trial began after a 100 ms inter trial interval.

The level of stimulus expansion/contraction of the test stimulus used in each trial was guided by a double interleaved 1-up-1-down adaptive staircase routine. Hence, if a participant rated an image as appearing expanded (contracted), the next time that body was seen a more contracted (expanded) version was presented and vice versa. This continued until a "reversal" was obtained at which point a more expanded (contracted) version was presented. Beginning with a ±50% adjustment in expansion/contraction (e.g., going from −30% contraction to +20% expansion), the step size was reduced by 15% after each reversal until the minimum 5% step size was reached (after the third reversal). Each staircase terminated after eight reversals at which point the mean distortion level of each participant's last six reversals was calculated to represent the PSN. Each staircase proceeded until it reached a maximum of 30 trials. If participants did not reach eight reversals, with consistent responses to the extreme test stimulus, the PSN was recorded at the appropriate extreme of the range of test stimulus values. Two interleaves were run concurrently in a randomized fashion for each test body with one starting at −30 and one at +30%, with each participant's baseline PSN calculated as the average of the PSNs from each staircase.

Adaptation data collection followed the baseline block. This progressed in the same manner as baseline testing with the following exceptions. Testing began with a 120 s initial adaptation phase, during which participants were exposed to alternating images of self and other (2 s each presentation: 60 s total for each stimulus type). In between each trial, 6 s of top up exposure, with the two alternating adaptation stimuli visible for 1 s each (total 3 s), ensured that adaptation was maintained before the next test body was presented.

#### Results and Discussion

The main dependent variable—the change of PSN between baseline and adaptation blocks—is plotted in **Figure 3**. Means, calculated across participants, are all positive following adaptation to expanded stimuli (i.e., the "self " test condition for the "self-expanded/other-contracted" adaptation group, and the "other" test condition for the "self-expanded/other-contracted" adaptation group). Hence, unmanipulated bodies appeared to be abnormally thin, and therefore needed to be expanded to appear normal. The opposite pattern of results is observed following adaptation to contracted stimuli (the "other" test stimulus for the "self-expanded/other-contracted" adaptation group, and the "self " test stimulus for the "self-expanded/other-contracted" adaptation group). These informal observations were confirmed by a statistically significant interaction between adaptation condition and test stimulus in a 2-way ANOVA [F(1, 22) = 10.19; p = 0.004; η 2 <sup>p</sup> <sup>=</sup> 0.317], demonstrating the presence of a contingent aftereffect. As expected, neither the main effect of adaptation condition [F(1, 22) = 0.003; p = 0.958; η 2 <sup>p</sup> < 0.0005] nor of test stimulus [F(1, 22) = 0.113; p = 0.740; η 2 <sup>p</sup> <sup>=</sup> 0.005] proved to be significant.

Although the size of the shifts in PSN were small (compare with examples of stimuli given in **Figure 1**), the significant effects for these data demonstrate, for the first time, the presence of simultaneous aftereffects in opposite directions contingent on stimulus identity. This supports the notion that the perception of body size is mediated, at least to some extent, by dissociable neural mechanisms that can be adapted independently of each other. In addition, this is consistent with the idea that different bodies are compared to different perceptual norms.

#### EXPERIMENT 2: CROSS ADAPTATION

The observation that contingent body size aftereffects can be established suggests that the neural processes underlying the perception of body size for self and other are at least to some degree separate. However, the degree of separation is not clear. While these results are consistent with a system in which the processing of bodies of self and other are mediated by completely independent processes, it is also possible that they represent the operation of a system with overlapping mechanisms. For example, if perceived body size is determined by neural structures that are strictly selective for self and other in concert with others that are not selective for stimulus type, only the latter pool of neurons would experience any "cancelation" of the opposite

adaptation effects. This would lead to a small net aftereffect demonstrating the effects in stimulus selective neurons. We sought to further investigate the degree of independence of the mechanisms underlying the processing of these stimulus categories using a cross adaptation paradigm, using a single adapting stimulus, and assessing its effects on both stimulus types.

If the same neural structures are involved when judging the size of the self and other, then aftereffects of the same magnitude would be expected regardless of whether the same stimulus is used for adaptation and test (e.g., adapt with self, test with self; or adapt with other, test with other), or whether these two stimuli are dissimilar (e.g., adapt with self, test with other; or adapt with other, test with self). This is known as cross adaptation. However, if the neural mechanisms for the perception of body size in self and other are entirely independent, then aftereffects established with one adaptation stimulus should be seen when testing with the same stimulus type only, with no transfer of this aftereffect to a test stimulus of a different type. The observation of partial transfer, i.e., significant, but nevertheless smaller aftereffects when adapting and testing with different stimulus types, would indicate an intermediate degree of overlap between the neural populations.

Although previous experimental reports of body size aftereffects evince a causal link between exposure and the perception of body shape and size, for this aftereffect to be considered a realistic model of real-world cases of body image distortion, it would be necessary to demonstrate transfer between the identity of other, typically unfamiliar bodies (e.g., models in fashion magazines) and images of one's self. In addition to reinforcing the link between laboratory results and real-world body image distortion, such a cross adaptation effect would allow us to infer a degree of overlap between the neural substrates that are involved in processing each stimulus type that could not be revealed by the contingent adaptation paradigm. Although one study (Hummel et al., 2012b) has assessed the degree of transfer between identities, this study used grayscale images of the self that did not show any explicit identifying information (torso only), and which the participants may not have perceived as representing their own body. Furthermore, simple adaptation and cross adaptation experiments were performed under different conditions (e.g., substantial differences between the clothing worn by bodies serving as adaptation stimuli) in Hummel and colleagues' research, rendering comparisons problematic and making inferences regarding common/independent mechanisms equivocal. In the following experiment, we use easily identifiable color photographs of the full body including the photographic subject's head and face, captured under identical conditions, to compare the magnitude of simple and cross adaptation aftereffects.

#### Methods

Methodological details for this experiment were identical to those for Experiment 1 except in the following respects. We used a 2 × 2 × 2 mixed factorial design with one within-subjects and two between-subjects independent variables. The within-subjects variable—test identity—involved two levels: "self " and "other." The first between-subjects variable—adaptation identity—had the same two levels: "self " and "other," while the second betweensubjects variable—adaptation direction—also had two levels: "expanded" and "contracted." Thirty-five female participants were randomly allocated to each of these four conditions. Initial adaptation lasted 120 s, during which participants viewed a single adaptation stimulus featuring a particular combination of adaptation direction (expansion or contraction) and adaptation identity (self or other). In between each trial, 6 s of top up exposure ensured that adaptation was maintained before the next test body was presented.

As in experiment 1, we expect to see a main effect of adaptation direction, as adaptation to an expanded and a contracted image are expected to yield opposite effects. While complete transfer of the aftereffect between others and the self would appear unlikely given the results of experiment 1, it would be revealed by equivalent adaptation effects for both test stimuli regardless of which adaptation stimulus is used, and hence no additional significant ANOVA effects. However, partial transfer (i.e., observations that aftereffects are less intense when adapting and testing with different stimuli, compared to adapting and testing with the same stimuli) would be represented by a crossover interaction between adaptation identity and test identity for the expanded adaptation conditions, with the mirrorimage pattern of results for contracted adaptation conditions. This corresponds to a three-way interaction between the three independent variables.

#### Results and Discussion

The change of PSN from baseline that is caused by adaptation is plotted in separately for the different adaptation directions in **Figure 4**. Means, calculated across participants, are all positive following adaptation to expanded stimuli, indicating that unmanipulated stimuli appeared to be abnormally thin, and therefore needed to be expanded to appear normal (**Figure 4A**). The opposite pattern of results is shown following adaptation to contracted stimuli (**Figure 4B**). In addition, aftereffects appear to be smaller when different identities are used as adaptation and test stimuli. These observations were confirmed in a 2 × 2 × 2 mixed ANOVA, which showed a significant main effect

of adaptation direction [F(1, 31) = 94.498; p < 0.0005; η 2 <sup>p</sup> <sup>=</sup> 0.753] and a significant three-way interaction between adaptation direction, adaptation identity and test identity [F(1, 31) = 11.731; p = 0.002; η 2 <sup>p</sup> <sup>=</sup> 0.275].

While the three-way interaction demonstrates that aftereffects have a different magnitude when adapting and test stimuli involve the same body identity (self or other), this can be more clearly communicated by replotting the data. Given that the different adaptation directions (expanded and contracted) are expected to produce different directions of aftereffect, these conditions can be combined by multiplying all PSNs from the expanded adaptation conditions by −1. Further, the remaining four conditions involving the use of self or other as adaptation and test stimuli can be combined into two conditions which involve adapting and testing with the same, or with different body identities. Such a plot is shown in **Figure 5**, demonstrating that the change in PSN is smaller when adapting and testing with different stimuli, such that the transfer of the aftereffect is only partial, at 65%. A paired-samples t-test confirms the significance of this finding [t(34) = 3.378, p = 0.002].

While the effects detailed above are relevant to the issue of the independence of neural mechanisms, to test the viability of visual adaptation as a model for various types of real-world body image distortion, we wished to assess the presence of significant aftereffects in each individual condition. For this reason, 8 one-sample t-tests were performed. These yielded significant results in all but one instance, namely the condition in which participants viewed an expanded image of themselves, but were tested with an image of an other's body (p = 0.249). All other conditions produced p-values in the range 0.00008–0.012, as shown in **Figure 4**.

Overall, these results confirm that adaptation to images that have been manipulated to appear thinner or fatter than normal are effective in creating aftereffects of perceived body size. Further, effects caused by prolonged exposure to distorted

images of the self can transfer to the bodies of others and, more importantly, exposure to thin or fat unfamiliar others can transfer to the self. The latter observation may have implications for the occurrence of body image distortion in real-world settings, including clinical psychology settings.

### GENERAL DISCUSSION

In two experiments, we have demonstrated the phenomenon of body image misperception through exposure to bodies that had been digitally manipulated to appear fat or thin. The aftereffect of exposure to such images for a period as short as 1 min was sufficient to cause subsequently seen images to have a contrasting appearance, such that exposure to thin images made unmanipulated images appear enlarged, and vice versa. While effects such as these are not unprecedented (Winkler and Rhodes, 2005; Glauert et al., 2009; Hummel et al., 2012a,b, 2013; Robinson and Kirkham, 2014; Brooks et al., in press; Mohr et al., 2016), experiment 1 is the first to show simultaneous aftereffects of exposure to fat and thin images contingent on the identity (self vs. other) of the test stimulus. This adaptation effect was successful regardless of whether images of the self were thin and images of the other identity were fat or vice versa. This suggests that the two identity types are compared, at least to some extent, against different perceptual/psychological norms, while establishing at least a partial dissociation of the neural mechanisms encoding body size for self and other. Extending this finding, experiment 2 investigated the extent to which the mechanisms encoding the body size of self and other are separate. The results demonstrate substantial transfer of the aftereffect between images that explicitly depict the participant's own body and those of other bodies (see also Hummel et al., 2012b). This cross adaptation effect is reciprocal: the effect transfers from exposure to others' bodies affecting the perception of one's own body, and from exposure to one's own body influencing the perception of another's body. However, since the effect was smaller when the body used for exposure and the body used for testing were different, compared to when the same stimulus is used for both, the transfer is not complete. That the aftereffect showed only partial transfer (in this case, a 65% transfer) implies that the neural mechanisms responsible for encoding body size for one's own body and the body of others show a degree of overlap.

While the current study presents evidence of dissociable neural populations processing the size of one's own and others' bodies, it is not able to locate the relevant structures. Previous studies have identified various cortical areas that are particularly active when bodies are viewed (compared to their responses to other stimuli). These areas include the extrastriate body area (EBA), the fusiform body area (FBA), the right middle occipital gyrus (rMOG), the lateral occipital cortex (LOC), the middle frontal gyrus (MFG) and the inferior parietal lobe (IPL), amongst others. Some of these areas [EBA and to some extent FBA: (Hodzic et al., 2009; Aleong and Paus, 2010); LOC and MFG: (Mohr et al., 2011)] appear to show a modulation of activity when viewing bodies of different sizes, making them candidates for the mediation of body size perception. Other areas (right IPL, right EBA, left FBA: Hodzic et al., 2009) show selectivity for different identities. Further, changes in the neural responses that accompany aftereffects of perceived body size have been demonstrated by Hummel et al. (2013). These authors found reductions in BOLD responses in the FBA and the rMOG with repeated presentation of images of their own bodies, while EBA showed no such adaptation effect. However, as this experiment lacked a condition wherein bodies were presented but no size aftereffect would be expected (e.g., using a normal body as the adaptation stimulus), it is not clear whether the imaging results reflect body size judgments, or simply the expected general reduction in activity that accompanies stimulus repetition. Aleong and Paus (2010) also demonstrated adaptation to body stimuli depicting other unfamiliar individuals in areas EBA and FBA, although the relationship between this activity and size aftereffects is similarly unclear. Importantly, these authors also demonstrated a partial release from this adaptation effect when the subject's identity was changed. The latter finding indicates some sensitivity of EBA and FBA to this attribute of bodies, and raises the possibility that this activity underlies the separate processing of the body size of self and other identified in the current study.

While the current research employed many different identities, each participant saw only two: their own body and the body of an unfamiliar other. This leaves open the possibility that the effects seen here are not specific to the self, but rather would be observed regardless of the two specific identities used, so long as these are unambiguously distinct. It is noteworthy that similar caveats have been acknowledged for a related study involving the perception of the faces of the self and other (Rooney et al., 2012). While further research replacing the body of the self with the body of a second unfamiliar individual would address this question, we chose to concentrate on the examples of aftereffect transfer that may be most relevant to real-world examples of body image misperception in the current study. The observation that adaptation to thin others causes observers to regard their own objective bodies as larger than normal corresponds to the established, real-world phenomenon whereby some individuals—including those with eating disorders and other clinical manifestations of body image disturbance—tend to over-estimate their body size (Groesz et al., 2001; Levine and Murnen, 2009). In this case, where exposure to the thin ideal in the mainstream media takes the place of our 2-min adaptation period, and self-observation in a mirror takes the place of our on-screen test stimuli. We have also presented evidence that equal and opposite effects can be seen when the exposure is to larger individuals, such that one's own normal body appears abnormally thin. This size underestimation of body size may correspond to effects reported for overweight adults residing and/or working in areas with high obesity prevalence (Burke et al., 2010; Powell et al., 2010), and perhaps children and adolescents with overweight friends and/or parents (Maximova et al., 2008). While experimental tests of these effects have been conducted (e.g., Robinson and Kirkham, 2014), to our knowledge the current study is the first to demonstrate the transfer of the aftereffect between exposure to the large bodies of others and judgments of one's own body.

While the duration of exposure to adapting images in the present study was relatively brief, yet sustained (e.g., 1 min of constant stimulation), it is clear that patterns of exposure for images in daily life are rather different. Media saturation may lead to many years of exposure to "ideal" bodies, although exposure is unlikely to be constant during this period. However, these differences do not constitute a serious challenge to the viability of visual adaptation as an experimental model of body image distortion. Although we chose a time course that suited our experimental investigation, in general, adaptation effects have been demonstrated across many different timescales, involving exposure and aftereffect durations ranging from milliseconds to months (Webster, 2011, 2015). In general, longer durations of adaptation are associated with longer lasting aftereffects, while frequent re-exposure has been identified as a factor that tends to further prolong the duration (Ditye, 2015). Hence, it is reasonable to hypothesize that "real world" body adaptation effects shown by some individuals may be larger than those induced in the current research.

A potential limitation involves our participant sample, who were all young women taken from an undergraduate sample. In part, this sub-group may be the most relevant, as they are considered to be at higher risk of developing body dissatisfaction issues, eating disorders and associated pathologies. This was deemed appropriate for an initial study, given that body size misperception and associated pathologies are overrepresented in this demographic (Mond et al., 2013; Mitchison and Mond, 2015). Further, there are specific reasons for the exclusion of certain other groups. Older participants were excluded in view of the fact that body shape changes with age, as does the way in which bodies change as they increase in mass. Although the effect of adaptation to self and other has not yet been tested in males, other data from related studies in our laboratory show that young males are also susceptible to body size aftereffects (Brooks et al., in press; Stephen et al., in submission). While we have no reason to believe that the results reported here would be any different either for older female observers or for males, further research will be needed to test this hypothesis.

The fact that participants with high levels of body image concern were not specifically targeted in the current study also needs to be considered when interpreting the current study findings. It has recently been reported that, despite success for normal observers, experimenters were not able to adapt eating disorder patients to images of thin bodies to create an aftereffect that their own body is larger (Mohr et al., 2016). While there was variance in results within this group, there was also a significant negative correlation between aftereffect size and scores on eating disorder questionnaires. However, the same group of participants did show aftereffects in the opposite direction when exposed to fat stimuli. As the authors of this study noted, these findings are consistent with the idea that those with clinical issues may already be "pre-adapted" through their usual daily diet of body images, precluding further adaptation effects.

While there are substantial differences between the strict laboratory conditions that were imposed for this experiment and real-world examples of body image distortion, and although our participants were clearly very different from those who suffer from clinical forms of body image disturbance, we believe that this paradigm may have utility in terms of the development of an experimental model of body image distortion. With further experimentation to confirm the viability of the paradigm, it may be possible to make observations that can inform health promotion programs and/or clinical practice. As a preliminary example, if exposure to extreme body types can contribute to body image misperception, it may be considered inadvisable to assemble sufferers with the same extreme body types, such as those with the eating disorder anorexia nervosa, for group treatment. Although some authors have already voiced concerns over this relatively common practice, objections usually relate to examples of weight loss competition, increased social pressure from within the inpatient group, or to collaboration over weight loss strategies between sufferers of anorexia nervosa (Colton and Pistrang, 2004; Vandereycken, 2011; Allison et al., 2014). The results of this study suggest that such congregations could also be harmful in the sense of exacerbating individuals' body image disturbance, further reducing motivation to gain weight. While visual adaptation may be substantial under such circumstances,

#### REFERENCES


its effects are unlikely to be elucidated through the use of selfreport questionnaires, diagnostic interviews or other methods that are commonly used in these situations.

#### AUTHOR CONTRIBUTIONS

KB: responsible for the conception and design; the acquisition, analysis and interpretation of data; original drafting and final approval of the submitted version. JM: substantial contributions to the interpretation of data, critical revisions for important intellectual content, and final approval of the submitted version. RS: substantial contributions to the interpretation of data, critical revisions for important intellectual content, and final approval of the submitted version. IS: substantial contributions to the interpretation of data, critical revisions for important intellectual content, and final approval of the submitted version.

#### ACKNOWLEDGMENTS

All authors were supported by a Macquarie University Research Development Grant (MQRDG 9201401548). Thanks to Kristy Kolc and Trudy Green for assistance with data collection.


High-Level Vision, eds C. W. G. Clifford and G. Rhodes (Oxford: Oxford University Press), 17–45.


Mollon, J. D. (1974). After-effects and the brain. New Sci. 61, 479–482.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Brooks, Mond, Stevenson and Stephen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# No Effect of Featural Attention on Body Size Aftereffects

Ian D. Stephen1,2,3 \*, Chloe Bickersteth<sup>1</sup> , Jonathan Mond<sup>4</sup> , Richard J. Stevenson1,3 and Kevin R. Brooks1,3

<sup>1</sup> Department of Psychology, Macquarie University, Sydney, NSW, Australia, <sup>2</sup> ARC Centre of Excellence in Cognition and its Disorders, Macquarie University, Sydney, NSW, Australia, <sup>3</sup> Perception in Action Research Centre, Faculty of Human Sciences, Macquarie University, Sydney, NSW, Australia, <sup>4</sup> Centre for Health Research, School of Medicine, Western Sydney University, Campbelltown, NSW, Australia

Prolonged exposure to images of narrow bodies has been shown to induce a perceptual aftereffect, such that observers' point of subjective normality (PSN) for bodies shifts toward narrower bodies. The converse effect is shown for adaptation to wide bodies. In low-level stimuli, object attention (attention directed to the object) and spatial attention (attention directed to the location of the object) have been shown to increase the magnitude of visual aftereffects, while object-based attention enhances the adaptation effect in faces. It is not known whether featural attention (attention directed to a specific aspect of the object) affects the magnitude of adaptation effects in body stimuli. Here, we manipulate the attention of Caucasian observers to different featural information in body images, by asking them to rate the fatness or sex typicality of male and female bodies manipulated to appear fatter or thinner than average. PSNs for body fatness were taken at baseline and after adaptation, and a change in PSN (1PSN) was calculated. A body size adaptation effect was found, with observers who viewed fat bodies showing an increased PSN, and those exposed to thin bodies showing a reduced PSN. However, manipulations of featural attention to body fatness or sex typicality produced equivalent results, suggesting that featural attention may not affect the strength of the body size aftereffect.

#### Edited by:

Stefania D'Ascenzo, University of Bologna, Italy

#### Reviewed by:

Srimant Prasad Tripathy, University of Bradford, UK Nicolas Davidenko, University of California, Santa Cruz, USA

> \*Correspondence: Ian D. Stephen ian.stephen@mq.edu.au

#### Specialty section:

This article was submitted to Perception Science, a section of the journal Frontiers in Psychology

Received: 27 May 2016 Accepted: 02 August 2016 Published: 22 August 2016

#### Citation:

Stephen ID, Bickersteth C, Mond J, Stevenson RJ and Brooks KR (2016) No Effect of Featural Attention on Body Size Aftereffects. Front. Psychol. 7:1223. doi: 10.3389/fpsyg.2016.01223 Keywords: body perception, adaptation aftereffects, featural attention, body fatness, body size misperception

## INTRODUCTION

Body size misperception is the perceptual effect wherein a person's view of their body size is inaccurate. That is, they view themselves as larger or smaller than they really are. This has implications for people who fail to recognize that they are overweight, and thus are less likely to take steps to lose weight, increasing their risk of diabetes and hypertension (Powell et al., 2010). Body size misperception is also associated with anorexia nervosa (Stice, 2002), body dissatisfaction, negative affect, eating-disordered behavior and poor mental health (Stice et al., 2003; Paxton et al., 2006) in underweight and normal weight people who perceive themselves to be overweight (McCreary et al., 2004).

Body misperception is often attributed to exposure to unrealistic body ideals (thin for women, lean and muscular for men; Furnham et al., 2002), such as those presented in the mass media. Malkin et al. (1999) found that 94% of covers in a survey of 69 American women's magazines featured a thin-idealized subject. Research has found that exposing women to extremely thin or fat bodies significantly alters their perception of body normality and ideals (Glauert et al., 2009).

This is supported by other studies demonstrating that both longterm and short-term exposure to idealized stimuli (i.e., thin bodies for women and muscular bodies for men) is associated with negative body image in women (Groesz et al., 2002) and men (Barlett et al., 2008). More recent evidence suggests that exposure to such images can affect observers' perceptions of what makes an attractive female body (Stephen and Perera, 2014).

Whilst the effects that occur after exposure to these images have been repeatedly demonstrated, little is known about the perceptual mechanisms behind body misperception. Recently, studies under controlled laboratory conditions have shown a causal link between exposure to thin bodies and body size misperception, implicating a visual adaptation effect in the process (Winkler and Rhodes, 2005; Glauert et al., 2009; Brooks et al., 2015; Sturman et al., submitted). Adaptation is the wellstudied perceptual phenomenon whereby prolonged exposure to an "extreme" visual stimulus leads to an aftereffect such that a "normal" stimulus appears distorted in the other direction. For example, in motion perception, exposure to downward motion results in a subsequently viewed stationary scene appearing to drift upward (Addams, 1834). In color perception, adapting to the color green results in neutral stimuli being perceived as red because red and green are perceptually opposed in the human visual system (Padgham, 1953; Hurvich and Jameson, 1957). These are examples of low-level aftereffects, as they involve simple stimulus attributes that are known to be processed early in the visual system.

However, in recent research the adaptation phenomenon has been demonstrated in higher-level properties of stimuli, including the gender (Webster and MacLeod, 2011; Hummel et al., 2012b), race (Webster et al., 2004), identity (Rhodes et al., 2013), and geometric structure (Gwinn and Brooks, 2013, 2015a,b) of faces. These face aftereffects are found even when the adaptation and test stimuli are presented at different locations on the screen, or at different sizes, indicating that they are not retinotopic, as would be expected of low-level effects (Leopold et al., 2005; Rhodes et al., 2011).

Aftereffects are also found in body stimuli, including adaptation to higher-level percepts such as gender (Palumbo et al., 2013, 2014). Many studies finding that after exposure to thin or fat bodies, perceptions of body normality and ideals are significantly changed (Winkler and Rhodes, 2005; Glauert et al., 2009; Hummel et al., 2012a,b). For example, observers who adapted to a picture of their own body that had been manipulated to appear thinner perceived a subsequently presented picture of their undistorted body as being fatter than it actually was, and vice versa (Hummel et al., 2012b). Body size aftereffects cannot be induced by exposure to wide or narrow rectangles (Hummel et al., 2012a), suggesting that, as for faces, body size aftereffects are attributable to adaptation of higher level brain mechanisms, rather than to simple mechanisms earlier in the visual system representing stimulus shape.

While humans in a given geographical area are exposed to approximately the same 'set' of bodies in the environment, not all individuals suffer from body size misperception. The reasons for this difference in susceptibility have yet to be established. One possible candidate may be the degree of attention that is paid to the environmental stimuli, with some individuals selectively attending to the size of bodies more than others. More recently, it has been shown that, while eating disordered patients show normal adaptation aftereffects following exposure to images of fat bodies, they do not show the expected aftereffects following exposure to images of thin bodies, suggesting that "pre-adaptation" to thin bodies may be a characteristic of eating disorders (Mohr et al., 2016).

Research suggests that increased attention to a particular stimulus increases the neural responses to that stimulus (Pestilli et al., 2007). This increased neural activity is accompanied by increased levels of neural adaptation, as shown by demonstrations that the motion aftereffect (Rezec et al., 2004), the figural aftereffect (Yeh et al., 1996), and the direct tilt aftereffect (Spivey and Spirn, 2000) are all increased by spatial attention (attention directed to the location of the stimulus). Object attention (attention directed to a particular object within a space) has also been shown to enhance the direct tilt aftereffect (Spivey and Spirn, 2000), and the figural aftereffect (Shulman, 1992). Similarly, featural attention has been found to increase the strength of the motion aftereffect (Lankheet and Verstraten, 1995; Boynton et al., 2006).

In faces, increased object attention enhances the strength of the identity adaptation effect (Rhodes et al., 2011), suggesting that other high-level aftereffects may also be susceptible to enhancement by attention. However, a recent report in the current volume shows that featural attention to the ethnicity or gender of faces does not impact the strength of aftereffects along ethnicity and gender dimensions (Davidenko et al., 2016). Yet little is known about the impact of featural attention to body fatness on body size aftereffects.

Here, we assess the impact of featural attention on the body size adaptation effect. All observers will see the same adaptation stimuli, but in the fatness attention condition they will rate the bodies according to their adiposity, whereas in the sex typicality attention condition observers will rate the bodies according to their sex typicality. We predict that selective attentional bias (i.e., attending different featural aspects of bodies) will modulate the magnitude of body size aftereffects. Observers engaging in a task explicitly rating body fatness are predicted to experience stronger body size adaptation effects than those who rate the same bodies for a non-fatness attribute (sex typicality).

Previous studies on the body size adaptation effect have tended to use stimuli that are either computer generated bodies (Glauert et al., 2009), or used simple width (Winkler and Rhodes, 2005) or simulated surface area (Hummel et al., 2012a) as a proxy for body fatness. Here, we use photographs of real people, manipulated along an empirically measured body fat axis to enhance ecological validity.

### MATERIALS AND METHODS

All work was approved by the Macquarie University Human Research Ethics Committee. All observers gave prior informed consent in writing.

### Participants

fpsyg-07-01223 August 18, 2016 Time: 15:3 # 3

Throughout this paper, we refer to participants who were photographed for the stimuli as "subjects," and to participants who give perceptual responses as "observers." One hundred and ninety two Caucasian subjects aged 18–30 (M = 20.76, SD = 5.35) were recruited (128 females, 64 males). Subjects received course credit or \$20 for their time. Eighty-nine Caucasian observers, aged 18 and 29 (M = 21.4, SD = 3.02) were recruited (45 males, 46 females). Observers were compensated for their time with course credit or \$10.

### Design

This experiment used a 2 × 2 × 2 between-subjects design, with three independent variables: attention (fatness or sex typicality), adiposity of adaptation stimuli (high or low fat) and sex. Male observers saw only male images and female observers saw only female images, as we are primarily interested in the adaptation to own sex stimuli. The dependent variable was change in the Point of Subjective Normality (1PSN) from baseline to adaptation testing. PSN was measured using a method of adjustment task that allowed observers to manipulate the adiposity of a body to make them look as normal as possible.

### Stimuli and Apparatus

Full body photographs of the subjects were taken and used to create the stimuli. Subjects wore a standard gray tight fitting singlet and gray tight fitting shorts provided by the researcher. They were asked to pull their hair back, to remove all jewelry and skin make-up, and to pose in anatomical position (upright, legs shoulder width apart, facing the camera with arms by their side, palm facing forward) with a neutral facial expression. Markings on the floor were used to ensure a standardized position for all subjects.

Photographs were taken in a light booth painted Munsell N5 neutral gray and illuminated with 15 Verivide F20 T12/D65 daylight simulation bulbs mounted in high-frequency fittings to reduce the effects of flicker (Verivide, UK; Stephen et al., 2011). No other light source was present in the room. Photographs were taken using a Canon D50 digital SLR camera with the settings held constant.

A Tanita SC 330 body composition analyser was used to accurately measure body fat and muscle mass. Height was measured manually using a fixed measuring tape. One hundred and thirty landmark points were delineated on the body images using Psychomorph (Tiddeman et al., 2001; **Figure 1**).

Since body fat mass is correlated with muscle mass and height, linear regression was used to estimate the fat mass of the subjects, controlling for muscle mass and height, following Brierley et al. (2016).

The 10 female bodies with the highest and the 10 female bodies with the lowest fat mass (controlling for muscle mass and height) were selected. Independent sample t-tests showed that these two groups differed significantly in fat mass [Mdiff = 12kg; t(18) = 4.10; p = 0.001], but not muscle mass [Mdiff = 1.5kg; t(18) = 0.96; p = 0.350] or height [Mdiff = 2.27 cm; t(18) = 0.94; p = 0.362].

FIGURE 1 | A female body image delineated with 130 landmark points.

The 10 female bodies with the highest fat mass (controlling for muscle mass and height) were used to produce a high fat composite image in PsychoMorph. This procedure involved finding the mean coordinates for each landmark across the 10 subjects to form an average shaped template. Each subject image was then warped into the shape of the average template, and the mean color was calculated across the 10 subject images at each pixel to form an average high fat mass residual female image. A similar procedure was used to create an average low fat mass residual female image from the 10 lowest fat mass residual subject images. These average images formed endpoints for the transformations.

All images were aligned in PsychoMorph to remove variation in translation and rotation. They were also resized to 600 × 900 pixels. The background of all images was changed to be a uniform gray using Photoshop.

Following Stephen and Perera (2014), 50 female subject images were then transformed in PsychoMorph according to the difference between the two endpoint composite images. This was achieved by calculating the difference in location between the two endpoint images for each landmark point to form a vector. For

each subject image, each landmark point was moved along the corresponding vector to simulate increased or decreased fat mass. These transformations were conducted in 13 equidistant steps. The smallest body image was thinner than the original by 100% of the difference between the endpoint images (−12 kg), while the largest was fatter by 100% of the difference between the endpoint images (+12 kg). The middle image was the unmanipulated original photograph, and each step represented a change of 2 kg fat mass (see **Figure 2**).

This process created images that retained the identity cues of the original images, but varied in terms of apparent body fat composition. Faces were blurred using the Photoshop pixelate mosaic function (which averaged pixel color over a square 45 × 45 pixels in size), to render individuals' faces unidentifiable and to obscure facial shape changes introduced by the transforms.

The process was then repeated for the 10 highest and 10 lowest fat male images (controlling for muscle mass and height). Independent samples t-tests from the male data showed that these two groups differed significantly in fat mass [Mdiff = 12 kg; t(18) = 2.48; p = 0.023], but not muscle mass [Mdiff = 0.04 kg; t(18) = 0.01; p = 0.995] or height [Mdiff = −2.21 cm; t(18) = 0.61;

p = 0.548]. The transformations were conducted on 50 male identities in the same way as the female images.

Ten male and 10 female subject identities were randomly selected to be used in the PSN method of adjustment tasks, with the remaining identities to be piloted for use as adaptation stimuli.

#### Pilot

For the 40 remaining male and 40 remaining female identities, seven of the 13 frames per identity (excluding every other image in the sequence) were used in a pilot study. Observers were asked to rate how fat each body appears, using the full range of a 7-point Likert scale ranging from very thin to very fat. Ten Caucasian males (aged 18–30) completed a pilot of male images and 10 Caucasian females (aged 18–30) completed a pilot of female images. Each observer therefore rated 280 (7 frames × 40 identities, presented in random order) of his/her own gender.

GraphPad Prism (Version 6; 2015) was used to assess the distribution of fatness ratings across the frames for each subject identity separately. In the first step, cumulative Gaussian functions were fitted to the fat and thin ends of the continua separately (excluding the original image), and the values and 95% confidence intervals for the best fitting curve parameters were compared, to ensure that the ratings for each end were symmetrical (i.e., that one step at the thin end was perceptually equivalent in magnitude to one step at the fat end of the transform). For three male and three female identities, functions fitting the fat and the thin ends of the data set differed significantly. These identities were not used in the main experiment.

A second Gaussian function was then fitted to the whole of the continuum for each subject identity separately. The point at which the line crossed the middle of the perceptual rating scale was identified as the perceived "normal" fatness. The transformations in PsychoMorph were then rerun using the "normal" point attained from the Prism data as the new "zero."

In instances where the "normal" point for the identity was greater than ±45% of the original transform (14 females, 11 males), that identity was not used for the adaptation phase. 20 female and 20 male subject identities were used in the adaptation phase. Images used for fat adaptation were 10 kg of apparent fat mass above the "normal" point for the identity, and images for the thin adaptation were 10 kg of apparent fat mass below the normal point for the identity.

#### Procedure

#### Baseline Phase

A method of adjustment app was coded in Matlab with the Psychophysics Toolbox, which allowed observers to cycle through the frames of transforms by moving the mouse horizontally, giving the appearance that observers were manipulating the adiposity of the subject identities. Observers were presented with 10 same sex subject identities (twice each) and were informed that moving the mouse horizontally would change the appearance of the body (they were not told it was a fat transform) and were instructed to "make each body look as normal as possible" before clicking to save the data and move onto the next identity. The mean fat mass chosen as "normal" was defined as the baseline PSN for the observer. Identities were presented in random order, and the starting frame was randomized.

#### Adaptation

In the adaptation phase, observers viewed 20 either high fat or low fat own-sex images, presented twice each in a random order. While viewing these images, they were asked to either rate the fatness (fatness attention condition) or the sex typicality (sex typicality attention condition) of the image using the whole range of a 13-point Likert scale. In the fatness attention condition, the scale ranged from very thin (1) to very fat (13). In the sex typicality attention condition, the scale ranged from very feminine (1) to very masculine (13). Immediately following this, observers completed another PSN method of adjustment task consisting of 10 same sex subject identities twice each in a randomized order. Observers were again asked to "make the body as normal as possible" and by moving the mouse from left to right they were able to adjust the bodies to establish their adapted PSN. After every three PSN trials, a "top up" adaptation stimulus was presented, for which the observer was asked to make the same Likert scale rating as during the initial adaptation phase. No set time limits were used for trials, but participants took approximately 5 s per adaptation or "top up" trial and 11 s per PSN trial. Thus, the adaptation phase lasted approximately 3.5 min and, during the post-adaptation phase, participants experienced a 5 s "top up" trial for every approximately 15 s of PSN trials.

### RESULTS

#### PSN Data

The change in PSN (1PSN) between baseline and adaptation was calculated and used as the dependent variable (see **Figure 3**). Positive values represent a shift to a fatter PSN, while negative values represent a shift to a thinner PSN. The responses of one outlier in the thin adaptation condition (1PSN greater than 2.5 standard deviations from the thin mean), were excluded from further analysis, though the pattern of results was similar with the outlier included.

SPSS Version 22 was used for all analyses. A 2 (attention) × 2 (sex) × 2 (adiposity) between-groups ANOVA was conducted with 1PSN as the dependent variable.

A significant main effect of adiposity was found [F(1,81) = 98.78, p < 0.001, η 2 <sup>p</sup> = 0.55]. One samples t-tests confirmed that observers in the fat adaptation condition had a significantly positive 1PSN [M = 2.30, SD = 2.52, t(44) = 6.11, p < 0.001], while the 1PSN for observers in the thin adaptation condition was significantly negative [M = −2.07, SD = 1.66, t(43) = 8.28, p < 0.001], demonstrating aftereffects in the predicted directions (**Figure 3**).

A significant main effect of gender was found (F1,81) = 5.78, p = 0.018, η 2 <sup>p</sup> = 0.07, with males' mean 1PSN being positive (M = 0.69, SD = 3.27), while females' was negative (M = −0.42, SD = 2.75). The interaction between gender and adiposity was

are in the predicted directions.

marginally significant [F(1,81) = 3.88, p = 0.052, η 2 <sup>p</sup> = 0.05]. Independent samples t-tests showed that men (M = 3.23, SD = 2.06) showed a stronger aftereffect than women (M = 1.33, SD = 2.64) in the fat adaptation condition [t(43) = 2.71, p = 0.010] (though correcting for multiple comparisons made this result non-significant. No significant differences were found between the magnitude of men's (M = −1.98, SD = 1.87) and women's (M = −2.17, SD = 1.47) aftereffects in the thin adaptation condition [T(62) = 0.38, p = 0.707].

The interaction between adiposity and attention condition was not significant [F(1,81) = 0.07, p = 0.792, η 2 <sup>p</sup> = 0.00], suggesting that featural attention does not affect the magnitude of the body size adaptation effect.

All other main effects and interactions were non-significant and not relevant to our hypotheses (all F < 2.6, all p > 0.11).

To test whether the magnitude of aftereffects differed depending on whether observers adapted to fat or thin stimuli, the 1PSN data for the thin condition were multiplied by (−1), while data for the fat condition were left untransformed. Thus positive numbers represented a change in the predicted direction in both fat and thin adaptation conditions. An independentsamples t-test showed no significant difference in the magnitude of fat (M = 2.30, SD = 2.53) and thin (M = 2.07, SD = 1.66) aftereffects for both sexes combined, [t(76.27) = 0.50, p = 0.616]. However, since differences were seen between men and women in the magnitude of aftereffects, separate independent samples t-tests were performed for male and female observers. No significant difference was seen between the magnitude of fat (M = 1.33, SD = 2.64) and thin (M = 2.17, SD = 1.47) aftereffects for women [t(32.81) = 1.31, p = 0.200]. For men, the fat aftereffect (M = 3.23, SD = 2.06) was significantly larger than the thin (M = 1.98, SD = 1.87) aftereffect [t(43) = 2.14, p = 0.038]. However, the p-value for this effect in males is non-significant when alpha is Bonferroni adjusted for multiple comparisons.

Since the time course of body size aftereffects is not well known, we repeated the analysis on just the first five adapted PSN trials (25%) and on just the last five adapted PSN trials (25%) performed by each participant. The pattern of results did not differ from that presented above, though the interaction between gender and adiposity (marginally significant p = 0.52 above) was non-significant for both the first five trials (p = 0.139) and the last five trials (p = 0.636). Next, the 1PSN data for the thin condition were multiplied by (−1), while data for the fat condition were left untransformed. Thus positive numbers represented a change in the predicted direction in both fat and thin adaptation conditions. A paired samples t-test showed no significant difference between the magnitude of aftereffects in the first and last five trials [t(71) = 0.467, p = 0.642]. This suggests that the "top up" trials were sufficient to prevent any significant decay of the aftereffect over the time course of the adaptation PSN trials.

#### Rating Data

Two separate 2 (time) × 2 (size) mixed ANOVAs were performed on the first 10 trials (25%) and last 10 trials (25%) of the body rating data from the adaptation phase – one for the fat attention condition, and one for the sex typicality attention condition. For the sex-typicality attention condition, the main effects of size and time, and the interaction between the two were all non-significant (all F < 0.66, all p > 0.42), suggesting that adapting to fat or thin bodies did not affect the perception of the sex typicality of the bodies.

For the fat attention condition, the main effect of size was significant (F1,35) = 71.10, p < 0.001, η 2 <sup>p</sup> = 0.67), showing that bodies in the thin condition were rated as thinner than the bodies in the fat condition. The main effect of time [F(1,35) = 1.61, p = 0.213, η 2 <sup>p</sup> = 0.04] was non-significant. A significant interaction between time and size was found [F(1,35) = 16.60, p < 0.001, η 2 <sup>p</sup> = 0.32]. Follow-up independent-samples t-tests

showed that thin bodies were rated as thinner than fat bodies for both the first 10 trials [thin M = 4.37, SD = 1.11, fat M = 8.41, SD = 1.42, t(35) = 9.66, p < 0.001] and the last 10 trials [thin M = 5.47, SD = 1.06, fat M = 7.83, SD = 1.60, t(35) = 5.30, p < 0.001]. Paired samples t-tests showed that, for the thin condition, bodies were rated as less thin in the last 10 trials (M = 5.48, SD = 1.06) than in the first 10 trials [M = 4.37, SD = 1.11, t(18) = 3.91, p = 0.001]. For the fat condition, a marginal effect suggested that bodies were rated as less fat in the last 10 trials (M = 7.83, SD = 1.60) than in the first 10 trials [M = 8.41, SD = 1.42, t(17) = 1.92, p = 0.072]. This suggests that participants became adapted to the size of the bodies that they were rating over the adaptation phase, viewing the bodies as less extreme at the end of the phase than at the beginning.

### DISCUSSION

We report the results of an experiment in which observers' featural attention was manipulated by asking them to rate either the fatness (fatness attention condition) or sex typicality (sex typicality attention condition) of fat or thin adaptation stimuli, and measured the resultant change in PSN. Aftereffects were found in the predicted directions in both attention conditions and in male and female observers, in line with previous studies (Winkler and Rhodes, 2005; Glauert et al., 2009; Hummel et al., 2012a,b). These adaptation effects were symmetrical in magnitude for female observers. A marginal interaction effect between gender and adiposity may have raised suspicions that fat and thin adaptation aftereffects may not be symmetrical in magnitude for male observers, who showed a larger mean PSN change after adapting to fat stimuli compared to thin stimuli. However, the post hoc t-tests were not significant when Bonferroni corrected, and this effect was not found when analyzing the initial five adapted trials, or the final five adapted trials. Therefore, we have no evidence to suggest that aftereffects are larger for fat or thin adaptors.

Manipulations of featural attention to the fatness or sex typicality of bodies had no discernible effect on the strength of the body size aftereffect. While both spatial attention (Yeh et al., 1996; Spivey and Spirn, 2000; Rezec et al., 2004) and object attention (Shulman, 1992; Spivey and Spirn, 2000) have been shown to affect the strength of low level aftereffects, and object attention has been shown to impact the strength of face identity aftereffects, our results suggest that featural attention does not have the same effect on body stimuli. While featural attention has been found to affect the strength of the low level motion aftereffect (Lankheet and Verstraten, 1995; Boynton et al., 2006), our results add to recent findings that featural attention does not affect the strength of adaptation aftereffects in higher level stimuli, such as faces (Davidenko et al., 2016). It may be that when the different complex featural dimensions of the target object (such as fatness vs. sex typicality) are differentiated by variations in the appearance of the same local aspects of the image (such as width of hips or width of waist), attention cannot selectively modulate adaptation to the two complex featural dimensions.

Our results suggest that increasing featural attention to body size (by rating fatness vs. rating sex typicality) does not increase the size of the body size aftereffect. It may be the case, however, that our attentional manipulation was ineffective at inducing participants to attend to cues to fatness (in the fatness attention condition) and non-fatness cues (in the sex typicality attention condition), due to overlapping concepts of fatness and sex typicality. While there is an association between sex and fat mass (the healthy range for young Caucasian women is 21–33% fat, but for men it is 8–21% fat; Gallagher et al., 2000; Frankenfield et al., 2001), the variation in body shape associated with sex typicality (femininity is associated with wider hips, narrower waist, larger breasts; Furnham et al., 1998) is substantially different to the shape variation associated with body fat variation within sex (fatter people have wider hips, wider waist, larger breasts, larger stomach; Cornelissen et al., 2009). This suggests that different body cues are important when judging fatness and sex typicality, and that the two are unlikely to be conflated. Further, the analysis of the rating data from the adaptation phase showed that, while exposure to fat or thin bodies was associated with a reduction in how fat or thin the bodies were perceived to be across the adaptation phase, this exposure to fat or thin bodies was not associated with a change in the perceived sex typicality of the bodies across the adaptation phase. This suggests that fatness and sex typicality may be processed by separate channels in the brain.

Our results also have a number of implications for the techniques used in research into body aftereffects. The method of adjustment task was successful in measuring the PSN for body size, allowing us to successfully detect aftereffects in all conditions. This technique provides a quicker, more efficient alternative to the staircase tasks that are typically employed in research into aftereffects (e.g., Hummel et al., 2012a,b; Gwinn and Brooks, 2013, 2015a,b; Brooks et al., 2015). We also used a more realistic technique for manipulating the apparent fatness of bodies than has been used in previous body adaptation studies. By manipulating images of real people along an empirically derived body fat axis, our stimuli give a more realistic depiction of how people's bodies change as their body fat levels change (Brierley et al., 2016; Sturman et al., submitted). We suggest that this technique provides a more ecologically valid method for producing stimuli than those that rely on simple geometric transforms, such as widening the bodies or by graphics manipulations that simulate an increase of the surface area.

### CONCLUSION

In a visual adaptation paradigm, we have demonstrated the utility of a more ecologically valid technique for manipulating stimuli along a biologically relevant, empirically derived body fat axis (Brierley et al., 2016; Sturman et al., submitted), and established a novel method of adjustment task to quickly and efficiently measure PSNs (Sturman et al., submitted). By using these techniques, we have detected body size aftereffects in the predicted directions, in men and women observing fat and thin adaptation stimuli. Featural attention toward body fatness (vs. sex typicality) was not found to affect the strength of the aftereffect, suggesting that object attention is sufficient to produce the effect. Indeed, passive viewing of high or low fat bodies has been reported to induce body size adaptation aftereffects (Brooks et al., 2016).

### AUTHOR CONTRIBUTIONS

fpsyg-07-01223 August 18, 2016 Time: 15:3 # 8

Conception of the project: IS, CB, JM, RS, and KB. Design of the project: IS, CB, JM, RS, and KB. Acquisition of data: IS, CB, and KB. Interpretation of data: IS, CB, JM, RS, and KB. Drafting of manuscript: IS, CB, JM, RS, and KB. Final approval of manuscript: IS, CB, JM, RS, and KB. Agreement to be accountable for the work: IS, CB, JM, RS, and KB.

### REFERENCES


### FUNDING

This project was funded by a Macquarie University Research Development Grant held by IS and JM. The funders exerted no control at any stage of the research.

### ACKNOWLEDGMENTS

The authors wish to acknowledge Dan Sturman for programming support; Kristy Kolc, Dan Sturman, Rachael Johnston, Mary-Ellen Brierley, and Robyn Ordman for contributing to stimulus collection.



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Stephen, Bickersteth, Mond, Stevenson and Brooks. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Emotional Actions Are Coded via Two Mechanisms: With and without Identity Representation

Joanna Wincenciak <sup>1</sup> \* † , Jennie Ingham<sup>1</sup> , Tjeerd Jellema<sup>1</sup> and Nick E. Barraclough<sup>2</sup> \*

*<sup>1</sup> Department of Psychology, University of Hull, Hull, UK, <sup>2</sup> Department of Psychology, University of York, York, UK*

#### Edited by:

*Rocco Palumbo, Schepens Eye Research Institute, Massachusetts Eye and Ear, USA*

#### Reviewed by:

*Christopher P. Benton, Bristol University, UK Chiara Fini, CNC Centre of Cognitive Neuroscience, Italy*

#### \*Correspondence:

*Joanna Wincenciak joanna.wincenciak@ncl.ac.uk; Nick E. Barraclough nick.barraclough@york.ac.uk*

#### †Present Address:

*Joanna Wincenciak, Institute of Neuroscience, Newcastle University, Newcastle upon Tyne, UK*

#### Specialty section:

*This article was submitted to Perception Science, a section of the journal Frontiers in Psychology*

Received: *15 January 2016* Accepted: *25 April 2016* Published: *11 May 2016*

#### Citation:

*Wincenciak J, Ingham J, Jellema T and Barraclough NE (2016) Emotional Actions Are Coded via Two Mechanisms: With and without Identity Representation. Front. Psychol. 7:693. doi: 10.3389/fpsyg.2016.00693*

Accurate perception of an individual's identity and emotion derived from their actions and behavior is essential for successful social functioning. Here we determined the role of identity in the representation of emotional whole-body actions using visual adaptation paradigms. Participants adapted to actors performing different whole-body actions in a happy and sad fashion. Following adaptation subsequent neutral actions appeared to convey the opposite emotion. We demonstrate two different emotional action aftereffects showing distinctive adaptation characteristics. For one short-lived aftereffect, adaptation to the emotion expressed by an individual resulted in biases in the perception of the expression of emotion by other individuals, indicating an identity-independent representation of emotional actions. A second, longer lasting, aftereffect was observed where adaptation to the emotion expressed by an individual resulted in longer-term biases in the perception of the expressions of emotion only by the same individual; this indicated an additional identity-dependent representation of emotional actions. Together, the presence of these two aftereffects indicates the existence of two mechanisms for coding emotional actions, only one of which takes into account the actor's identity. The results that we observe might parallel processing of emotion from face and voice.

Keywords: action perception, emotion, adaptation, social perception, social cognition

### INTRODUCTION

Information about the identity and emotional state of other individuals can be derived from a variety of social cues, including the face, the voice, the body, and whole body actions. The accurate perception of these cues underlies our ability to function successfully in a complex social environment. To date, models have been developed to explain how identity and emotion are derived from faces (Bruce and Young, 1986; Haxby et al., 2000) and voices (Belin et al., 2004; Campanella and Belin, 2007). However, our understanding of how we process identity and emotion derived from bodily actions is limited, despite the importance of action information in social cognition (de Gelder, 2006; de Gelder et al., 2010; Jellema and Perrett, 2012).

Early theories of face perception have argued that recognition of emotional expression and identity is performed independently, and processed in separate subsystems (Bruce and Young, 1986). However, during social interaction, it is important to be able to evaluate the differing changes in the emotional expressions of specific individuals engaged in the interaction (Baseler et al., 2012) and thus link identity and expression together. Indeed various lines of evidence have suggested an interaction between the processing of facial expression and face identity (see Calder and Young, 2005).

For example, single unit recordings in the monkey have shown that a proportion of neurons in the temporal cortex (Perrett et al., 1984; Hasselmo et al., 1989) and within the amygdala (Gothard et al., 2007) respond selectively to the conjunction of specific facial expressions and specific facial identities. Furthermore, psychophysical evidence (Schweinberger and Rudiger, 1998; Fox and Barton, 2007; Ellamil et al., 2008; Campbell and Burke, 2009) has supported the notion of a partially integrated processing of emotion and identity information. In addition, recent neuroimaging (e.g., Gobbini and Haxby, 2007; Vuilleumier and Pourtois, 2007; Baseler et al., 2012) and neuropsychological studies (e.g., de Gelder et al., 2003) have indicated that processing of facial emotion and facial identity can interact; also see Calder and Young (2005) for a discussion.

In a similar vein, voice identity and voice emotion have been proposed to be processed by separate, functionally independent systems (Belin et al., 2004; Spreckelmeyer et al., 2009). As for faces, these systems are not wholly independent, and the interactions between them may allow for voice emotion being processed with reference to the identity of the speaker (Belin et al., 2004).

In contrast, our understanding of how we conjointly process information about emotion and identity derived from body actions is still unclear and models of visual processing of actions ( e.g., Giese and Poggio, 2003; Fleischer et al., 2013) have not been developed to take into account these factors. Therefore, in the current study, we investigated if identity played a role in the perception of emotional actions using visual adaptation paradigms. Visual adaptation results from prolonged exposure to a specific visual stimulus causing an aftereffect consisting of a bias in perception. Typically, perception of a test stimulus is biased toward the opposite of the adapted stimulus. Examining the characteristics of the aftereffects allows us to determine the properties of the underlying visual processing mechanisms, as has been used previously to understand the representation of other complex social stimuli like faces (e.g., Leopold et al., 2001; Webster et al., 2004; Fox and Barton, 2007), biological motion (e.g., Troje et al., 2006; Roether et al., 2009), implied motion (Lorteije et al., 2007), and actions (Barraclough et al., 2009; Barraclough and Jellema, 2011).

In our experiments, participants adapted to whole-body actions performed in either a happy or sad manner and then subsequently categorized test actions that were carried out in a neutral manner. Both physical characteristics of the body movement such as speed, acceleration or force, and more subtle relationships between the position of the joints inform about the type of expressed emotion and thus distinguish movements performed in happy or sad manner (e.g., Dittrich et al., 1996; Pollick et al., 2001, 2003; Roether et al., 2009). We chose happy and sad emotions for our adapting stimuli as they are opposite in valence with distinctive emotion-specific features (see Roether et al., 2009 for details). We predicted that emotional action adaptation would result in subsequent test actions to appear opposite in emotion to the emotion conveyed by the adapting action. We tested the role of identity in the emotional action aftereffects by varying the identity of the adapting and test actors. Following previous research (Kohn, 2007; Verhoef et al., 2008), we predicted that the magnitude of the aftereffect would depend upon the degree of similarity between the adapting and test stimuli. Additionally, in order to differentiate any putative emotional action aftereffect from other potential mechanisms, we performed three control experiments. First, to rule out the potential contribution of post-perceptual response and decision biases (cf. Hsu and Young, 2004) to the emotional action aftereffects we employed a 4AFC task in the adaptation experiment. Here we tested whether participants categorized test actions as conveying the opposite (cf. Leopold et al., 2001; Skinner and Benton, 2010), rather than a different emotion to that of the adapting action. Second, as adaptation aftereffects show characteristic build-up and decay over time (e.g., Hershenson, 1989; Leopold et al., 2005; Barraclough et al., 2012, but see Storrs, 2015), we examined these factors in two further experiments.

### GENERAL METHOD

### Participants

Participants were University of Hull staff and students and received course credit or payment for participating. Participants had normal or corrected to normal vision. Number of participants tested in each experiment was based upon previously reported sample size in action adaptation studies (Experiment 1; c.f. Barraclough and Jellema, 2011; Barraclough et al., 2012) and determined by the power analysis on data obtained in pilot studies (Experiments 2–4). All participants were naïve to the purpose of the study except authors JI (Experiment 1) and JW (Experiments 2 and 4). Excluding authors' data from the analysis did not influence the significance of any results and thus any conclusions drawn. Experiments were approved by the ethics committee of the Department of Psychology, University of Hull, and performed in accordance with the ethical standards laid down in the 1990 Declaration of Helsinki.

### Stimuli

We created two sets of emotional action stimuli by filming actors performing different actions whilst conveying happy and sad emotions in varying degrees of intensity, as well as the same actions conveyed with no emotion (neutral). Emotion was conveyed by the way the actions were executed rather than by executing specific emblematic actions (e.g., see Atkinson et al., 2004; Keefe et al., 2014). Actors were free to interpret and express each emotion as they wished. We did not want to instruct actors to move in a specific way to ensure that a full range of naturalistic bodily expressions were represented in our stimulus sets. Furthermore, by allowing non-stereotyped bodily expressions of emotion from our actors, it was possible to maintain the unique kinematics of actions that help characterize the identity of the individual actors.

#### Stimulus Set 1

Sixteen (eight females, eight males) lay actors with no specific acting experience were filmed. All actors were Caucasian and were students at the University of Hull. We recruited lay actors as previous studies (e.g., Roether et al., 2009; Keefe et al., 2014) show emotions portrayed by professional and lay actors are not significantly different from each other; lay actors portray emotional expressions that are easily recognized and similar to expressions of professional actors (e.g., Wallbott, 1998; Atkinson et al., 2007; Grèzes et al., 2007). Actors were filmed (Canon XL1s, 720 × 576 pixels, 25 fps, progressive scan) performing four different actions: lifting a box, putting down a box, sitting down on a chair, and walking to the right. Actors were asked to perform these actions in an emotionally neutral manner and whilst conveying happy and sad emotions in two levels of increasing intensity: moderate (somewhat happy/sad) and high (very happy/sad). Actors wore plain clothes (e.g., jeans and a dark t-shirt or jumper) and were filmed with their head at a threequarter rear view (the face turned away from the camera), so that facial expressions could not be seen. This was done to ensure that the emotional expression and actor's identity are both conveyed only by bodily cues (shape and action kinematics) and not by the clothing or by the face. Recordings were edited to generate 320 (16 actors × 4 actions × 2 emotions × 2 intensity levels, plus neutral) 2 s long movies. These movies were subsequently rated by 14 independent observers (11 females, 3 males; mean age M = 21.9 years, SD = 4.58 years) on a five-point Likert scale, where 5 = very happy and 1 = very sad. The 64 actions (16 actors × 4 actions) rated as being most happy (M = 4.27, SD = 0.50), and the 64 actions (16 actors × 4 actions) rated as being most sad (M = 1.65, SD = 0.46) were used as adapting stimuli. The 64 neutral actions (16 actors × 4 actions; M = 2.88, SD = 0.28) were used as test stimuli.

#### Stimulus Set 2

Stimuli in set 2 were selected from the Action Database (Keefe et al., 2014; http://www-users.york.ac.uk/~neb506/databases. html). This database contains movies of semi-professional actors with moderate acting experience (i.e., drama students at the University of York) and lay actors with no previous acting experience (i.e., non-drama students and staff at the University of York). As the lay actors were as able to portray different emotions through their bodily action similarly to the semi-professional actors (see Keefe et al., 2014 for details), for this study we selected sixteen (eight females and eight males) Caucasian actors. The full details of the filming and validation procedure for these stimuli are described in Keefe et al. (2014). Briefly, actors were filmed (Panasonic, HDC-TM900, HD format 1920 × 1080 pixels, at 50 fps progressive scan) in a chroma-green room walking diagonally to the right so that facial expressions could not be seen. They portrayed this action in an emotionally neutral manner whilst conveying emotions in three levels of increasing intensity: low (slightly happy/sad), moderate (somewhat happy/sad), and high (very happy/sad). All actors wore a gray t-shirt and dark trousers to ensure that actor's identity was conveyed by the bodily cues (shape and action kinematics) and not by the clothing. For this stimulus set, a mood induction procedure consisting of an autobiographical recall was used to help actors convey specific emotion during filming, and actors were free to convey the emotion as they wished. Film was edited to key out the green screen, and generate 112 (16 actors × 2 emotions × 3 intensity levels, plus neutral) 2 s long movies. Movies were subsequently rated by 10 (6 females, 4 males; mean age M = 27.1 years, SD = 9.6 years) independent observers on two independent ninepoint Likert scales, where 1 = neutral, 9 = very happy/very sad, to ensure that the adapting actions conveyed the appropriate emotion (for detials of this rating procedure see Keefe et al., 2014).

For our purposes we chose the 16 actors from the database that conveyed the happy and sad walking actions with the highest intensity. For each actor, actions rated as happy (16 movies, M = 5.31, SD = 1.34) and sad (16 movies, M = 4.56, SD = 1.02) were selected as adapting stimuli. Neutral actions (16 movies, also rated for happiness: M = 2.77, SD = 0.47, as well as sadness: M = 2.89, SD = 0.56) were used as test stimuli.

### Procedure

A PC running MATLAB 2006a (The MathWorks Inc., Natick, MA) and the Cogent toolbox (www.vislab.ucl.ac.uk/cogent.php) was used to control the experiment, display movies in the center of a 22" CRT monitor (Philips 202P40, 1600 × 1200 pixels, 100 Hz refresh rate), and record participant responses. Participants sat ∼57cm from the screen. Movies from stimulus set 1 were presented such that they filled the entire screen and subtended ∼41.3 degrees × 31.5 degrees at the eye. Movies from stimulus set 2 were presented on a black background such that the full width of the HD movie fitted the horizontal extent of the screen. To help preclude low-level retinotopic dependent adaptation, participants freely viewed all stimuli limiting stereotypic eye-movements. All experiments followed a standard adaptation procedure (**Figure 1**) where adapting movies were followed by a blank inter-stimulus interval (ISI) containing a small yellow fixation cross, followed by a single test movie. Participants were required to indicate on a labeled keypad whether the test movie conveyed happy or sad emotions (2 AFC: Experiments 1, 3, and 4) or happy, sad, disgusted or surprised emotions (4 AFC: Experiment 2).

#### Data Analysis

For each experiment and each participant and condition, we calculated the mean aftereffect (see individual sections). Even though aftereffects with complex social stimuli are typically quite variable (Barraclough and Jellema, 2011; Barraclough et al., 2012) a very small number of participants showed exceptionally large aftereffects. These effects may reflect a potential problem in the conduct of the task by these individuals (for example rating the adapting stimulus, and not the test stimulus). Data was therefore discarded from those outlying individuals with aftereffects more than 2 SD from the mean aftereffect. Whenever data from outlying participants have been excluded, and whether its exclusion influenced the significance of the results, has been described in the respective results section.

### EXPERIMENT 1: ADAPTATION TO EMOTIONAL ACTIONS

Experiment 1 tested if exposure to movies of actors portraying happy and sad emotions through their bodily actions resulted in emotional action aftereffects: a change in the perception of neutral actions biased toward the opposite of the adapted emotion. The adaptation paradigm was adapted from Fox and Barton (2007), who investigated the role of identity in emotion perception from faces. The effect of stimulus similarity on the magnitude of emotional action aftereffects was tested by varying the similarity between actions executed in the adapting and test stimuli, and by varying the similarity between the identities of the actors executing the actions.

### Method

#### Participants

Twenty participants (14 females, 6 males, mean age = 23.85 years, SD = 8.2 years) took part in the experiment.

#### Stimuli

Stimuli were selected from stimulus set 1 (see General method section): two females and two males actors performing three actions (lifting a box, sitting down, and walking). Actions performed in a happy and sad manner were used as adapting stimuli, while actions performed in a neutral manner were used as test stimuli.

#### Procedure

Stimuli were presented across five blocks, each containing 24 trials. In an initial control block without adaptation, 12 test stimuli (depicting four actors performing three actions in a neutral manner) were presented twice in a pseudorandom order. Participants categorized the emotion conveyed by the actor as either happy or sad (scored as: 1 = happy, 0 = sad). The control block was later used as a baseline for calculating the aftereffects. The control block was followed by four counterbalanced adaptation blocks, where 4 different pairings of adapting and test stimuli were employed, where the adapting and test stimuli had the same identity/same action, same identity/different action, different identity/same action, or different identity/different action. Adapting stimuli (happy and sad actions) were presented for 16 s (each movie was repeated eight times), followed by a 0.3 s ISI, and then a test stimulus presented for 2 s. Each test stimulus was presented following both happy and sad adaptation stimuli, where this factor was interleaved within the block. Participants categorized the emotion conveyed by the test actor as either happy or sad (scored as: 1 = happy, 0 = sad). Following the participant response there was an inter-trial interval (ITI) of 5 s before the start of the next trial. Mean estimates of the emotion perceived in the test stimuli were calculated separately for the control, and happy and sad adaptation conditions across the 4 stimulus similarity conditions. "Happy" and "sad" aftereffects were calculated by subtracting estimates of test stimuli in the control condition from the happy and sad adaptation conditions, respectively.

#### Results

Data from two participants (both female, both age 20) were discarded from the analysis as one showed outlying data (>2 SD from mean aftereffect) and one did not engage in the task. Removal of this data did not change the significance of the following analysis. Aftereffects were tested using a within-subject Analysis of Variance (ANOVA) with factors adaptation emotion (happy, sad) and stimulus similarity (same identity/same action, same identity/different action, different identity/same action, different identity/different action). Adaptation to happy and sad emotional action movies generated significantly different aftereffects {F(1, 17) = 11.261, p < 0.005, η<sup>p</sup> <sup>2</sup> = 0.40, 95% CIs: adapt happy [−0.21, −0.015], adapt sad [−0.01, 0.14], see **Figure 2**}, where test actions were judged as conveying the opposite emotion to the adapting stimuli, similar to other emotional aftereffects seen with biological motion stimuli (Roether et al., 2009) and facial expressions (Fox and Barton, 2007). As the similarity between the adapting and test stimuli decreased, so did the aftereffect magnitude {interaction between adaptation emotion and stimulus similarity: F(2.1, 35.7) = 4.25, p < 0.05, η<sup>p</sup> <sup>2</sup> = 0.20, 95% CIs adapt happy [−0.26, −0.04], [−0.18, 0.01], [−0.23, 0.0], [−0.20, 0.02], adapt sad [0.03, 0.19], [−0.03, 0.14], [−0.03, 0.22], [−0.11, 0.08] for each level of similarity respectively, Greenhouse-Geisser correction applied}; there was no main effect of stimulus similarity {F(2.1, 35.8) = 0.83, p > 0.05, η<sup>p</sup> <sup>2</sup> = 0.05, 95% CIs [−0.09, 0.05], [−0.08, 0.05], [−0.10, 0.08], [−0.13, 0.03], Greenhouse-Geisser correction applied}.

Happy and sad aftereffects were significantly different only in those conditions where the adapting and test stimuli contained the same action, irrespective of identity: same identity/same action {t(17) = 4.48, p < 0.001, 95% CIs [0.14, 0.39]}; different identity/same action {t(17) = 2.85, p = 0.011, 95% CIs [0.05, 0.36], Bonferroni correction applied}. For conditions where the adapting and test stimuli contained different actions, happy and sad aftereffects were not significantly different from each other, indicating that adaptation to emotional action adaptation does not transfer from one action type to another.

### EXPERIMENT 2: CONTROL FOR CHANGE IN RESPONSE CRITERION

appear sadder. Error bars represent standard error of the mean (SEM).

In Experiment 2 we clarified whether the emotional action aftereffects were due to biases in perception rather than postperceptual processes, for example biases in participant responses or decisions. We reasoned that aftereffects resulting from a perceptual process would result in test actions being categorized as conveying the opposite emotion following adaptation (see Hsu and Young, 2004; Webster et al., 2004). Here, perceptual adaptation to one emotion (e.g., happy) would result in an increase in the likelihood of only the opposite emotion (sad), rather than other possible emotions. If, however, the aftereffects resulted from either a bias in response or decision then participants would be likely to indicate that the test actions conveyed "any other" emotion to that of the adapting stimulus. In these cases, participants could in principle have reasoned that since the test stimulus was clearly different from the adapting stimulus, they should select a response option that was clearly different from the adapting stimulus.

We tested these predictions across the two conditions that generated significant aftereffects in Experiment 1 (same action/same identity and same action/different identity). Further, to check that these two emotional action aftereffects were not dependent upon any particular set of stimuli, we selected movies from stimulus set 2 (see General method).

## Method

#### Participants

Fifteen new participants (six females, nine males, mean age = 24.9 years, SD = 3.2 years) took part in the experiment, all were naïve to the purpose of the experiment except author JW, and one individual (male, age 28) tested also during Experiment 4. Removal of these data did not change the significance of the following analysis.

#### Stimuli

Stimuli were obtained from stimulus set 2 (see General method). Eight females and eight males actors performing walking actions were selected for the experiment. Walking actions performed in a happy and sad manner were used as adapting stimuli, while walking actions in a neutral manner were used as test stimuli.

#### Procedure

Stimuli were presented across four blocks; an initial control block without adaptation (the 16 test stimuli presented twice in a pseudorandom order), followed by two counterbalanced adaptation blocks (where on each trial adapting and test stimuli were paired to create same identity/same action and different identity/same action combinations of stimulus similarity), and a final control block identical to block 1. The control condition was split across two blocks (before and after adaptation), in order to obtain a stable measure of test stimulus emotion over the duration of the experiment. Adapting stimuli (happy and sad actions) were displayed for 8 s (by repeating the movie four times); the ISI was 0.4 s. Each test stimulus was presented following happy and sad adaptation. Participants categorized each test movie as happy, sad, surprised or disgusted (a 4AFC task). Surprise and disgust were selected as additional options as they vary from happiness and sadness in valence and arousal (Russell, 1980). Following the participant response there was an inter-trial interval of 5 s before the start of the next trial. For each participant, and each adaptation block, we calculated percentage of responses for each of the 4 possible emotions (happy, sad, surprised, and disgusted; cf. Skinner and Benton, 2010). We averaged the responses from both control blocks and calculated the percentage of each response as for the adaptation blocks. Aftereffects were calculated by subtracting the percentage of each response from the control blocks from percentage of each response to the same stimuli following happy and sad adaptation for same and different identity conditions separately.

#### Results

Aftereffects were tested using ANOVA with the within-subject factors of adaptation emotion (happy, sad), stimulus similarity (same identity/same action, different identity/same action) and response emotion (happy, sad, surprised, disgusted). Both the same and different identity aftereffects resulted from shifts in the perception of action emotion (rather than a bias in response or decision) as participants categorized test movies as conveying the opposite to the adapted emotion rather than "any other" emotion {interaction adaptation emotion × response emotion, F(1.81, 25.4) = 27.20, p < 0.001, η<sup>p</sup> <sup>2</sup> = 0.66; 95% CIs adapt happy [−22.05, −7.00], [7.01, 22.51], [−3.54, 2.71], [−4.72, 1.39], adapt sad [−1.07, 17.32], [−14.59, −2.07], [−2.82, 3.23], [−4.34, 5.17] for happy, sad, surprised, disgusted response option respectively, Greenhouse-Geisser correction applied, **Figure 3**}. As suggested by **Figure 3**, the same identity aftereffects were significantly bigger (M = 15.8) than the different identity aftereffects (M = 6.0). This was confirmed by a three-way interaction (adaptation emotion × stimulus similarity × response emotion, ANOVA, F(3, 42) = 14.49, p < 0.001, η<sup>p</sup> <sup>2</sup> = 0.50, see Appendix for 95% CIs). No other main effects or interactions reached significance. Post-hoc paired t-tests revealed that adaptation only affected the categorization of test stimuli as happy {same identity: t(14) = 5.88, 95% CIs [−41.51, −19.32] different identity: t(14) = 3.11, 95% CIs [23.17, 42.66], both ps < 0.0125, Bonferroni correction applied}, and sad {same identity: t(14) = 7.25, 95% CIs [−18.28, −3.37], different identity: t(14) = 3.38, 95% CIs [4.87, 21.78], both ps < 0.0125, Bonferroni correction applied} and had no effect on the choice of surprised or disgusted labels (all ps > 0.27).

### EXPERIMENT 3: DYNAMICS OF EMOTIONAL ACTION AFTEREFFECTS

The first two experiments indicated that adaptation influences perceptual mechanisms that represent emotional actions, and that these mechanisms are selective for the executed action. We find that when the identities of the adapting and test actors

emotions conveyed by test stimuli following happy and sad adaptation (positive values indicate more often selected emotion following adaptation; negative values indicate less often selected emotion following adaptation). The identity of the actors in the adapting and test stimuli were either the same (A) or different (B). Error bars represent standard error of the mean (SEM).

are the same, aftereffects are greater than when the identities of the adapting and test actors are different (**Figures 2**, **3**). Similar to the face aftereffects observed by Fox and Barton (2007) this indicates that action emotion can be represented irrespective of actor identity, but also actor identity can influence the representation of action emotion. These results suggest that there are two potential mechanisms underlying the coding of action emotion that are both susceptable to percetual adaptation. Adaptation aftereffects seen with other stimuli show a characteristic logarithmic build-up and exponential decay over time, e.g., for motion (Hershenson, 1989), faces (Leopold et al., 2005; Rhodes et al., 2007), biological motion (Troje et al., 2006), and actions (e.g., Barraclough et al., 2009). To further test whether our two emotional action aftereffects show the same characteristic dynamics as previously reported perceptual aftereffects we examined their build-up and decay over time. We tested the effect of adapting stimulus duration and the duration of the inter-stimulus interval (ISI) between the adapting and test stimuli on aftereffect magnitude in two separate, fully factorial experiments (Experiment 3a: same identity/same action aftereffect; Experiment 3b: different identity/same action aftereffect). Furthermore, these results would help further distinguish the perceptual aftereffects we observe from other post-perceptual processes.

### Method

#### Participants

Nineteen new participants (11 females, 8 males, mean age = 20.4 years, SD = 3.8 years) took part in Experiment 3a; a further 18 new participants (12 females, 6 males, mean age = 24.7 years, SD = 6.7 years) took part in Experiment 3b. All participants were naive to the purpose of the experiment.

#### Stimuli

Stimuli were obtained from stimulus set 1 (see General method). Eight females and eight males actors performing four different actions (lifting a box, putting down a box, sitting down on a chair, and walking to the right) were selected for the experiment. Actions performed in a happy and sad manner were used as adapting stimuli, while actions in a neutral manner were used as test stimuli.

#### Procedure

In both experiments we varied the duration of the adapting stimulus (2, 4, 8, 16 s) by repeating the adapting movie a number of times (1, 2, 4, 8 repeats), and varied the ISI between the adapting and test stimuli (200, 400, 600, 1600 ms in Experiment 3a; 400, 800, 1600, 3200 ms in Experiment 3b). In each experiment, therefore, there were 32 different adaptation conditions (4 adaptation repeats × 4 ISIs x 2 emotions) that occurred in a pseudorandom order. The choice of adaptation durations and ISIs in Experiment 3a was based upon previous measures of the duration of face identity aftereffects (Leopold et al., 2005) and whole body aftereffects (Barraclough et al., 2012) where significant aftereffect declines were observed over 1600 ms. Following Experiment 3a we conducted Experiment 3b, here we decided to extend the range of tested ISIs to 3200 ms

to better sample the possible decline of aftereffects over time. In Experiment 3a the identity of the actor and the type of action performed in the adapting and test movies were identical on every trial. In Experiment 3b, the type of action performed by the actor in the adapting and test movies was identical on every trial, however, the identity (and gender) of the actor was always different. Participants completed 320 trials in total over two testing sessions. Participants categorized the emotion conveyed by the test actor as either happy or sad (scored as: 1 = happy, 0 = sad). Following the participant response there was an inter-trial interval (ITI) of 5 s before the start of the next trial.

Aftereffects were calculated by subtracting the mean ratings of the test stimuli following adaptation to sad stimuli from the mean ratings of the test stimuli following adaptation to happy stimuli. This difference value represented the magnitude of the aftereffect. Here, positive values indicated typical, repulsive aftereffects, where following adaptation the emotion conveyed by the test actors looked like the opposite of the emotion conveyed by the adapting actors. Negative aftereffects values indicated that following adaptation, the emotion conveyed by the test actors looks similar to the emotion conveyed by the adapting actors. Although this method precludes identifying the relative contribution of happy and sad actions to emotional aftereffect dynamics, it reduces any systematic biases, maximizes the magnitude of any aftereffects and eliminates the necessity for a control condition without adaptation, thereby allowing us to test many experimental conditions within a limited testing period.

#### Results

Mean aftereffects for the 16 different conditions were calculated in both Experiments 3a and 3b and are plotted in **Figure 4**. Data from one participant (female, age 20) in Experiment 3a was discarded from the analysis as outlying data. Repeating the analysis with this data did not change the significance of the results when testing with ANOVA. For both Experiments 3a and 3b, all aftereffects were positive, indicating that under all conditions adaptation had a repulsive effect. The dynamics of the emotional action aftereffects, however, appeared to be different in the two experiments (compare **Figures 4A,B**). As the groups of participants and the durations of the ISIs between the two experiments were different, we analyzed the two experiments separately using 2-way ANOVAs (4 adaptation repeats × 4 ISI durations).

For Experiment 3a (same identity) increasing the duration of the adapting stimulus results in significantly larger aftereffects [main effect of adaptation repeats: F(3, 51) = 5.72, p < 0.005, ηp <sup>2</sup> = 0.25, see Appendix for 95% CIs]. There was, however, no significant effect of ISI duration [main effect of ISI: F(3,51) = 1.04, p = 0.38, η<sup>p</sup> <sup>2</sup> = 0.06, see Appendix for 95% CIs] nor a significant interaction between adaptation repeats and ISI duration [adaptation repeats × ISI: F(9, 153) = 0.90, p = 0.53, η<sup>p</sup> <sup>2</sup> = 0.05, see Appendix for 95% CIs]. For Experiment 3b (different identity) increasing the duration of the adapting stimulus results in significantly larger aftereffects [main effect of adaptation repeats: F(3, 51) = 9.94, p < 0.001, η<sup>p</sup> <sup>2</sup> = 0.37, see Appendix for 95% CIs]. Although a reduction in aftereffect

magnitude with ISI was apparent from **Figure 3B** (i.e., overall reduction in the size of bars across ISI durations), this was not significant [main effect of ISI: F(3, 51) = 1.95, p = 0.132, η<sup>p</sup> 2 = 0.10, see Appendix for 95% CIs] nor was the interaction between adaptation repeats and ISI significant [adaptation repeats × ISI: F(9, 153) = 0.90, p = 0.52, η<sup>p</sup> <sup>2</sup> = 0.05, see Appendix for 95% CIs].

The reason for the lack of significance of ISI on aftereffect magnitude in Experiment 3b may be due to a number of factors. First, following 1 repeat of the adapting stimulus there was little aftereffect at any ISI. Indeed, significant aftereffects were only found following 4 or 8 adaptation stimulus repeats (one-sample t-tests, p < 0.0031, Bonferroni correction applied). This contrasts with the analysis of the same identity aftereffects measured in Experiment 3a, where significant aftereffects were found following 1, 2, 4, and 8 adaptation stimulus repeats (onesample t-tests, p < 0.0031, Bonferroni correction applied). The small and non-significant aftereffects in Experiment 3b obtained after 1 and 2 repeats may have contributed to the non-significant main effect of ISI. Second, the range of ISIs that we tested for both Experiments 3a and 3b may have been too restricted, with aftereffects tested over a greater range of ISIs we may have found a significant decline in the magnitude of both same identity and different identity aftereffects.

In order to better illustrate the build-up and decay of the emotional action aftereffects we re-plotted the aftereffects for same and different identity conditions on a semi-log scale, as a function of adaptation repeats (**Figures 5A,B**) and ISI duration (**Figures 5C,D**; cf. Leopold et al., 2005). The lack of decay of the

aftereffects in the same identity condition raised a possibility that same identity and different identity aftereffects show a difference in storage. Storage occurs when a period of non-stimulation (e.g., darkness) is imposed between the adapting and the test stimulus. In consequence, the decay of an aftereffect is delayed as it persists for a period of non-stimulation (Thompson and Wright, 1994). Storage has previously been reported for motion aftereffects (e.g., Thompson and Wright, 1994; Watamaniuk and Heinen, 2007) and face aftereffects (Leopold et al., 2001; Hsu and Young, 2004) suggesting that emotional action aftereffects may also exhibit a storage effect. Therefore in Experiment 4 we tested the decay of emotional action aftereffect over a short (400 ms) and long (10.8 s) ISI durations.

### EXPERIMENT 4: AFTEREFFECT DURATION TESTED WITH LONGER INTER-STIMULUS INTERVALS

In Experiment 4 we wanted to assess whether the lack of significant decline with time observed in Experiments 3a and b was a function of the ISIs originally chosen, and whether the similarity of the identity of the actor in the adapting and test stimuli had an influence on the duration of aftereffects. Experiments 3a and 3b gave an indication for the duration of emotional action aftereffects following 4 repeats of adapting stimuli. At short ISIs (400 ms) we expected to see significant aftereffects in the same identity/same action condition, as well as in the different identity/same action condition. At long ISIs (10.8 s), however, we expected to see no decline in the same identity/same action condition, but a decline in the different identity/same action condition.

## Methods

#### Participants

Twenty-four new participants (16 females, 8 males, mean age = 21.5 years, SD = 3.1 years) took part in the experiment; all were naïve to the purpose of the experiment except author JW.

#### Stimuli

Stimuli were obtained from stimulus set 1 (see General methods). Four females and 4 males actors performing four different actions (lifting a box, putting down a box, sitting down, and walking) were selected for the experiment. Actions performed in a happy and sad manner were used as adapting stimuli, while actions performed in a neutral manner were used as test stimuli.

#### Procedure

Stimuli were presented across two counterbalanced blocks, where on each trial adapting and test stimuli were paired up to create same identity/same action or different identity/same action stimulus similarity conditions. Adapting stimuli (happy or sad actions) were presented for 8 s (repeated four times). Interstimulus intervals, determined in pilot testing, were 400 ms and 10.8 s. Each test stimulus was presented following happy and sad adaptation and each ISI. Participants categorized the emotion conveyed by the test actor as either happy or sad (scored as: 1 = happy, 0 = sad). Following the participant response there was an inter-trial interval (ITI) of 5 s before the start of the next trial. We calculated mean estimates of test stimuli following happy and sad adaptation separately for each of the four different conditions (identity × ISI). We then calculated aftereffect magnitudes by subtracting responses following sad adaptation from responses following happy adaptation.

#### Results

Data from two participants (male, age 19; female, age 20) were discarded from the analysis as they showed outlying data (>2 SD from mean aftereffect). Results with and without the outlying data are described below. Aftereffects were analyzed using ANOVA with within-subject factors of identity (same, different) and ISI duration (0.4, 10.8 s). The aftereffects in the same identity/same action condition were significantly larger than the aftereffects in the different identity/same action condition, as seen in the previous two experiments {**Figure 6**; ANOVA, main effect of identity, F(1, 21) = 5.74, p < 0.05, η<sup>p</sup> <sup>2</sup> = 0.25, 95% CIs [0.20, 0.44], [0.09, 0.30] for same and different identity respectively}. Importantly there was a significant interaction between identity and ISI {F(1, 21) = 4.77, p < 0.05, η<sup>p</sup> <sup>2</sup> = 0.18, 95% CIs; same identity [0.16, 0.40], [0.21, 0.51]; different identity [0.13, 0.37], [−0.01,0.28] for 0.4 s and 10.8 s ISI respectively}, in which the aftereffect in the different identity/same action condition declined over time, whereas the aftereffect in the same identity/same action condition appeared to persist. There was no main effect of ISI {F(1, 21) = 0.07, p = 0.79, η<sup>p</sup> 2 = 0.00, 95% CIs [0.16, 0.37], [0.14, 0.36] for same and different identity respectively}. Including the outlying data from the two participants who showed strong priming effects influenced only the identity × ISI interaction which became a trend {F(1, 23) = 3.66, p = 0.06, η<sup>p</sup> <sup>2</sup> = 0.14, 95% CIs; same identity [0.07, 0.36], [0.14, 0.46]; different identity [0.06, 0.33], [−0.05, 0.25] for 0.4 and 10.8 s ISI respectively}. The aftereffects in the different identity/same action condition therefore appeared to show a similar decline over time as for other high level and action aftereffects (Leopold et al., 2005; Troje et al., 2006; Barraclough et al., 2012). The aftereffects in the same identity/same action condition, however, showed a different pattern.

### DISCUSSION

Here we show that adaptation to naturalistic movies of wholebody actions results in emotional aftereffects. Adaptation to an actor conveying either happiness or sadness biases subsequent perception of emotion conveyed by either the same or another individual to appear like the opposite emotion. Importantly, we see that the characteristics of the aftereffects are dependent upon whether the subsequent individual has the same or a different identity to the adapting actor.

The effects observed here are likely to result from adaptation of high level action coding mechanisms that represent the emotions of actors. The emotional action aftereffects show several of the characteristics of previously observed high-level aftereffects. First, similarity between adapting and test stimuli determined aftereffect magnitude (see Kohn, 2007; Verhoef et al., 2008). Aftereffects were largest when the identity and the action of the adapting and test stimuli were the same. As the identity or the action differed between these stimuli, aftereffects became smaller. In addition, aftereffects failed to transfer across different actions suggesting that the aftereffects did not result from cognitive biases, where decision regarding the test stimuli could be based purely on the valence information. Second, the adaptation aftereffects observed here did not result from a change in the response or decisions made by the participants during the experiments. Experiment 2 showed that participants did not alter their response following adaptation; rather, they selectively categorized the emotion of the test stimuli as the emotion opposite to the adaptor's emotion (discussed in Hsu and Young, 2004). Third, aftereffect magnitude increased with adaptation exposure as for other high-level action and face

aftereffects (e.g., Leopold et al., 2005; Barraclough et al., 2012, but see Storrs, 2015). Fourth, the aftereffects that resulted from the different identity/same action condition appeared to decline over time, similar to other high-level action and face aftereffects (e.g., Leopold et al., 2005; Barraclough et al., 2012). The aftereffects that resulted from the same identity/same action condition, however, appeared to show a storage effect, similar to motion (e.g., Thompson and Wright, 1994; Watamaniuk and Heinen, 2007) and face aftereffects (Leopold et al., 2001; Hsu and Young, 2004).

Finally, the dominating effect here was the effect of highlevel emotion adaptation, as the aftereffect was predominantly determined by the emotion conveyed by the adapting actor. Lowlevel retinotopic dependent adaptation is unlikely to explain the effects that we observe here, as adapting and test stimuli were dynamic with naturally occurring differences in expressions of emotions and kinematics across actors. Furthermore, actors were never presented in precisely overlapping retinotopic locations (comparable to e.g., Butler et al., 2008; Xu et al., 2008). Together, these results rule out other possible low-level or postperceptual explanations for emotional action aftereffects that might account for some other demonstrations of high-level perceptual aftereffects.

We cannot, however, rule out the contribution of adaptation in motion processing mechanisms, which is likely to occur simultaneously when viewing complex action stimuli (e.g., Benton et al., 2016). Action speed is an important cue in the emotion recognition from body actions (e.g., Dittrich et al., 1996; Pollick et al., 2001, 2003; Roether et al., 2009), and is likely to have influenced the aftereffects observed here, but is unlikely to be a dominating factor. Other postural and dynamic features that do not rely on speed are also critical for expression and for perception of emotional actions (e.g., Atkinson et al., 2007; Roether et al., 2009). These postural (e.g., head angle) and dynamic (e.g., expressiveness) features present in happy and sad actions, even when these are matched for speed, are sufficient to elicit emotional action aftereffects (Roether et al., 2009). It is therefore plausible that the aftereffects observed here resulted from adaptation at multiple levels of action visual processing.

Importantly, our studies with emotional action aftereffects indicate that identity plays a role in the representations of emotional actions, and identity can modulate the way that emotional actions are processed. When the identities of the adapting and test actors were the same aftereffects were larger than when the identities of the adapting and test actors were different. Furthermore, when the identities of the adapting and test actors were the same, the aftereffects did not appear to decline over the time period we tested (up to 10.8 s). In contrast, when the identities of the adapting and test actors were different, the aftereffects declined over time. Thus, emotional action aftereffects show different magnitudes and decay functions dependent upon the relationship between the identity of adapting and test actors.

One explanation for our results is that emotional actions are represented in both an identity-dependent and identityindependent way. The aftereffects observed following adaptation in the different identity/same action condition might result from adaptation of a mechanism that represents emotional actions in an entirely identity-independent fashion. Whilst the aftereffects observed following adaptation in the same identity/same action condition might result from adaptation of mechanisms that represent emotional actions in both an identity-independent as well as an identity-dependent way. When adapting and test actors have the same identity, the aftereffects increase in magnitude and appear to last much longer than typical high-level action or face aftereffects. The increase in aftereffect magnitude could be explained by an increase in visual similarity between the adapting and test stimuli. However, this increase in visual similarity could not abolish the decay of the aftereffect over time in the way that we observe here. A parsimonious explanation is that an identitydependent mechanism is additionally adapted when adapting and test stimuli have the same identities, resulting in a larger aftereffect and a longer-lasting influence on the representations of the emotions of individual actors. Such long-lasting aftereffects are not necessarily unusual, as they have been previously reported with other social stimuli such as gaze direction (e.g., Kloth and Schweinberger, 2008) and face configuration (e.g., McKone et al., 2005; Carbon and Ditye, 2011, 2012). These previous effects, however, were observed with much longer adaptation periods than we have used in our experiment reported here. Identity has also been shown to affect the decay of face aftereffects (Kiani et al., 2014). Brief presentation of an unrelated, intervening face in the period between the adaptation and the test phase, accelerated the decay of face aftereffects by 85%, whilst presentation of a face of an opposite to the adapted identity did not affect the decay (Kiani et al., 2014). However, it is not yet clear whether other inferences could have contributed to the quick decay of aftereffects in the different identity/same action condition observed here.

For both identity-dependent and identity-independent aftereffects, adaptation never transferred across different actions. Different actions have different kinematics, and bodily expression of emotion is dependent upon the characteristic kinematics of the action (e.g., see Roether et al., 2009). The failure of adaptation to one set of characteristic movements to exert an influence on the perception of a very different set of movements, suggests that actor emotion may be coded within action specific neural mechanisms.

In some ways, these results parallel findings of face emotion aftereffects. Evidence from face adaptation studies suggest the existence of a common visual representation that underlies the coding of face identity and face expression (Ellamil et al., 2008; Rhodes et al., 2015). This common representation may contain one dimension that codes both the identity and expression, and the other dimension that is selective for identity or expression (Calder, 2011; Rhodes et al., 2015). These findings are also consistent with Haxby et al.'s (2000) model of face perception, where neural substrates that underlie processing of invariant (e.g., identity) and changeable (e.g., expression) aspects of the face are thought to be disocciable, although the model also suggests they may interact. Specifically, the fusiform face area (FFA), predominantly involved in the coding of the invariant aspects of the face, may also play a supportive role in the coding of emotional expressions (Haxby et al., 2000).

Similarly, processing of voice identity and voice emotion is thought to be processed by seperate, but interacting systems (Belin et al., 2004; Spreckelmeyer et al., 2009), where voice identity is processed within the anterior STS (Belin et al., 2004) and voice emotion is processed by several separate regions including the amygdala, anterior insula, and temporo-medial and inferior prefrontal cortex (Imaizumi et al., 1997; Buchanan et al., 2000). While early analysis of vocal input (from primary auditory cortex to the middle part of the STS) is shared for different types of vocal information, at the highest level of voice processing the pathways become independent (Belin et al., 2004).

The similarity between the aftereffects we observed with emotional action adaptation and our understanding of how faces (Haxby et al., 2000; Ellamil et al., 2008; Campbell and Burke, 2009) and voices (Belin et al., 2004) are processed suggest that similar, parallel processing systems might underlie the coding of emotion and identity information from these different social signals. Action representation within the STS can be modulated by emotional body language (Grèzes et al., 2007), and adaptation within this region (cf. Grossman et al., 2010) might underlie the emotional action aftereffect showing identity-independence. The neural substrates underlying our ability to recognize actor identity from body form and motion have not been well delineated, but may involve the Extrastriate Body Area (EBA; Saxe et al., 2006; Urgesi et al., 2007) along with regions in the anterior temporal lobe (although see Downing and Peelen, 2011). Adaptation in more anterior regions of the temporal cortex that takes into account actor identity may underlie the emotional action aftereffect that is influenced by the identity of the individual actors; although this suggestion is more speculative. Anterior cortical regions do not show the same adaptation characteristics as the more posterior cortical regions (cf. Verhoef et al., 2008). The less characteristic longer lasting adaptation effects observed here, and also seen in some face adaptation experiments (e.g., Carbon and Ditye, 2011, 2012), may result from the effects of stimulus repetition on later stages of processing inducing a longer term recalibration of the representation of individuals. Such long lasting aftereffects may shift the observer's reference point according to their recent experience in order to optimize the sensory processing of the external world over a longer period of time.

### REFERENCES


### CONCLUSIONS

This study has demonstrated that prior exposure to emotional whole-body actions influences the perception of the emotion conveyed by subsequent actions via a perceptual adaptation mechanism. Our visual adaptation experiments reveal two separate processing mechanisms for emotional actions with different characteristics: one mechanism that processes actor emotion irrespective of actor identity, and one that processes actor emotion taking into account actor identity. This organization parallels recent data on the processing of face information that suggest that rather than completely separate processing of emotion and identity, representations of emotion, and identity can interact. These mechanisms we study here would not only help us to determine the emotions of individuals around us from their actions and behavior, but also critically ensure the identity of the individual is linked to the specific emotions expressed.

### AUTHOR CONTRIBUTIONS

JW, TJ, and NB designed the research, JW, and JI carried out data collection, JW, JI, TJ and NB analyzed the data, and JW, TJ, and NB wrote the paper.

### FUNDING

This research was supported by the Economic and Social Research Council (grant number RES-062-23-2797) awarded to NB; JW was supported by an 80th Anniversary Scholarship from the University of Hull. We thank Andy Young and Bruce Keefe for comments on an earlier draft of this manuscript.

### DATA ACCESS STATEMENT

All data supporting this study are provided as supplementary information accompanying this paper.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fpsyg. 2016.00693


functional neuroimaging. Neuropsychologia 45, 174–194. doi: 10.1016/j.neuropsychologia.2006.06.003


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Wincenciak, Ingham, Jellema and Barraclough. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Adaptation to Complex Pictures: Exposure to Emotional Valence Induces Assimilative Aftereffects

Rocco Palumbo1,2 \*, Stefania D'Ascenzo<sup>3</sup> , Angelica Quercia<sup>4</sup> and Luca Tommasi<sup>2</sup>

<sup>1</sup> Schepens Eye Research Institute, Harvard Medical School, Boston, MA, USA, <sup>2</sup> Department of Psychological Science, Humanities and Territory, "G. d'Annunzio" University, Chieti, Italy, <sup>3</sup> Department of Philosophy and Communication, University of Bologna, Bologna, Italy, <sup>4</sup> Department of Neuroscience, Imaging and Clinical Sciences, "G. d'Annunzio" University, Chieti, Italy

Aftereffects have been documented for a variety of perceptual categories spanning from body gender to facial emotion, thus becoming an important tool in the study of high-level vision and its neural bases. We examined whether the perceived valence of a complex scene is subject to aftereffects, by observing the participants' evaluation of the valence of a test picture preceded by a different picture. For this study, we employed an adaptation paradigm with positive and negative images used as adapters, and positive, negative, and neutral images used as tests. Our results show that adaptation to complex emotional pictures induces assimilative aftereffects: participants judged neutral tests more positively following positive adapters and more negatively following negative adapters. This depended on the prolonged adaptation phase (10 s), as the results of a second experiment, in which adapters lasted for 500 ms, did not show aftereffects. In addition, the results show that assimilative aftereffects of negative and positive adapters also manifested themselves on non-neutral (negative and positive) targets, providing evidence that the global emotional content of complex pictures is suitable to induce assimilative aftereffects.

Keywords: adaptation, aftereffect, positive valence, negative valence, complex images

### INTRODUCTION

Aftereffects produced by the prolonged exposure to certain stimuli are a well-known phenomenon in the perceptual literature. Since Aristotle, it has been observed that maintaining fixation for a long time (adaptation) on a specific stimulus' property produces a perceptual change (aftereffect) of that specific property in a following stimulus. These effects can be subdivided into assimilative aftereffects or contrastive aftereffects (Howard and Rogers, 1995). The former type is a perceptual change on the second stimulus (named here the test) that runs in the same direction of the first stimulus (named here the adapter). The later type is a perceptual change on a test stimulus that runs in the opposite direction of the adapter one. One relevant factor to determine whether assimilation or contrastive effects prevail is stimulus type. Strong, salient, high-contrast, longduration stimuli tend to lead to contrast effects, while brief, less salient low-contrast stimuli lead to assimilative effects (Pantle et al., 2000; Kanai and Verstraten, 2005; Yoshimoto and Takeuchi, 2013). In particular, contrastive aftereffects are ubiquitous in both low- and high-level perception: the well-known "waterfall illusion," an example of a motion aftereffect, is just an example. The biological explanation of this phenomenon assumes that prolonged exposure to an object's

### Edited by:

Lars Muckli, University of Glasgow, UK

#### Reviewed by:

Victor De Lafuente, National Autonomous University of Mexico, Mexico Justin Michael Gaetano, Southern Cross University, Australia

\*Correspondence:

Rocco Palumbo roccopalumbo@gmail.com

#### Specialty section:

This article was submitted to Perception Science, a section of the journal Frontiers in Psychology

Received: 31 May 2016 Accepted: 10 January 2017 Published: 30 January 2017

#### Citation:

Palumbo R, D'Ascenzo S, Quercia A and Tommasi L (2017) Adaptation to Complex Pictures: Exposure to Emotional Valence Induces Assimilative Aftereffects. Front. Psychol. 8:54. doi: 10.3389/fpsyg.2017.00054

perceptual feature (such as its motion direction) reduces the responses of the neurons tuned to that particular feature, facilitating the perception of the feature in the opposite direction along its continuum (e.g., Barlow and Hill, 1963; Srinivasan and Dvorak, 1979; Huk et al., 2001; Thompson and Burr, 2009; Glasser et al., 2011). Many simple properties have been investigated besides motion direction, such as color (e.g., McCollough, 1965), orientation (e.g., Gibson and Radner, 1937; Dekel and Sagi, 2015), size (e.g., Blakemore and Campbell, 1969; Blakemore and Sutton, 1969), contrast, and shape (e.g., Suzuki and Cavanagh, 1998).

Aftereffects, together with other visual illusions and phenomena, represent a powerful instrument in understanding the processes that intervene among sensation, neural coding of the environment, and perception. For this reason, in recent decades, several research studies have extended the investigation to observe whether visual aftereffects occur for more complex properties, not strictly related to the physical features of the stimulus, but for more high-level cognitive representations, in particular for socially relevant properties of faces and bodies, such as gender (e.g., Webster et al., 2004; Palumbo et al., 2013; D'Ascenzo et al., 2014), age (e.g., Schweinberger et al., 2010), body representation (Cazzato et al., 2016), and emotional expression (e.g., Fox and Barton, 2007). For example, after prolonged exposure to male faces, neutral faces that were subsequently presented appeared female to the observer and vice versa (Webster et al., 2004). To further support the evidence that aftereffects can be unrelated to the physical features of the stimulus to which the subject is adapted, numerous studies have tried to investigate whether aftereffects occur across categories as well (e.g., Davidenko et al., 2008; Ghuman et al., 2010; Javadi and Wee, 2012; Kessler et al., 2013). In a study by Palumbo et al. (2015), for instance, the authors found that 10 s of exposure to male or female faces produces a gender aftereffect on the following presentation of androgynous bodies. This example suggests that adaptation to gender is not related to the physical features of the stimulus, but to the representation of gender as a more global property of bodies and faces. It should be noted that Kovács et al. (2006) investigated gender aftereffects using faces and body parts as stimuli, finding that prolonged adaptation to hands or faces produces aftereffects on hand or face test stimuli, respectively. However, no aftereffects were found when the adaptor and test stimuli belonged to different categories, suggesting that the underlying adaptation mechanisms are category-specific. Although several studies on aftereffects of socially relevant stimuli have strengthened the evidence that these effects extend to complex perceptual contents, no study has investigated the role of adaptation to more complex content on subsequent perceptions. The perceived emotional content of pictures appears as a potential candidate in the study of this phenomenon, one in which would possibly allow to carefully exclude the well-known contribution of expressive faces, that may drive the presence of aftereffects.

The ability to perceive and classify stimuli as positive or negative is one of the most crucial abilities for survival (Damasio, 1994). In order to quickly detect hazards and respond properly to the environment, it is important to make sense of the surroundings by assigning emotional value to the sensory input (Greene et al., 2015; Zebrowitz et al., 2015; Altamura et al., 2016; Mammarella et al., 2016a). That means, we relentlessly create an internal mental representation of the stimuli immediately present in our surroundings by categorizing the scene as positive or negative in valence (for a review, see Brosch et al., 2010; Mammarella et al., 2016b). Regarding the temporal dimension, which is a crucial factor in assessing any perceptual content, the effects due to the duration of exposure and the order of presentation when multiple items are presented, are aspects that have been investigated very thoroughly. However, they have been investigated in rather compartmentalized traditions belonging to perception, cognition, learning, and in applied domains, such as clinical psychology or even marketing studies. Picture valence is no exception: the effects of previous exposure to emotional stimuli have been widely investigated through several experimental paradigms, such as priming (e.g., Meyer and Schvaneveldt, 1971; Neely, 1991; Klauer and Musch, 2003), habituation (e.g., Leventhal et al., 2007), and mood conditioning (e.g., Sutton et al., 1997; Di Domenico et al., 2015, 2016). Furthermore, exposure therapy (Abramowitz, 2013), can be listed as one of the potential clinical applications of these phenomena.

However, there is a scarcity of studies that have investigated whether and how the perceived valence of complex images might be affected by adaptation to previous images that have themselves a positive or a negative valence. These types of adaptation studies would provide evidence that the emotional impression (valence) arising from any window on the environment may represent a high-level category in perceptual analysis, beyond the wellknown evidence concerning the specific aspect of facial emotion adaptation. The fact that adaptation to primary and secondary features, such as orientation or motion of simple stimuli, can produce aftereffects or, that adaptation to higher features, such as facial expressions, similarly produces aftereffects, can have a twofold relevance. Firstly, it could help in defining which features, or combinations thereof, can qualify as genuine perceptual categories (if adaptation to body gender is observed, then gender should have a high likelihood to be deemed in itself a perceptual category). Secondly, if neural mechanisms are invoked to explain the existence of adaptation phenomena across a wide hierarchy of content complexity (from low-level to high-level), then one can conclude that specific neural units or regions must be responsible for encoding those types of content, providing indirect evidence of content-specific brain specialization. Although not all would necessarily endorse the second argument, we believe that measuring adaptation of the perceived valence arising from pictures could be a testbench for exploring the upper limits of perceptual adaptation, thus providing inputs for better understanding the notion of adaptation itself, its generality, and possibly its neural realization.

For this reason, in the present study we aimed to investigate whether adaptation to complex scenes with emotional content would affect the perception of subsequent scenes. We hypothesized that the attribution of valence to a complex scene, due to its relevance in eliciting quick behavioral responses

crucial for survival (Damasio, 1994), may qualify as a genuine perceptual category. It has to be highlighted that valence is a continuous dimension, ranging from extremely negative to extremely positive. However, the valence of a complex pictorial input might be synthetically attributed by assigning one single value along that continuum. With these premises in mind, we conjectured that testing the susceptibility of valence attribution to adaptation, could act as a proxy of testing whether valence is a perceptual category. Furthermore, if the perceived valence is subject to aftereffect it would then be possible to hypothesize that the perceiver creates a representation of the valence contained in the scene. In order to test our hypothesis, we used an adaptation paradigm where we asked participants to look for 10 s (adaptation phase) at a complex scene, which had either a positive or negative emotional valence, and to rate the valence of a following scene (which could be positive, neutral, or negative). In current literature on adaptation and aftereffects, it is still unclear whether aftereffects can be considered due to a decisional bias. Based on this approach, the adapter itself is responsible for modulating the participant's strategies of categorization, thus generating a sort of strategy rule where the subject tends to judge the test as opposite to the adapter just because it is different from it (Storrs, 2015). In order to exclude a possible decisional bias, we addressed this issue by asking our participants to rate the images, after the adaptation phase, rather than categorize them. Using this method, we thought that any change in the participant's judgment would be attributed to the effect of the adaptation and not to a change of strategy in the categorization process. Moreover, because valence is a continuous dimension that can be quickly processed by individuals, we based our study on the fact that it could be used as a dimension onto which aftereffects could be precisely assessed as contrastive or assimilative, using a typical adaptation paradigm. Although a typical adaptation paradigm would use test stimuli that are neutral on a chosen continuum (e.g., an androgynous face in a male-female space), we used positive, negative, and neutral test stimuli (see also Fujimura et al., 2013; Czekóová et al., 2015). Differently from previous studies on consecutive effects on affective evaluation of pictures (Flaisch et al., 2008; Schupp et al., 2012; Fujimura et al., 2013; Czekóová et al., 2015), the pictures chosen as adapters and tests were deliberately selected to not include images of faces. This was crucial in order to avoid the possibility that any effect possibly observed could be driven by this specific category.

### EXPERIMENT 1

### Methods

#### Participants

Forty-nine volunteers (26 females; mean age: 20.42; SD: 3.11) were recruited at the University of Chieti (Italy) to take part in the experiment. All of the participants had normal or correctedto-normal vision. The local ethics committee approved this study and written informed consent was obtained from all participants. All procedures employed in the study comply with ethical standards on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.

#### Stimuli

All stimuli were images taken from the International Affective Picture System (IAPS; Lang et al., 2008) with a positive, negative, or neutral valence. Furthermore, it should be noted that images containing faces were explicitly excluded to avoid the well-known effect of these stimuli on adaptation (e.g., Leopold et al., 2001).

Twenty-six independent raters (13 females; mean age: 22.15, range: 18–32) evaluated the valence of 176 images by means of a visual analog scale (VAS): the ends of the scale represented maximum negative value (−100%) on one side and maximum positive value (+100%) on the opposite side. The centre of the VAS represented neutrality (0). Following this rating, we selected 28 images: 11 positive, 11 negative, and 6 neutral; namely, the images that had received extreme mean scores in terms of positive and negative valence and the centremost mean scores in terms of neutral valence (the range of values of the positive, negative, and neutral images selected were +60% to +100%, −60 to −100%, and −4 to 4%, respectively). Among these selected images, five were used as positive adapters, five as negative adapters, and the remaining images were used as tests (six neutrals, M = −0.81%; six positive, M = +66.41%; six negative, M = −80.94%).

#### Procedure

Participants were seated in a dark experimental room and images were presented on a 17-inch Philips 170S monitor via an E-prime 1.2-controlled script. In a representative trial of the experiment (**Figure 1**), a fixation cross was presented in the center of the screen for 250 ms. The fixation cross was followed by the presentation of the adapter full screen for 10 s. The adapter was followed by the presentation of a fixation cross (duration 250 ms). Subsequently, the test image was presented for 500 ms. After the test image disappeared, the participant evaluated the valence of the image just presented by providing the response on a VAS, i.e., a line (length 31◦ , thickness 0.2◦ ) presented horizontally in the center of the screen, by moving a slider on the line with a mouse. The line represented a double-ended continuum where the two ends, labeled as "extreme negative" and "extreme positive," indicated the maximum negative value (−100%) on one side and the maximum positive (+100%) value on the opposite side. For each trial, the slider was positioned in the center of the line (neutral value; 0%) in order to not influence the participant's decision. The positive/negative or negative/positive spatial polarity of the continuum was balanced across participants. Each of the ten adapters (positive or negative) were presented in a random order and followed by a one of the 18 different tests stimuli (positive, negative, or neutral), for a total of 180 trials. Each new trial started after the participant's last response. The experiment lasted approximately 60 min overall.

### Statistical Analyses

Scores were coded on a continuum, ranging from −100 (negative) to +100 (positive), and converted to percentage values. They were submitted to a mixed factorial Analysis of Variance (ANOVA) with Valence of the Adapter (levels: negative; positive) and Test (levels: negative; neutral; positive) as withinparticipant factors and Gender as between-participant factor. A post hoc analysis using the Duncan test was used for multiple

comparisons. The level of statistical significance was set at 5%

to avoid showing International Affective Picture System (IAPS) pictures.

(p < 0.05). In addition, single sample t-tests were computed separately for each test type (according to adaptation to positive and negative stimuli), in order to compare the judgment to the mean value obtained during the preliminary rating for each valence level. This analysis was carried out to assess whether the scores obtained following adaptation depended upon the presence of the adapter.

### Results

The analysis showed a significant main effect of Valence of the Adapter, F(1,48) = 5.51, MSE = 2363, p < 0.05, η 2 <sup>p</sup> = 0.11, with a more negative value after adaptation to a negative image (M = −5.84%; SD = 54.76) with respect to adaptation to a positive image (M = −0.06%; SD = 55.01). The main effect of Test was also significant, F(1,48) = 447.27, MSE = 375113, p < 0.001, η 2 <sup>p</sup> = 0.90, showing that irrespective of the Valence of the Adapter, participants gave a negative score to the negative test images (M = −67.09%; SD = 18.14), which significantly differed from that of the neutral test images (M = 1.20%; SD = 17.91; p < 0.001) and from that of the positive test images (M = 57.03; SD = 25.22; p < 0.001). Additionally, the score of the positive test images differed significantly from that of the neutral test images (p < 0.001).

The interaction between Valence of the Adapter and Test was significant, F(1,48) = 3.61, MSE = 167.71, p < 0.05, η 2 <sup>p</sup> = 0.07 (**Figure 2**). Post hoc comparisons showed that, considering the neutral test, the positive adapters shifted the judgment toward a more positive judgment (M = 5.38%; SD = 17.15) with respect to the negative adapters that shifted the judgment toward a more negative judgment (M = −2.98%; SD = 17.86; p < 0.001). The negative test was evaluated as more negative following the negative adapter (M = −68.60%; SD = 18.29) compared to the positive adapter (M = −65.58%; SD = 18.05; p < 0.05). Finally, the positive test was given a more positive judgment following the positive adapter (M = 60.01%; SD = 12.14) compared to when it was followed by the negative adapter (M = 54.03%; SD = 27.88; p < 0.001). Neither the main effect of Gender, F(1,48) = 1.4, p = 0.25, nor the interactions involving this factor reached significance Valence of the Adapter and Gender: F(1,48) = 0.5, p = 0.50; Test and Gender: F(1,48) = 1, p = 0.36, Valence of the Adapter, Test and Gender: F(1,48) = 1.2, p = 0.32.

In addition, single sample t-tests showed that the judgment given to the neutral tests after being adapted to the positive adapters was significantly more positive from the mean score of the neutral tests as obtained in the preliminary rating, t(48) = 2.53, p < 0.05. The judgment given to the positive tests after being adapted to the positive and to the negative adapters was significantly less positive from the score given to the positive tests as obtained in the preliminary rating, t(48) = −2.02, p < 0.05 and t(48) = −3.10, p < 0.01, respectively. Moreover, the judgment given to the negative tests after being adapted to the positive adapters and after being adapted to the negative adapters were significantly less negative from the score given to the negative tests as obtained in the preliminary rating, t(48) = 5.95, p < 0.001 and t(48) = 4.72, p < 0.001, respectively. Finally, the judgment given to the neutral tests after being adapted to the negative adapters failed to reach significance when it was compared to the neutral tests' score as obtained in the preliminary rating, t(48) = −0.85, p = 0.39.

### EXPERIMENT 2

In order to ensure that the findings of Experiment 1 were the result of the prolonged exposure to adapter stimuli, we performed

FIGURE 2 | Valence scores obtained in the preliminary rating and after 10 s of exposure to the adapter (Experiment 1), as a function of Valence of the Adapter and Test type. Positive percentages values represent perceived valence and negative percentages represent perceived negative valence. The box plots show: the lower and upper quartile (the 25th and 75th percentiles, respectively); the whiskers, that refer to the minimum and the maximum values (excluding outliers) and the bold line, that refer to median value. Dots above the box plots are the mean rating of each participant. Dots outside whiskers indicate outliers.

a second (control) experiment reducing the duration of the adapter to 500 ms. In this experiment, both the adapter and test had the same duration. However, the temporal duration of the adapter was too short to qualify the paradigm as an adaptation paradigm, but too long to qualify it as a priming paradigm.

## Methods

#### Participants

Twenty-seven new volunteers (15 females; mean age: 20.74; SD: 2.52) were recruited at the University of Chieti (Italy) to take part in the experiment. All of the participants had normal or corrected-to-normal vision. The local ethics committee approved this study and written informed consent was obtained from all participants. All procedures employed in the study comply with ethical standards on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.

#### Procedure

Stimuli, procedure, and statistical analysis were the same of that in Experiment 1, with the only exception that the adapter was presented for 500 ms.

### Results

The analysis showed a significant main effect of Test, F(1,26) = 536.71, MSE = 25479, p < 0.001, η 2 <sup>p</sup> = 0.95: irrespective of the Valence of the Adapter, participants gave a negative score to the negative test images (M = −79.81%; SD = 3.49), which significantly differed from that of the neutral test images (M = −0.69%; SD = 1.94; p < 0.001) and from that of the positive test images (M = 65.52%; SD = 2.39; p < 0.001). Additionally, the score of the positive test images differed significantly from that of the neutral test images (p < 0.001).

The main effect of the Valence of the Adapter and its interaction with Test were not significant, F(1,26) = 0, p = 0.99, F(1,26) = 0.1, p = 0.92, respectively. Neither, the main effect of Gender, F(1,26) = 0, p = 0.92, nor the interactions involving this factor reached significance, Valence of the Adapter and Gender: F(1,26) = 0.3, p = 0.58; Test and Gender: F(1,26) = 0.6, p = 0.58, Valence of the Adapter, Test, and Gender: F(1,26) = 0.7, p = 0.50. In addition, single sample t-tests were performed to compare judgments given to the neutral, negative, and positive tests after adaptation to the negative and positive adapters with the mean score to the neutral, negative, and positive tests as obtained in the preliminary rating and none of them were significant (**Figure 3**).

### DISCUSSION

Adaptation and aftereffects have been widely investigated by employing low- (e.g., Suzuki and Cavanagh, 1998) and high-level stimuli (Rhodes et al., 2004). To assess how stimulus properties can be global in an ideal ordering of visual content that produce perceptual aftereffects, in the current study, two experiments were carried out to investigate affective adaptation and valencespecific aftereffects. Specifically, participants were exposed to images with positive and negative affective value and had to rate the valence of subsequent test pictures. Moreover, we used as tests not only images with a neutral affective value, but also images with a positive and a negative value themselves. The presence of faces was intentionally excluded from the sets of pictures employed as adapters and tests, in order to prevent the possibility that any potential effect of affective adaptation could be driven latently by the well-known aftereffects produced by facial emotional expressions. Our results show that participants judged a neutral test picture as more positive after having been exposed to a positive image adapter, while the judgment was shifted to a more negative value after having been previously adapted to a negative image adapter. Moreover, a negative test image was evaluated as more negative following a negative adapter compared to a positive adapter and a positive test image was evaluated as more positive following a positive adapter compared to a negative adapter. It must be acknowledged that these results should be taken with some caution, considering differences in statistical power. The main effect of the test image valence was statistically powerful and that of the adapter image valence was moderately supported. Nevertheless, the interaction between the two factors – central to the current study – was rather weakly supported by statistical power.

These results seem to confirm previous evidence obtained in a variety of methodological studies (priming studies, context processing, rapid serial visual presentation), which covered a number of temporal features of the "adapter" and "test" stimuli (usually short presentations as in priming or rapid serial visual presentation), as well as various levels of stimulus repetition in respect to the nature of the "adapter" (from isolated primes to multiple stimuli streams used as adapters). In the present study, using a typical adaptation procedure, the exposure to a complex picture, with a given valence for 10 s, influenced the subsequent evaluation of the valence of a test picture presented for 500 ms. Consequently, the aftereffect was always assimilative in nature, in that the positive or negative valence of the adapter picture "carried over" the appreciation of the test picture valence. Interestingly, this was manifested in the typical adaptation sequence embedded in our experimental design, where the neutral test pictures were rated as more positive following positive adapters and more negative following negative adapters, in which case it would suggest anassimilative aftereffect of valence. The present results also show that assimilative aftereffects of negative and positive adapter pictures manifested themselves also on non-neutral test pictures. This result might be better framed as a "congruence effect" in the extant literature on priming or affective context processing. Notably, these aftereffects were not driven by the well-known influence of facial expression in emotion adaptation, inasmuch the picture sets were carefully selected to not include faces.

The interpretation that the assimilative aftereffects could be attributed to prolonged stimulus exposure (i.e., adaptation) was supported by the evidence obtained in Experiment 2, wherein the presentation time of the prior stimulus was set to 500 ms. This value is far too short for an adaptation paradigm, and far too long for a priming paradigm. Thus, the null result in Experiment 2 suggests that the adaptation effects observed in Experiment 1 were genuine. Further, it does not seem likely that the long and repetitive nature of the task in the adaptation paradigm had induced effects of learning and fatigue on our participants, because the repetition structure of the non-adaptation experiment was the same as in the adaptation experiment. Adaptation to the emotional content of complex pictures, does not seem to produce contrastive aftereffects as those that have been widely reported for face emotion adaptation

(e.g., Rutherford et al., 2008), but rather assimilative "carryover" or "context" effects as already documented in other methodological studies (Fujimura et al., 2013; Czekóová et al., 2015). It could be speculated that the appraisal of valence, as observed in a complex picture, cannot be considered a "global" feature of automatic perceptual analysis depending on specialized neural units along the visual system. Typically, when such units exist, their (neural) adaptation can explain the phenomenological experience of contrastive aftereffects observed in perceptual adaptation paradigms (Webster and MacLeod, 2011), with stronger aftereffects at earlier levels of visual analysis, corresponding to neural circuitry specialized for hierarchically simpler features. However, contrastive aftereffects have often been observed also for complex visual content or for more global features of certain stimuli (but see Kovács et al., 2006, for a counterexample).A neural locus responsible for the effects of adaptation cannot be assumed as easily as in the case of simpler features, due to the fact that higher-level visual analysis engages multiple and increasingly interconnected regions of the visual brain and cannot depend strictly on the adaptation properties of single neurons. In particular, this is true for the adaptation to the emotional expression of faces, which typically exerts aftereffects that are contrastive in direction, despite the encoding of these stimuli that entails neural networks involving connections extending deeply in the visual brain, to the socalled "affective" and "social" brain (Brancucci et al., 2009). The fact that the typically contrastive aftereffects observed following adaptation to emotional expression of faces are replaced by assimilative effects when the visual content is complex and does not entail the presence of faces as documented in the present study, suggests the idea that prototypical stimulus categories (e.g., faces) do provide a structural framework for parametric encoding along the pathway of perceptual analysis, which is in turn more suitable to support contrastive aftereffects due to neural adaptation of dedicated circuitry (Webster and MacLeod, 2011). In other words, since neural encoding depends on a relatively low number of parameters, it might be hypothesized

#### REFERENCES


that for some stimulus categories (like color or facial expression) contrastive aftereffects would reveal the neural adaptation with respect to a predictable distribution of parameters in the space of stimuli (such as the face space) to which the observer is usually exposed. This would not be applicable for the overall perceived valence derived from a complex visual image. In such a case, due to the unpredictable number of objects and their relationships in the picture itself, parameters might be too numerous and unstructured, in which the neural representation might be much sparser than in the case of faces.

Finally, since previous evidence of assimilative context effects using emotional pictures (Fujimura et al., 2013; Czekóová et al., 2015) had not controlled for the relative contribution of the face vs. non-face pictures that were included in the stimulus material, the present results make deeper investigations on the contentspecificity of emotional adaptation necessary. This might involve, for instance, comparing the effects of adaptation to two identical sets of visual images differing only for the presence or absence of a person expressing a given emotion.

### ETHICS STATEMENT

University of Chieti: The local ethics committee approved this study, and written informed consent was obtained from all participants. All procedures employed in the study comply with ethical standards on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.

### AUTHOR CONTRIBUTIONS

RP: experiment design, data collection, data analysis, and manuscript writing. SD: data collection, data analysis, and manuscript writing. AQ: data collection and data analysis. LT: experiment design and manuscript writing.



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Palumbo, D'Ascenzo, Quercia and Tommasi. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# **Longer prime presentation decreases picture–word cross-domain priming**

*Kiyofumi Miyoshi\*, Yusuke Kimura and Hiroshi Ashida*

*Laboratory of Psychology, Department of Letters, Kyoto University, Kyoto, Japan*

A short prime presentation has been shown to provide a greater priming magnitude, whereas a longer prime presentation results in a lower priming magnitude. In Experiment 1, we attempted to replicate the decrease of priming using word stimuli. Words were presented in both prime and test sessions, and participants judged whether each stimulus was natural or manmade. In Experiment 2, we employed a cross-domain priming paradigm to assess the impact of prime duration on non-perceptual processes. Pictures were presented in prime sessions, and their semantically matched words were presented in test sessions. We did not observe a significant decrease in priming in Experiment 1. However, we found that 2000 ms of prime exposure led to weaker cross-domain priming when compared with 250 ms of the exposure in Experiment 2. The results suggest that the longer presentation of pictures causes a non-perceptual adaptation effect. This effect may occur at conceptual, linguistic, and/or response-related levels.

#### *Edited by:*

*Rocco Palumbo, Schepens Eye Research Institute – Harvard Medical School, USA*

#### *Reviewed by:*

*Britt Anderson, University of Waterloo, Canada David A. Ross, University of Massachusetts Amherst, USA*

#### *\*Correspondence:*

*Kiyofumi Miyoshi, Laboratory of Psychology, Department of Letters, Kyoto University, Yoshida-Honmachi, Sakyo, Kyoto 6068501, Japan miyoshi80@gmail.com*

#### *Specialty section:*

*This article was submitted to Perception Science, a section of the journal Frontiers in Psychology*

*Received: 15 May 2015 Accepted: 08 July 2015 Published: 21 July 2015*

#### *Citation:*

*Miyoshi K, Kimura Y and Ashida H (2015) Longer prime presentation decreases picture–word cross-domain priming. Front. Psychol. 6:1040. doi: 10.3389/fpsyg.2015.01040*

**Keywords: adaptation, habituation, priming, implicit memory, reaction time**

## **Introduction**

Priming refers to improved processing of a repeated stimulus [e.g., greater accuracy or faster reaction time (RT)] through unconscious memory retrieval (see Schacter et al., 2007, for a review). Many studies suggest that presenting stimuli for a short duration provides positive priming, whereas presenting stimuli for a long duration provides lower or even negative priming (Huber and O'Reilly, 2003; Zago et al., 2005; Voss and Gonsalves, 2010; Faivre and Kouider, 2011; Miyoshi and Ashida, 2014). Huber and O'Reilly (2003) proposed a neural network model of perceptual identification to explain the impact of prime duration on short-term priming. In this model, short prime presentation leads to the pre-activation of stimulus representations and provides a "head start" for stimulus processing, resulting in a positive priming effect. However, longer prime presentation leads to the adaptation of stimulus representations and reduces the magnitude of pre-activation, resulting in a lower or negative priming effect. In this model, pre-activation and adaptation are considered to be two different mechanisms which can simultaneously coexist. Note that considering the evidence that short- and long-term priming were associated with different neural repetition suppression effects (Epstein et al., 2008), this model may be only applicable to data from short-term priming experiments.

Zago et al. (2005) investigated the impact of prime duration on long-term repetition priming in a natural versus manmade decision task for pictures of familiar objects. Their results demonstrated that the magnitudes of behavioral priming were lower for longer presentation of primes (350–1900 ms) than for shorter presentations (250 ms). Consistent with the behavioral results, the magnitude of functional magnetic resonance imaging (fMRI) repetition suppression effects was lower for longer prime presentations than shorter presentations. It should be noted that these researchers conducted three different versions of the experiment, each of which has different inter-repetition intervals between the prime and target: average time between presentations for the block design fMRI experiment was 40–58 s (9–18 intervening stimuli); for the event-related fMRI experiment was 2 s–14 min; (1–374 intervening stimuli); and for the behavioral experiment was 2 s–2 min (2–60 intervening stimuli). Notably, these different versions of the experiment yielded highly similar results, suggesting the robustness of results.

Although the focus of their study was to investigate how visual exposure shapes perceptual representations of objects, the researchers found fMRI repetition suppression effects in cortical regions typically associated with perceptual processes (posterior occipital–temporal regions) and non-perceptual (e.g., conceptual, linguistic, and response-related) processes (anterior temporal and inferior frontal regions). Thus, the need for additional research to investigate the extent to which the priming decrease with longer prime presentation reflects general operating characteristics in different levels of processing was emphasized. For that purpose, behavioral evidence about the impact of prime duration on high level priming is needed, because fMRI repetition suppression and behavioral priming are not always associated (Sayres and Grill-Spector, 2006).

The precise mechanisms for the "rise and fall" pattern of long-term priming are still unclear (see Zago et al., 2005; Voss and Gonsalves, 2010; Miyoshi and Ashida, 2014). However, we hypothesized that bottom-up changes in stimulus processing are responsible for the pattern (Miyoshi and Ashida, 2014); short stimulus presentation times lead to the fine tuning of the neural activities coding the stimulus (sharpening; Desimone, 1996; Wiggs and Martin, 1998), but longer stimulus presentation times lead to the fatigue of neurons (adaptation) and make stimulus representations less efficient.

In the present study, we conducted two experiments. In Experiment 1, we attempted to assess the generality of the rise and fall of long-term priming reported in Zago et al. (2005). Specifically, we attempted to replicate the decrease of long-term priming with longer prime duration using word stimuli. Words were presented in both prime and test sessions, and participants were required to identify them as natural or manmade. Following the method of Voss and Gonsalves (2010), the prime duration was set at 250 and 2000 ms. In Experiment 2, we employed a long-term cross-domain priming paradigm to investigate the effect of stimulus duration on highlevel processes beyond the perceptual level. Stimuli comprised pictures of real-world objects and semantically matched words. Pictures were presented in prime sessions, but the semantically matched words were presented in test sessions, which should exclude perceptual priming/adaptation and involve only nonperceptual (e.g., conceptual, linguistic, and response-related) priming/adaptation (see Schacter et al., 2004, for a review of different types of priming). This cross-domain paradigm serves the purpose of investigating the temporal dynamics of high-level stimulus representations.

## **Materials and Methods**

#### **Participants and Design**

Sixteen undergraduates (eight men, eight women; ages 19–24 years) volunteered to participate in both Experiments. The sample size was determined according to above mentioned previous studies (12–20 in Zago et al., 2005; 14 in Voss and Gonsalves, 2010). The order of Experiments 1 and 2 was counterbalanced across the participants. The participants received 1000 yen for their participation. All participants had normal color vision; an informed written consent was obtained from them before the experiments. All data were collected in accordance with the ethical principles of the Japanese Psychological Association. The study was a 2 (experiment: Experiments 1 and 2) *×* 3 (prime condition: novel, 250 ms, and 2000 ms) design with both variables as within-participant factors.

### **Materials and Procedure**

We used 300 color pictures of real-world objects and 300 of their semantically matched Japanese words. We organized two stimulus lists, each including 150 semantically matched pairs of a picture and a word. Each list included an equal number of natural and manmade pairs, and the number of letters of the word stimuli was counterbalanced between the lists. The lists were counterbalanced such that they were used in Experiments 1 and 2 for an equal number of times across the participants. The stimulus pairs in each list were randomly assigned to each prime condition (novel, 250 ms, and 2000 ms) for each participant. The stimuli were displayed on a dark background on a computer monitor using the software Presentation (Neurobehavioral Systems). The distance between the monitor and participant was 48 cm. Each picture was framed in a white window measuring 7.5 (width) cm *×* 7.5 (height) cm (8.9° *×* 8.9° of visual angle). Each word measured 2.2–5.5 *×* 1.1 cm (2.6–6.6° *×* 1.3°of visual angle).

The participants took part in both Experiments. Each experiment contained two consecutive sets of a prime–test session. Each prime session included 50 trials, and each test session included 75 trials. Thus, the participants experienced a total of 250 trials in each experiment. The order of Experiments 1 and 2 was counterbalanced across the participants, and there was a 3-min interval between the two experiments.

In Experiment 1, 50 words were presented one by one; this was immediately followed by a mask in the prime session. The participants were required to answer as quickly and accurately as possible whether each word was natural or manmade with a button press. Half the words were presented for 250 ms, and the remaining were presented for 2000 ms. The duration of the mask was either 2250 or 500 ms, thereby ensuring that the total duration of each trial was 2500 ms. Trial presentation order was randomized, and intertrial intervals were 2000 ms.

The test session began 15 s after the prime session. In each test session, 25 novel words, 25 250-ms-primed words, and 25 2000-ms-primed words were presented one by one for 500 ms in a random order; this was immediately followed by the mask. The duration of the mask was 1000 ms. The participants were again required to answer as quickly and accurately as possible whether each word was natural or manmade with a button press; intertrial intervals were 2000 ms.

The procedure of Experiment 2 was the same as that of Experiment 1, except that pictures were presented in the prime sessions and their semantically matched words were presented in the test sessions.

### **Results**

**Table 1** presents mean response accuracies recorded in the test sessions. RT analyses included only trials with correct responses. We conducted a repeated-measures ANOVA on the mean RTs with experiment (Experiments 1 and 2) and prime condition (novel, 250 ms, and 2000 ms) as within-participant factors (**Figure 1**). There was no significant main effect of experiment [*F*(1,15) = 1.98, *p* = 0.18, η 2 *<sup>p</sup>* = 0.117], but there was a significant main effect of prime condition [*F*(2,30) = 27.70, *p <* 0.001, η 2 *<sup>p</sup>* = 0.649]. The interaction between these two factors was significant [*F*(2,30) = 5.13, *p* = 0.01, η 2 *<sup>p</sup>* = 0.255].

There was a significant simple main effect of prime condition in Experiment 1 [*F*(2,30) = 18.63, *p <* 0.001, η 2 *<sup>p</sup>* = 0.554]. Paired *t*-tests with Shaffer's modified sequentially rejective Bonferroni procedure (Shaffer, 1986) revealed that the mean RT in the novel condition was significantly longer than those in the 250 ms condition [*t*(15) = 5.93, *p <* 0.001, *d ′* = 1.48] and the 2000 ms condition [*t*(15) = 5.85, *p <* 0.001, *d ′* = 1.46]. There was no significant difference in the mean RTs of the 250 and 2000 ms conditions [*t*(15) = 0.35, *p* = 0.73, *d ′* = 0.09]. The results revealed



*Standard deviation (SD) is shown in parentheses.*

that the longer presentation of word primes did not decrease the priming magnitude in Experiment 1.

A simple main effect of prime condition was also significant in Experiment 2 [*F*(2,30) = 12.09, *p <* 0.001, η 2 *<sup>p</sup>* = 0.446]. Paired *t*-tests with Shaffer's modified sequentially rejective Bonferroni procedure revealed that the mean RT in the novel condition was significantly longer than those in the 250 ms condition [*t*(15) = 5.27, *p <* 0.001, *d ′* = 1.32] and the 2000 ms condition [*t*(15) = 2.30, *p* = 0.04, *d ′* = 0.58]. Notably, the mean RT in the 2000 ms condition was significantly longer than that in the 250 ms condition [*t*(15) = 2.50, *p* = 0.02, *d ′* = 0.63], thereby suggesting that the longer presentation of picture primes decreased the magnitude of long-term cross-domain priming.

Finally, we conducted a repeated-measures ANOVA on the mean response accuracies with experiment and prime condition as within-participant factors. There was no significant main effect of experiment [*F*(1,15) = 1.06, *p* = 0.32, η 2 *<sup>p</sup>* = 0.066] and prime condition [*F*(2,30) = 0.35, *p* = 0.71, η 2 *<sup>p</sup>* = 0.023]. The interaction between these two factors was not significant [*F*(2,30) = 0.03, *p* = 0.97, η 2 *<sup>p</sup>* = 0.002]. The results suggest that RT differences cannot be attributed to a speed versus accuracy trade-off.

### **Discussion**

In Experiment 1, mean RTs were highly matched between the 250 and 2000 ms conditions. Namely, we did not observe the decrease of long-term word repetition priming with longer prime duration. Contrary to the present results, Zago et al. (2005) demonstrated that longer prime presentation decreases the magnitude of longterm picture repetition priming. These different result patterns probably arise because the processing of words and pictures depends on (at least partially) different underlying mechanisms. However, it is unsafe to further infer about the difference between our study and Zago et al. (2005) on the basis of the negative results reported here. Considering the relatively small sample size of the present study, the non-significant RT difference between the 250 and 2000 ms conditions could simply result from a lack of statistical power. The power to detect this RT difference is 0.65, if we set the significance level at 0.05 and the population effect size (δ *′* ) at 0.63 by reference to the results of Experiment 2.

To the best of our knowledge, Experiment 2 provides the first behavioral evidence that the longer presentation of picture primes decreases the magnitude of picture–word cross-domain priming. The present behavioral results are consistent with the fMRI results of Zago et al. (2005) that longer presentations of picture primes decrease the magnitude of neural repetition suppression in cortical regions typically associated with non-perceptual processes (anterior temporal and inferior frontal regions). Taken together, these results provide strong evidence about the temporal dynamics of high-level stimulus representations. As mentioned in the introduction, we hypothesized that bottomup changes in stimulus processing are responsible for the decrease of long-term priming with longer prime duration (Miyoshi and Ashida, 2014). According to the hypothesis, the present results suggest that short presentation of pictures leads to the fine tuning of the neural activities which code higher-level information beyond the perceptual level; however, longer presentation times lead to adaptation at higher-levels, such as for conceptual, linguistic, and response-related processes. The combination of these two may be responsible for the long-lasting changes in stimulus representations.

Alternatively, Miyoshi and Ashida (2014) suggested that another possible cause of the priming decrease is interference from explicit memory retrieval. We hypothesized that longer prime presentation enables an elaborate explicit memory encoding and that explicit recollection during the test might interfere with implicit memory processes and decrease the magnitude of priming. In fact, explicit recollection is considered to be dependent on slow and effortful processes (Hasher and Zacks, 1979; Jacoby, 1991). From this perspective, this study's

### **References**


results suggest that explicit memory retrieval interferes with non-perceptual implicit memory.

The present study provides the novel finding that the longer presentation of pictures reduces the magnitude of long-term cross-domain priming, thereby suggesting the effect of high-level adaptation. However, the present findings do not identify the exact level of processing at which the present adaptation effect occurs. Future research is required to further elucidate this point.

## **Acknowledgments**

This work was supported by a JSPS Grant-in-Aid for scientific research (22220003) and a Grant-in-Aid for JSPS Fellows (15J06742).


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2015 Miyoshi, Kimura and Ashida. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Beyond the Sensorimotor Plasticity: Cognitive Expansion of Prism Adaptation in Healthy Individuals

#### *Carine Michel1,2\**

*<sup>1</sup> Unité de Formation et de Recherche en Sciences et Techniques des Activités Physiques et Sportives, Campus Universitaire, Université de Bourgogne, Dijon, France, <sup>2</sup> INSERM, U 1093, Cognition, Action et Plasticité Sensorimotrice, Dijon, France*

Sensorimotor plasticity allows us to maintain an efficient motor behavior in reaction to environmental changes. One of the classical models for the study of sensorimotor plasticity is prism adaptation. It consists of pointing to visual targets while wearing prismatic lenses that shift the visual field laterally. The conditions of the development of the plasticity and the sensorimotor after-effects have been extensively studied for more than a century. However, the interest taken in this phenomenon was considerably increased since the demonstration of neglect rehabilitation following prism adaptation by Rossetti et al. (1998). Mirror effects, i.e., simulation of neglect in healthy individuals, were observed for the first time by Colent et al. (2000). The present review focuses on the expansion of prism adaptation to cognitive functions in healthy individuals during the last 15 years. Cognitive after-effects have been shown in numerous tasks even in those that are not intrinsically spatial in nature. Altogether, these results suggest the existence of a strong link between low-level sensorimotor plasticity and high-level cognitive functions and raise important questions about the mechanisms involved in producing unexpected cognitive effects following prism adaptation. Implications for the functional mechanisms and neuroanatomical network of prism adaptation are discussed to explain how sensorimotor plasticity may affect cognitive processes.

Keywords: prism adaptation, cognition, after-effects, pseudoneglect, neglect

### INTRODUCTION

Sensorimotor plasticity allows producing an appropriate motor response in reaction to environmental changes or bodily evolution during the life. Prism adaptation is one of the oldest paradigms to study sensorimotor plasticity. It consists of pointing to visual targets while wearing prismatic lenses that shift the visual field laterally. The pointing errors made in the direction of the optical shift are gradually corrected. After prism removal, the pointing movements are shifted in the direction opposite to the optical deviation. The entire process can be explained by proprioceptive, visual, and motor control changes (e.g., Kornheiser, 1976). Prism adaptation has been described for more than one century (e.g., Stratton, 1896) but the interest taken in this procedure was considerably increased since the publication of the article of Rossetti et al. (1998) showing the therapeutic impact of prism adaptation in neglect rehabilitation (Newport and Schenk, 2012). Neglect is described as a failure to report, respond, or orient to novel or meaningful stimuli presented to the side opposite to a brain lesion usually in the right hemisphere

#### *Edited by:*

*Luca Tommasi, Università degli Studi "G. d'Annunzio" Chieti – Pescara, Italy*

#### *Reviewed by:*

*Igor Christian Schindler, University of Hull, UK Giorgia Committeri, Università degli Studi "G. d'Annunzio" Chieti – Pescara, Italy*

> *\*Correspondence: Carine Michel carine.michel@u-bourgogne.fr*

#### *Specialty section:*

*This article was submitted to Perception Science, a section of the journal Frontiers in Psychology*

*Received: 25 September 2015 Accepted: 10 December 2015 Published: 05 January 2016*

#### *Citation:*

*Michel C (2016) Beyond the Sensorimotor Plasticity: Cognitive Expansion of Prism Adaptation in Healthy Individuals. Front. Psychol. 6:1979. doi: 10.3389/fpsyg.2015.01979*

(e.g., Heilman et al., 1993). In their comprehensive review, Jacquin-Courtois et al. (2013) set out the therapeutic interest of prism adaptation on a broad range of clinical and experimental measures beyond the framework of sensorimotor coordination. The present revue focuses on the cognitive after-effects of prism adaptation in healthy individuals and presents this procedure as a robust tool for simulating neglect in normals. In the light of the recent results, it offers potential insights into the understanding of the expansion of low-level sensorimotor processes to cognitive functions.

### Cognitive After-Effects of Prism Adaptation in Space Representation

The first demonstration of prism-induced cognitive after-effects in space representation (mental image of the space mapped across the brain) in healthy individuals was shown Colent et al. (2000). Line bisection task is an invaluable tool to assess space representation. In the manual version, subjects place a mark at the center of a horizontal line. In the perceptual judgment version (landmark test), they are requested to judge whether a line has been transected to the left or the right of its true center. Performance in healthy subjects is characterized by a leftward pseudoneglect bias due to a mental over-representation of the left part of space and an under-representation of the right part of space (e.g., McCourt and Jewell, 1999). Neglect patients show a rightward bias because they exhibit a mental under-representation of the left part of space and an overrepresentation of the right part of space (e.g., Halligan, 1995). When adaptation was developed by visuo-manual pointings with the right dominant hand during prism exposure (20 min under 15◦ leftward or rightward deviation), only adaptation to a leftward optical deviation produced representational aftereffects. They appeared to be a faithful qualitative simulation of neglect, i.e., a rightward bias in perceptual line bisection (Colent et al., 2000). Complementary studies confirmed the effects of adaptation on both manual and perceptual bisection tasks (Michel et al., 2003a; Nijboer et al., 2010; Striemer and Danckert, 2010; Fortis et al., 2011; Michel and Cruz, 2015) and showed that they lasted for at least half an hour (Schintu et al., 2014). The occurrence of cognitive after-effects in healthy individuals seems to depend on the baseline expression of pseudoneglect. A greater leftward magnitude at baseline is associated with greater rightward after-effects (Goedert et al., 2010; Herlihey et al., 2012). Therefore, individuals with leftward bias due to right hemisphere dominance (e.g., Fink et al., 2001) are particularly sensitive to adaptation to leftward optical deviation that may act on right hemisphere functioning (see paragraph 5). This could also be the reason why experimental conditions involving the right hemisphere, as left-sided location of the line, favor the occurrence of cognitive after-effects (Michel et al., 2003a) and why no aftereffect was observed in manual line bisection in the absence of baseline pseudoneglect (Colent et al., 2000).

Cross-modal after-effects of prism adaptation are observed on haptic tasks where participants are required to locate the center of a haptically explored circle or a visually perceived circle. Prism adaptation induced a rightward shift of performance similar to the bias shown in neglect patients (McIntosh et al., 2002; Girardi et al., 2004). Cognitive after-effects also extend to extrapersonal representation, i.e., beyond the immediate region (arm reach) within which visuomotor adaptation takes place. They were shown in the landmark task (Berberovic and Mattingley, 2003) and in goal-oriented locomotion (Michel et al., 2008). They are similar to neglect-like locomotor bias (Robertson et al., 1994; Berti et al., 2002). Cognitive after-effects even occur in mental scales. The mental number line is thought to have a left-to-right organization whereby low and high numbers are represented along a spatial continuum from left to right (Dehaene et al., 1993). As a result, when judging the distance between two numbers, without using arithmetic, normal subjects misbisect the mental distance toward the smaller number (i.e., to the left) (Longo and Lourenco, 2007; Loftus et al., 2009a). Adaptation to a leftward optical deviation was responsible for a shift in bisection toward the high numbers (i.e., to the right) (Loftus et al., 2008) that could be compared to a mild neglect-like behavior (Vuilleumier et al., 2004; Zorzi et al., 2002, 2006). The mental alphabetic line has also a left-to-right organization with early letters on the left side and later letters on the right side of space (Gevers et al., 2003; Zorzi et al., 2006). Normal subjects misbisect the mental distance toward early letters (i.e., to the left) (Zorzi et al., 2006; Nicholls and Loftus, 2007). Adaptation to a leftward optical deviation produces a shift in bisection toward the later letters (i.e., to the right) (Nicholls et al., 2008) that could be compared to a mild neglect-like behavior (Gevers et al., 2003; Zorzi et al., 2006). Cognitive after-effects also concern body representation. The analysis of the center of pressure (point of application of the ground reaction force vector) is a useful but indirect tool to assess the internal model of the body (Gurfinkel and Levick, 1991). Adaptation to a leftward optical deviation produced a rightward shift of the center of pressure in an eyes closed condition which correlated negatively with a counterclockwise estimation of the visual vertical (encoded within an egocentric frame of reference) (Michel et al., 2003b). These results on posture and subjective visual vertical are similar to mild symptoms following right brain lesion and more particularly neglect manifestations (Bohannon et al., 1986; Brandt et al., 1994; Rode et al., 1997, 1998; Kerkhoff, 1999; Tilikete et al., 2001). Altogether, prism adaptation affects a supramodal level of space representation in both explicit and non-explicit spatial tasks.

We could mention here that prism adaptation affects also spatial remapping that enables the construction of a stable representation of the visual environment despite constantly changing retinal images. Prism adaptation to a leftward optical deviation induces impairment in spatial remapping for left visual field targets in the double-step saccade paradigm (Bultitude et al., 2013c). These after-effects could be viewed as mild neglect behavior (Pisella et al., 2011).

### After-Effects of Prism Adaptation in Attention and Hierarchical Processing

Because space representation depends on orientation of attention (e.g., Milner et al., 1992), the rightward representational after-effects described above could be partly explained by a redistribution of spatial attention to the detriment of the left space. Prism adaptation affects covert attention (Striemer et al., 2006) and has lateralised effects on spatial attention. Judgment of luminance (grayscales task) requires a forced choice judgment between two mirror-reversed luminance gradients. Participants usually select the stimulus that is darker on the left despite the fact that both stimuli are equiluminant (Nicholls et al., 1999; Okubo and Nicholls, 2006). This leftward bias is reversed following adaptation to a leftward optical deviation (Loftus et al., 2009b) mimicking neglect-like behavior in grayscales (Sarri et al., 2011).

The influence of prism adaptation extends also to hierarchical processing as first demonstrated by Bultitude and Woods (2010). When healthy participants are presented with figures in which small letters are arranged to form a large letter (Navon, 1977), they are faster to identify the global-level than the local-level information, and have difficulty ignoring global information when identifying the local level. After adaptation to a leftward optical deviation, there was a significant reduction in global interference similar to the processing bias demonstrated in patients with right temporo-parietal junction lesions (Bultitude et al., 2009). Reed and Dassonville (2014) demonstrated that adaptation to a leftward optical deviation increased the susceptibility to a subset of visual illusions known to be driven by local contextual processing. However, adaptation failed to influence performance in the composite face task that is supposed to evaluate the automatic global-level processing of faces (Bultitude et al., 2013a). Negative results were also observed on spatial attention in space-based or object-based attention (Bultitude et al., 2013b), in a temporal order judgment task (Berberovic et al., 2004), in saccade latencies or antisaccade errors (Nijboer et al., 2010) and in visual search (Morris et al., 2004; Saevarsson et al., 2009). Even if a lack of sensitivity cannot be excluded for several tasks, these negative results could be explained by the absence of pseudoneglect behavior in baseline performance. It has been proposed that any aspects of performance that have been altered by prism adaptation are ones for which the behavior is already biased toward pseudoneglect (Nijboer et al., 2010; Bultitude et al., 2013b) (see Paragraph 1). Therefore, the influence of prism adaptation could be viewed as reducing pseudoneglect or inversing pseudoneglect to produce mild neglect.

### Why aren't Cognitive After-Effects Explained by Sensorimotor After-Effects?

The occurrence of cognitive after-effects is even more interesting when considering that the development of prism adaptation relies on active motor behavior and that cognitive after-effects cannot be explained in terms of sensorimotor after-effects. Several arguments could be presented here. (1) If cognitive aftereffects were explained by sensorimotor after-effects they would be mainly observed in tasks involving visuo-manual coordination. On the contrary, they are shown in tasks requiring verbal responses (Berberovic and Mattingley, 2003; Michel et al., 2003a; Loftus et al., 2008, 2009b; Nicholls et al., 2008; Nijboer et al., 2010; Schintu et al., 2014). (2) If cognitive after-effects were explained by sensorimotor after-effects the amplitude of the bias in bisection task would be independent of the spatial location because sensorimotor after-effects generalize homogeneously over space (Bedford, 1989). The bias would have also the same amplitude irrespective of the length of the line because relative position coding is not altered by wedge prisms (Redding and Wallace, 1996). On the contrary, line bisection bias is greater for left-sided locations than for right sided-locations and it increases with the line length (Michel et al., 2003a). (3) If cognitive aftereffects were explained by sensorimotor after-effects they would be symmetric in amplitude following exposure to left and right optical deviations (Cohen, 1967; Loftus et al., 2008; Michel et al., 2003b, 2008; Schintu et al., 2014). On the contrary, cognitive after-effects on pseudoneglect behavior occur only following adaptation to a leftward optical deviation (Colent et al., 2000; Michel et al., 2003b; Loftus et al., 2008, 2009b; Nicholls et al., 2008; Bultitude and Woods, 2010; Goedert et al., 2010) (4) If cognitive after-effects result from sensorimotor after-effects they could be partly explained by visual after-effects. Prism adaptation is responsible for visual displacement of the gaze in the direction of the optical deviation during and after prism exposure (e.g., Wallace et al., 1973) due to eye muscle potentiation (Ebenholtz, 1974) and visual recalibration (Craske, 1975; Craske and Crawshaw, 1975). Because line bisection performance is biased toward the start location of the scanning direction (Brodie and Pettigrew, 1996; Chokron et al., 1998) a leftward bias could be expected after adaptation to a leftward optical deviation. On the contrary, a rightward bias is observed in line bisection (e.g., Colent et al., 2000; Striemer and Danckert, 2010) and in ocular exploration (Ferber and Murray, 2005) showing a reorganization of the visual functions that cannot be explained by visual after-effects. (5) If cognitive after-effects were explained by sensorimotor after-effects there would be a correlation between the amplitude of sensorimotor and cognitive after-effects. On the contrary, all studies analyzing the link between sensorimotor and cognitive after-effects showed no correlation (Berberovic and Mattingley, 2003; Girardi et al., 2004; Fortis et al., 2011; Herlihey et al., 2012; Guinet and Michel, 2013; Schintu et al., 2014).

Even if cognitive after-effects cannot be directly explained by sensorimotor after-effects, their occurrence strictly depends on the development of adaptation (spatial realignment) which needs active pointing movements during prism exposure (Michel et al., 2003a). Furthermore the spatial realignment must be strong enough (by using at least 10◦ optical deviation) to observe cognitive after-effects (Michel and Cruz, 2015). Nevertheless, the attempt to increase cognitive after-effects by combining neck muscle vibration with prism adaptation to increase the misperception of the target in the direction of the prismatic shift is unfruitful in healthy individuals (Guinet and Michel, 2013).

### Sensorimotor and Cognitive After-Effects: Where is the Boundary?

The term 'cognitive' refers to the fact that effects take place beyond the usual framework of compensatory sensorimotor after-effects and involves mental abilities. Cognitive after-effects are mainly assessed by 'paper-pencil' tests or need verbal responses as manual line bisection or mental number bisection, respectively. They result from higher cognitive processes involved in judgment and comparison. In contrast, sensorimotor aftereffects are exclusively shown in tasks assessing sensorimotor coordination as visuo-manual open-loop pointing. They result from adaptive changes in perception and motor command. Nevertheless, under certain circumstances, when a motor response is required for a cognitive task, both types of aftereffects can coexist. This could be the case for the manual line bisection but the use of slow movements under visual guidance to set the bisection mark abolishes any sensorimotor influence (Redding and Wallace, 1996). The absence of correlation between cognitive and sensorimotor after-effects indicates that sensorimotor after-effects do not influence cognitive responses (see Paragraph 3). On the contrary, goal-oriented locomotion to a memorized visual target allows the coexistence of both types of after-effects because it involves memori*z*ed representation of a target in space for a few seconds (before and during the displacement) which favors the appearance of representational after-effects in far space (Michel et al., 2008). Moreover, as mentioned in Paragraph 3, the optical deviation used to produce after-effects is to consider. Cognitive aftereffects are asymmetric whereas sensorimotor after-effects are symmetric.

### From Pseudoneglect to Neglect-Like Behavior: How is it Possible?

Attempts to produce neglect in normals are based on the specialization of the human right hemisphere for visuo-spatial functions (e.g., Benton and Tranel, 1993). Interhemispheric changes to the detriment of the right hemisphere (or in favor of the left hemisphere) produce a mild neglect-like behavior or reduce pseudoneglect. For clarity and conciseness, only examples from line bisection studies are presented here. Leftward pseudoneglect bias in line bisection decreases when the right hand (versus left hand) is used (Scarisbrick et al., 1987; Fukatsu et al., 1990; Brodie and Pettigrew, 1996; Jewell and McCourt, 2000), when lines are located in right hemispace (Reuter-Lorenz et al., 1990; Luh, 1995; McCourt and Jewell, 1999) or when attention is oriented to the right extremity of the line (Milner et al., 1992; Harvey et al., 2000). Aging is also characterized by hemispheric changes. Faster aging of the right hemisphere (Meudell and Greenhalgh, 1987; Robinson and Kertzman, 1990) or reduced hemispheric asymmetry with aging (Dolcos et al., 2002; Reuter-Lorenz, 2002; Cabeza et al., 2004), may be responsible for a rightward bias in line bisection (Fujii et al., 1995; Schmitz and Peigneux, 2011; Benwell et al., 2014). Direct modulation of the cerebral activity by using transcranial magnetic stimulation (TMS) or transcranial direct current stimulation (tDCS) produces also neglect-like symptoms

### REFERENCES

Bedford, L. B. (1989). Constraints on learning new mappings between perceptual dimensions. *J. Exp. Psychol. Hum. Percept. Perform.* 15, 232–248. doi: 10.1037/0096-1523.15.2.232

(Sack, 2010). The use of repetitive TMS over right frontal and right posterior parietal cortices produces a rightward neglectlike bias in line bisection (Fierro et al., 2000, 2001; Brighina et al., 2002; Ellison et al., 2004; Bjoertomt et al., 2009), in mental number line (Göbel et al., 2006) or in visual target detection (Thut et al., 2005; Muggleton et al., 2006). When right cathodal (hyperpolarization) and left anodal (depolarization) tDCS were simultaneously applied over homolog posterior parietal cortices or when only right cathodal tDCS was used, a rightward bias in landmark task was shown (Giglia et al., 2011; Benwell et al., 2015). Interestingly enough, when neglect-like bias was produced during TMS over the right parietal cortex, functional imaging studies showed a decreased activity within the site of stimulation and in interconnected right hemisphere structures and even enhanced BOLD signal in the left parietal and visual cortices (Sack et al., 2007; Heinen et al., 2011; Ricci et al., 2012).

Therefore, interhemispheric imbalance to the detriment of the right parietal cortex may explain prism-induced cognitive after-effects (Michel, 2006). Studies in brain damaged patients and healthy individuals underline the involvement of the parieto-cerebellar network in the development of adaptation (Clower et al., 1996; Pisella et al., 2005; Luauté et al., 2006, 2009; Danckert et al., 2008; Chapman et al., 2010; Crottaz-Herbette et al., 2014). The temporal cortex, involved during realignment, might also account for some of cognitive after-effects (Luauté et al., 2009). Otherwise, the hand used during prism exposure may have a potential influence on the hemispheric imbalance following adaptation. Except two studies (Michel et al., 2008; Reed and Dassonville, 2014) using both hands for visuo-manual pointings during prism exposure, the right dominant hand (involving the left hemisphere) is always used.

### CONCLUSION

Prism adaptation is undoubtedly a fascinating phenomenon that urges us to revisit our conception of sensorimotor plasticity and questions us on the reciprocal relations between cognition and action. Prism adaptation is a powerful non-invasive method for neglect rehabilitation (e.g., Rossetti and Rode, 2002; Rode et al., 2015) that is able to mirror neglect in normal. Nevertheless the neural substrate of after-effects during a cognitive task following prism adaptation to a leftward optical deviation (conditions known to express neglect-like behavior) has not yet been studied in healthy individuals. Furthermore, the nature of the aftereffects needs to be further investigated because it does not limit to sensorimotor and cognitive domains (e.g., Sumitani et al., 2007).

Benton, A., and Tranel, D. (1993). "Visuoperceptual, Visuospatial, and visuoconstructive disorders," in *Clinical Neuropsychology*, eds K. M. Heilman and E. Valensteinn (New York, NY: Oxford University Press), 165–213.

Benwell, C. S., Learmonth, G., Miniussi, C., Harvey, M., and Thut, G. (2015). Non-linear effects of transcranial direct current stimulation as a function of individual baseline performance: evidence from biparietal tDCS influence on lateralized attention bias. *Cortex* 69, 152–165. doi: 10.1016/j.cortex.2015.05.007


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Michel. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*