# BRIDGING THE THEORIES OF AFFORDANCES AND LIMB APRAXIA

EDITED BY: Antonello Pellicano, Anna M. Borghi and Ferdinand Binkofski PUBLISHED IN: Frontiers in Human Neuroscience

#### *Frontiers Copyright Statement*

*© Copyright 2007-2017 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.*

*The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.*

*Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.*

*Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.*

*As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.*

> *All copyright, and all rights therein, are protected by national and international copyright laws.*

*The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.*

ISSN 1664-8714 ISBN 978-2-88945-225-5 DOI 10.3389/978-2-88945-225-5

# About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

# Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

# Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

# What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# **BRIDGING THE THEORIES OF AFFORDANCES AND LIMB APRAXIA**

Topic Editors:

**Antonello Pellicano,** RWTH Aachen University, Aachen, Germany **Anna M. Borghi,** Sapienza University of Rome & Institute of Cognitive Sciences and Technologies, Italian National Research Council, Italy **Ferdinand Binkofski,** RWTH Aachen University, Germany

"A spoonful of grasps" © 2017 ANTONELLO PELLICANO All Rights Reserved

Affordances are meaningful relations between the features of observed objects and the observer's action systems with its proper abilities. The notion of affordance integrates perceptual, cognitive and motor functions, so that perceiving an object, conducting cognitive operations on it, and executing motor actions with it cannot be considered as independent functions. Limb apraxia is a higher-order motor disorder that refers to disturbance of one or more of three domains: imitation of meaningless gestures, pantomime of meaningful gestures, and disturbance of interaction with objects. The first aim of the Research Topic was to put together theoretical and research contributions on affordance mechanisms to highlight their role in explaining apraxia deficits. The second aim was to clarify how studies on apraxia have implications for theories of affordances.

**Citation:** Pellicano, A., Borghi, A. M., Binkofski, F., eds. (2017). Bridging the Theories of Affordances and Limb Apraxia. Lausanne: Frontiers Media. doi: 10.3389/978-2-88945-225-5

# Table of Contents


*66 Response: Commentary: Viewing Photos and Reading Nouns of Natural Graspable Objects Similarly Modulate Motor Responses*

Giovanni Buccino and Barbara F. M. Marino

*68 Components of Action Representations Evoked When Identifying Manipulable Objects*

Daniel N. Bub, Michael E. J. Masson and Terry Lin

*80 Dissociation between Semantic Representations for Motion and Action Verbs: Evidence from Patients with Left Hemisphere Lesions*

Lawrence J. Taylor, Carys Evans, Joanna Greer, Carl Senior, Kenny R. Coventry and Magdalena Ietswaart

# **Affordances and Implications for Apraxia**


Carys Evans, Martin G. Edwards, Lawrence J. Taylor and Magdalena Ietswaart

*145 The Representation of Objects in Apraxia: From Action Execution to Error Awareness*

Loredana Canzano, Michele Scandola, Valeria Gobbetto, Giuseppe Moretto, Daniela D'Imperio and Valentina Moro

# Editorial: Bridging the Theories of Affordances and Limb Apraxia

Antonello Pellicano<sup>1</sup> \*, Anna M. Borghi 2, 3 and Ferdinand Binkofski <sup>1</sup>

<sup>1</sup> Division for Clinical and Cognitive Sciences, Department of Neurology Medical Faculty, RWTH Aachen University, Aachen, Germany, <sup>2</sup> Department of Dynamic and Clinical Psychology, Sapienza University of Rome, Rome, Italy, <sup>3</sup> Institute of Cognitive Sciences and Technologies, Italian National Research Council, Rome, Italy

Keywords: affordances, apraxias, stable/variable affordances, attention, language and affordances, handle-tohand correspondence effect

**Editorial on the Research Topic**

#### **Bridging the Theories of Affordances and Limb Apraxia**

Affordances are meaningful relations between the features of observed objects and the observer's action systems with its proper abilities. The notion of affordance integrates perceptual, cognitive and motor functions, so that perceiving an object, conducting cognitive operations on it, and executing motor actions with it cannot be considered as independent functions. Limb apraxia is a higher-order motor disorder that refers to disturbance of one or more of three domains: imitation of meaningless gestures, pantomime of meaningful gestures, and disturbance of interaction with objects. The first aim of the Research Topic was to put together theoretical and research contributions on affordance mechanisms to highlight their role in explaining apraxia deficits. The second aim was to clarify how studies on apraxia have implications for theories of affordances. Here we provide a summary of the contributions to the Research Topic. We will first discuss three issues related to the mechanisms underlying affordances and their implications for apraxia, then we will describe the studies directly focusing on apraxia.

# BROKEN HANDLES AND ATTENTION

Edited and reviewed by: Hauke R. Heekeren, Freie Universität Berlin, Germany

> \*Correspondence: Antonello Pellicano apellicano@ukaachen.de

Received: 09 January 2017 Accepted: 13 March 2017 Published: 29 March 2017

#### Citation:

Pellicano A, Borghi AM and Binkofski F (2017) Editorial: Bridging the Theories of Affordances and Limb Apraxia. Front. Hum. Neurosci. 11:148. doi: 10.3389/fnhum.2017.00148

Two studies investigated the role of attention in affordance perception for objects with broken handles. Ambrosecchia et al. investigated the handle-to-hand correspondence effect (CE) to support the affordance activation account, or the location coding account (attention-based Simon effect, see Pellicano et al., in press). A discrimination task was performed on graspable objects with intact and broken handles, preceded by a spatial Stimulus-Response Compatibility task with incompatible S-R mapping. The CE was eliminated with broken-handle objects, whereas it stayed significant with intact-handle objects. Thus, CE seems to depend on both affordance and attention mechanisms. Wulff and Humphreys also presented single objects and object-pairs (e.g., teapot + cup) with broken handles to patients with left visual extinction. In object-pairs the broken handle reduced the degree to which it captured attention, especially when the tool-object fell within the ipsilesional side. Thus, to facilitate affordance perception, patients should be trained on the contralesional side with action-pairs. Overall, both studies showed affordance effects that cannot be reduced to simple attentional effects.

# STABLE AND VARIABLE AFFORDANCES

The second conceptual node addressed in the Topic revolves around the notion of stable/variable affordance, and its eventual implications for apraxia. Borghi and Riggio proposed this distinction, Mizelle and Wheaton defended it; Osiurak argued instead that apraxia is not a matter of affordances. Indeed, on the base of three assumptions, Osiurak claimed that his mechanical knowledge hypothesis represents an alternative to the manipulation knowledge/stable affordances hypothesis (Binkofski and Buxbaum, 2013). First: The conception of tool use is based on allocentric knowledge of abstract mechanical principles. Second: The semantic knowledge of objects and tools is another form of allocentric knowledge, linking together different tools and objects when used in the same context or for the same target. Third: Affordances only translate the allocentric representation of the tool action into precise egocentric motor programs. Osiurak concluded that tool use apraxia is not a matter of affordances, but also that there is no distinction between variable and stable affordances: affordances are necessarily stable, because they must fit to human, biomechanical capacities, but are also temporary because they are perceived only as part of a specific goal. Mizelle and Wheaton commented on their model for Modular Selection for Action Goals (MSAG) in light of Osiurak's and Pellicano et al.'s (2011) articles. They contended that their MSAG model provides a preliminary framework to relate conceptual and motor "faults" to each other, which would reflect conceptual and ideomotor apraxias.

Borghi and Riggio presented the distinction between stable and variable affordances (Borghi and Riggio, 2009; Sakreida et al., 2016), and responded to the objections raised by Osiurak: even if they are not dichotomous, stable affordances (represented in the ventro-dorsal stream) emerge from characteristics less variable across contexts, as objects' size, whereas variable affordances (dorso-dorsal stream) from characteristics that change across contexts, as objects orientation. They reported that, during offline linguistic tasks, stable rather than variable affordances are recruited (Borghi, 2012): in line with the theory of reuse (Gallese, 2008) language recruits and modifies pre-existent mechanisms of the motor system. The authors also discussed whether automatic activation of affordances is challenged by task and context modulations: Automaticity and contextual dependency/flexibility are not necessarily in conflict, since the context can operate as a late filter. Importantly, the stable/variable distinction can address the automaticity issue: in offline tasks stable affordances are automatically activated, but also modulated by the task/context; in online tasks variable affordances are first activated. Overall, the authors of the contributions of this section debated to what extent the distinction between stable and variable affordances has implications for apraxia.

# LANGUAGE AND AFFORDANCES

Marino et al. investigated whether pictures and words of manipulable objects recruit the motor system in a similar way. They found slower responses with manipulable compared to non-manipulable objects independently from the responding hand. This cost is likely due to two concurrent tasks (i.e., stimulus processing and response production). The authors speculated that similar performance with nouns and photos can be either due to stable affordances being only coded, or to the fact that natural objects rather than tools were used. In his commentary, Makris contended that 150 ms after stimulus presentation is too early for an affordance effect to emerge. Makris argued that the effect could be attentional, and suggested an affordance competition interpretation (Cisek, 2007): graspable objects immediately catch exogenous attention, which are then redirected to non-graspable objects 150 ms after stimulus onset, leading to a rebalance of affordance-driven motor plans. Buccino and Marino recognize that the attentional hypothesis cannot be completely ruled-out; however it is unclear why attention would be captured only by graspable objects, since also non-graspable ones were presented abruptly. Bub et al. examined the influence of holding planned hand-actions in working memory for the time taken to identify handled objects. Their result suggested that the representation of the appropriate grasping action for one object is based on its canonical orientation, rather than on its contingently depicted orientation.

From their side, Taylor et al. found selective deficits in understanding motor action verbs in patients with lesions involving posterior, parietal, and lateral occipitotemporal cortex. In contrast, deficits in understanding motionless action verbs were found in patients with more anterior lesions sparing posterior parietal and lateral occipitotemporal cortex. They speculated that semantic representations for motion and motionless actions are behaviorally and neuro-anatomically dissociable. The findings presented in this section provide a hint toward the role of perceptual and motor regions in processing modality-specific semantic knowledge.

# AFFORDANCES AND IMPLICATIONS FOR APRAXIA

Michałowski and Króliczak criticized the fact that the understanding of tool representations is provided by investigations of right-handed individuals and their typical organization of cognitive and manual skills. They claimed that tool-related processing in left-handers with greater incidence of right-sided or bilateral (atypical) lateralization of functions is not just mirror reversed. Therefore, caution is required in neurorehabilitation directed at left-handed patients. Rounis and Humphreys based their mini review on the affordance competition hypothesis (Cisek, 2007). According to them, some aspects of apraxia may reflect an abnormal sensitivity to competition when multiple affordances are present and/or a poor ability to exert cognitive control over this competition when it occurs. This framework would help overcoming the distinction between ideomotor and ideational apraxia, and account for mixed symptoms from the two disorders.

Randerath and Frey scrutinized the role of affordance perception on feedback learning. Participants judged whether their hand would fit into a given aperture, and whether objects were reachable. Performance resulted worst for openings or distances close to the individual's physical limits. Feedback improved performance in both tasks suggesting a rapidly trainable affordance perception. Furthermore, feedback experience could transfer between hands.

Evans et al., tested the assumption that in Apraxia, stored object knowledge from the ventral stream is less readily available to incorporate into the action plan; leading to an over-reliance on visual affordances in object-directed motor behavior. Left-hemisphere stroke-patients, apraxiapatients, and healthy controls grasped cylindrical objects of varying weight distribution. Object weight was indicated by either a memory-associated or a visual-spatial cue. Apraxiapatients suggested impaired integration of visible and known object properties attributed to the ventro-dorsal stream. In learning to grasp the weighted object, they applied neither pure knowledge-based information (memory-associated condition) nor higher-level information (visual-spatial cue condition).

Canzano et al. review focused on how objects use helps to better understand apraxia. They considered transitive vs. intransitive action dissociation, and the less frequent constructive and magnetic apraxia. They also considered pantomime and objects imitation within a view to dissociating the various components involved in upper limb apraxia. They concluded that object knowledge and sensory-motor representations are further supported by spatial and body representations, executive functions, and monitoring systems.

In summary, the recent revival of the idea of affordances has led to further refinement of the concept, and opened new avenues to the understanding of the interaction with objects and tools.

# REFERENCES


This development kindled the research on brain representations of affordances. This special Topic provides emerging evidence that affordances code flexibly the dynamic interaction with objects. It seems that Apraxia-patients are capable only to utilize very basic affordances. We can thus speculate that observable apraxic deficits derive from the inability to utilize the flexible features of affordances.

# AUTHOR CONTRIBUTIONS

AP, AB, and FB gave a substantial contributions to the conception and the design of the work, drafted the work, revised the manuscript critically for important intellectual contents, agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved, gave a final approval of the version to be published. AP edited the final version of the manuscript.

# FUNDING

This work was supported by the START-Programme der Medizinischen Fakultät der RWTH-Aachen (START-Projekt 691240, 144/12).


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Pellicano, Borghi and Binkofski. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Spatial stimulus-response compatibility and affordance effects are not ruled by the same mechanisms

*Marianna Ambrosecchia1, Barbara F. M. Marino1,2, Luiz G. Gawryszewski3 and Lucia Riggio1\**

*<sup>1</sup> Sezione di Fisiologia, Dipartimento di Neuroscienze, Università di Parma, Parma, Italy, <sup>2</sup> Dipartimento di Psicologia, Università di Milano-Bicocca, Milano, Italy, <sup>3</sup> Neuroscience Program, Universidade Federal Fluminense, Rio de Janeiro, Brazil*

Stimulus position is coded even if it is task-irrelevant, leading to faster response times when the stimulus and the response locations are compatible (spatial Stimulus– Response Compatibility–spatial SRC). Faster responses are also found when the handle of a visual object and the response hand are located on the same side; this is known as affordance effect (AE). Two contrasting accounts for AE have been classically proposed. One is focused on the recruitment of appropriate grasping actions on the object handle, and the other on the asymmetry in the object shape, which in turn would cause a handle-hand correspondence effect (CE). In order to disentangle these two accounts, we investigated the possible transfer of practice in a spatial SRC task executed with a S–R incompatible mapping to a subsequent affordance task in which objects with either their intact handle or a broken one were used. The idea was that using objects with broken handles should prevent the recruitment of motor information relative to object grasping, whereas practice transfer should prevent object asymmetry in driving handle-hand CE. A total of three experiments were carried out. In Experiment 1 participants underwent an affordance task in which common graspable objects with their intact or broken handle were used. In Experiments 2 and 3, the affordance task was preceded by a spatial SRC task in which an incompatible S–R mapping was used. Inter-task delays of 5 or 30 min were employed to assess the duration of transfer effect. In Experiment 2 objects with their intact handle were presented, whereas in Experiment 3 the same objects had their handle broken. Although objects with intact and broken handles elicited a handle-hand CE in Experiment 1, practice transfer from an incompatible spatial SRC to the affordance task was found in Experiment 3 (broken-handle objects), but not in Experiment 2 (intact-handle objects). Overall, this pattern of results indicate that both object asymmetry and the activation of motor information contribute to the generation of the handle-hand CE effect, and that the handle AE cannot be reduced to a SRC effect.

#### *Edited by:*

*Antonello Pellicano, RWTH Aachen University, Germany*

#### *Reviewed by:*

*Sanjay Kumar, Oxford Brookes University, UK Heath Eric Matheson, University of Pennsylvania, USA*

#### *\*Correspondence:*

*Lucia Riggio, Sezione di Fisiologia, Dipartimento di Neuroscienze, Università di Parma, via Volturno 39, 43125 Parma, Italy riggio@unipr.it*

> *Received: 22 December 2014 Accepted: 29 April 2015 Published: 18 May 2015*

#### *Citation:*

*Ambrosecchia M, Marino BFM, Gawryszewski LG and Riggio L (2015) Spatial stimulus-response compatibility and affordance effects are not ruled by the same mechanisms. Front. Hum. Neurosci. 9:283. doi: 10.3389/fnhum.2015.00283*

Keywords: affordance effect, Simon effect, spatial S–R compatibility, transfer of practice, intact and broken handle

# Introduction

Several studies corroborated the notion that the environment is perceived not only in terms of object visual properties or qualities, but also in terms of object possibilities for action (affordances; Gibson, 1977, 1979/1986). To date, there is ample neurophysiologic evidence demonstrating that the mere observation of common graspable objects recruits the fronto-parietal circuits for object manipulation, both in monkeys (Jeannerod et al., 1995; Murata et al., 1997; Raos et al., 2006; Umilta et al., 2007) and in humans (Grafton et al., 1997; Chao and Martin, 2000; Grèzes et al., 2003; Buccino et al., 2009; Cardellicchio et al., 2011).

At a behavioral level, Tucker and Ellis (1998, 2004), using a Stimulus–Response Compatibility (SRC) paradigm, have presented evidence that visual objects lead to activation of specific components of actions they afford. In a seminal experiment, participants had to judge the vertical orientation (upright or inverted) of common objects with a graspable part (their handle), emitting lateralized key-press responses. Faster reaction times (RTs) were found when the handle and the response key corresponded than when they were on opposite sides, resulting in an affordance effect (AE, Tucker and Ellis, 1998).

Spatial SRC effect (Fitts and Seeger, 1953) refers to faster RTs, in a two-choice key-press task when both the locations of the stimulus and of the response correspond. It happens even if the encoding of the spatial features is not relevant for the response, as in the Simon effect (SE, Simon and Rudell, 1967; Simon, 1969). In a typical Simon task, geometrical shapes are presented on the right or left of the fixation point and participants are instructed to respond to a non-spatial stimulus dimension, such as shape or color, with right or left responses. If, for example, they have to respond with the right response key to green stimuli and with the left response key to red ones, they are faster when green and red stimuli appear on the right and the left side, respectively. As assumed by the dual-process model of the SE (e.g., Kornblum et al., 1990; De Jong et al., 1994), two different response codes would be activated by the stimulus: a code that is automatically activated (primed) by the stimulus spatial location, and a code that is activated by the instructions given to the participants. The corresponding trials lead to more efficient performance because both codes activate the same response. Performance on non-corresponding trials, in contrast, is slower and less accurate because competing responses are activated at the response selection, thus generating a conflict that must be solved before response execution.

It has been suggested that common mechanisms may underlie both SE and AE, given that both effects are based on the spatial relation between the response and the location of the object, or part of it (the handle). Indeed, the object handle could prime responses in its side because of its saliency (e.g., Anderson et al., 2002; Cho and Proctor, 2010, 2011), facilitating the corresponding responses but not the non-corresponding ones, and giving rise to a handle-hand SE. Some authors, indeed, found evidence supporting the hypothesis that AE, far from being the product of the potentiation of appropriate actions to graspable objects, can be the result of an attentional bias toward the handle side of the object. This bias would be produced by the asymmetry of the object, which renders the handle more salient than other object parts (e.g., Anderson et al., 2002; Matheson et al., 2014), thus capturing attention. In keeping with the attentionshift account of the AE (Nicoletti and Umiltà, 1994; Rubichi et al., 1997), this attentional bias would generate a spatial response code, priming the corresponding response (Anderson et al., 2002; Cho and Proctor, 2010; Kostov and Janyan, 2012) as in the typical SE (Proctor and Vu, 2012). However, as showed by Pappas (2014), the results of these studies (e.g., Cho and Proctor, 2010) might be due to the nature of the used stimuli. The author, indeed, compared participants' performance with naturalistic (photographs) and non-naturalistic stimuli (silhouettes). His findings indicate that the amount of internal details of the objects and the environmental information might be critical to dissociate between SE and AE.

Contrary evidence has also been collected in favor of actionbased mechanisms. For example Riggio et al. (2008) reported data supporting the independence between AE and SE. They found that the AE, when evident, was always relative to the target object, irrespective of its attentional capturing properties, whereas the SE occurred relative to the event capturing attention (see also Phillips and Ward, 2002; Symes et al., 2005; Janyan and Slavcheva, 2012). Some studies found that the two effects seem to depend on the stimulus properties being processed in order to perform the task (Pellicano et al., 2010), and that their interaction relies upon the type of action that is required (Iani et al., 2011). Using the same task in which graspable objects were presented, and varying the instructions (i.e., to respond to the color of the object vs. its vertical orientation), either a Simon-like effect or an AE emerged (Pellicano et al., 2010). These two effects interacted in response times but not in reaching movements time (Iani et al., 2011). Furthermore, in a TMS study, Buccino et al. (2009) found that the recruitment of the motor system depends on the graspability of the handle itself, which clearly supports hypotheses based on an action-based role of the handle. In fact when a visual object with a broken handle was presented, that is when the object most important feature relevant for action was violated, motor programs triggered by the handle were violated as well, resulting in significantly reduced MEPs area than when objects were presented with an intact handle.

To investigate whether SE and AE are ruled by common mechanisms, we took advantage of a peculiar feature of SE, that is its susceptibility to the influence of previous practice with an incompatible spatial SRC task (Tagliabue et al., 2000; Vu et al., 2003; Vu, 2007; Creekmur and Vu, 2012). In the transfer of practice paradigm, participants first perform a spatial compatibility task in which they are required to respond with a S–R compatible or incompatible mapping to the stimulus right–left location and then the Simon task. It has been shown that a S–R incompatible mapping eliminates or even reverses the SE (Tagliabue et al., 2000; Vu et al., 2003; Proctor et al., 2007). This demonstrates that the spatial associations between stimulus and response locations defined in the practice task, when stimulus location was relevant, remain active during the subsequent Simon task, when stimulus location is irrelevant. More importantly, this result also shows that in both tasks the same mechanisms are at work since the strategy acquired in the spatial compatibility task (i.e., the strengthening of opposite sides S–R association) transfers to the Simon task.

We explored the possible transfer of practice effect from a spatial SRC task executed with a S–R incompatible mapping to a subsequent affordance task (Tucker and Ellis, 1998). We also manipulated the time between the two tasks (Tagliabue et al., 2000) to assess the possible modulation of time on the duration of the transfer of practice.

Stimuli were common graspable objects with an intact or a broken handle. In the first condition the crucial feature for the expression of AE is preserved, in the second condition it is missing, but the asymmetry of the object and the saliency of the handle are still present. Since only objects with an intact handle should activate specific grasping motor programs, we predicted different practice effects from a spatial SRC task to an affordance task according to the status of the object handle, if the AE and the SE are related to different mechanisms. In order to dissociate between AE and SE we ran three experiments. In Experiment 1 participants had to decide, by pressing a right or a left key, whether pictures depicting common objects, either with an intact or a broken handle, were upright or inverted (affordance task).

In Experiment 1 participants underwent only the affordance task, while in Experiments 2 and 3 the affordance task was preceded by a spatial SRC task executed with a S–R incompatible mapping. In Experiment 2 the objects displayed an intact handle, whereas in Experiment 3 they had a broken handle. In this way we explored the possible practice transfer from an incompatible spatial SRC task to a subsequent intact or broken handle affordance task. If the AE is a Simon-like effect we should expect that it would be nullified or reversed after an incompatible practice as it is for the SE (Proctor and Lu, 1999; Tagliabue et al., 2000; Vu et al., 2003; Vu, 2007) either when the handle is intact or broken. In contrast, if AE and SE are the result of different mechanisms then we should not expect transfer of practice from a spatial SRC to an affordance task, at least when objects with the intact handle are used. The inter-task delay was manipulated in both experiments (5 vs. 30 min).

# Materials and Methods

# Experiment 1

Experiment 1 was designed to test if the handle-hand correspondence effect (CE) occurs relative to the object's handle, and if its magnitude depends on the graspability of the handle. To this aim, participants were required to respond to upright or inverted graspable objects presented at the center of a computer screen. Objects could have their handle intact or broken.

# Participants

Twenty-four undergraduate students (16 females; mean age = 22.5 ± 4.5) from the University of Parma volunteered to take part in this experiment. All participants were right-handed as assessed by the Edinburgh Handedness Inventory (Oldfield, 1971), had normal or corrected-to-normal vision and were naïve as to the purpose of the experiment. The experimental protocol

was approved by the Ethics Committee of the University of Parma. The experiment was conducted in accordance with the ethical standards of the 1964 Declaration of Helsinki and all participants gave written informed consent.

# Procedure

The experiment was carried out in a sound-attenuated room, dimly illuminated by a halogen lamp directed toward the ceiling. The participants were tested individually. They sat comfortably in front of a computer screen (Philips monitor with a resolution of 1024 × 768 pixels, interfaced with a Pentium 2.80 GHz computer equipped with a NVIDIA GeForce 7300 LE video Board), mounted in a wooden frame and covered by a gray cardboard, except for a 18 cm × 25.5 cm window where the stimuli were displayed. Participants had their head supported by a chin rest in order to maintain a stable head position and keep their eyes at a constant distance from the screen (about 57 cm). Eye height was adjusted to the level of fixation.

Stimuli presentation and response collection were controlled by E-Prime software system. The stimuli consisted of a series of images of four common objects (a coffee pot, a milk jug, a teacup, and a coffee cup; see **Figure 1**) presented in two vertical orientations (upward or inverted). Each object was inserted in a 157 × 126 pixels matrix and was displayed in gray scale on a black background at the center of the screen. All objects had a handle, intact or broken, oriented to the right or the left (suitable for a right-hand or a left-hand grasp).

Responses were executed by pressing the "P" or the "Q" key of the QWERTY keyboard with the left and the right index finger, respectively. The response keys were in symmetrical locations to the right and the left of the body midline. Participants were requested to keep their index fingers on the keys during the experiment.

Each trial began with the presentation of the fixation cross (22 × 22 pixels), replaced after 500 ms by an upward or inverted object in the center of the screen, with its handle oriented to the right or to the left. Twelve participants executed the task with intact handled objects and twelve with broken handled objects. Objects were displayed until a response was given; if the response did not occur within 1000 ms, the object disappeared. Half of the participants were instructed to make a left key-press (Q) for upright objects and a right key-press (P) for inverted objects. The remaining participants experienced the reverse mapping. Visual

Ambrosecchia et al. S–R compatibility and affordance effects

feedback on speed and accuracy was provided for 500 ms in the center of the screen after each response.

The experiment consisted of 160 experimental trials divided into two blocks of 80 trials each. The first block was preceded by sixteen familiarization trials with the same stimuli used in the experimental trials. For each handle condition (intact or broken) an equal number of trials was provided for each combination of the following variables: object Orientation (upward vs. inverted) and handle-hand Correspondence (corresponding vs. non-corresponding).

The correct mean response latencies (RTs) and accuracies (following arcsine transformations) were entered into two analyses of variance (ANOVAs), with object Orientation (up vs. down) and hand-handle Correspondence (corresponding vs. non-corresponding) as within-subjects variables, and Handle (intact vs. broken) as a between-subjects variable. Whenever appropriate, *post hoc* analyses were conducted using the Tukey's HSD (honest significant difference) test in order to control for both the Type I and Type II errors, since it is not only more conservative (e.g., the Newman–Keuls method), but also more powerful than other procedures (e.g., the Bonferroni method; Perneger, 1998).

# Results and Discussion

Familiarization trials were discarded from the analysis. Overall errors (wrong responses and missing responses) were 7.4% of the dataset. Responses either longer or shorter than two SDs from the individual mean were treated as outliers and not considered in the analysis (2% of the data set).

The ANOVA on Accuracy revealed that only the main effect of Correspondence was significant (*F*1,<sup>22</sup> = 15.81; *p* < 0.01; <sup>η</sup><sup>2</sup> <sup>=</sup> 0.42), indicating that participants were more accurate for corresponding (mean = 95%, SE = 1.43) than for noncorresponding trials (mean = 90.16%, SE = 1.42)

Similarly, the ANOVA on RTs revealed that only the main effect of Correspondence was significant (*F*1,<sup>22</sup> = 43.91; *p* < 0.001; η<sup>2</sup> = 0.66). Importantly both the analysis on Accuracy and the analysis on RTs showed no real difference on overall responses between objects that had an intact or a broken handle pointing out that they were equally recognizable. The interaction between Correspondence and Handle was not significant (*F*1,<sup>22</sup> <sup>=</sup> 0.012; *<sup>p</sup>* <sup>&</sup>gt; 0.9; <sup>η</sup><sup>2</sup> <sup>=</sup> 0.001) demonstrating that the magnitude of the handle-hand CE when the handle was either intact or broken is very similar [- RTs (non-corresponding – corresponding) = 29.7 ms, SE = 4.4 vs. 28.7 ms, SE = 7.6; see **Figure 2**].

As evidenced by a large number of studies, Experiment 1 confirms that when objects with intact handles oriented to the right or to the left are displayed, responses are faster if the location of the response corresponds to the location of the handle. Notably, the same effect is also obtained when objects with a broken handle are shown. Since in this case the handle does not afford grasping, the handle-hand CE cannot be regarded as an AE, suggesting that it might be produced by the asymmetry of the object more than the pragmatic role of the handle. Given the similarity of results when objects with an intact or broken handle are presented, a parsimonious explanation might refer both effects to

an attentional bias toward the asymmetrical part of the object, i.e., the handle intact or broken. Matheson et al. (2014) reached a similar conclusion in a study in which asymmetrical manipulable artifacts and non-manipulable animals were compared in a SRC paradigm. With both types of stimuli, compatibility effects were reported. However, as the same authors argued, the fact that compatibility effects were obtained both with artifacts and animals does not exclude *per se* that different mechanisms could be at work. Therefore, on the basis of studies showing that the AE is not merely a kind of SE (Symes et al., 2005; Riggio et al., 2008; Buccino et al., 2009; Pappas, 2014), we think that the similarity between the two handle-hand CE, found in the present study, is only apparent. Thus solely the observed handle-hand CE when objects with their intact handle were displayed might be due to the recruitment of handle grasping information. The study of Buccino et al. (2009), which to our knowledge is the unique study that used objects with intact and broken handles, seems to support this interpretation. In fact, it clearly demonstrated that the status of the handle is critical in the recruitment of the motor system. Hence Experiments 2 and 3 specifically try to dissociate the handle-hand CE due to the intact and the broken handle.

#### Experiment 2

In order to investigate whether SE and AE depend on the same mechanisms, in this experiment we explored the possible practice transfer from a spatial SRC task executed with a S–R incompatible mapping to a subsequent affordance task with objects having their intact handle (Tucker and Ellis, 1998). The time between the two tasks has been manipulated (5 vs. 30 min).

# Participants

Twenty-eight undergraduate students (12 females; mean age 21.66 ± 3.71) volunteered to take part in this experiment. None of them took part in the previous experiment. They were righthanded, had normal or corrected-to-normal vision and were naïve as to the purpose of the experiment. All participants gave written informed consent. Experiments were conducted in accordance with the ethical standards of the 1964 Declaration of Helsinki.

# Procedure

In this experiment participants had to perform two successive tasks: a spatial SRC task with an incompatible S–R mapping and, after a delay of 5 or 30 min, an affordance task. Fourteen participants executed the affordance task with objects having their intact handle with the inter-task interval of 5 min and fourteen executed the task with the inter-task interval of 30 min.

# The Spatial SRC Task

Each trial began with the presentation of a central fixation cross (22 × 22 pixels), followed after 500 ms by a white circle (30 × 30 pixels) presented 10◦ (center to center) to the right or to the left of the fixation. The circle remained until a response was given, but anyway no longer than 1000 ms.

Participants were instructed to respond to the location (left or right) of the circle by pressing the key on the opposite side, that is the left response key (Q) when the circle compared to the right and the right response key (P) when it compared to the left. Visual feedback on speed and accuracy was provided after response was given.

The task was composed by 160 experimental trials divided into two blocks of 80 trials each. Sixteen familiarization trials preceded the first block. An equal number of trials was provided for the left and right stimulus location.

A two-tailed independent samples *t*-test was carried out both on mean RTs for correct responses and accuracies (following arcsine transformations), comparing the performance in the practice task between the two groups of participants (participants who executed the affordance task after 5 or 30 min the practice task).

#### The Intact Handle Affordance Task

Stimuli and procedure of the affordance task were the same as in the Experiment 1. Mean RTs for correct responses and accuracies (following arcsine transformations) were submitted to two ANOVAs with Orientation (up vs. down) and handlehand Correspondence (corresponding vs. non-corresponding) as within-subjects variables, and inter-tasks Time (5 vs. 30 min) as a between-subjects variable. A further ANOVA was conducted on - RTs comparing the magnitude of the handle-hand CE among the three practice conditions used in Experiments 1 and 2. The handle-hand CE was entered in a one-way ANOVA with Practice (absent vs. 5 min before the task vs. 30 min before the task) as a between-subject variable. *Post hoc* analyses were conducted using the Tukey's HSD method.

### Results and Discussion *Results of the Spatial SRC Task*

Familiarization trials were discarded from the analysis. Errors (wrong responses about the position of the circle and missing responses) were 1% of the total trials. Responses either longer or shorter than two SDs from the individual mean were treated as outliers and not considered (3.8% of the data set).

Both *t*-test comparisons on accuracy (*t*<sup>26</sup> = 0.07, *p* > 0.9; 2-tailed) and RTs (*t*<sup>26</sup> = −1.43, *p* > 0.1; 2-tailed) showed that the performance in the practice task did not differ between the group of participants assigned to the 5 min condition (mean RTs = 388 ms, SE = 10; mean accuracy = 98%, SE = 1) and the group assigned to the 30 min condition (mean RTs = 417 ms, SE = 4; mean accuracy = 98%, SE = 0.5).

# *Results of the Intact Handle Affordance Task*

Familiarization trials were discarded from the analysis. Errors (wrong responses about orientation of the object and missing responses) were 5% of the total trials. Responses either longer or shorter than 2 SDs from the individual mean were treated as outliers and not considered (0.3% of the data set).

The ANOVA on Accuracy showed that only the main effect of Correspondence was significant (*F*1,<sup>26</sup> = 23.17; *p* < 0.01; <sup>η</sup><sup>2</sup> <sup>=</sup> 0.47), with more accurate responses in corresponding trials (mean = 96%, SE = 1.84) than in non-corresponding ones (mean = 90%, SE = 1.98).

The ANOVA on RTs revealed that the main effect of Time was significant (*F*1,<sup>26</sup> = 10.99; *p* < 0.01; η<sup>2</sup> = 0.30) showing that responses were faster (mean = 469 ms, SE = 10.4) when the two tasks were separated by 5 min than by 30 min (mean = 518 ms, SE = 10.4). Also the main effect of Correspondence was significant (*F*1,<sup>26</sup> = 50.08; *p* < 0.01; η<sup>2</sup> = 0.61) with faster RTs in corresponding (mean = 481 ms, SE = 7.10) than non-corresponding trials (mean = 506 ms, SE = 8). Moreover Correspondence interacted significantly with Time (*F*1,<sup>26</sup> <sup>=</sup> 6.55; *<sup>p</sup>* <sup>&</sup>lt; 0.05; <sup>η</sup><sup>2</sup> <sup>=</sup> 0.2). *Post hoc* comparisons revealed a handle-hand CE of a smaller magnitude in the shorter than in the longer inter-tasks condition (-<sup>=</sup> 16 ms, SE <sup>=</sup> 4 vs. 35 ms, SE <sup>=</sup> 6; *<sup>p</sup>* <sup>&</sup>lt; 0.03 see **Figure 3**).

The ANOVA on - RTs showed that the main effect of Practice was significant (*F*2,<sup>37</sup> <sup>=</sup> 3.88; *<sup>p</sup>* <sup>&</sup>lt; 0.03; <sup>η</sup><sup>2</sup> <sup>=</sup> 0.2). The magnitude of the handle-hand CE in the 5 min delay before the Affordance task was smaller than the 30 min delay condition (mean = 16, SE = 4.85 vs. mean = 35 ms, SE = 4.85; *p* < 0.03) and largely reduced than the no practice condition (mean = 16 ms, SE = 4.85 vs. mean = 30 ms, SE = 5.24; *p* = 0.16) even if the difference was not significant. Moreover, the magnitude of the handle-hand CE in the 30 min delay condition did not differ significantly from the no practice condition (mean = 35 ms, SE = 4.85 vs. mean = 30 ms, SE = 5.24; *p* > 0.4).

FIGURE 3 | Mean RTs during the affordance task in Experiment 2 for corresponding and non-corresponding trials as a function of inter-tasks time. Error bars depict SEs of the means. The asterisk indicates a statistical significance between the means.

The data demonstrated that the handle-hand CE is modulated by previous practice with a smaller magnitude in the short but not in the long inter-tasks time condition (see **Figure 4**).

Although, our results may be connected to a briefer effect of incompatible practice on AE than on SE, previous studies have found that the magnitude of the handle-hand CE depends on the response speed (see Supplementary Material), with an increasing magnitude over time (Phillips and Ward, 2002; Fischer and Dahl, 2007; Proctor et al., 2011). This means that participants would exhibit a handle-hand CE smaller in fast than in slow RTs. Since in this experiment participants' performance was significantly faster in the 5 min than in the 30 min time condition, it follows that the reduction of the magnitude of the handle-hand CE in the

5 min condition may be due to response speed rather than to a short transfer effect of practice on AE. To disentangle the role of practice and response speed, following the Vincentization procedure introduced by Ratcliff (Ratcliff, 1979), we divided the RT distributions for each participant, and for the two levels of Time variable, into four quantiles (bins) and we computed mean RTs for each quantile (for further analyses on Experiments 1 and 2, see Supplementary Material).

This RT distribution analysis indicated that in order to eliminate the effect due to differences in response speed, it was necessary to compare only overlapping bins: the first three bins of the 30 min condition with the last three bins of the 5 min condition. To make this kind of comparisons we adopted three two-tailed independent samples *<sup>t</sup>*-test (see **Figure 5**; all *<sup>p</sup>*<sup>s</sup> <sup>&</sup>gt; 0.10). Based on the results of these analyses, mean data for the second, third, and fourth bins of the 5 min condition and the first, second, and third bins of the 30 min condition were entered in the ANOVA. Bin and Correspondence were within-subjects variables, and Time (5 vs. 30 min) was a between-subjects variable. As before, when necessary, *post hoc* comparisons were performed using the Tukey's HSD method.

Besides the main effect of Bin, (*F*2,<sup>52</sup> = 801.90, *p* < 0.001; <sup>η</sup><sup>2</sup> <sup>=</sup> 0.97), the analysis revealed the main effect of Correspondence (*F*1,<sup>26</sup> = 52.72, *p* < 0.01; η<sup>2</sup> = 0.68). The interaction between Bin and Correspondence was also significant (*F*2,<sup>52</sup> = 5.22, *p* < 0.005; η<sup>2</sup> = 0.17), showing that the magnitude of the handle-hand CE increases as reaction times become slower (Bin 1: - = 19.27 ms; Bin 2: - = 27.87 ms; Bin 3: - = 32.3 ms; all *p*s < 0.001), as typically shown for the AE (Phillips and Ward, 2002; Fischer and Dahl, 2007; Proctor et al., 2011). However, the interaction among Bin, Correspondence and Time was not significant (*F*12,<sup>52</sup> <sup>=</sup> 0.72, *<sup>p</sup>* <sup>&</sup>gt; 0.5; <sup>η</sup><sup>2</sup> <sup>=</sup> 0.27), revealing that

the reduction of the magnitude of the hand-handle CE in the 5 min was due to the response speed rather than to the previous practice in incompatible spatial SRC. Hence, although one possible interpretation of the larger handle-hand CE at 30 min is that the interference of the prior practice has worn off by then in this task, our results support, instead, the absence of a transfer effect of the incompatible practice on the AE.

#### Experiment 3

Experiment 2 showed no transfer effect from an incompatible SRC task to a subsequent intact Handle Affordance Task supporting different acting mechanisms in the two tasks. Experiment 3 was set to assess the transfer effect when objects having a broken handle are presented in the affordance task.

# Participants

Twenty-eight undergraduate students (19 females; mean age 20.82 ± 4.26) volunteered to take part in this experiment. All students were right-handed, had normal or corrected-to-normal vision and were naïve as to the purpose of the experiment. All participants gave written informed consent. Experiments were conducted in accordance with the ethical standards of the 1964 Declaration of Helsinki.

### Procedure

As in Experiment 2, participants had to perform two tasks: the incompatible spatial SRC task and, after a delay of 5 or 30 min, the affordance task. However, in this case all the objects were presented with the broken handle. Fourteen participants executed the task with the inter-task interval of 5 min and 14 executed the task with the inter-task interval of 30 min.

For the practice task, a two-tailed independent samples *t*-test was carried out both on mean RTs for correct responses and accuracies (following arcsine transformations) comparing the performance to the practice task between the two group of participants (participants who executed the affordance task after 5 or 30 min the practice task).

Regarding the Broken Handle Affordance Task, correct responses (Accuracy) and mean RTs for correct responses were submitted to an ANOVA with Orientation (up vs. down) and Correspondence (corresponding vs. non-corresponding) as within-subjects variables, and inter-tasks Time (5 min vs. 30 min) as a between-subjects variable. A further ANOVA was conducted on -RTs comparing the magnitude of the handle-hand CE in the two inter-task Time (5 min vs. 30 min) conditions. As before *post hoc* comparisons were performed using the Tukey's HSD method.

# Results and Discussion

# *Results of the Spatial SRC Task*

Familiarization trials were discarded from the analysis. Errors (wrong responses about the position of the circle and missing responses) were 1% of the total trials. Responses either longer or shorter than 2 SDs from the individual mean were treated as outliers and not considered (3.9% of the data set).

Both *t*-test comparisons on accuracy (*t*<sup>26</sup> = 0.65, *p* > 0.5; 2-tailed) and RTs (*t*<sup>26</sup> = 1.65, *p* > 0.1; 2-tailed) showed that the performance in the practice task did not differ between the group of participants assigned to the 5 min condition (mean RTs = 419 ms, SE = 16; mean accuracy = 98%; SE = 0.5) and the group assigned to the 30 min condition (mean RTs = 380 ms, SE = 16; mean accuracy = 98%; SE = 0.5).

# *Results of the Broken Handle Affordance Task*

Familiarization trials were discarded from the analysis. Overall errors (wrong responses about orientation of the object and missing responses) were 5.6% of the total trials. Responses either longer or shorter than 2 SDs of the individual mean were treated as outliers and not considered (0.6% of the data set).

Both the analyses on Accuracy and RTs did not reveal any significant effect or interaction. In particular, neither the handlehand Correspondence (*F*1,<sup>26</sup> <sup>=</sup> 1.37; *<sup>p</sup>* <sup>=</sup> 0.064; <sup>η</sup><sup>2</sup> <sup>=</sup> 0.05), nor the interaction between handle-hand Correspondence and Time (*F*1,<sup>26</sup> = 0.25; *p* = 0.61; η<sup>2</sup> = 0.01) were significant in the RT analysis, although corresponding trials were slightly faster than non-corresponding ones (512 ms, SE = 11.07 vs. 518 ms, SE <sup>=</sup> 11.21; see **Figure 6**).

# Experiment 2 vs. Experiment 3

In order to compare the handle-hand CE in Experiments 2 and 3, we ran a bin distributional analysis, computing correct means RTs from the first to the fourth bin of the individual rank-ordered raw data separately for the Handle Status (intact vs. broken) and Time (5 min vs. 30 min) variables using the same procedure adopted in the Experiment 2.

Mean data were entered into an ANOVA in which we considered only overlapping bins (see **Figure 7**; all *<sup>p</sup>*<sup>s</sup> <sup>&</sup>gt; 0.05; pairwise *t*-test comparisons). Correspondence (corresponding vs. non-corresponding) and Bin were within-subjects variables and Handle Type (Intact vs. Broken) and Inter-task Time (5 vs. 30 min) were between-subjects variables. As before, when necessary, *post hoc* comparisons were performed using the Tukey's HSD method.

Besides the main effect of Bin, (*F*2,<sup>104</sup> = 1220.40; *<sup>p</sup>* <sup>&</sup>lt; 0.001; <sup>η</sup><sup>2</sup> <sup>=</sup> 0.96), the analysis revealed the main effect of Correspondence (*F*1,<sup>52</sup> = 25.02; *p* < 0.01; η<sup>2</sup> = 0.33). The interaction between Correspondence and Handle was also significant (*F*1,<sup>52</sup> <sup>=</sup> 17.97; *<sup>p</sup>* <sup>&</sup>lt; 0.01; <sup>η</sup><sup>2</sup> <sup>=</sup> 0.26). *Post hoc* comparisons showed that after an incompatible spatial SRC practice, the handle-hand CE occurs only in the intact handle affordance task

(broken handle; in black). Error bars depict SEs of the means.

(intact handle affordance task = 478 ms, SE = 9 vs. 505 ms, SE = 9, *p* < 0.001; broken handle affordance task = 476 ms, SE = 9 vs. 478 ms, SE = 9, *p* = 0.5; for corresponding and non-corresponding trials, respectively; see **Figure 8**).

# General Discussion

The present study aimed at disentangling between the two main accounts of the handle-hand CE: the recruitment of motor programs for interacting with the object and the orienting of attention toward the asymmetrical part of the object. To this end, we assessed the possible transfer of practice from a prior incompatible spatial SRC task to an affordance task in which objects, with an intact or broken handle, were presented. Indeed, while the presence of the intact handle makes the recruitment of motor programs for handle grasping possible, a broken handle prevents it.

Results of the Experiment 1 showed a handle-hand CE of the same magnitude regardless of the handle status (30 vs. 29 ms for the intact and broken handle, respectively). This result may support the orienting attention hypothesis (e.g., Anderson et al., 2002; Cho and Proctor, 2010), and hence the fact that the AE could be simply reduced to a SE, due to the asymmetry of the object; or, in other words, to the spatial relation between the location of the handle, either when it is intact or broken, and the location of the response. If this was true, and hence the two effects would be based on the same mechanisms, we should expect that both of them would be nullified or reversed after an incompatible practice (Proctor and Lu, 1999; Tagliabue et al., 2000; Vu et al., 2003; Vu, 2007).

This is what we observed in Experiment 3, where objects with their broken handle were presented. The handle-hand CE after the incompatible practice was eliminated with an inter-task interval of both 5 and 30 min (no practice = 29; 5 min = 5; 30 min = 6 ms). In contrast, in Experiment 2, in which objects with their intact handle were used, no real transfer effect was found. In fact, although the handle-hand CE apparently diminished after a delay of 5 min in comparison to both the 30 min delay and the task executed without prior practice (no practice = 30; 5 min = 16; 30 min = 35 ms), it may be that this reduction depends on different response speed between the two delay time conditions. It is well known that AE increases as the RTs increase (Phillips and Ward, 2002; Fischer and Dahl, 2007; Proctor et al., 2011). Thus, to assess the modulation of the prior practice on the subsequent affordance task, it is important to consider the response speed by comparing similar RTs. After such a RT adjustment, the handle-hand CE seems not to be influenced by the prior incompatible practice when objects having their intact handle were presented, in agreement with an AE account. Conversely, the handle-hand CE was eliminated by the prior incompatible practice when objects with their handle broken were shown, as was expected for a SE. These results suggest that the AE and SE may be considered independent.

However, since the handle makes objects asymmetric, determining a bias for attentional shift, it is reasonable to wonder whether the handle-hand CE comprises also a SE, as it could be deduced from the presence of the handle-hand CE in Experiment 1 with objects with a broken handle. In this condition indeed, the broken handle, because of its non-graspability, does not trigger manual motor programs. There are more than one reason against this idea: the magnitude of the handlehand CE observed in Experiment 1 does not differ between intact and broken handle objects and, when the handle is intact and graspable, the practice does not modulate it. On the contrary the handle-hand CE becomes null when the handle is broken suggesting the presence of a SE. Therefore, the two effects (AE and SE) are not additional but seem to compete.

These results are in line with Symes et al. (2005) and Riggio et al. (2008). In the first study the authors by manipulating the orientation and the location of the objects found two separate SRC effects at different levels of attentional demand; the first one was a SE, which appeared alone when the attentional demand was low; the second one, due to the orientation of the handle, required that the object was coded as an object. In the second study, the event capturing attention was manipulated to assess the role of attention in the emergence of AE and SE. The authors found that the AE, when evident, was always relative to the target object, irrespective of its attentional capturing properties; while the SE was present in relation to the event capturing attention. A recent study by Wilf et al. (2013) gave additional evidence of the independence between the SE and AE. These authors, by measuring button-press and electromyography (EMG) responses, found the presence of spatial SRC from the earliest stages of movement preparation and throughout the different stages of movement execution. In contrast, the AE was evident only in the early stages of movement execution, although this effect has been only related to a general motor system activation, and not specifically connected to a body-part. They tested also a small group of unilateral amputees using EMG and found residual spatial SRC but no AE.

Unlike our results, a recent study by Ottoboni et al. (2013) found that an incompatible practice eliminated the AE. Although these authors did not aim at directly dissociate between SE and AE, they interpreted this result as an evidence supporting that AE and SE share some similarities (Iani et al., 2011; McBride et al., 2012). A reason for the difference between their and our study could regard the different amount of practice in the two studies (160 vs. 600 trials in the present and in Ottoboni et al.' (2013) study, respectively).

Several studies (e.g.,Tagliabue et al., 2000; Vu et al., 2003; Vu, 2007) showed that the transfer from a spatially incompatible practice to a subsequent Simon task is already evident after 72 practice trials and it lasts up to 1 week (Tagliabue et al., 2000). Vu (2007) indicated, however, that while a short practice (e.g., 72 trials) could be sufficient to give rise to a "within-dimension transfer effect" (e.g., from an incompatible horizontal SRC practice to a subsequent horizontal Simon task), a "between-dimension transfer effect" (e.g., from an incompatible horizontal SRC practice to a vertical Simon task) needs up to 600 trials of incompatible practice to emerge. Some authors (Wiegand and Wascher, 2005; Vu, 2007) suggested that the within-dimension transfer effect may be due to the short-term spatial S–R associations, acquired with short practice, overriding the long term associations, and the between-dimension transfer effect may be due to the acquisition of a more general strategy of giving a response opposite to stimulus location (Vu, 2007). Also, Marini et al. (2011) reported that after 600 trials, practice transfer is present even if the two tasks do not share any spatial irrelevant dimension. They found a significant reduction of the subsequent color Stroop effect (Stroop, 1935; see MacLeod, 1991 for a review) after an incompatible spatial SRC practice. However, this transfer effect was absent if the subsequent Stroop task did not require the same response modality of the practice task (i.e., vocal responses instead of bimanual responses). These findings demonstrated that in order for the transfer effect to appear, rather than the dimensional overlap between stimuli and responses of the two tasks, it needs the dimensional overlap between the responses of the two tasks. It seems that after 600 trials of practice participants learned to emit the response alternative to the one automatically activated and that such a rule transferred into the following task. This cognitive strategy could be also responsible of the elimination of the AE found by Ottoboni et al. (2013). Although Ottoboni et al. (2013) demonstrated that, as for the SE, the conflict at the basis of the AE is not unavoidable, this is only an indirect index of the mechanisms at the basis of the two effects. These results alone, indeed, do not allow us to disentangle between the two accounts of the handle-hand CE.

In the present study, using the same amount of practice, we compared the transfer effect between objects having their intact or broken handle. As described above, such an amount of practice is enough to eliminate the SE. If it was true that the AE is a SE, we should have found the elimination of the handle-hand CE both in Experiments 2 and 3. Furthermore, the transfer effect should have remained even after 30 min.

The fact that the handle-hand CE for objects with their intact graspable part was not influenced by the prior practice, unlike what happens for objects with their broken handle, is compelling evidence that the two observed effects are due, at least in part, to different mechanisms.

# Conclusion

Our findings support the motor-based nature of handle-hand CE in spatial SRC paradigms. We found that both the activation of motor programs and the asymmetry of the object because of the handle, can contribute to the generation of the handle-hand CE, but while the former leads to the generation of the AE, the latter leads to the generation of a Simon like CE when the handle is not graspable because it is broken. Hence the graspable part of an object is a condition not necessary to generate a SE, but it is necessary to generate an AE.

# Acknowledgments

This study was supported by the European Community Grant ICT-216125-ROSSI, and CNPq – Proc. 450588/2009-2, Proc.

# References


309544/2009-2, Proc. 483611/2009-3, Proc. 455353/2012-3, Proc. 312189/2012-5, Proc. 478834/2013-6 CAPES. FAPERJ – Edital FAPERJ n◦ 06/2013 (Pró-Idoso) – Proc. 110.079/2013, E-26/101.958/2010, E-26/101.061/2011, E-26/100.661/2013, E-26/102.160/2013. PIBIC/UFF/CNPq, PIBITI/UFF/CNPq, PIBINOVA, FOPESQ/PROPPi/UFF. We would like to thank Paola Ricciardelli for helpful comments concerning this study, and Daniele Zavagno for his help in improving the manuscript.

# Supplementary Material

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnhum. 2015.00283/abstract


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2015 Ambrosecchia, Marino, Gawryszewski and Riggio. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Effects of broken affordance on visual extinction

#### Melanie Wulff <sup>1</sup> \* and Glyn W. Humphreys <sup>2</sup>

<sup>1</sup> School of Psychology, University of Birmingham, Birmingham, UK, <sup>2</sup> Department of Experimental Psychology, University of Oxford, Oxford, UK

Previous studies have shown that visual extinction can be reduced if two objects are positioned to "afford" an action. Here we tested if this affordance effect was disrupted by "breaking" the affordance, i.e., if one of the objects actively used in the action had a broken handle. We assessed the effects of broken affordance on recovery from extinction in eight patients with right hemisphere lesions and left-sided extinction. Patients viewed object pairs that were or were not commonly used together and that were positioned for left- or right-hand actions. In the unrelated pair conditions, either two tools or two objects were presented. In line with previous research (e.g., Riddoch et al., 2006), extinction was reduced when action-related object pairs and when unrelated tool pairs were presented compared to unrelated object pairs. There was no significant difference in recovery rate between action-related (object-tool) and unrelated tool pairs. In addition, performance with action-related objects decreased when the tool appeared on the ipsilesional side compared to when it was on the contralesional side, but only when the tool handle was intact. There were minimal effects of breaking the handle of an object rather than a tool, and there was no effect of breaking the handle on either tools or objects on single item trials. The data suggest that breaking the handle of a tool lessens the degree to which it captures attention, with this attentional capture being strongest when the tool appears on the ipsilesional side. The capture of attention by the ipsilesional item then reduces the chance of detecting the contralesional stimulus. This attentional capture effect is mediated by the affordance to the intact tool.

#### Edited by:

Anna M. Borghi, University of Bologna, Italy

#### Reviewed by:

Lewis A. Wheaton, Georgia Institute of Technology, USA Solene Kalenine, Universite Lille 3, France

#### \*Correspondence:

Melanie Wulff, School of Psychology, University of Birmingham, Hills Building, Edgbaston, Birmingham, B15 2TT, UK mxw127@bham.ac.uk

Received: 07 July 2015 Accepted: 04 September 2015 Published: 23 September 2015

#### Citation:

Wulff M and Humphreys GW (2015) Effects of broken affordance on visual extinction. Front. Hum. Neurosci. 9:515. doi: 10.3389/fnhum.2015.00515 Keywords: affordance, action relation, visual extinction, attention, tools, objects

# Introduction

Previous studies have shown that the perceptual properties of single objects ''afford'' certain actions, and this in turn influences visual attention and perception. This effect (Gibson, 1979) reflects the action possibilities offered by the environment to the observer, depending upon the observer's current goal and his/her action capabilities. For example, a cup will strongly afford a drinking action when we are thirsty and are able to grasp it, but not if we just have quenched our thirst and the cup is positioned inappropriately for the action (e.g., Humphreys and Riddoch, 2001). Such affordances are determined by the perceptual properties of the object such as the size and orientation of the cup. Thus for a right-handed person a cup is more likely to afford an action when its handle is oriented to the right than when it is oriented to the left, even though the object can be recognized equally efficiently in the different orientations (Riddoch et al., 1998).

The affordance effect is of particular relevance for patients showing visual extinction, a neuropsychological disorder commonly observed following damage to the right posterior parietal cortex (Karnath et al., 2003; Chechlacz et al., 2014). Extinction patients are able to detect a single contralesional stimulus presented in isolation but frequently fail to detect a contralesional stimulus when an ipsilesional stimulus appears simultaneously. Several behavioral studies have demonstrated that extinction can be modulated by grouping on the basis of Gestalt principles such as similarity and collinearity (e.g., Gilchrist et al., 1996). There are also higher-order influences on extinction which act even in the absence of Gestalt grouping factors. For example, extinction can be reduced when patients view pictures displaying objects oriented for left-hand or righthand actions. Di Pellegrino et al. (2005) first showed that the orientation of an object handle influenced stimulus detection, with less extinction when the contralesional object afforded a left-hand rather than a right-hand grasp. Di Pellegrino et al. (2005) suggested that affording an action to the left reduced extinction.

Apparently similar affordance effects on extinction can be observed with pairs of objects. Riddoch et al. (2003) presented pictures of objects either positioned to interact with each other or not. There was less extinction when objects appeared in the correct co-locations for action (a fork and knife facing each other) relative to when the same objects were positioned incorrectly for action (a knife facing away from a fork). Riddoch et al. (2003) concluded that interacting objects offer an affordance which groups the objects for attentional selection, enabling the constituent stimuli to be selected as a single unit. As a result, the perceptual report of both stimuli is improved and extinction is less severe. Several studies have reported similar results with healthy participants, with performance improving when objects are action-related compared to when they are unrelated (Green and Hummel, 2006; Adamo and Ferber, 2009; Roberts and Humphreys, 2011a,b; Borghi et al., 2012; McNair and Harris, 2012). For example, Roberts and Humphreys (2011a) showed healthy participants briefly presented objects and found improved identification performance when objects were in correct relative to incorrect co-locations for action.

Several behavioral studies have demonstrated that affordance effects for single (graspable) objects can be manipulated by factors such as object size (e.g., Tucker and Ellis, 2001), object location in space (e.g., Costantini et al., 2010), object orientation (e.g., Tucker and Ellis, 1998; Goslin et al., 2012) and handobject congruence (e.g., Girardi et al., 2010). However, it seems that the position and graspability of the object handle is particularly important for affordance effects (cf. Symes et al., 2007; Matheson et al., 2014). Notably, the spatial location of the handle influences stimulus identification as demonstrated in neglect patients (Humphreys and Riddoch, 2001), extinction patients (di Pellegrino et al., 2005) and healthy participants (e.g., Tucker and Ellis, 1998). In addition, performance can also be affected by disrupting graspability by breaking the handle of an object. Buccino et al. (2009) applied transcranial magnetic stimulation (TMS) over the left motor area in healthy participants. Participants viewed pictures of objects with an intact and a broken handle oriented to the right and the left side. Objects with an intact right oriented handle evoked a larger motor response compared to objects with a broken right oriented handle. The decrease in the motor response with broken handles relative to intact handles suggests that not only the handle orientation but also whether it is intact or not is crucial for the perception of affordance. Buccino et al. (2009) proposed that the graspability of an object may be processed in the motor cortex. Objects with an intact handle will be processed as being graspable and the corresponding motor representations will be automatically activated, whereas objects with a broken handle will be coded as less graspable and thus there will be reduced activation of the motor cortex. Graspability also influenced responses in a probe detection task (Garrido-Vásquez and Schubö, 2014), with faster probe detection times when the cued object was graspable (a cup) compared to when the cued object was non-graspable (a cactus). Whether such effects also occur in extinction patients has not been examined, nor is it clear whether effects of breaking a handle modulate how we attend to objects. It is possible that the coding of action-related pairs of objects operates using relatively coarsely coded visual representations, where the graspability of individual objects (and the presence of a broken handle) is less critical. Here we might expect a broken handle to reduce attentional responses to paired, action-related objects.

There are also data indicating that attention can be biased within pairs of action-related objects. Notably, when only one member of an object pair is reported by patients showing extinction, this tends to be the object that was ''active'' in an action (typically the tool that was used to act on the other object; Riddoch et al., 2003; Wulff and Humphreys, 2013). This bias can occur even when the active object falls in the contralesional field. In addition, normal participants tend to judge that the active member of an action-related pair appears first, when asked to make temporal order judgements (Roberts and Humphreys, 2010). Both findings are consistent with attention being attracted to the active tool, within an action-related pair. The preferential report for tools has subsequently been replicated with healthy participants using various experimental paradigms (Roberts and Humphreys, 2010, 2011a; McNair and Harris, 2014; Laverick et al., 2015; Wulff et al., 2015; Xu et al., 2015). Thus, breaking the handle of the tool may have a greater effect on report than breaking the handle of the passive, action recipient. For example, the attentional bias to the active tool may be reduced.

In the present study, we assessed the impact of a broken handle on the effects of affordance on extinction. To do this, we evaluated whether the effect of action relations on visual extinction holds when object pairs appear and one of the stimuli has a broken handle. In contrast to other studies (e.g., Humphreys et al., 2010a), we only presented pairs of objects in correct co-locations for action. We predicted that the affordance effect is stronger for familiar (action-related) rather than for unfamiliar (unrelated) pairs of objects (cf. Riddoch et al., 2006). Also, if the graspability of individual objects is important, we expected that the affordance effect would be reduced with broken object pairs compared to intact object pairs as previous studies have shown that viewing non-graspable (broken-handled) objects can eliminate motor-based affordance effects (Buccino et al., 2009). We further predicted differences according to whether a tool or an acted-upon object had a broken handle (cf. Riddoch et al., 2003; Wulff and Humphreys, 2013). Breaking the handle of a tool should be more disruptive to performance than breaking the handle of a passive object, in an action-related pair.

# Materials and Methods

#### Patients

Eight patients with visual extinction from 55–78 years of age (2 females, M = 66.88; SD = 8.15) were recruited from the volunteer panel at the University of Birmingham. Six patients had right unilateral lesions and two had bilateral lesions (clinical details are given in **Table 1**). All the patients showed left visual extinction on the BCoS Cognitive Screen (Humphreys et al., 2012).<sup>1</sup> The patients did not have visual field defects on visual confrontation testing or suffered from optic ataxia. Three patients (P1, P3, and P6) showed mild apraxia on the BCoS (see **Table 1**). However, the extinction data for these patients were not clearly different from the results of the other patients; similarly there were no differences between the extinction results for the unilateral and bilateral cases. All reported normal or correctedto-normal vision. Informed consent was obtained from all patients and the study was approved by a national NHS research ethics committee.

### Apparatus and Stimuli

Four colored photographs of common drinking containers were used (flask, teapot, cup, and beaker). Each item was photographed on a table with the handle orienting to the right, and then flipped within the horizontal plane in Microsoft Office Picture Manager (Version 12) to create a mirror image of each item. Thus, an item with a right-oriented handle was turned into an item with a left-oriented handle. A second set of images in which each item had a broken handle was created using Paint.NET (Version 3.5.10). This resulted in 2 (handle: intact, broken) × 2 (handle orientation: right, left) × 2 (stimulus type: object, tool) images. The tools and non-tool objects were not matched for size as this manipulation might have disrupted the effect of action relation (cf. Riddoch et al., 2011). However, variations across the individual stimuli should not have been critical as items were counter-balanced across conditions.

The individual items were organized into pairs with the items positioned to interact with each other with their handles facing outwards. There were three conditions in which the object pairs were varied (see **Figure 1**). The objects were: (i) action-related: a tool and an object that were commonly used together (teapot and cup; beaker and flask); (ii) an unrelated pair in which two tools were presented (teapot and flask); and (iii) an unrelated pair in which two objects were presented (beaker and cup). For the action-related pair, each object within the pair was classified as being either the active or the passive member of the pair (cf. Riddoch et al., 2003). In the ''intact handle condition'', all the objects had an intact handle, while in the ''broken handle condition'' one item within the pair had a broken handle. This was the active tool for half of the stimuli, and the passive object for the other half. The items were arranged either with: (i) the tool on the right side and the object on the left side; or with (ii) the tool on the left side and the object on the right side. Note that the side of extinction could correspond to the side of the tool or not. Each item pair was presented simultaneously, one item to the right and the other item to the left side of fixation. The stimuli appeared on a black background.

One-item trials were randomly intermingled with the twoitem trials. Here, an item (either with an intact or a broken handle) was paired with a blank table on the other side of fixation (to maintain approximate levels of visual stimulation), and it was presented at the same location and for the same duration as it appeared on two-item trials.

Items were displayed on a 19-inch monitor at a viewing distance of approximately 50 cm. The monitor provided a frame refresh rate of 60 Hz with a spatial resolution of 1024 × 768 pixels. The stimuli subtended 10.29◦ × 8.56◦ of visual angle and were located 0.86◦ either to the left or right side of central fixation. We positioned the items very centrally to imply a joint action between the two objects in the action-related condition. The average distance between the center of both items was 12 cm (see also di Pellegrino and De Renzi, 1995; Ptak et al., 2002).

#### Design and Procedure

A similar design to Humphreys et al. (2010a) and Wulff and Humphreys (2013) was used. The experiment consisted of two conditions (Intact objects and Broken objects), which were administered to each patient in an ABAB order across three sessions, with at least 1 week apart. The order of the conditions was counterbalanced across patients.

The two conditions were identical with the exception that in the Broken handle condition, one member of the pair had a broken handle, whereas in the Intact handle condition the handles of both stimuli were intact. The Broken handle condition consisted of eight bilateral conditions [condition (action-related, unrelated tool, unrelated object) × handle (tool broken, object broken) × side of tool (contralesional, ipsilesional)] and eight unilateral conditions [stimulus type (object, tool) × handle (tool broken, object broken) × side (ipsilesional, contralesional)]. There were 768 trials which were presented in 12 blocks of 64 trials; 48 trials for each condition. The Intact condition

<sup>1</sup>All patients were also impaired on a short computer-based test of visual extinction, defining their inclusion in the study. In this test, we presented the letters A to D, 0.5◦ × 0.5◦ centered at locations 3◦ to the left or right side of fixation. The letters were presented for 200 ms unmasked either alone (randomly in the left or right field) or bilaterally. Patients had to identify the letters presented. There were 24 single left trials, 24 single right and 48 two-item trials. Patients were classified as having extinction if they showed a lateralized difference of more than 2 when reporting items under bilateral relative to unilateral conditions. A group of 12 age-matched control patients were able to report all the items under these presentation conditions. All the patients met this definition for extinction.



Note: F, female; M, male; L, left; R, right.

consisted of six bilateral conditions [condition (action-related, unrelated tool, unrelated object) × side of tool (contralesional, ipsilesional)] and four unilateral conditions [stimulus type (object, tool) × side (ipsilesional, contralesional)]. There were 384 trials which were presented in six blocks of 64 trials; 48 trials for each condition. Each stimulus was repeated eight times within one block. In prior studies of the effects of affordance on extinction only a small number of items have been used (e.g., di Pellegrino et al., 2005) in order to allow a clear and controlled manipulation of the main factors of interest. In both the Intact and the Broken handle conditions one-item and two-item trials were fully randomized.

Patients had to identify and name the item(s) on each trial by verbal report. Patients were tested individually in a quiet room. Responses were recorded as correct if either the single item was correctly named, or if both items were correctly named on bilateral trials. It was also noted whether one item on two-item trials was correctly reported, while we did not separately record: (i) whether patients reported the presence of a second item which they could not name; or (ii) named the second item incorrectly; or (iii) whether they thought only one item was present. However, we also recorded whether any item on two-item trials was reported. Before each session, pictures of the stimuli were presented individually on a monitor to each patient to ensure that the patients could recognize and correctly identify the items. Additionally, patients were given at least 14 practice trials to ensure adequate performance in the task and the stimuli on these practice trials were different from those employed in the experimental trials to avoid carry-over effects. During these practice trials, stimulus exposure time was adjusted to ensure that each patient achieved a performance level of roughly 70–90% correct for single items in the contralesional hemifield (**Table 2**) before the experimental trials began. The practice trials were repeated until this level was achieved across a block of 14 trials; the exposure duration was then fixed for a patient for each session.

Each trial began with a white central fixation cross presented on a black background for 2000 ms, which was replaced by a red fixation cross for 500 ms to inform patients that the stimulus was about to appear. Next a single object or an object pair was presented. For all patients (except P1, P6, and P7) a 100 ms mask followed the object(s) to maintain the same level of task difficulty across patients. Responses were manually recorded by the experimenter, and after that the next trial was initiated.

# Results

We analyzed accuracy data as well as error data. Accuracy data reflect correct naming of a single item in unilateral and of two items in bilateral trials. These data were used to contrast report on one- and two-item trials. For two-item trials, error data were then computed by counting how many times only one of two items was correctly named (either on the left or right visual field), or no item was reported and whether the reported item fell on the ipsi- or contralesional side.<sup>2</sup> Note that errors when only one item was reported included three different response types: identification of one item and not reporting the second, identification of one item and reporting the presence of the second item which could not be named, and incorrect identification of the second item; cf. method section.<sup>3</sup> On average, patients made errors on 40% of the two-item trials,

<sup>2</sup>Note that the accuracy data could not be used since these data failed to distinguish which item was reported on an error trial.

<sup>3</sup>Unfortunately we failed to record the type of error when only a single item was reported. However it should be noted that by far the majority of such errors involved patients reporting one item and making no response to the other.

of which 38% were errors when patients only named one item correctly, while on just 2% of the trials patients failed to report any item. The former error type was used for all subsequent analyses. We report the results in several sections.


In all analyses, we included patient as a between-subject factor (with sessions as subjects) to test whether there are variations in the size of the effects across patients. Greenhouse-Geisser correction for degrees of freedom was used when the assumption of sphericity was not met. Significant differences between conditions were further assessed with paired t-tests (p < 0.05).

#### The Presence of Extinction

We compared performance on one-item trials with performance on two-item trials to confirm that patients suffered from


TABLE 2 | Stimulus exposure times for the Intact (unbroken handles) and the Broken handle condition.

Note: M, mean. Mask, visual backward mask.

on the ipsilesional side (right panels).

extinction, with extinction being present when patients' identification performance was significantly better on one-item than on two-item trials. The accuracy data from one-item trials and from the different two-item conditions (pooled across conditions), based on the number of items correctly reported on the ipsilesional or contralesional side, were entered into an ANOVA with the within-subject factors being number of objects (one-item, two-items) and side of item being reported (ipsilesional, contralesional); patient was treated as a betweensubject factor.

#### Intact Condition

Performance on one-item trials was significantly better than performance on two-item trials, confirming that visual extinction was present, F(1,16) = 674.86, p < 0.001, η 2 <sup>p</sup> = 0.977. The main effects of side, F(1,16) = 55.10, p < 0.001, η 2 <sup>p</sup> = 0.775 (ipsilesional > contralesional stimuli) and patient, F(7,16) = 9.33, p < 0.001, η 2 <sup>p</sup> = 0.803, were significant. The number of objects by side interaction, F(1,16) = 6.64, p = 0.020, η 2 <sup>p</sup> = 0.293, reached significance. The side effect was slightly larger in the twoitem trial conditions compared to the one-item trial conditions, though it was reliable for both, t(23) = 4.96, t(23) = 4.63, both p < 0.001, respectively (see **Figure 2A**). There were also significant interactions between the number of objects and patient, F(7,16) = 3.70, p = 0.014, η 2 <sup>p</sup> = 0.618, between side and patient, F(7,16) = 3.44, p = 0.019, η 2 <sup>p</sup> = 0.601, and between number of objects, side and patient, F(7,16) = 14.87, p < 0.001, η 2 <sup>p</sup> = 0.867 (**Figure 2B**). These interactions indicate that the extinction effect was larger for some patients than for others, though all patients showed extinction and patients' performance varied as a function of the side of stimulus.

#### Broken Handle Condition

The same ANOVA was conducted with broken object pairs. As with intact object pairs, identification performance was significantly better on one-item than on two-item trials, F(1,16) = 1395.25, p < 0.001, η 2 <sup>p</sup> = 0.989 (**Figure 2C**). There were significant main effects of side, F(1,16) = 75.21, p < 0.001, η 2 <sup>p</sup> = 0.825 (ipsilesional > contralesional stimuli) and patient, F(7,16) = 8.34, p < 0.001, η 2 <sup>p</sup> = 0.785. The number of objects by side interaction, F(1,16) = 4.81, p = 0.043, η 2 <sup>p</sup> = 0.231, was also significant. As before, the side effect was slightly larger in the two-item trial conditions compared to the one-item conditions, t(23) = 4.74, t(23) = 4.17, both p < 0.001, respectively. There were also significant interactions between the number of objects and patient, F(7,16) = 3.55, p = 0.017, η 2 <sup>p</sup> = 0.608, between side and patient, F(7,16) = 6.55, p = 0.001, η 2 <sup>p</sup> = 0.741, and between number of objects, side and patient, F(7,16) = 11.50, p < 0.001, η 2 <sup>p</sup> = 0.834. The variations across patients are shown in **Figure 2D**; however the one item advantage was present for all patients.

# Effects of Object Pair Type on Two-Item Report (Intact Handles)

#### Accuracy Data

To investigate whether the type of object pair affected identification performance when both handles were intact, the

data from action-related (object-tool) pairs were compared with unrelated tool-tool and with unrelated object-object pairs. **Figure 3A** shows the mean performance for each object pair condition. The main effect of condition, F(1.9,30.3) = 65.64, p < 0.001, η 2 <sup>p</sup> = 0.804, reached significance. Bonferroni corrected multiple comparisons showed that accuracy was significantly higher for action-related objects and for unrelated tools than for unrelated object pairs (both p < 0.001), whereas there was no difference between the report of actionrelated objects and unrelated tool pairs. The benefit for the related (object-tool) pair condition over the unrelated objectobject pair condition indicates that the presence of the tool (in the action-related object-tool condition) benefitted report of the other (non-tool) object, and that action relatedness can benefit report (cf. Riddoch et al., 2003). There was also a benefit for two tools compared with two objects, indicating a general advantage for reporting tools. There was a significant main effect of patient, F(7,16) = 5.19, p = 0.003, η 2 <sup>p</sup> = 0.694. The interaction between condition and patient, F(13.3,30.3) = 9.00, p < 0.001, η 2 <sup>p</sup> = 0.797 (see **Figure 3A**), was reliable. This indicates that the magnitude of the effect of condition varied across individuals, but all patients showed the effect.

#### Error Data

We compared the error data from these two-item trials when only one item of an object pair was correctly reported based on the side of the reported item (either on the ipsilesional or the contralesional side). A chi-square test indicated that the type of the object pair modulated the side of the reported item, χ 2 <sup>2</sup> = 7.203, p = 0.027, Cramer's V = 0.127. As can be seen in **Figure 3B**, the number of reported items on the ipsilesional relative to the contralesional side was higher for unrelated objects compared to action-related pairs and unrelated tools. This suggests that there is more ''weight'' placed during selection on the spatial position of the target when two objects are present relative to when one of the stimuli is a tool.

# Role of Broken Handles on Two-Item Trial Performance

Several separate ANOVAs were conducted with the factors being handle (both handles intact/one handle broken) and side of broken handle (contra- vs. ipsilesional); patient was treated as between-subject factor. Separate ANOVAs were conducted because the make-up of the conditions (e.g., two objects, two tools, object-tool—each sometimes having a broken handle) meant that the factors could not be nested in a single ANOVA.

### Effects with Action-Related Objects Only

First we assessed effects of having a broken tool handle; then we assessed effects of having a broken object handle. Finally, we analyzed error trials to examine whether tools or objects are reported more often in error trials when only one item was correctly reported.

### **Tool handle broken (Figure 1A(i) vs. Figure 1A(iii))**

There were reliable main effects of side of tool, F(1,16) = 9.33 p = 0.008, η 2 <sup>p</sup> = 0.368 (ipsilesional > contralesional) and patient, F(7,16) = 6.08 p = 0.001, η 2 <sup>p</sup> = 0.727. The interaction between intact/broken handle and side of tool was reliable, F(1,16) = 12.90, p = 0.002, η 2 <sup>p</sup> = 0.446. When both handles were intact, there was better performance when the tool was presented on the contralesional side relative to when it was presented on the ipsilesional side, t(23) = 3.84, p = 0.001 (**Figure 4A**), while there was no reliable effect of the positioning of the tool when the tool handle was broken. The side of tool by patient interaction, F(7,16) = 2.84, p = 0.040, η 2 <sup>p</sup> = 0.554, was also significant (**Figure 4B**). Patients differed in the degree to which they reported more stimuli when the tool was on the ipsilesional compared to when the tool was on the contralesional side; these effects were present for all but one patient (P1).

#### **Object handle broken (Figure 1A(ii) vs. Figure 1A(iii))**

There were significant main effects of intact/broken handle, F(1,16) = 4.90, p = 0.042, η 2 <sup>p</sup> = 0.234 (broke > intact), side of broken handle, F(1,16) = 38.72, p < 0.001, η 2 <sup>p</sup> = 0.708 (ipsilesional > contralesional) and patient, F(7,16) = 5.36 p = 0.003, η 2 <sup>p</sup> = 0.701. The effects of having a broken object handle and the side of the broken object handle were additive, F(1,16) = 0.634 p = 0.438, η 2 <sup>p</sup> = 0.038 (see **Figure 4C**). Note that the effect of the side of the broken object handle here fits with the effect of the tool position (above). Performance was better when the broken object handle was on

the ipsilesional side (and the tool was on the contralesional side in the action-related pair) than when the broken object was on the contralesional side (and the tool was on the ipsilesional side). The interaction between the side of the broken object and patient was also reliable, F(7,16) = 5.04, p = 0.004, η 2 <sup>p</sup> = 0.688 (**Figure 4D**). The effect of whether the broken object handle was on the ipsi- or contralesional side varied across patients but was present in all except in one patient (P1).

These analyses indicate that the report of action-related pairs changed as a function of the position of the tool when the tool handle was intact, with performance generally being worse when the tool was on the ipsilesional side relative to when it fell in the contralesional field. This effect of tool position was eliminated when the tool handle was broken. This interpretation is supported by the error data (**Figure 5**, see below).

#### **Error data**

The error data from two-item trials when only one item of an object pair was correctly reported were entered into a loglinear analysis, with the factors being handle (intact/broken), side of tool (either on the ipsilesional or contralesional side) and side of reported item (either on the contralesional or on the ipsilesional side). The analysis produced a final model with the highest order interaction (handle × side of tool) and a main effect of reported item, χ 2 <sup>3</sup> = 3.508, p = 0.320. There was similar performance in reporting tools on the ipsilesional and contralesional sides, but this held only for the broken tool condition. In contrast, there were more reports of the tool occurring on the ipsilesional than the contralesional side when the tool was intact. There was better performance in reporting tools compared to objects, and the report was better for ipsilesional compared with contralesional tools (**Figure 5**).

#### Action-Related Objects vs. Unrelated Tools (with Broken Tool Handle; Figure 1A(i) vs. Figure 1B(i))

The within-subject factors were condition (action-related objects vs. unrelated tools) and location of the broken tool (contralesional vs. ipsilesional field). Patient was treated as a between-subject factor. The only reliable effects were the main effect of patient, F(7,16) = 9.57, p < 0.001, η 2 <sup>p</sup> = 0.807, and the interaction between condition and patient, F(7,16) = 6.96, p = 0.001, η 2 <sup>p</sup> = 0.753. The difference in overall report between action-related pairs and tool pairs varied unsystematically across patients (**Figure 6**). The effects of breaking the handle of the tool were the same for action-related pairs and unrelated tools, consistent with the effect of breaking the handle being largely driven by the tool, in action-related pairs.

#### Action-Related Objects vs. Unrelated Objects (with Broken Object Handle; Figure 1A(ii) vs. Figure 1C(i))

The within-subject factors were condition (action-related objects vs. unrelated objects) and location of the broken object (contralesional vs. ipsilesional). Patient was treated

as a between-subject factor. The main effects of condition, F(1,16) = 133.36, p < 0.001, η 2 <sup>p</sup> = 0.893 (action-related objects > unrelated objects), side of broken object, F(1,16) = 9.22, p = 0.008, η 2 <sup>p</sup> = 0.365 (ipsilesional > contralesional stimuli), and patient, F(7,16) = 3.77, p = 0.013, η 2 <sup>p</sup> = 0.623, were reliable. There was a significant interaction between condition and side of broken object, F(1,16) = 12.46, p = 0.003, η 2 <sup>p</sup> = 0.438 (**Figure 7A**). In the action-related condition, performance was increased when the broken object was on the ipsilesional side and the intact tool was on the contralesional side compared to when the stimuli were in the opposite positions, t(23) = 3.14, p = 0.005. In contrast, there was no reliable effect of the side of the broken object with unrelated object pairs. There were also interactions between condition and patient, F(7,16) = 7.57, p < 0.001, η 2 <sup>p</sup> = 0.768 (**Figure 7B**), and side of broken object and patient, F(7,16) = 2.63, p = 0.051, η 2 <sup>p</sup> = 0.535 (**Figure 7C**). There was an overall advantage for action-related pairs over unrelated object pairs and for intact tools/broken object handles on the contralesional compared with the ipsilesional side, but these effects varied in size although in the same direction across patients.

#### Effect of Stimulus Type on One-Item Report

The accuracy data from unilateral trials were also analyzed in order to assess whether there were any differences between the report of tools and other objects when presented in isolation (equivalent to the active and passive members within an object pair; see Methods). The within-subject factors were stimulus type (object, tool), side of stimulus (contra- vs. ipsilesional) and handle (broken, intact); patient was treated as a betweensubject factor. There were significant main effects of stimulus type, F(1,16) = 24.44, p < 0.001, η 2 <sup>p</sup> = 0.604 (tools > objects), side of stimulus, F(1,16) = 38.92, p < 0.001, η 2 <sup>p</sup> = 0.709 (ipsilesional > contralesional stimuli), and patient, F(7,16) = 4.67, p = 0.005, η 2 <sup>p</sup> = 0.671. There was also an interaction between stimulus type and side of stimulus, F(1,16) = 6.35, p = 0.023, η 2 <sup>p</sup> = 0.284. Patients tended to report more stimuli on the ipsilesional than the contralesional side (tools, t(23) = 4.17, p < 0.001; objects, t(23) = 3.77, p = 0.001 (**Figure 8A**). In addition, the interaction between side of stimulus and patient was also significant, F(6,16) = 5.09, p = 0.003, η 2 <sup>p</sup> = 0.690 (**Figure 8B**); patients varied in the magnitude of the side effect but they all showed the same direction. This analysis indicates that the effect of having a broken handle had little effect when single objects were presented (i.e., when there was no spatial competition for selection).

# Discussion

It is well-established that positioning familiar objects for action promotes recovery from visual extinction (Riddoch et al., 2003). Similarly, extinction can be affected by the position of the actionrelated part of a single object (di Pellegrino et al., 2005). Also, within pairs of action-related objects, attention tends to be drawn to the object that would be grasped to perform the action (the active tool), rather than the passive object (Riddoch et al., 2003). These effects have been attributed to the affordance offered by the objects, which helps to draw attention to the contralesional side (for recent reviews, see Humphreys et al., 2010b, 2013) and

to the active object in a pair (Roberts and Humphreys, 2010). The present study investigated whether recovery from extinction held on trials when the affordance was disrupted by presenting objects with a broken handle, and whether the presence of the broken handle altered any bias to attend to the active object in a pair. There were several effects to note, some of which did not relate to the presence of a broken handle and some of which did.

### Effects Independent of the Broken Handle

We will initially consider effects that were assessed independent of the presence of a broken handle. Firstly, there was an overall effect of extinction. Patients were able to report more items on one-item trials than on two-item trials. Secondly, patients did benefit overall more when action-related (object-tool) stimuli were presented relative to when unrelated object-object pairs were presented. This is in line with previous studies showing that extinction patients are better at attending to object pairs which have the potential to interact with each other (object-tool pairs here) compared to when this is unlikely (with unrelated

objects; e.g., Riddoch et al., 2006; Wulff and Humphreys, 2013). Interestingly, there was no advantage for action-related (objecttool) pairs compared to when two tools were presented. Contrary to our expectation, however, it might be that the two tools themselves afforded a common action together, even though they were unfamiliar as a pair. Familiarity does not appear to be critical here. This interpretation matches the results from the error trials, where only one item of the object pair was reported. There was better report of ipsilesional items for unrelated objects compared to ipsilesional stimuli presented with action-related and unrelated tool pairs. Based on this result, we cannot exclude the possibility that the presence of a tool rather than its relationship to the other non-tool object in a pair is what matters for the affordance effect. This argument seems plausible as the error data revealed that patients reported tools over objects, irrespective of whether the tool appeared on the ipsilesional or contralesional side (**Figure 5**). In addition, with intact handles, performance was better when the tool was on the contralesional relative to the ipsilesional side (**Figure 4A**). We speculate that either the presence of the tool helped to cue attention to the contralesional field (cf. di Pellegrino et al., 2005) or that presenting the tool on the ipsilesional side tended to attract attention and led to attentional capture, ipsilesional, and thus increased extinction (e.g., Shalev and Humphreys, 2000). We consider this further below.

#### Effects when a Handle was Broken

When the handle of one of the objects was broken, some of the results changed. Notably, when the tool handle was broken, there was now no longer an effect of the position of the tool for actionrelated objects (**Figure 4A**). The direction of this effect was that performance improved relative to when the tool handle was intact and when the tool fell on the ipsilesional side (**Figure 4A**). This is consistent with an account of attentional capture by an ipsilesional tool with an intact handle—reducing this capture by breaking the handle of the ipsilesional tool led to better report of both items (see above). This argument about attentional capture fits well with the results from the error analysis. Here we observed that patients reported more broken tools, regardless of their location in space (**Figure 5B**).

When the handle of the object (rather than of the tool) was broken, there was no interaction with whether actionrelated objects or unrelated objects were presented, and the advantage for action-related (object-tool) pairs was maintained (**Figure 7A**). This suggests that breaking the handle of the object has a weaker effect on any affordance-based response to the stimuli, so that the effect of action relatedness is maintained even when a handle is broken (cf. **Figure 5**). There were also effects of whether the broken handled object appeared on the contralesional or ipsilesional side (better report when it fell on the ipsilesional side, in action-related pairs; **Figure 4C**). However, this result can also be explained in terms of the location of the intact tool, which fell in the contralesional field in the former case (broken handled object in the ipsilesional field). Presenting a tool on the ipsilesional side disrupted performance relative to when the tool fell in the contralesional field, in line with the error analysis (**Figure 5A**).

However, if there was only a detrimental effect of presenting an intact tool on the ipsilesional side, we would not expect to see the overall advantage for action-related objects compared to the unrelated baseline (unrelated tools, unrelated objects) since the tool, in the action-related trials, would disrupt performance. Instead, we suggest that, on top of any attentional capture by the tool, the report of both items was enhanced by coding an action relation between the stimuli, which facilitated attention across both presented items.

Riddoch et al. (2003) and Wulff and Humphreys (2013) both noted that, on trials where patients only reported one item in an interacting pair, the tool was typically identified. Roberts and Humphreys (2010) also showed that, in normal participants, there is a ''prior entry'' effect for tools over objects; when the stimuli are presented in co-locations for action, participants tend to identify the tool as appearing before the object (cf. Rorden et al., 1997; see also Laverick et al., 2015; Wulff et al., 2015). This is consistent with attention being biased towards the tool (Handy et al., 2003; Matheson et al., 2014). We speculate that, in the present study, this biasing of attention would be exacerbated when the tool falls in the ipsilesional (attended) field and allocating attention to the ipsilesional tool can then disrupt the report of the contralesional object. The interesting result here was that the effect of position of the tool was eliminated when the tool handle was broken but not when the object handle was broken. This observed result for broken tools in our study fits well with the TMS results from healthy participants using single objects. Buccino et al. (2009) presented pictures of intact tools and tools with a broken handle and found that only intact stimuli evoked a motor response. We found a similar pattern with intact paired objects, but not when the handle of one object was broken. This result confirms that viewing nongraspable objects can eliminate motor-based affordance effects. The data further support the assumption that the active tool, rather than the passive recipient of the action has a higher weight within a pair (see e.g., Riddoch et al., 2003; Wulff and Humphreys, 2013; Xu et al., 2015). Taken together, the results indicate that the response to an affordance is modulated by the graspability of the object (the tool in case of action-related object pairs).

In addition to these effects on two-item trials, we found an advantage for reporting single tools over single objects. However, and perhaps in contrast with the study by Buccino et al. (2009), this result was unaffected by whether the tool handle was broken. In the present study, the major constraint on perceptual report was on whether there was competition for attention from an ipsilesional item on the selection of a contralesional stimulus, and this was mediated by whether the tool handle was broken. However, the effects of breaking the handle on attentional competition should be lessened with single objects, as we observed. The data do suggest though that individual items were equally identifiable irrespective of whether or not the handle was broken, and this was not a major factor on report (for a similar result using a spatial stimulus-response compatibility paradigm, see Ambrosecchia et al., 2015). Thus, the results on two-item trials may more clearly reflect whether tools capture attention, and the effects of attentional capture by tools appear to be lessened when the handle is broken.

Interestingly, there was also a suggestion in the data that the effect of the tool could also have been moderated by the handedness of the patients. P1 and P8 were formerly left-handed. These patients tended to show weaker effects of whether the tool was positioned on the contralesional or ipsilesional side, relative to the other patients (see **Figures 4B,D**). We may speculate that the drive to attend to the tool when it fell on the ipsilesional side was reduced in these patients, perhaps because it reflects a motorbased response to tools. Since the present patients all had right hemisphere lesions and left-sided extinction, an attentional drive to the right side tool (in the ipsilesional field) would be reduced in the left-handed patients. Clearly, the number of patients here is too small to make strong conclusions, but the effects of handedness on performance remain an interesting question to examine.

A final point to note is that the present result appears to be driven largely by whether an intact tool falls on the ipsilesional side, and attentional capture by this item is moderated by whether the handle is broken. The evidence is consistent with the affordance from the tools being coded in an attended region of field (on the ipsilesional side), but there is not strong evidence for the tool-related affordance being critical when the tool is in the contralesional field. We conclude that performance here is modulated by two factors: (i) an overall effect of having a tool within an object pair (action-related objects = unrelated tools); (ii) coding an action relation between stimuli (action-related objects > unrelated objects); and (iii) attentional capture by an intact tool on the ipsilesional side (overall report better for tool on the contralesional side vs. tool on the ipsilesional side). Only this attentional capture effect was moderated by breaking the handle of the tool.

The present data may have clinical implications. Attentional capture by the active object in the action (the tool) could be used to improve patients' performance in everyday tasks. For example, training everyday tasks such making a sandwich or preparing a hot drink could benefit by always presenting an action pair (e.g., knife and fork) and positioning the tool (the fork) on the contralateral side. Furthermore, our results indicate that drinking containers should have a handle to facilitate affordance perception. Whether patients with other neuropsychological deficits (e.g., apraxia, dementia) would benefit from affordance in a similar way to extinction patients would be an interesting question to follow up.

#### Study Limitations

We acknowledge that the limited stimulus set could have contributed to these results. The aim of the experiment was to investigate affordance effects with intact and broken objects. As previous studies have shown that the object handle and

# References


its orientation is the most prominent feature to guide visual attention (cf. Symes et al., 2007; Matheson et al., 2014), we chose drinking containers with handles to manipulate affordances (cf. Buccino et al., 2009; Garrido-Vásquez and Schubö, 2014; Ambrosecchia et al., 2015). In order to prevent guessing, we chose distinct drinking containers instead of using different cups or teapots. We do agree that the action pairs ''cup-teapot'' and ''flask-beaker'' have a stronger association than non-action pairs (cup-beaker or teapot-flask). We expected that action pairs, in contrast to unrelated pairs, would increase affordance-based responses. Furthermore, we chose highly familiar objects to avoid training effects. We did not observe any improvements across sessions as we adjusted the stimulus exposure time for each session to ensure a similar performance across sessions.

# Acknowledgments

We thank all the patients for their kind participation. This work was supported by grants from the European Union FP 7 CogWatch Project (grant number FP7-ICT-288912) and the NIHR to the Oxford cognitive health Clinical Research Facility.


intact motion perception after unilateral parietal damage. Neuropsychologia 35, 421–433. doi: 10.1016/s0028-3932(96)00093-0


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Wulff and Humphreys. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Stable and variable affordances are both automatic and flexible

#### Anna M. Borghi <sup>1</sup> \* and Lucia Riggio<sup>2</sup>

<sup>1</sup> Department of Psychology, University of Bologna and Institute of Cognitive Sciences and Technologies, Italian National Research Council, Rome, Italy, <sup>2</sup> Department of Neuroscience, University of Parma, Parma, Italy

The mere observation of pictures or words referring to manipulable objects is sufficient to evoke their affordances since objects and their nouns elicit components of appropriate motor programs associated with object interaction. While nobody doubts that objects actually evoke motor information, the degree of automaticity of this activation has been recently disputed. Recent evidence has indeed revealed that affordances activation is flexibly modulated by the task and by the physical and social context. It is therefore crucial to understand whether these results challenge previous evidence showing that motor information is activated independently from the task. The context and the task can indeed act as an early or late filter. We will review recent data consistent with the notion that objects automatically elicit multiple affordances and that top-down processes select among them probably inhibiting motor information that is not consistent with behavior goals. We will therefore argue that automaticity and flexibility of affordances are not in conflict. We will also discuss how language can incorporate affordances showing similarities, but also differences, between the motor information elicited by vision and language. Finally we will show how the distinction between stable and variable affordances can accommodate all these effects.

#### Edited by:

Agustin Ibanez, Institute of Cognitive Neurology, Argentina

#### Reviewed by:

Costantini Marcello, University of Chieti, Italy Mariella Pazzaglia, University of Rome "La Sapienza", Italy

#### \*Correspondence:

Anna M. Borghi, Department of Psychology, University of Bologna and Institute of Cognitive Sciences and Technologies, Italian National Research Council, Viale Berti Pichat 5, Bologna 40127, Italy anna.borghi@gmail.com

> Received: 23 March 2015 Accepted: 01 June 2015 Published: 19 June 2015

#### Citation:

Borghi AM and Riggio L (2015) Stable and variable affordances are both automatic and flexible. Front. Hum. Neurosci. 9:351. doi: 10.3389/fnhum.2015.00351 Keywords: affordances, language comprehension, canonical neurons, mirror neurons, automaticity, grasping, embodied cognition, tools

# Introduction

The study of affordances (Gibson, 1979), i.e., of the invitations to act objects offer to us, is becoming increasingly popular in the last years (for a review, Thill et al., 2013), also due to the increasing spread of embodied and grounded cognition views, according to which there is a strict interaction between perception, action, and cognition. The aim of this paper is to propose a novel view on affordances, which considers the way in which they are represented and vary depending on the context and the task, based on recent evidence obtained in our labs and in other labs.

The paper is organized in two main sections.

In the first section we will claim that different kinds of affordances exist, i.e., stable and variable ones. Both stable and variable affordances are flexible, but to a different extent. We will also consider how these affordances are modulated and constrained by language.

In the second section we will discuss whether affordances are always automatically activated or whether they are contextual dependent. Finally, we will consider some cases in which we might need to ''block'' affordance activation: the activation of multiple affordances, of broken affordances, and of affordances of dangerous objects.

Overall, we will defend a view according to which automaticity and flexibility are not in conflict.

# Stable and Variable Affordances

The notion of affordance, proposed by Gibson in 1979, has received a lot of interest in the last 15 years. In the ecological perspective adopted by Gibson, affordances consists in the invitation to action offered by the environment to living organisms. Gibson's theory of affordances is directly related to his overall view of direct perception. According to him, given that our species has evolved in a given ecological niche, the environment directly offers to us the possibility to perceive it correctly, without the mediation of mental representations. Affordances are perceived in a direct way: we do not need to activate objects knowledge to perceive their affordances. Importantly, according to Gisbon affordances are not properties of objects alone but they are relational, since they concern both the organism and the environment, both the subject and the object.

Given that affordances involve both perception and action, it is not surprising that this notion has had a lot of success within embodied and grounded views. However, depending on whether a more or less radical embodied perspective is adopted, this notion has been differently interpreted. While Gibson's perspective is fully externalist and anti-representational (see also Chemero, 2009), recent authors contributing to the spread of the notion of affordances in psychology and neuroscience have considered affordances as the product of the conjunction, in the brain, of visual and motor experiences (Ellis and Tucker, 2000).

We share this second perspective. This perspective has in our view two important implications. First, it implies that affordances are flexible and continuosly modified and updated thanks to novel experiences. In this respect, there is no discontinuity with Gibson's view. Second, it ascribes relevance to the interactions between the environment and the organisms as a whole, taking into account not only the dynamics of these interactions but also their neural representation: in this respect, this view departs from Gibson's externalist view, as a famous quote clarifies: ''Ask not what's inside your head, but what your head's inside of.'' Adopting this second view has led to an increased interest for the neural representation of affordances and has produced impressive behavioral and neural results in the last years.

In order to emphasize both the similarities and the differences between their perspective and Gibson's view, Ellis and Tucker (2000) have proposed to use the term microaffordance. For research reason, we find the inspiration leading to the proposal of microaffordance highly useful. In fact, in order to understand the processes that are going on during object processing it is very useful to refer to specific action components, as reaching and grasping; the importance of these specific action components is captured in the proposal to use the term micro-affordances (see also Masson et al., 2011). In the following pages we will however stick to the term affordance, since we think it is a good umbrella term, but we will try to formulate a theoretical proposal that takes into account specific action components related to the interaction with specific objects.

We propose (see also Borghi and Riggio, 2009) that, when specific action components are considered, affordances can be distinguished into stable and variable.

We consider stable affordances to derive from rather stable or invariant features or properties of objects, and from their relationship with organisms who interact with them. Imagine for example listening to somebody telling you ''Bring here some fruit'' or imagine seeing some fruits at a distance. For grasping affordances of fruit the property of size and the grasping action it evokes is rather stable: we may indeed prepare ourselves to interact with cherries using a precision grip, and with apples with a power grip, assuming that these fruits are within our reaching distance. Obviously the size of cherries (or apples) has a certain degree of variability: not all cherries have the same size, but we typically grasp all cherries with a precision rather than with a power grip. The associations between the visual aspects of cherries and apples and the motor response they produce can be incorporated into an object representation, stored in memory (e.g., we ''know'' that cherries are graspable with a precision grip).

Canonical affordances can be considered as a subset of stable affordances, characterized by a lower degree of stability and higher contextual dependence than purely stable affordances. They derive from properties that vary with respect to the interaction with us, such as orientation, but that can become more stable across multiple experiences. For example, we might consider cherries as having a canonical orientation: they are hardly grasped with the petiole on the lower side, since we typically pick them up from trees, from containers or from plain surfaces, and in all these cases they have the petiole on the upper side. A similar but slightly more complex case is given by cups. The complexity is due to the fact that, differently from natural objects, artifacts typically evoke both manipulative/grasping actions and actions related to object's use, i.e., they evoke two kinds of actions that do not necessarily coincide (Jax and Buxbaum, 2010). The way we use the term manipulation probably requires some clarification, since this term has been used as well to refer to haptic exploration of objects not guided by a specific goal (Menz et al., 2010). We use this term to refer to the hand posture and grip which are not aimed at using a given object but simply at interacting with it, for example to move it. We propose that canonical affordances are linked to the actions we more typically perform with objects, to the most frequent contexts in which they are embedded and to the most frequent goals with which we approach them. In the case of artifacts these actions, contexts and goals are usually related to their use. Even if we interact with cups in different orientations—for example when we wash them, move them, etc., when we grasp cups to use them, they typically have a specific orientation: they are upright, since we have to hold them to drink the liquid they contain. Due to the higher frequency of this upright orientation when we use cups, it might be useful to store information on cups' canonical orientation.

Referring to the context can help us to further clarify why canonical affordances can be seen as a subset of stable affordances, characterized by a lower degree of stability and a higher degree of contextual dependency. Stable affordances related to intrinsic properties of objects (Jeannerod et al., 1995), such as those emerging from object size, vary less across contexts and goals. For example, we typically grasp cherries and pencils with a precision grip, independently from the context, for the very simple fact that their dimension affords a precision grip. Canonical affordances, such as canonical orientation, are instead selected by the context. For example, the orientation of a knife for using it in order to cut something and for handling it to somebody else typically differs.

Differently from stable affordances, we consider variable affordances to derive, instead, from rather temporary object characteristics. Furthermore, variable affordances are strictly linked to the actions we are about to perform. Take the following example concerning object location: the location of cherries may vary—they can be on a tree, on a table, and their petiole might be upright but more or less inclined, thus we may need to adapt online our motor responses to the current location of the cherry we intend to grasp. Given the variability of this information, it wouldn't make sense to store in memory information on it.

Notice that the term ''stable'' should not be misleading: we do not see any incompatibility between the use of this term and the idea that affordances are processed and responded to online, thus that they might need a certain degree of adjustment of the organism in relationship to objects (see objections by Osiurak, 2013). At the same time, some stable parameters are needed to program actions, in particular if we have to program them offline, without having an object or an entity in front of us. This happens, for example, when someone tells us ''Grasp the cherry'' or ''Lift the hammer'' before we have seen or recognized the object we have to interact with. This stability is more the result of a dynamical process than of an a priori determination; furthermore, it is not given but it is subject to continuous updating. It could be argued that what we call ''stable'' affordances are not real affordances but rather consists in simple knowledge of the object. We do not think this is the case, because not only variable but also stable affordances dynamically evoke motor responses. Hence, given that in our view stable and variable affordances are arranged along a continuum, one could speak of ''more'' stable and ''more'' variable affordances.

As to their brain localization, some years ago Borghi and Riggio (2009) proposed that stable affordances are represented more ventrally compared to variable ones (see also Young, 2006). In particular, the bipartition of the dorsal stream into a dorso-dorsal and a dorso-ventral system, introduced by Rizzolatti and Matelli (2003), is crucial to capture how these two kinds of affordances are represented in the brain. According to them, the dorso-dorsal stream, corresponding to the dorsal stream as originally defined by Milner and Goodale (Goodale and Milner, 1992; Milner and Goodale, 1995) and contrasted with the ventral stream, is the only stream not related to perception and would be dedicated to the online control of action. The ventro-dorsal stream, instead, would be specifically involved in sensorimotor transformation for grasping, space perception and recognition of actions performed by others (see Gallese, 2007; Binkofski and Buxbaum, 2013; Maranesi et al., 2014). Notice however that these two streams are strictly interconnected and that all three streams—dorso-dorsal, ventro-dorsal and ventral—finish into cortical frontal areas.

A meta-analysis of fMRI studies (Sakreida et al., in preparation) has confirmed that variable affordances are represented more dorsally than stable ones, which are instead represented in bilateral inferior parietal and premotor cortices (dorso-dorsal vs. dorso-ventral stream), even if in the left hemisphere there are overlap areas between the two. This distinction is consistent with clinical observations. Optic ataxia implies indeed impairments during visual reaching of objects, which might be influenced by object orientation (variable affordance). Limb apraxia can be instead characterized by impairments in manipulation of objects, which might be related to object size (stable affordance).

Results obtained with single cell recordings on monkeys and brain imaging data with humans are also informative as to the specific circuits likely involved during processing of specific kinds of affordances (for a review, see Rizzolatti and Craighero, 2004). Data on the monkey's brain (Raos et al., 2004, 2006) indicate that the visuomotor transformations for grasping objects occur in the anterior intraparietal area (AIP)- F5 circuit, which is devoted to select the most appropriate motor schemas for the actions to be activated. Even if AIP-F5 are better conceived of as a whole, neurons of F5 are more motor and maintain memory of the object also in the dark, while neurons of AIP are more visual and likely render visual affordances available to the motor system. F5 canonical neurons are differently activated depending on the kind of grip objects require (e.g., precision, power) and are not influenced by changing the position of the object in space (Jeannerod et al., 1995). Recent evidence (Bonini et al., 2014) suggests that they are however influenced by the position with respect to the agent's body, namely they are responsive only when the object is in the peripersonal space. The role played by F5 neurons in motor object representation is consistent with fMRI and PET studies on humans showing that the ventral premotor cortex is activated during observation and imagery of manipulable objects and tools. For example, neuroimaging evidence has shown that images of tools, but not of houses, animals, and faces, activated the ventral premotor cortex (Grafton et al., 1997; Knight et al., 1999; Chao and Martin, 2000), and that the ventral premotor cortex was activated with manipulable objects but not with not manipulable ones (e.g., Gerlach et al., 2002; Kellenbach et al., 2003); the conjunct activation of the left posterior parietal and left premotor cortices can be considered as the human homolog of the canonical neuron system (see Martin, 2007, for a review).

Overall, according to our proposal two ventro-dorsal circuits concern more stable affordances in humans. The first is the phAIP circuit (Orban and Caruana, 2014), which corresponds in humans to the AIP-F5 circuit in monkeys. Both stable and canonical affordances would be represented in this first circuit: it has namely been shown that F5 neurons encode at the single neuron level both the grip and the wrist rotation (Raos et al., 2006). The second circuit, which is present only in humans, would be located in anterior supramarginal gyrus (aSMG), operates in parallel with phAIP and is specifically devoted to tool use (see Orban and Caruana, 2014; on tool use see also Johnson-Frey, 2004; Johnson-Frey et al., 2005).

As to variable affordances, as anticipated we propose they are represented more dorsally, in the dorso-dorsal stream of the dorsal pathway (Rizzolatti and Matelli, 2003). Considering data on the monkey brain, an area candidate to the processing of variable affordances is a third area where grasping neurons are located, namely area F2 (corresponding in humans to the dorsal PM cortex). This area, which receives visual input from the superior parietal lobe and area medial superior temporal (MST), encodes the orientation of the wrist for grasping the object under visual guidance, continuously adjusting online the grip to the object. Notice that the novelty of our proposal does not consist in the anatomical identification of novel neural circuits but rather in connecting these previously identified circuits with the distinction between stable and variable affordances, that hopefully will allow researchers to link behavioral evidence on affordances with such neural underpinnings.

# Similarities and Differences from Other Proposals

In the following section we will discuss the notion of stable and variable affordances in the framework of similar proposals advanced in the literature: the distinction between intrinsic and extrinsic properties, originally proposed by Jeannerod (1981, 1984; Jeannerod et al., 1995), the 2 Action Systems (2AS) proposal (Buxbaum and Kalénine, 2010) and a recent proposal formulated by Orban and Caruana (2014).

Jeannerod (1981, 1984) distinguished between intrinsic object properties, such as size, shape and texture, which are linked to grasping, and extrinsic properties, such as distance and direction, related to object transport in space, which determine the arm and hand position with respect to the object. According to the visuomotor channels hypothesis (Arbib, 1981; Jeannerod, 1981), the visuomotor transformation related to object transportation and to object grasping are namely independent. Different motor schemas would be activated: a circuit pertaining the arm, devoted to transportation, and another specifically focused on the hand, linked to grasping, i.e., to the preshape of the hand while approaching the object and to the enclosure of the hand on the object.

A first difference between the distinction between stable and variable affordances we propose and the distinction between intrinsic and extrinsic object property is that according to our proposal affordances cannot be assimilated to object properties. Affordances are instead relational constructs, i.e., they refer to brain representations of relationships between an organism and one or more objects/entities within a social and physical environment. As said, we agree with Ellis and Tucker (2000) as they consider affordances are patterns of associations, in the brain, of visual and motor experiences.

A second difference is that we do not distinguish between stable affordances related to grasping and variable affordances related to transport. As the examples above should clarify, stable (including canonical) and variable object affordances can emerge both during the transport and the grasping actions.

A further difference concerns the neural underpinnings: the difference between intrinsic and extrinsic properties is indeed linked to the distinction between the grasping and the transportation route (Jeannerod, 1994, 1997; Jeannerod et al., 1995). We propose instead that the distinction between stable and variable affordances is anchored to the difference, within the dorsal route, between the ventro-dorsal and the dorso-dorsal streams (Rizzolatti and Matelli, 2003). The two routes identified by Jeannerod and collaborators as dedicated to transport and to grasping are both represented in the ventro-dorsal route, which in our proposal is dedicated to stable and canonical affordances. Variable affordances would be instead represented in the dorso-dorsal route, mainly used for the online control of actions.

Our view has some similarities with the (2AS) proposal (e.g., Buxbaum and Kalénine, 2010). The main tenet of this proposal is that two different routes to action exist, the Structure and the Function one. The Structure system is bilateral, and it is specialized for visual information related to object shape, size, and location, which is continuously updated. The Function system is left-lateralized, and concerns more stable conceptual knowledge: for example, this system extracts the characteristics of a given action that remains constant over time, such as the typical features characterizing grasping regardless of the specific object to be grasped and of the kind of grip which is used. These systems are not independent but highly interactive and are likely mediated respectively by the dorsodorsal and by the ventro-dorsal route. According to the 2AS view, artifact objects might evoke at the same time both structural responses and functional ones (see also evidence by Bub et al., 2008), and these actions might interfere. For example, a knife might evoke both manipulation and functional information: a kind of grip adequate to hold it to put it into a drawer (structural response) and another adequate to use it, for example cutting something (functional response). The damage of one of the two systems can lead to impairment: for example, apraxia of tool use would be due to impaired manipulation (structural) knowledge. Jax and Buxbaum (2010) have also demonstrated that the structural and functional systems differ in activation and maintainance time: structural responses are typically quicker than functional ones, but they last less, whereas functional responses are slow and long lasting.

One similarity between our view and the proposal by Buxbaum and Kalenine (see also Binkofski and Buxbaum, 2013), is the assumption that observing a tool allows us to extract information that differs in content and time course, i.e., long term information such as that characterizing stable affordances, and online information such as that characterizing variable affordances. Another similarity is that, in both cases, the dorso-dorsal and the ventro-dorsal routes are the candidate areas for these representations. However, in our view stable affordances are not necessarily dedicated to functional information—for example, for natural objects such as cherries or apples stable affordances can concern the typical way in which we grasp and manipulate them.

Furthermore, while Buxbaum and collaborators stress the potential interference arising between the two action systems, and in particular between manipulation and function, we mainly underline the different temporal and content characteristics of stable and variable affordances. Stable and variable affordances derive indeed from different object characteristics, some of which are more keen to be maintained in long term memory compared to others.

Finally, our proposal has some similarities with the view by Orban and Caruana (2014) who propose that humans have two different parietal circuits: the first, located in phAIP, would be dedicated to grasping and manipulating all kinds of objects; the second, located in the left aSMG, would be specifically devoted to tool use and would be at the basis of technological development in our species. Both would send connections to the vPMC. Importantly, the parallel operation of phAIP and aSMG is a further elaboration of the ventrodorsal stream (Rizzolatti and Matelli, 2003). According to Orban and Caruana, affordances would refer only to the grasping component, which is related not only to tools but to objects as well, even if the aSMG can contribute in selecting the affordances for phAIP, as demostrated by evidence with patients with ideomotor apraxia. The phAIP component contributes in planning appropriate grasping actions considering objects size and shape, thus corresponding to the canonical neuron systems in monkeys (F5 and AIP), i.e., to the ventro-dorsal stream.

Our proposal is in line with Orban and Caruana's one since we do believe that it is important to distinguish between motor information related to manipulation and to use, even if this distinction is not at the core of our proposal, and the authors identified two different neural circuits for grasping affordances and tool use in humans. There is clear experimental behavioral and neural evidence supporting the view that grasping and use are different: for example, grasping the handle of an object to use it is disrupted by a semantic task, but not by a visuospatial one (Creem and Proffitt, 2001; see also Creem-Regehr and Lee, 2005). Despite some similarities the focus of our proposal, i.e., the distinction between stable and variable affordances, is obviously different from that of Orban and Caruana. Orban and Caruana limit the use of the term affordances to the activation of objectdirected actions that take into account objects' shape and size. In this sense they adopt a strictly Gibsonian view, according to which accessing to knowledge on the object is not necessary to respond to object affordances. We use the term affordance both to refer to grasping and to use (see Chaigneau et al., 2004, for an integrative view of affordances, intentionality and function in artifacts). The reason why we choose to use the term in both cases is that in humans there are situations in which the distinction is clear, but also many cases in which it is really hard to distinguish between the two. Take for example a fork: to what extent is the proficient use of a fork evoked by characteristics of the fork we process online, to what extent is it due to long-term visuomotor associations between the vision of that particular object and the action of bringing something to the mouth, i.e., to its use? Furthermore, limiting the use of the term affordances to online grasping of objects would not allow us to speak of affordances mediated by language. Given our use of the term affordances, we do not like to claim that affordances are egocentric, as both Orban and Caruana and as Osiurak (2013) seem to do: we report below experimental evidence showing that affordances are modulated by the context, for example that a pen affords different actions when it is presented close to a sheet of paper or to a stapler (Yoon et al., 2010; Borghi et al., 2012; Ellis et al., 2013).

# Stable and Variable Affordances and Language

So far we have illustrated the distinction between stable and variable affordances. One important issue concerns how these affordances are encoded in language (Kaschak and Glenberg, 2000). According to embodied theories of cognition, language is grounded in perception, action and emotional systems (see Gallese and Lakoff, 2005; Barsalou, 2008; Pulvermüller and Fadiga, 2010; Glenberg and Gallese, 2012; Meteyard et al., 2012; see the recent special issues: Borghi and Pecher, 2011; Cappa and Pulvermüller, 2012; Cangelosi and Borghi, 2014). Does this imply that language mirrors exactly the processes and structures of the motor system? We do not believe this is the case. Instead, in line with theories of resuse (Gallese, 2008; Anderson, 2010), we think that language recruits some mechanisms and processes of the perception and motor system, but not necessarily all of them are encoded (see Borghi, 2012, for a more detailed analysis). From this general view the hypothesis follows, that language acts as a sort of filter, encoding only certain kinds of affordances. We will describe some recent studies to clarify our points.

In a behavioral study we asked participants to read a sentence composed by an action vs. an observation verb followed by a noun (e.g., grasp/look at the brush), then they were presented with a photo of an object and had to decide whether the object was the one mentioned in the sentence or not (Borghi and Riggio, 2009). We found that during language processing a motor prototype is formed. This prototype includes stable and canonical affordances, related in this case to object size and canonical orientation: they are indeed encoded in language, while variable affordances are not (see Borghi, 2012, for theoretical development of this issue, and Ferri et al., 2011b; Myachykov et al., 2013, for further evidence). During real interaction with objects the role played by affordances might differ, and in particular the role played by stable affordances might be more marginal, compared to what happens with language.

This hypothesis is supported by evidence by Ferri et al. (2011b). The authors used 3D pictures of objects and asked participants to perform precision or power grips to determine their category (artifact vs. natural object); for artifacts they found a compatibility effect between the grip used to respond and the object size, but only when the objects were presented within the reachable space. In a further experiment participants were required to decide whether the 3D pictures corresponded to previously presented names: in this case the compatibility effect between the grip to respond and the object size was present, but it was not modulated by the space. These data suggest that objects and objects' names have different motor representations: while objects are characterized both by stable (i.e., shape and size) and variable affordances (i.e., orientation and distance with respect to the perceiver), objects' names seem to house only stable ones.

Further support to the view that language encoded primarily stable affordances comes from a recent study in which we used mouse tracking to investigate the real-time dynamics of compatibility effects (Flumini et al., 2014). We tracked the time course of a categorization experiment requiring subjects to categorize as natural or artifact pictures of big and small objects. Participants responded using either a big mouse (hand posture compatible with the grasping of big objects) or a small mouse (requiring a precision grip: a hand posture compatible with the grasping of small objects). We found a compatibility effect between the grip required by the mouse and the grip elicited by objects, even if it was irrelevant to the task. A further experiment in which images were substituted by words failed to reproduce the effects. The use of words in this study (as in the previous one) allows to test three different hypotheses, each leading to different predictions. According to the first, words are represented in an abstract, propositional and amodal way, thus no perceptual or motor effect should be found with words. According to the second hypothesis, derived from a purely embodied view, words are grounded in the sensorimotor system, hence the same compatibility effect found with images should be found with words. According to the third hypothesis, derived from theories of reuse (Gallese, 2008; Pezzulo and Castelfranchi, 2009; Anderson, 2010), words are grounded in perception and action systems, but language processing differs to some extent from processing of objects, hence the results with words should not necessarily mirror those obtained with objects. Apparently at odd with a purely embodied view, we did not find the compatibility effect between the grip elicited by the mouse and the grip evoked by the object with words. However, we found that while using the small mouse, and thus performing a precision grip, the processing of artificial small targets was inhibited and the processing of natural small targets was facilitated. This reveals that during language processing participants activated information on the size of the word referent. When the object was an artifact, it provoked an interference with the mouse they held, otherwise a facilitation. The interference is likely due to the fact that artifact words evoke use programs, which are in conflict with the manipulation posture required by the mouse. Such a conflict between manipulation and use is not present with natural objects. This sensitivity to size with linguistic stimuli is in keeping with our third hypothesis. Indeed, it suggests that, since language is a rather sophisticated ability, word processing might not reflect all the dynamics characterizing processing of their referents, in line with theories of reuse. Furthermore, it confirms that language recruits stable affordances, such as size.

# On Dynamic Aspects of Both Stable and Variable Affordances: Some Responses to Osiurak

In a recent paper, Osiurak (2013) criticizes the proposal of stable and variable affordances and proposes that apraxia is not a matter of affordances. Responding to Osiurak's objections will allow us to better outline our points, in order to avoid any misunderstanding, concerning some important issues. First, we will clarify that the proposal of stable and variable affordances was initially conceived in the context of studies on language processing, and we will show the consequences of this. Second, we will clarify that we are not inclined to use the terms allocentric and egocentric to refer to affordances. Third, we will address the relationship between apraxia and affordances. Fourth, we will try to differentiate between affordances and action goals. We will discuss these four points below.

1. Osiurak (2013) criticizes our view arguing that no affordances about the canonical manipulation of tools can be stored, due to the dynamic character of action. We agree with Osiurak that affordances are flexible and variable (We address this issue more in depth at point 4). At the same time, however, we believe that not only online information but also previous experience play a major role in affordance representation. The role of previous experience is particularly important when we consider affordances as processed offline, as it happens when they are mediated by words or by images. The proposal of stable and variable affordances was firstly advanced in such a context, while discussing the results found by Borghi and Riggio (2009) (see above), showing that during language comprehension we form a motor prototype encoding stable and canonical affordances. Without encoding some stable information no motor recruitment and linguistic comprehension would be possible. At the same time, however, accepting that some stable information must exist does not imply at all denying the importance of the flexibility and contextual dependence of affordances (Mizelle and Wheaton, 2010).

What happens with novel objects? To respond adequately to their affordances we might rely on the context as support (Pellicano et al., 2011). If the context is novel as well, we would in any case need to rely on previous experiences with objects endowed with similar affordances. Jacquet et al. (2012) clearly showed the role of probabilistic cues related to previous experience, together with that of biomechanical constraints, in predicting interaction with novel objects under conditions of visual uncertainty. Overall, we believe that it is difficult to think of objects, entities and situations which are completely novel, in which current experience cannot be traced back to similar previous experiences.

2. Osiurak (2013) claims that manipulation knowledge and stable affordances would be egocentric, since they specify the relationship between the user and the tool. The problem, he argues, is that patients need to form an allocentric representation to solve mechanical problems. Differently from Osiurak, we are not really keen to use the distinction between egocentric and allocentric representation while referring to affordances (see Osiurak, 2013). Rather, we prefer to see affordances as the product of repeated experiences, with a given object or with objects structurally similar to it. This experience is not necessarily an individual experience, but we can benefit from others' experience. We might observe other people interacting with an object to ''capture'' the object's affordance, or we might even see an object or two objects interacting to simulate interaction with them. In this respect, recent evidence on canonicalmirror neurons can be informative. Canonical neurons, which are thought to be the neural underpinnings of affordances (Murata et al., 1997), and mirror neurons (Gallese et al., 1996) were typically considered as segregated: the first ones fire not only when individuals interact directly with objects but also when they observe manipulable objects and the second ones during observation of others' actions. Recent evidence challenges this dychotomic view (Bonini et al., 2014), showing that canonical-mirror neurons exist. Interestingly, while canonical neurons do not code 90◦ rotated objects, canonical-mirror neurons do. Furthermore, canonical neurons fire only when objects are located in the peripersonal space, likely due to the connections between area F5 and area F4 (Fogassi et al., 1996; Matelli et al., 1996). Canonical-mirror neurons seem instead to code object as target for both one's own and other's action, thus they are not selective only to objects presented in the peripersonal space (for consistent behavioral evidence, see Costantini et al., 2011a,b). This suggests that they could play a role in predicting others' actions (for a review see Maranesi et al., 2014). In sum, we do not think that affordances are necessarily egocentric, because we can perceive also affordances for others, as demonstrated by recent evidence. Furthermore, we think that to respond to objects' affordances we take into account the context and the relationships between objects: for example, a cup affords a different action when located near to a coffeepot (we might need to hold it firmly to pour the coffee), near to a spoon (we might need to hold its handle to turn the spoon) or near to an object we want to hide (we might want to turn it upside down). In all these cases, we need to consider the relationship between the objects in the context.

3. As to the relationship between apraxia and affordances, we think, differently from Osiurak, that the possibility that apraxia depends on difficulty in affordances processing is an interesting research avenue that should be explored. For example, disturbed object use and disturbed pantomime in apraxia (Goldenberg, 2009) can be linked to difficulties in responding properly to affordances. In cases of disturbed object use patients typically grasp the object in a wrong way and use it for a wrong purpose: for example, they may grasp a hammer and move it to and fro over the table. In cases of disturbed pantomime the patients either perform the wrong movement or use a body part as object. For example, a patient with disturbed ability to pantomime, when asked to show how to use a comb will scratch his head. Especially interesting are body part as object errors. Here the patient, when asked to use scissors, will move his fingers as it were scissors instead to show that he/she is holding scissors and using them.The fact that such disturbances in object use do not always lead to impairments in object recognition can be explained by the fact that in our view not only object use, but also object manipulation when directed to use can implicitly activate object knowledge (see also Ellis and Tucker, 2000, for a similar view).

4. Finally, with respect to Osiurak we do more clearly differentiate between affordances (i.e., interactions between hand and object) and action goal.

In line with some influential proposals on action representation, such as those derived from ideomotor views (e.g., Prinz, 1997; Hommel et al., 2001), empirical evidence has shown that during action planning the action goal dominates over the hand grip (e.g., van Elk et al., 2011). According to hierarchical views of action representation (e.g., Grafton and Hamilton, 2007), specific motor programs are selected on the basis of the outcome of the action (see however Bonini et al., 2012). We are totally in line with this view. On similar basis, we agree with Osiurak as he highlights that the context and the goal select affordances in a variable and flexible way. But once this has happened, the selected affordances for a given context might be stable.

To clarify with an example: neural and behavioral studies have shown that the way in which we grasp objects might differ depending on the context/goal, and that we are sensitive to this information when we observe others. Fogassi et al. (2005) have shown in a study on the monkey parietal cortex that motor acts, such as ''grasping'', are coded differently depending on the action goal (e.g., ''grasping for eating'' vs. ''grasping for placing'') (see also Iacoboni et al., 2005, for an fMRI study on humans). In a similar vein, recent kinematics evidence by Scorolli et al. (2014) demonstrated that subtle variations in the hand posture can suggest whether an individual vs. a cooperative action will be performed (e.g., grasp a cup of tea to drink/to offer it to someone else). The context and the action goal are therefore not independent from affordances. But some aspects remain rather stable: for example, during both grasping for eating and grasping for placing a cherry we use a precision grip, even if the grip orientation and the action preparation vary depending on the action goal.

In sum: we do not intend to deny the flexible interplay between stable and variable aspects that occurs both when we interact with objects and when we process images or words referring to objects. This interplay might however be different depending on whether affordances are processed online, during direct interaction with objects, or whether they are processed through images and words. In the first case stable affordances might play a more marginal role compared to the second.

In particular, we propose that language understanding is tied to and constrained by object affordances, but that language recruits primarily some kinds of affordances, i.e., stable and canonical affordances rather than variable ones (Borghi and Riggio, 2009; Borghi, 2012). As we have seen, evidence from our labs and other labs clearly supports this view.

# Affordances Automaticity Questioned

As discussed above, the notion of affordances has been object of growing interest in the last years, in particular in the framework of embodied and grounded theories of cognition. A variety of studies have been conducted, the majority of which using compatibility effects. For example, in one of their seminal studies Tucker and Ellis (1998) asked participants to decide by pressing with the two hands two different keys on the keyboard to decide whether objects were upright or reversed. Results showed a compatibility effect between the location of the handle of the object (left, right) and that of the key to press (left, right). The results suggest that handles evoke affordances, even if the task does not require to pay attention to them. Evidence like this has been taken as demonstration that observing objects activate affordances, and that affordances are activated automatically, independently from the task at hand (for similar evidence, see Tucker and Ellis, 2001). However, recent evidence suggests more caution in approaching the issue of whether affordances are automatically activated or not; the issue of automaticity contrasted with topdown processing does not seem to be solved but is rather hotly debated (see Buxbaum and Kalénine, 2010; for a recent critical review, see van Elk et al., 2014). First of all, some recent work (Yu et al., 2014) failed to replicate compatibility effects when participants were not explicitly instructed to imagine picking up the pictured objects. More crucially, recent studies have challenged the view according to which affordances are automatically activated, showing that their activation is modulated by the task and the context (e.g., Girardi et al., 2010).

Some results by Riggio et al. (2008) are useful to understand whether affordance effects can be qualified as automatic. The authors presented participants with pictures of two objects with a handle; one object remained on the screen and the other disappeared. They used a modified version of Tucker and Ellis (1998) task, asking half of the participants to judge whether the object that disappeared and the other half to decide whether the object that remained on the screen were upright or reversed. Since disappearing stimuli are dynamic events capturing attention, the target object could or could not be the event capturing attention. The objects were shown above and below or to the left and to the right of a fixation point in order to dissociate the affordance effect (correspondence between handle left-right orientation and response location) and the Simon effect (correspondence between stimulus and response position). The results showed that, while the Simon effect occurred relative to the event capturing attention, the affordance effect, when evident, was always relative to the target object, irrespective of its attentional capturing properties. This result is in keeping with the view that the affordance effect is the consequence of encoding the pragmatic properties of the target, and rules out the possibility that the effect is generated by the attentional capture of the object (or part of it) per se. Moreover, these findings suggest that automatic and controlled processes of visual attention may play a differential role in the occurrence of the affordance and Simon effects. In particular, the affordance effect seems to depend on the selection and processing of the objects that are relevant to the task. This finding raises the issue of the automaticity of the activation of affordances, even if the result is not necessarily incompatible with an initial automatic activation of affordances followed by the selection of the affordances relevant to their current task.

Further recent studies, which we will briefly overview below have shown that affordances activation is not independent from the task and is modulated by the context.

# Automatic but Task and Context Dependent Task

A first series of studies has shown that affordances effects are present only when the task requires deep processing of the objects characteristics: for example, shape categorization typically leads to affordance effects, while color categorization does not. To the best of our knowledge, Tipper et al. (2006) were the first who showed that the affordance effect was modulated by the task, since it was present only when participants were required to categorize handles as to their shape, not to their color. In keeping with this result, Pellicano et al. (2010) used torches and demonstrated that when categorizing them on the basis of color (blue vs. red), a Simon effect (compatibility between the goal-directed tip of the object and the location of the key to press to respond) was found, but no affordance effect was present. The affordance effect, intended as the compatibility between the position of the object handle (left, right) and the location of the key to press (left, right), was present only when participants had to decide whether a given torch was upright or reversed. Crucially, the effect was more marked when the torches were switched on, in line with the idea that participants formed a motor simulation of the action of grasping the handle and holding the torch to illuminate. The absence of the affordance effect with a color judgment task, likely due to the fact that color categorization requires superficial processing, challenges the view according to which affordances are activated automatically, i.e., independently from the task at hand (for a computational model on this, see Simione et al., submitted).

### Context

A further series of studies has started to emphasize the importance of context in responding to affordances.

#### **Single objects: near and far space**

Recent studies demonstrated that object affordances are only activated when objects are located in the near (peripersonal) but not in the far (extrapersonal) space. Costantini et al. (2010) showed with 3D pictures of everyday objects (e.g., bottle, cup) that objects evoke actions only when they are presented within the portion of the near space that is reachable by the participants.

Further studies with a similar paradigm investigated whether the modulation of the affordance effects due to their location in the near and far space held also with linguistic stimuli. Costantini et al. (2011a) showed participants with 3D objects located in peripersonal vs. extrapersonal space followed by function, manipulation or observation verbs (e.g., ''to drink'', ''to grasp'', ''to look at''). Participants were required to respond releasing a key and performing a simulated grasp when the verb they read was compatible with the presented object. Responses with both function and manipulation verbs were faster when objects were presented in reachable than in the far space, while no difference between the near and far space was present for observation verbs. Results suggest that, during simulation of an action evoked by manipulation and function verbs, objects affordances are primarily activated when objects appear in the participants' reachable and operational space. Ambrosini et al. (2012) confirmed and extended the previous finding. They used the same paradigm, but introduced a variation: they distinguished between actual and perceived (explicitly estimated by participants) near and far space. Their results confirmed that responses to verbs related to actions were faster in the near than in the far space, while responses to pointing and observation verbs did not differ. Importantly, responses to function and manipulation verbs were faster in the actual near space compared to the perceived near space. This finding suggests that activation of affordances when objects were followed by action verbs is modulated by objects' location in space with respect to the body. Importantly, this location is computed online and it is not reflected in explicit representations, as the distance estimations participants were required to provide. It should be noted that these results are quite in agreement with the neural counterpart of affordances computation. As already said the AIP-F5 and ventral intraparietal area (VIP)-F4 circuits are interconnected. In particular the VIP-F4 circuit codes the peripersonal space, that coincides with the motor space for arm reaching, regardless of the eye location, thus explaning why the space modulation is limited to action verbs.

One possible question that might rise is the following: given that according to your proposal language encodes stable and (eventually) canonical affordances, but not variable ones, why did you find that variable affordances as those emerging from object's location and orientation with respect to our own body influence language processing? Notice that the present study does not make use of solely linguistic stimuli, but of linguistic stimuli in combination with pictures—this is likely the reason why action verbs encoded variable affordances as well (see below).

Even if the effect was present with action verbs, it disappeared when object names were presented. Ferri et al. (2011b) used a similar but sligthly different paradigm, in which participants after presentation of the 3D images had to decide by making a reach-to-power or a reach-to-precision response whether the object was congruent with a previously displayed word. They found a compatibility effect between the response and the grip evoked by the object, but, differently from what happened with objects, the effect was not limited within the reaching space. However an important difference exists between the two studies. Costantini et al. (2011a) and Ambrosini et al. (2012) first presented visual objects and then verbs: as already said, in these cases object perception recruits motor information related to their possible interaction only when objects are presented in the near space, determining a priming effect when verbs expressing such an interaction are then presented. Ferri et al. (2011b), conversely, first presented nouns and then visual objects. Results demonstrated that noun processing recruits motor information too, but in this case the motor recruitment is not limited to objects that are then presented in the near space since noun processing cannot specify variables features such as the distance between the object denoted by the noun and the body. Indeed when a noun is presented the VIP-F4 circuit cannot be activated since a physical object is absent. However the size-grasp motor recruitment is still present even if it is not modulated by the space; stressing, in line with embodied theories, that concepts incorporate motor information.

Overall, the studies presented reveal that: (a) affordance activation is modulated by the distance of the object from the body; (b) that this information is encoded in language only to a certain extent, probably due to the fact that the object location is a variable affordance. Further studies with solely linguistic stimuli are probably necessary to better understand how different kinds of affordances are linguistically encoded.

# **Single objects in scene, or more than one object**

For a while the majority of studies on affordances have focused on how we respond to single affordances, and to single objects. It is however important to focus also on objects that might evoke different affordances. In everyday life we are indeed typically exposed to multiple affordances. Many objects typically surround us—for example, I might choose to write with the laptop or with a pen I have on my desk. Even the same object can evoke multiple affordances: for example, different parts of an ice cream might evoke grasping and licking. While grasping and licking can be performed at the same time, sometimes objects evoke conflicting actions: for example, a sofa might invite us to sit but also to jump on it, or the same object can elicit different kind of grips. Studies mostly performed in Laurel Buxbaum's lab have shown that structural information and functional information may conflict while planning actions with objects. Interestingly, these two kinds of information have a different time course: functional information may last longer generating long-term interference, as information in long term memory, while structural information has a rapid decay (Jax and Buxbaum, 2010). Recent work by Kalénine et al. (2014) demonstrated that, depending on the kind of scene in which it is embedded, the same object can evoke a manipulative or a functional grip. They presented images of ''conflict'' objects, i.e., objects associated with move (clench posture) vs. use (pinch posture) hand postures, as for example a corkscrew. The objects were displayed within everyday scenes, as a kitchen or an office. The results revealed a compatibility effect between the move scene (e.g., drawer for corkscrew) and the clench posture and a more marked compatibility effect between the use scene (e.g., on a bottle for corkscrew) and the pinch response. This result suggests that the same object can evoke different affordances depending on the context. However, the time-course of the process needs to be explored, since the result is compatible with two possibilities: an automatic activation of all object affordances followed by a selection, triggered by the context, of the affordance relevant for the current context, or an early selection determined by the context.

As we have seen, the same object can evoke different affordances, and the context selects which one to activate. Apart from this, objects might be embedded in contexts where multiple objects are present, hence where multiple affordances are activated. Pezzulo et al. (2010) analyzed how expert and novice climbers memorize multiple affordances, i.e., sequences of holds organized in routes of varying difficulty. They found that climbers simulated ascending the route: thus they represented affordances in context, and this influenced their recall. Aside from this study, the great part of evidence concerns online processing of objects or images rather than recall tasks. To our knowledge the only notable exception are the studies recently conducted by Diane Pecher and collaborators, who investigated the role played by affordances in working memory using interfering paradigm (e.g., Pecher, 2013). The authors failed to find that affordances played a role in working memory. This could be due to the fact that, in order to be activated, affordances linked to memory would need deeper processing compared to the more superficial one required by working memory. This is in line with our view, according to which stable and canonical affordances are encoded in long-term memory, while temporary affordances decay rather soon. Their rapid decay can contribute in explaining why variable affordances require continuous monitoring of the relationship between the hand and the object.

So far we used the term ''context'' referring only to the physical context (e.g., scenes, presence of other objects, etc.). It is however important to determine the influence on affordance activation of both the physical and the social context. To investigate this, some authors have introduced the presence of social cues, for example of an effector interacting or potentially interacting with the object, or of a more complex social context.

# **Physical and social context**

Yoon et al. (2010) have demonstrated affordances effects presenting participants with object pairs. Right-handed participants made faster classification responses to pairs of objects displayed in standard co-locations for right-handed actions compared to when the objects are shown in reflected locations. These effects were more marked when participants' task consisted in deciding if the two objects are typically used together, rather than if objects typically occur in a given context. The effects, which are stronger when an agent is shown holding the objects, disappear when the objects are not viewed from the first-person perspective and when words are presented rather than objects. The data suggest that: (a) participants are sensitive to whether objects are positioned correctly for their own actions; (b) the position information is coded within an egocentric reference frame; (c) the critical representation involved is visual and not semantic; and (d) the effects are enhanced by a sense of agency. The authors interpret the results within a dual-route framework for action retrieval in which a direct visual route, the dorsal one, is influenced by affordances for action, while the ventral route is not. If we consider the further distinction between the dorso-dorso and ventro-dorsal stream, however, we could hypothesize that the process pertains the dorso-dorsal route.

Borghi et al. (2012) presented images of pairs of objects linked by different kinds of relations. They could be functionally related (e.g., scissors-paper) (functional context), thematically related (e.g., scissors, stapler) (spatial context) or not related (e.g., scissors-bottle). The object to be used was positioned on the right. In <sup>3</sup> 4 of the trials a hand appeared, which could be simply close to the object, or interacting with it either with a functional grip, i.e., grasping the object as to use it, or a manipulative grip. Participants were required to decide by pressing a different key on the keyboard whether the two objects were related or not. The results showed a clear effect of the context. Overall, the functional context was processed faster than the spatial one, consistently with the view that artifacts are represented in terms of their use. Most importantly, the interaction between the hand posture and the context was significant. A compatibility effect between context and grip was found: response times were indeed slower when a manipulative grip was presented in a functional context, and when a functional hand posture was displayed in a Spatial context.

The neural mechanisms underlying the described interaction were further investigated by an EEG study with the same stimuli (Natraj et al., 2013). While both Functional and Manipulative postures in the Functional context activated predominantly an early left parietofrontal circuit, the Manipulative posture alone engaged a late right parietofrontal network. Furthermore, bilateral parietofrontal activation increases with the Spatial context, supporting our previous interpretation that, when no functional use of the object is allowed, the motor system tries to make sense of the scene. These EEG results suggest that, when action affordances are not immediately apparent and hand posture does not support action (Manipulative) as well as when the context does not immediately evoke tool use (Spatial context), bilateral activation is increased.

Overall, the two previously described studies highlight the relevance for affordance activation of both the physical context (relations between objects) and of the social cues allowing to detect the intention of the agent, given by the different hand postures. A number of recent studies focus on affordances in a social context (e.g., Sartori et al., 2009; Ferri et al., 2011a; Ellis et al., 2013). We will here illustrate a recent kinematics study by Scorolli et al. (2014) who investigated the role of the physical and social context more in depth, engaging participants in an interaction with real objects and with a real other person (the experimenter). Real objects were presented, which could be linked, as in the previous studies, by no relation, by Spatial relations (e.g., cup-knife) or by Functional relations. The Functional relations could be of two different kinds, i.e., functional-individual or functional-cooperative. With functional-individual relations the two objects are typically used together to perform an individual action (e.g., I typically put the teabag in my own cup), while with Functional-cooperative relations (e.g., cup-teapot) the two objects are used to perform an action that can typically involve somebody else: for example, I typically pour the tea from the tea-pot in the cup of somebody else. Further manipulation of the social intention of the experimenter were introduced: to move the objects the experimenter used either a functional grip or a manipulative grip (e.g., grasping the cup to drink from it or to put it away), and he could observe or not the other (direct vs. indirect gaze). The participants were submitted to two different conditions: in the give condition they had to move the target object toward the experimenter, while in the get condition they moved it toward themselves. The analysis of the kinematic parameters revealed that, during the give condition, the wrist acceleration peak was reached earlier when the other used a functional posture, and the maximal fingers aperture was reached faster when the objects were linked by functional individual than by functional cooperative relations. In the get condition, during visual contact the maximal fingers aperture increased when the experimenter has executed a manipulative grip, as if the participant felt entitled to take the object. This reveals that participants are highly sensitive to cues that might lead to a social or cooperative action. These cues can be found in the relations between objects as well in the characteristics of others that can be indicative of a social intention, such as the gaze and even the hand posture—participants seem indeed to interpret the direct gaze and manipulative grip as leading to a social action.

# Avoiding to Respond to Affordances

Affordances allow us to respond adequately to objects: objects invite us to perform actions with them, in order to reach our goals. However, there might be cases in which, instead of responding to affordances, we may need to avoid responding to the ''invitations'' we have received.

We will outline a series of cases, which may differ in intensity and specificity, in which such a situation might occur.

# Multiple Affordances

We have addressed the issue of multiple affordances in the previous session. As we have seen, the studies focusing on multiple affordances are not many, since most of recent experimental work has focused on the interaction with single objects. There are some studies on affordance effects elicited by different parts of the same object (Riggio et al., 2006; Borghi and Riggio, 2009; Pellicano et al., 2010), as well as studies on objects evoking conflicting actions (e.g., Jax and Buxbaum, 2010; Kalénine et al., 2014). Furthermore, some studies present participants with pairs of objects, the affordances of which can be combined to obtain an aim, as for example scissors and papers (Humphreys et al., 2010; Borghi et al., 2012; Natraj et al., 2013; Scorolli et al., 2014), and studies on multiple objects of the same kind (Pezzulo et al., 2010).

Our overall impression is that research so far has not clarified what happens when multiple affordances are activated. It is currently still unclear, whether we activate all possible affordances, or not. There are some possible scenarios: (a) all affordances are automatically activated, and some of them decay, because they are not selected as they are not relevant to the current context/situation and to the current goals of the observer; (b) all affordances are automatically activated, and some of them are actively inhibited to avoid interference between them, since not relevant to the current context and goal; (c) only the single affordance or the subset of affordances relevant to the current context/situation and to our present goals are activated. In the last case the context and the goals would work as an early filter. Both (a) and (b) are in keeping with the influential neural model of action selection described by Cisek (2007). In Cisek's model, the different objects activate multiple afforded actions automatically, with a later stage of competition in which only one of these actions is selected to be executed.

# Broken Affordances

A different case is when objects present affordances but they cannot be used, for example because they are broken. It is possible that, in this case, the mechanism is different, i.e., that the affordance is activated but then actively inhibited. In a recent TMS study Buccino et al. (2009) stimulated the left hemisphere hand motor area of participants who observed everyday objects, centrally presented, with a complete or broken handle, positioned to the right or to the left. Results revealed that the Motor Evoked Potential area was larger when the handle was on the right side of the object, but only when the handle was complete. The absence of a difference between right and left when the handle was broken is compatible with the absence of affordance activation when the pragmatic conditions to perform an action are not met. These data suggest that the handle affordances are not activated or that they are activated and then inhibited when the handle is broken, leading to a reduction of activation in the cortical areas typically involved in performing action when the handle is intact. The possibility to inhibit affordances was, for example, shown by Riggio et al. (2006) using an inhibition of return (IOR) paradigm. They presented first whole objects, in which the distinction between the graspable and the ungraspable parts was clearly defined (for instance, in a knife, although we can distinguish between blade and handle, only the handle is used for grasping the object), and then graspable or ungraspable parts of the objects. Participants had to ignore whole objects and to respond to objects parts. Results showed greater inhibitory effects for graspable than for ungraspable parts, specific for the most appropriate action necessary to grasp a specific object. Therefore results suggest distinct inhibitory effects related to the pragmatic features of objects, possibly activated by the neural substrates responsible for sensorimotor transformations required to act properly on an object. If this is the case, the mechanism active with broken affordances would be rather similar to what happens when processing negative action sentences (Tettamanti et al., 2008): the areas typically involved in action representation are recruited and then actively inhibited. A less probable alternative interpretation of the results is that affordances related to the broken handle are inhibited from the very start, hence not activated at all. This would be a case in which the context works as an early filter. Further studies on the time course of the process are necessary to better understand the mechanisms underlying broken affordances activation.

# Dangerous Objects

A special case is represented by dangerous objects. As in the case of broken affordances, with dangerous objects it is possible that we activate affordances, and then actively inhibit them, or alternatively that we directly avoid to activate them. We will illustrate and discuss below some recent studies we performed in which we contrasted neutral and dangerous objects.

In a first series of studies we presented images of graspable or of dangerous objects, preceded by a hand (a male hand, a female hand and a robotic grasping-hand; a male and a female static-hand) or by a control object, and asked participants to categorize target-objects into artifacts and natural objects by pressing two different keys on the keyboard. Across different experiments, performed with children and adults, we found that response times with dangerous objects were slower compared to those with neutral objects. Let us call this phenomenon a form of inhibition; we will discuss it later in more details. Interestingly, this inhibition was modulated by the hand prime. In a first study with children, we found that the inhibition effect was more marked when the perceived vulnerability of the hand was higher: female hands induced the strongest inhibition, followed by male hands, while robotic hands elicited the lowest one (Anelli et al., 2012); moreover, analyses indicated an effect of motor resonance: the more children and adults perceived the hand as similar to their own hand, the higher was the inhibition. The results of these studies, however, do not allow us to fully disentangle the effects of affordances from the effects of the prime. In addition, it remains unclear whether the slower reactions times (RTs) associated to dangerous objects are due to a late occurring blocking mechanism or the presence of aversive affordances, i.e., to the fact that dangerous objects are perceived as such from the start.

Further studies with different paradigms were performed to better understand the mechanisms underlying affordance activation, deactivation or not activation, in case of dangerous objects. We used a line bisection task (Anelli et al., 2013b). This paradigm is interesting because it allowed us to observe sensitivity to dangerous stimuli with a task not requiring stimulus response compatibility and where the object stimuli did not need to be processed to perform the task (Ohman et al., 2001); furthermore, compared to the above illustrated studies, the object was presented without a hand in potential interaction with it, thus it was easier to capture the effect of the object on its own. In a first study a line was flanked by neutral graspable and by dangerous objects of similar size (e.g., bulb vs. broken bulb; spoon vs. knife; cat vs. porcupine); we found that adolescents and adults tended to misperceive the line midpoint away from the dangerous objects. To understand whether the result was due to an affordance effect (the tendency to approach the graspable object) or to an avoidance effect (the tendency to refrain from the dangerous one) we asked adults to bisect lines flanked by dangerous and neutral objects matched on graspability (both graspable or ungraspable). The results indicate that graspable dangerous objects evoke aversive affordances characterized by the motor tendency to step back and escape. Time course analyses would be necessary to capture precisely how the process unfolds in time.

In a further study (Anelli et al., 2013a) we presented participants, children and adults, with artifact and natural objects, both neutral and dangerous, and asked them to categorize the objects, either by pressing or by releasing two different keys on the keyboard. The critical manipulation consisted in presenting the objects as moving away or toward the participant. Results were rather straightforward: neutral objects responded to faster when they performed an approaching movement, while dangerous objects when they moved away from the participant. No effect of the response typology was present.

To better understand the time course of the process we presented static images of the objects, which were displayed in different sizes (large-medium-small size), as if closer or further away from participants. In Experiment 2, 1 s passed between the presentation of a first image and the displacement of the second, that could be larger, smaller or of the same size, and that represented the go signal; in Experiment 3 the second image was immediately following the first one. The results can be interpreted relying on the different timing of the two experiments: when participants had time to prepare their response (i.e., 1 s passed before the presentation of the go signal) they responded immediately, faster, to larger objects, the most dangerous ones. When they did not have time to prepare themselves, instead, response times were longer, in particular with larger objects. We interpreted this result as due to a sort of freezing effect (see Eder et al., 2014), which was larger the bigger, hence more dangerous were the stimuli.

Overall, all these studies reveal that we are sensitive to dangerous affordances. We respond faster to them when objects or entities with dangerous affordances move toward us. Similarly, we tend to avoid graspable objects with dangerous affordances, as evidence on line bisection reveals. This evidence is in keeping with studies on approach avoidance effects, which show that we tend to attract positively connoted words and to withdraw from negative ones (Chen and Bargh, 1999; van Dantzig et al., 2008; Freina et al., 2009).

As to response times, across the experiments and populations (children, adolescents, adults) we found that responses to dangerous objects are slower than responses to affordances of neutral objects. Further results in the literature are consistent with our findings. Studies on the emotional Stroop effect have shown a general RTs slowdown with aversive stimuli (e.g., Algom et al., 2004). Algom et al. (2004) have proposed that the threatening character of stimuli determines a generic slowdown of responses.

The longer RTs we found with dangerous stimuli can be explained in terms of the mechanisms highlighted by Caligiore et al. (2013) in their TRoPICAL model (see also Caligiore et al., 2010). The model explains negative compatibility effects occurring when participants are required to respond to target-objects while refraining from responding to distractors. According to the model the dorsal and ventral pathways process information related to both the target-object and the distractor. Caligiore et al. (2013) have shown that the prefrontal cortex (PFC) plays a double role, exerting both an inhibitory and an excitatory control (Munakata et al., 2011). In Caligiore et al. (2013), this inhibitory control allows the model to refrain from executing the actions suggested by the distractors; similarly, since PFC can receive inputs from the emotional circuits, it may allow participants to inhibit the tendency to respond to affordances of dangerous objects.

In terms of time course, the slower responses with dangerous objects could be due to two different processes: (a) A two-stages process: we would perceive objects affordances, and plan our actions as a consequence of this; then we would realize that the objects are dangerous and block the planned responses; (b) A more automatic process: we would immediately perceive aversive affordances as such, and we would inhibit any motor response, adopting a freezing behavior. This outcome would occur in particular when dangerous objects are very close to us and we have no time to prepare an exit strategy.

Our data speak in favor of the second hypothesis. At the same time, they reveal that our responses to objects are highly flexible and dependent on the spatial context (near vs. far space), and on the presentation modality of the stimuli, dynamic vs. static (with dynamic objects no effect of the motor response—key press vs. key release—was found, while with static objects clear differences between the two motor responses were observed).

In sum: we have outlined three cases in which we might activate affordances, and then need to suppress them: the cases of multiple affordances, of broken affordances and of dangerous objects. As to multiple affordances, further evidence is needed to understand whether all affordances are automatically activated or whether only affordances relevant to the current context and situation are selected. What is certain is that an increasing number of studies are showing the importance of context for affordances activation. In the case of broken affordances, some of our results suggest that it is possible that the observer actively inhibits the activated affordances or that the affordances are not activated. As to dangerous objects, our results suggest that we do not activate their affordances and then block them, but that we respond directly to aversive affordances.

# Conclusion

Affordances represent an important aspect of our physical and social environment. Our interaction with the surrounding environment is namely potentiated and constrained by them. It is therefore particularly important to understand the mechanisms underlying their activation.

In the first part of the paper we have proposed that two different kinds of affordances exist: stable and variable affordances. As to their brain representation, these two kinds of affordances activate overlapping areas within the dorsal stream but have also different neural underpinnings, since the first activate mainly the dorso-ventral stream, while the second engage primarily the dorso-dorsal one (Sakreida et al., in preparation). We have seen that our proposal is related to Jeannerod's distinction between intrinsic and intrinsic properties, and that it is strictly linked, but not correspondent, with the view that there might be affordances dedicated to object manipulation and others to object use. Both variable and stable affordances are flexible, even if to a different extent. When it comes to language, we propose that language incorporates only certain kinds of affordances, i.e., stable and canonical rather than variable ones (see Borghi, 2012, for an extensive discussion of this issue). Current evidence obtained in our and other labs supports this view (Borghi and Riggio, 2009; Ferri et al., 2011b; Myachykov et al., 2013; Flumini et al., 2014).

# References


In the second part of the paper we have shown that recent evidence on contextual dependence of affordances may challenge the idea that they are automatically activated. We have briefly reviewed studies showing that the activation of affordances is modulated by the task (superficial vs. deep processing, as in color vs. shape categorization) and by the physical and social context, i.e., by the distance of objects from the body, by the relations between objects, by the scenes in which they are embedded, by the presence of others and by the intentions of others we infer from their behavior.

As we have seen, the data on conceptual dependence are not incompatible with the view that affordances are automatically activated, provided that the selection of the relevant affordances occurs late.

The two parts of this paper, the first concerning kinds of affordances and the second concerning their automaticity, might seem separate and independent, because focused on different aspects. However, we think they are deeply interconnected. The distinction between stable and variable affordances can indeed provide new ways to think of affordances automaticity, and can help advancing new predictions. It is indeed possible that all affordances are automatically activated, and that a competition among them is differently solved depending on the task and the stimuli. We can hypothesize that, when the task and the stimuli are linguistic, functional information ''wins'' over manipulation, unless the linguistic context clearly primes manipulation (see Lee et al., 2013, for a study highlighting the role of the linguistic context). Similarly, stable affordandes would ''win'' over variable ones. When the stimuli are not linguistic but consist of real objects and the task involves interaction with them the advantage of stable over variable affordances would disappear. As far as affordances related to manipulation and function are concerned, instead, the competition will be solved differently depending on the context.

In all cases, further evidence on the time course of these processes is needed. In addition, computational models of these processes would be really helpful in providing a synthetic framework and in refining predictions (for current models, see Bonaiuto and Arbib, 2010; Caligiore et al., 2010).

# Acknowledgments

The two authors contributed equally to the ideas presented in this paper. We would like to thank Leonardo Fogassi for the insightful comments and observations on a first draft of this manuscript and to Ferdinand Binkofski for the suggestions on apraxia and affordances.

reaching space. Conscious. Cogn. 21, 1551–1557. doi: 10.1016/j.concog.2012. 06.010


situations. Front. Hum. Neurosci. 7:344. doi: 10.3389/fnhum.2013. 00344


Jeannerod, M. (1997). The Cognitive Neuroscience of Action. Oxford: Blackwell.


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Borghi and Riggio. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

#### *J. C. Mizelle1,2 and Lewis A. Wheaton1 \**

*<sup>1</sup> Atlanta Veterans Affairs Medical Center, Decatur, GA, USA <sup>2</sup> School of Applied Physiology, Georgia Tech, Atlanta GA, USA*

*\*Correspondence: law@gatech.edu*

#### *Edited by:*

*Antonello Pellicano, RWTH Aachen University, Germany*

#### *Reviewed by:*

*Chiara Bozzacchi, Istituto Italiano di Tecnologia, Italy*

**Keywords: apraxia, affordances, motor control, tool use, tool use behavior**

#### **A commentary on**

#### **Apraxia of tool use is not a matter of affordances**

*by Osiurak F. (2013). Front. Hum. Neurosci. 7:890. doi: 10.3389/fnhum.2013.00890*

A recent opinion article published in Frontiers in Human Neuroscience (Osiurak, 2013) points out several challenges of the study of "affordances" related to investigations of apraxia. In 2010, we published in Frontiers a review and theoretical proposal that addresses our concerns about affordances and grounded cognition (Mizelle and Wheaton, 2010b). Central to our premise was the argument that, for tool use, action goals were the grounded invariant elements as opposed to the action representations of the tools themselves. Further, parameters of the behavior(s) undertaken to achieve the action goal (tool used to accomplish the task, usage context of the tool, and the motor variables to accomplish the task) are affording to the goal, inherently variable, but are driven by the fixed action goal itself. Our model (Modular Selection for Action Goals, MSAG) incorporates the idea that stored representational knowledge of tools can be broadly adapted by usage context so that action goals can be achieved, emphasizing the adaptability of tool contextual and usage representations and the fixedness of the overall action goal. In commentary to our MSAG model, Pellicano et al. (2011) considered an alternative view where affordances (stable and variable) function to align tools to action goals. The core difference is that Pellicano and colleagues proposed that potentiation of tool-action goal alignment is mediated by affordances (certain properties of tools). We proposed that the fulfillment of the action goal defines affording properties of any possible combination of tools and motor variables, where some combinations are more or less affording to the action goal than others. At any rate, we certainly agree that further studies need to be considered to appreciate and refine specifics of any models, whether ours or those of Pellicano and colleagues.

While Osiurak emphasized the alternative commentary (Pellicano et al., 2011), we feel it is worth noting that many of the ideas presented by Osiurak (2013) reflect core concepts of our 2010 MSAG theory. For example, in Figure 2 of the MSAG proposal paper (Mizelle and Wheaton, 2010b), selection of alternative tools when the canonical tool is not available (we use the example of tools within a reasonable workspace) is not necessarily driven by a broad range of stable affordances (the adaptive grounded view). Rather, an alternative tool is selected based on the properties which best allow for the accomplishment of the action goal based on known mechanical/functional properties of tools. This embodies the first two assumptions of Osiurak (2013). Under MSAG, interconnected modules are triggered by an action goal that afford semantic flexibility of tools; tool (selection), usage context (refinement, as tools have multiple uses), and neurobiomechanics (motor specifics).

Further, our contention has been that the elements that best fulfill the action goal become the relevant affordances for tool selection and motor performance, not necessarily the "grounded" or stable affording properties of tools. This embodies the third assumption presented by Osiurak. The action goal defines the cooperatively determined usage context of the tool. Chiefly, this allows for creativity and adaptability in how action goals are accomplished, especially when canonical tools—those with grounded action representations coincident with the action goal—are not available.

Work in our lab has sought to understand how people "connect" tools and objects for action goals. Using electroencephalography (EEG), we have suggested that erroneous tool-object pairings generates ventral activation, which seems to precede parietofrontal activation typical of tool-object encoding for action (Mizelle and Wheaton, 2010a,c). Further, in a multimodal neuroimaging study, we used EEG and functional MRI (fMRI) to propose that contextual understanding of incorrect/impossible actions (via ventral pathways) precedes the activity of correct/possible actions (via parietofrontal pathways) that may suggest how both conceptual and ideomotor type apraxias could occur (Mizelle and Wheaton, 2010d). Indeed, this was reflected in MSAG as we proposed that ventral damage could corrupt the ability to align a tool with the action goal and the canonical usage context of that tool. In this case, a failure to *deselect* inappropriate tools would result, but the inappropriate tool would be used in motor-relevant ways in an attempt to achieve the desired behavior. It is our proposal that such ventral pathway damage could help distinguish clinico-anatomical correlates of motor versus conceptual apraxias.

Core to the goals of this Research Topic ("Bridging the theories of offordances and limb apraxia"), what does MSAG have to do with apraxia? We have had interest in focusing MSAG on the conceptual level, to better understand neural circuits that could be vulnerable in persons with conceptual apraxias. We have recently refined the MSAG proposal, suggesting that the dorsal parietofrontal areas encode possible "functional affordance," where qualities of seen (or desired) actions of tools are encoded based on relevance to behaviors for achievement of an action plan (Mizelle et al., 2013). In this work, we chose tool-object pairs that were always correct/possible to fulfill an action goal, but modified the functional affordance by changing how the tool interacted with the object. When functional affordance is high (correct tool-object pairs are used correctly) parietofrontal activation is dominant. Yet, when functional affordance is low (correct tool-object pairs used incorrectly), ventral brain areas show significant activations. Thus, functional affordance may be similarly driven, at least in part, by the mechanical/physical alignment (Mizelle et al., 2013) and contextualization (Mizelle and Wheaton, 2010a,d) of tool-object pairs. This helps to underscore a common problem in conceptual apraxia, where tool selection for a task is impacted. If ventral networks are largely affected, conceptual errors can become predominant, yet the MSAG model does not stop there.

As MSAG predicts, successful fulfillment of the action goal is paramount and a certain amount of inherent flexibility exists to accomplish the goal. As we proposed in MSAG, accomplishing the goal without ending in a fault is of primary concern. MSAG proposes a preliminary framework of how both conceptual and motor "faults" may occur, which would reflect conceptual and ideomotor apraxias. While many of our studies have focused on the conceptual errors, we are continuing work on expansion into the motoric domain, and the interactions between conceptual and motor properties of action. We anticipate that such refinement will be a pivotal step in being able to better detail the neural systems involved in apraxia.

We are continuing to study how we encode and understand action goals, which is core to shaping MSAG and highly relevant to the opinion article by Ousirak. In the context of goal-based tool use, our own work suggests that affordances are complex, possibly dynamic entities. We propose that research should focus on the varied properties of affordance and how these varied properties might interact. A good place to start would be seeking to align the various proposals of affordance, and their relevance to apraxia, through collaborative research efforts.

#### **REFERENCES**

Mizelle, J. C., Kelly, R. L., and Wheaton, L. A. (2013). Ventral encoding of functional affordances: a neural pathway for identifying errors in action. *Brain Cogn.* 82, 274–282. doi: 10.1016/j.bandc.2013.05.002


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 09 April 2014; accepted: 22 July 2014; published online: 08 August 2014.*

*Citation: Mizelle JC and Wheaton LA (2014) How can we improve our understanding of skillful motor control and apraxia? Insights from theories of "affordances". Front. Hum. Neurosci. 8:612. doi: 10.3389/ fnhum.2014.00612*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2014 Mizelle and Wheaton. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# *François Osiurak\**

*Laboratoire d'Etude des Mécanismes Cognitifs (EA 3082), Université de Lyon, Lyon, France \*Correspondence: francois.osiurak@univ-lyon2.fr*

#### *Edited by:*

*Antonello Pellicano, RWTH Aachen University, Germany*

*Reviewed by: Anna M. Borghi, University of Bologna, Italy*

**Keywords: affordance, apraxia, manipulation knowledge, mechanical knowledge, tool use**

Until recently, most of our understanding about human tool use has come from left brain-damaged patients, particularly those who show difficulties in actually using familiar tools (hereafter referred to as apraxia of tool use). These difficulties have been suggested to result from impaired sensorimotor knowledge about manipulation (Rothi et al., 1991; Buxbaum, 2001; Buxbaum and Kalénine, 2010; Binkofski and Buxbaum, 2013). The manipulation knowledge hypothesis is very close to the stable affordance hypothesis, that is, the idea that the mere observation of a tool is sufficient to automatically extract stable affordances, namely, invariant features of the tool (i.e., its functional meaning), leading to the activation of the canonical motor action (e.g., Bub et al., 2008; Borghi and Riggio, 2009; Pellicano et al., 2011). In this article, I discuss the viability of the hypothesis that impaired manipulation knowledge/stable affordances might be the core deficit of apraxia of tool use.

Manipulation knowledge is supposed to contain information about the movements associated with the canonical manipulation of a familiar tool (e.g., for a hammer, a broad oscillation of the elbow joint; Buxbaum, 2001). This information is viewed as egocentric, because it specifies the user-tool relationship. On this basis, a parallel has been drawn between the notions of manipulation knowledge and motor affordances (Bub et al., 2008; Borghi and Riggio, 2009; Pellicano et al., 2011). To interact with a tool, some information such as the orientation of the tool has to be processed online, because it does not represent a permanent characteristic. Thus, orientation can be considered as an instance of temporary affordance, processed by the dorso-dorsal stream. Nevertheless, we also have to determine what is the typical orientation of a tool to use it (e.g., the canonical orientation of a book to read it). This typical orientation would be rather based on stable/permanent/canonical affordances, such as shape and size. These stable affordances would involve information stored in memory and might be processed by the ventral, or more particularly, the ventro-dorsal stream (see Borghi and Riggio, 2009; Ferri et al., 2011; Borghi, 2012; Myachykov et al., 2013). In sum, whereas the manipulation knowledge hypothesis focuses on the motor parameters associated with tool manipulation, the stable affordance hypothesis emphasizes the tools' properties useful for a specific manipulation. Whatever, both hypotheses assume that canonical/permanent/stable stored information can be associated with the manipulation of a specific tool, and is a potential basis for the conception of tool actions<sup>1</sup> (**Figure 1**).

Growing evidence indicates that left brain-damaged patients with apraxia of tool use are impaired to solve mechanical problems, consisting in selecting among several novel tools the one appropriate to lift a cylinder or to extract a target out from a box (Goldenberg and Hagmann, 1998; Goldenberg and Hagmann, 1998; Hartmann et al., 2005; Goldenberg and Spatt, 2009; Jarry et al., 2013; Osiurak et al., 2013; see also Osiurak et al., 2009). These difficulties are associated neither with a dysexecutive syndrome, nor with frontal lobe damage (Goldenberg and Hagmann, 1998; Hartmann et al., 2005; Goldenberg et al., 2007; Jarry et al., 2013; Osiurak et al., 2013). In other words, the ability to use both familiar and novel tools might be supported by a common cognitive process. An important question is whether the manipulation knowledge/stable affordances hypothesis is a good candidate for this common process. At least two theoretical arguments can be offered to conclude that the answer is no.

First, manipulation knowledge and stable affordances are supposed to be associated with a specific tool, more particularly with its canonical manipulation. Given that mechanical problems consist of novel tools, there is no reason that manipulation knowledge supports the solving of these problems. Pellicano et al. (2011) proposed a somewhat more subtle perspective, by assuming that, in some cases, the canonical, familiar tool associated with a usual action can be absent (e.g., to stir coffee in the absence of a spoon). In this case, the usage context might help the user to select among the available tools (e.g., a knife) the one with the most similar structure to the canonical tool (e.g., a spoon). Again, this proposal cannot be applied to the use of novel tools to solve mechanical problems, given that there is no usual context, and no canonical tool associated with the solution of the problem.

Second, manipulation knowledge and stable affordances are thought to be egocentric, in that they specify the relationship between the user and the tool.

<sup>1</sup> In recent years, a distinction has been made between volumetric/structural vs. functional gestures (e.g., Bub et al., 2008; Buxbaum and Kalénine, 2010; Jax and Buxbaum, 2010). Volumetric/structural gestures correspond to the hand postures used to grasp an object/tool to move it whereas functional gestures correspond to the manipulation of a tool in accordance with its conventional use. This distinction does not fully mirror the stable versus temporary affordances distinction because both volumetric/structural and functional gestures are based on the perception of stable affordances, such as shape or size. Therefore, I will only focus on stable affordances that can be perceived to use tools in a conventional way (i.e., the canonical manipulation of a tool).

For instance, Pellicano et al. (2011, p. 1) defined stable affordances as "the potentiation of motor interactions consistent with the conventional use of a perceived tool." The problem is that, to solve mechanical problems, patients have to form an allocentric representation of the tool solution (e.g., a hooking action involves the relationship between a hook and something that can be hooked). So at a theoretical level, the manipulation knowledge/stable affordances hypothesis cannot explain how this allocentric representation can be formed.

An alternative to the manipulation knowledge/stable affordances hypothesis can be proposed [Osiurak et al., 2010, 2011; Osiurak, 2013; for a somewhat similar view, see Goldenberg (2013)]. This alternative is based on three assumptions. First, when people intend to use tools, the conception of the tool action is not supported by knowledge about the egocentric user-tool relationship. Rather, the conception is based on mechanical knowledge, that is, knowledge about abstract mechanical principles, such as hooking, lever, and percussion. This knowledge is thought to be allocentric, because it specifies the relationship between the different elements of the environment. After all, once people understand the lever principle, they do not need to get a hypothetical, canonical tool to carry out a lever action. Instead, they seek among the different "available" tools, which are immediately within the workspace or not, the one appropriate to the present situation. Said differently, this proposal is the inverse of what Pellicano et al. (2011) suggested: It is not the representation of the stable affordances linked to a canonical tool that guides the search of the appropriate tool; rather, it is the representation of the physical properties useful for achieving the present goal that guides the search of the appropriate tool, whether the canonical tool is within the workspace or not. Interestingly, evidence indicates that the ventro-dorsal stream supports mechanical knowledge (Goldenberg and Spatt, 2009; Goldenberg, 2013). And, impaired mechanical knowledge might be the core deficit of apraxia of tool use.

This leads me to the second assumption: The ability to get appropriate tools that are not within the workspace is supported by what is commonly called semantic knowledge about tool function (Rothi et al., 1991; Buxbaum, 2001; Buxbaum and Kalénine, 2010). This knowledge specifies the purpose, recipient, and context wherein a tool can be used, and is commonly associated to the ventral stream. Evidence indicates that patients with a selective semantic deficit are able to actually use tools, when presented with the corresponding objects (e.g., a hammer with a nail; Buxbaum et al., 1997; Lauro-Grotto et al., 1997; Osiurak et al., 2008; Silveri and Ciccarelli, 2009). However, when the tool is presented in isolation, difficulties can occur, and are strongly linked to the semantic deficit (Sirigu et al., 1991; Hodges et al., 2000; Osiurak et al., 2008; see also Lesourd et al., 2013). In a way, those patients are able to determine through mechanical reasoning how the tools and objects can be used together. However, when tools are presented in isolation, they cannot determine the usual use, because knowledge about the social usages is impaired. Thus, those patients can attempt, on the basis of spared mechanical knowledge, to show that a key can be used for scrapping the chamfered edge of a wooden desk or a nail clipper can be used to attach several sheets of paper together (Sirigu et al., 1991; Osiurak et al., 2008). In other words, semantic knowledge about tool function can be viewed as another form of allocentric knowledge, linking the different tools and objects with the other tools and objects used for the same context or usage (Osiurak et al., 2010, 2011). Thus, when no tool is immediately available (i.e., within the workspace) to carry out an intended action, semantic knowledge can be requested to "mentally travel" over the different semantic categories in search of a potential appropriate tool. In sum, while mechanical knowledge specifies how tools and objects work together to carry out the utilization *per se*, semantic knowledge provides information about the different spaces wherein tools and objects can be found, thereby organizing the search in memory.

The third assumption is that the perception of affordances (and their actualization) only aims to translate the representation of the tool action elaborated through mechanical knowledge (e.g., that the hammer has to make a specific motion to pound a nail) into precise motor programs, linking in an egocentric way the user with the tool. This perspective is consistent with the ecological approach to perception, which assumes that affordances are animal-relative properties of the environment that are not created in the act of perception, but exist independent of it (Gibson, 1979). So people do not systematically or automatically perceive all the affordances that the environment offers to them, but rather only the affordances that are suitable for reaching a current goal [Shaw et al., 1982; Shaw, 2003; Osiurak and Badets, 2014; for a somewhat similar view, see also Tipper et al. (2006), Pellicano et al. (2010), and Ellis et al. (2013)]. In other words, the relevant affordances are *directly* perceived in accordance with the current goal. For instance, among the multitude of affordances that a hammer can offer, people can perceive it as move-able in a vertical plane, when attempting to pound a nail, but they can also perceive it as throw-able when attempting to defend themselves against attackers. Here, the move-ability and throw-ability of the hammer are affordances, but they are perceived only in function of the current goal. In this frame, affordances are necessarily stable, because they correspond to the "negative" of our biomechanical capacities. But, they are also temporary because they are perceived only in function of a specific goal. In sum, there are no stable neither temporary affordances, but only affordances. The corollary is that no affordances about the canonical manipulation of tools can be stored, because they are not engaged in the conception of the tool action *per se*, but are only a way for people to reify the conceptual representation of the action into the physical world. This perspective is much more consistent with the idea that the dorso-dorsal stream is precisely in charge of perceiving and actualizing affordances (see Young, 2006; **Figure 1**).

To conclude, apraxia of tool use, characterized by conceptual errors in the use of tools, might be not a matter of affordances. Rather, the perception/actualization of affordances would only be involved in the translation of the allocentric, tool action representation into specific, egocentric sensorimotor actions. In fact, in the field of apraxia, the only disorder that might be related to impaired perception/actualization of affordances might be motor apraxia, a disorder affecting the motor coordination mainly of distal movements. Motor apraxia is one of the clinical signs of cortico-basal degeneration (Zadikoff and Lang, 2005). Perhaps, an interesting avenue for future research would be to explore how those patients perceive affordances as usually assessed by the ecological approach to visual perception (e.g., Carello et al., 1989).

# **ACKNOWLEDGMENTS**

This work was supported by grants from ANR (Agence Nationale pour la Recherche; Project Deìmences et Utilisation d'Outils/Dementia and Tool Use, N◦ANR 2011 MALZ 006 03), and was performed within the framework of the LABEX CORTEX (ANR-11-LABX-0042) of Universiteì de Lyon, within the program "Investissements d'Avenir" (ANR-11- IDEX-0007) operated by the French National Research Agency (ANR).

# **REFERENCES**


1964–1972. doi: 10.1016/j.neuropsychologia.2013. 06.017


*Received: 28 October 2013; accepted: 06 December 2013; published online: 20 December 2013.*

*Citation: Osiurak F (2013) Apraxia of tool use is not a matter of affordances. Front. Hum. Neurosci. 7:890. doi: 10.3389/fnhum.2013.00890*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2013 Osiurak. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Viewing photos and reading nouns of natural graspable objects similarly modulate motor responses

# *Barbara F. M. Marino1,2, Miriam Sirianni 3,4, Riccardo Dalla Volta4, Fabio Magliocco4, Francesco Silipo4, Aldo Quattrone4 and Giovanni Buccino3,4\**

*<sup>1</sup> Dipartimento di Neuroscienze, Sezione di Fisiologia, Università di Parma, Parma, Italy*

*<sup>2</sup> Dipartimento di Psicologia, University Milano Bicocca, Milano, Italy*

*<sup>3</sup> IRCCS Neuromed, Pozzilli, Italy*

*<sup>4</sup> Dipartimento di Scienze Mediche e Chirurgiche, Università "Magna Graecia" di Catanzaro, Germaneto, Italy*

#### *Edited by:*

*Anna M. Borghi, University of Bologna and Institute of Cognitive Sciences and Technologies, Italy*

#### *Reviewed by:*

*Andriy Myachykov, Northumbria University, UK Joshua P. Salmon, Cognitive Health and Recovery Research Laboratory, Canada*

#### *\*Correspondence:*

*Giovanni Buccino, Dipartimento di Scienze Mediche e Chirurgiche, Università "Magna Graecia" di Catanzaro, Viale Salvatore Venuta, 88100, Germaneto, Italy e-mail: buccino@unicz.it*

It is well known that the observation of graspable objects recruits the same motor representations involved in their actual manipulation. Recent evidence suggests that the presentation of nouns referring to graspable objects may exert similar effects. So far, however, it is not clear to what extent the modulation of the motor system during object observation overlaps with that related to noun processing. To address this issue, 2 behavioral experiments were carried out using a go-no go paradigm. Healthy participants were presented with photos and nouns of graspable and non-graspable natural objects. Also scrambled images and pseudowords obtained from the original stimuli were used. At a go-signal onset (150 ms after stimulus presentation) participants had to press a key when the stimulus referred to a real object, using their right (Experiment 1) or left (Experiment 2) hand, and refrain from responding when a scrambled image or a pseudoword was presented. Slower responses were found for both photos and nouns of graspable objects as compared to non-graspable objects, independent of the responding hand. These findings suggest that processing seen graspable objects and written nouns referring to graspable objects similarly modulates the motor system.

**Keywords: embodiment, language processing, canonical neurons, affordances, motor responses**

# **INTRODUCTION**

It is known that hand-object interactions recruit a parieto-frontal circuit in the brain of both monkeys and humans subserving sensorimotor transformations (Rizzolatti et al., 1981, 1988, 2002; Kurata and Tanji, 1986; Taira et al., 1990; Hepp-Reymond et al., 1994; Jeannerod et al., 1995; Sakata et al., 1995; Binkofski et al., 1999; Grol et al., 2007; Hecht et al., 2013). Also the mere observation of objects that have the potential for being manipulated has been proven to be effective in modulating the activity of the motor system. Single-unit recording studies in monkeys have shown that a set of neurons known as "canonical neurons" discharges during the presentation of graspable objects (Rizzolatti et al., 1988; Murata et al., 1997; Raos et al., 2006; Umiltà et al., 2007). In keeping with this, brain imaging studies have shown the activation of fronto-parietal areas in the human brain during the observation of graspable objects (Chao and Martin, 2000; Grèzes et al., 2003a,b). The recruitment of the motor system during object observation is fine-tuned with the intrinsic features of objects that make them appropriate for manual action: for example motor evoked potentials (MEPs) recorded during the observation of graspable objects (e.g., a mug) with a broken handle were significantly different from MEPs recorded during the observation of a complete object (Buccino et al., 2009).

As far as language is concerned, the embodiment approach claims that language processing involves the activation of the same sensorimotor neural substrates recruited when one experiences the content of language material (Lakoff, 1987; Glenberg, 1997; Barsalou, 1999; Pulvermueller, 2001; Gallese, 2003; Gallese and Lakoff, 2005; Zwaan and Taylor, 2006; Fischer and Zwaan, 2008; Jirak et al., 2010). In recent years, there has been growing experimental evidence in favor of the embodiment. Much of this evidence comes from studies that used action verbs (individually presented or embedded in sentences) as stimuli (e.g., Pulvermueller et al., 2001, 2005; Hauk et al., 2004; Buccino et al., 2005; Tettamanti et al., 2005). Some works investigating the recruitment of the motor cortex during noun processing showed a modulation of the motor system activity according to manipulability of objects expressed by nouns (Glover et al., 2004; Tucker and Ellis, 2004; Lindemann et al., 2006; Myung et al., 2006; Bub et al., 2008; Cattaneo et al., 2010; Gough et al., 2012). Recently, slower hand motor responses have been shown during processing of nouns referring to hand-related objects (Marino et al., 2013; see also Sato et al., 2008; Dalla Volta et al., 2009 for similar results with verbs). Summing up, the studies reviewed so far clearly show that manipulation and observation of objects as well as processing of nouns referring to graspable objects modulate the activity of the motor system. It is not clear to what extent the modulation of the motor system during object observation overlaps with that related to noun processing. For example there is evidence that when some features of objects like the spatial location or the orientation are taken into account, then processing photos depicting graspable objects or nouns referring to those same objects differently modulate motor responses (Ferri et al., 2011; Myachykov et al., 2013).

Using a go-no go paradigm, we compared motor responses given while observing photos of graspable and non-graspable natural objects with those given while reading nouns of objects from the same categories. Given some evidence showing that tools and natural objects are differently represented in the brain and differently modulate the activity of the motor system (Boronat et al., 2005; Peeters et al., 2009; Rueschemeyer et al., 2010; Gough et al., 2012; Orban and Rizzolatti, 2012), we restricted our choice to natural objects. The experimental hypothesis was that if object and noun processing share the same neural substrates, as maintained by the embodiment approach, then objects and nouns should also exert a similar modulation of motor responses. In details, based on previous studies where a similar paradigm was used (e.g., Buccino et al., 2005; Sato et al., 2008; Marino et al., 2013), we expected slower motor responses for both types of stimuli with an early go-signal (150 ms). In Experiment 1 participants responded with the right hand while in experiment 2 participants responded with the left hand.

# **METHODS**

#### **PARTICIPANTS**

Forty (23 females; mean age = 22 years and 9 mo) and 43 (21 females; mean age = 23 years and 6 mo) undergraduate students from the University of Catanzaro took part in Experiment 1 and Experiment 2, respectively. They were right-handed according to the Edinburgh Inventory (Oldfield, 1971). None took part in both experiments. All participants were native Italian speakers, had normal or corrected-to-normal vision, and reported no history of language disorders. They were unaware of the purpose of the experiments and gave their informed consent before testing. The study was approved by the local Ethics Committee and conducted in accordance with the World Medical Organization (1996) and the procedure recommended by the Italian Association of Psychology (AIP).

## **STIMULI**

Forty Italian nouns (see **Table A1**) referring to natural objects and 40 pseudowords as well as 40 digital color photos (see **Table A2**) depicting natural objects and 40 scrambled images were used as stimuli. Twenty nouns referred to natural graspable objects (e.g., "foglia," "leaf") and 20 to natural non-graspable objects (e.g., "nuvola," "cloud"). **Figure 1** shows an example of each category. Nouns in the 2 categories were matched for word length [average values for nouns referring to graspable and non-graspable objects: 6.35 and 5.95; *F*(1*,* 38) = 0*.*68, *p* = 0*.*41], syllable number [average values: 2.5 and 2.6, *F*(1*,* 38) = 0*.*24, *p* = 0*.*63] and written lexical frequency [average values: 3.92 and 5.05 number of occurrences per million in Google search engine *F*(1*,* 38) = 0*.*31, *p* = 0*.*58; average values: 6.13 and 6.95 number of occurrences per million in CoLFIS (Corpus e Lessico di Frequenza dell'Italiano Scritto ∼3.798.000 words)—Laudanna et al., 1995—*F*(1*,* 38) = 0*.*08, *p* = 0*.*78; rGoogle*/*CoLFIS = 0*.*83, *p <* 0*.*0001]. Pseudowords were built by substituting one consonant and one vowel in two distinct syllables of each noun (e.g., "n*ip*ola" instead of "n*uv*ola").

With this procedure, pseudowords contained orthographically and phonologically legal syllables for the Italian language. In addition, nouns and pseudowords were matched for word length.

Photos depicted 20 graspable objects and 20 non-graspable objects. **Figure 1** shows an example of each category. The scrambled images were built by applying Adobe Illustrator distorting graphic filters (e.g., *twist* and *zigzag*) to the photos depicting both graspable and non-graspable objects so to make them unrecognizable and then meaningless. All photos and scrambled images were 440 × 440 pixels. The nouns of objects depicted in the photos and the 40 Italian nouns used as stimuli were matched for word length [average values for visual items and for verbal item: 6.45 and 6.15; *F*(1*,* 78) = 0*.*82, *p* = 0*.*37], syllable number [average values: 2.57 and 2.55; *F*(1*,* 78) = 0*.*04, *p* = 0*.*84] and written lexical frequency [Google average values: 4.98 and 4.49; *F*(1*,* 78) = 0*.*10, *p* = 0*.*75; CoLFIS average values: 7.74 and 6.54; *F*(1*,* 78) = 0*.*18, *p* = 0*.*67]. For further analysis on the stimuli, see also Supplementary Materials. The same set of stimuli served both Experiment 1 and 2.

### **EXPERIMENTAL DESIGN AND PROCEDURE**

The experiment was carried out in a sound-attenuated room, dimly illuminated by a halogen lamp directed toward the ceiling. Participants sat comfortably in front of a PC screen (LG 22 LCD, 1920 × 1080 pixel resolution and 60 Hz refresh rate). The eye-to-screen distance was 60 cm.

**Figure 2** shows the experimental procedure. Each trial started with a black (RGB coordinates = 0, 0, 0) fixation cross displayed at the center of a gray (RGB coordinates = 178, 178, 178) background. After a delay of 1000–1500 ms (in order to avoid response habituation), the fixation cross was replaced by a stimulus item, either a noun/pseudoword or a photo/scramble. Note that the delay could be at any time between 1000 and 1500 ms. Trial-by-trial a value between 1000 and 1500 was picked according to a uniform distribution. The verbal labels were written in black lowercase Courier New bold (font size = 24). Stimuli were centrally displayed and surrounded by a red (RGB coordinates = 255, 0, 0) 20 pixels-wide frame. The red frame changed to green (RGB coordinates = 0, 255, 0) 150 ms after the stimulus onset. The color change of the frame was the "go" signal for the response. Participants were instructed to give a motor response, as fast and accurate as possible, by pressing a key on a computer keyboard centered on participants' body midline with their right (Experiment 1) or left (Experiment 2) index finger. They had to respond when the stimulus referred to a real object, and refrain from responding when it was meaningless (go-no go paradigm). After the go signal, stimuli remained visible for 1350 ms or until participant's response. Stimulus presentation and response times (RTs) collection were controlled using the software package E-Prime 2 (Psychology Software Tools, Inc.).

The experiment consisted of 1 practice block and 1 experimental block. In the practice block, participants were presented with 16 stimuli (4 photos of graspable/non-graspable objects, 4 scrambled images, 4 nouns of graspable/non-graspable objects and 4 pseudowords) which were not used in the experimental block. During the practice block, participants received feedback ("ERROR") after giving a wrong response (i.e., responding to a meaningless or refraining from responding to a real item), as well as for responses given prior to go signal presentation ("ANTICIPATION"), or later than 1.5 s ("YOU HAVE NOT ANSWERED"). In the experimental block, the 160 items selected as stimuli were randomly presented with the constraint that no more than three items of the same kind (verbal, visual) or referring to objects of the same category (graspable, non-graspable, meaningless) could be presented on consecutive trials. No feedback was given to participants. Thus, the experiment, which lasted about 20 min, consisted of 80 go trials (40 nouns of objects, 50% graspable and 50% non-graspable, plus 40 photographs of objects, 50% graspable and 50% non-graspable) and 80 no-go trials (40 pseudowords plus 40 scrambled images), and 16 practice trials, for a total of 176 trials. To sum up, the experiment used a 2 × 2 repeated measures factorial design with Object Graspability (graspable, non-graspable) and Stimulus Type (nouns, photos) as within-subjects variables.

## **RESULTS**

**Figure 3** shows the result for both experiments.

Experiment 1. Trials with errors were excluded without replacement. Errors were not further analyzed given they were extremely rare (*<*5%). One participant was excluded from the analysis because his error rate exceeded 10%. RTs below 130 ms or above 1000 ms were omitted from the analysis (outliers). This cut-off was established so that no more than 0.5% of correct RTs were removed (Ulrich and Miller, 1994).

Median values of remaining RTs were calculated for each combination of Object Graspability (graspable and non-graspable) and Stimulus Type (photo and noun). These data entered a 2-way repeated measures analysis of variance (ANOVA) with Object Graspability and Stimulus Type as the within-subjects factors. *Post-hoc* comparisons were performed using the Newman-Keuls test with an alpha level of 0.05. Partial eta square values (η<sup>2</sup> p) are reported as an additional metric of effect size for all significant ANOVA contrasts.

The ANOVA revealed a main effect Object Graspability [*F*(1*,* 38) <sup>=</sup> <sup>73</sup>*.*90, *<sup>p</sup> <sup>&</sup>lt;* <sup>0</sup>*.*0001, <sup>η</sup><sup>2</sup> <sup>p</sup> = 0*.*66], reflecting slower responses for stimuli related to graspable objects (492 ms) as compared to those related to non-graspable objects (455 ms). Also the interaction between Object Graspability and Stimulus Type [*F*(1*,* 38) <sup>=</sup> <sup>25</sup>*.*01, *<sup>p</sup> <sup>&</sup>lt;* <sup>0</sup>*.*0001, <sup>η</sup><sup>2</sup> <sup>p</sup> = 0*.*40] was significant (**Figure 3A**). *Post-hoc* analysis showed that responses given to nouns referring to graspable objects were slower than responses to nouns referring to non-graspable objects (477 vs. 461 ms, *p <* 0*.*02). Similarly, responses given to photos referring to graspable objects were slower than those given to photos referring to non-graspable objects (507 vs. 448 ms, *p <* 0*.*0002). Moreover, responses to graspable objects were faster with nouns than with photos (477 vs. 507 ms, *p <* 0*.*0002) and, vice versa, for responses to non-graspable objects (nouns = 461 ms vs. photos = 448 ms, *p <* 0*.*04).

Experiment 2. Trials with errors and with outlier RTs were removed from the analysis as in Experiment 1. Four participants were excluded because their error rate exceeded 10%. Median values of correct RTs were computed and analyzed as in Experiment 1. The analysis revealed a main effect Object

Graspability [*F*(1*,* 38) <sup>=</sup> <sup>48</sup>*.*50, *<sup>p</sup> <sup>&</sup>lt;* <sup>0</sup>*.*0001, <sup>η</sup><sup>2</sup> <sup>p</sup> = 0*.*56], reflecting slower responses for stimuli related to graspable objects (510 ms) as compared to those related to non-graspable objects (470 ms). Also the interaction between Object Graspability and Stimulus Type [*F*(1*,* 38) <sup>=</sup> <sup>21</sup>*.*94, *<sup>p</sup> <sup>&</sup>lt;* <sup>0</sup>*.*0001, <sup>η</sup><sup>2</sup> <sup>p</sup> = 0*.*37] was significant (**Figure 3B**). *Post-hoc* analysis showed that responses given to nouns referring to graspable objects were slower than responses to nouns referring to non-graspable objects (500 vs.

484 ms, *p <* 0*.*03). Similarly, responses given to photos referring to graspable objects were slower than those given to photos referring to non-graspable objects (521 vs. 457 ms, *p <* 0*.*0002). Moreover, responses to graspable objects were faster with nouns than with photos (500 vs. 521 ms, *p <* 0*.*007) and, vice versa, for responses to non-graspable objects (nouns = 484 ms vs. photos = 457 ms, *p <* 0*.*001).

#### **DISCUSSION**

In the present study, participants gave slower motor responses when they were presented with natural graspable objects as compared to natural non-graspable objects. This was true for both nouns and photos. As for nouns, these findings are in keeping with previous data concerning verbs (Buccino et al., 2005; Boulenger et al., 2006; Sato et al., 2008; Dalla Volta et al., 2009; De Vega et al., 2013, 2014), hand-related relative to foot-related nouns (Marino et al., 2013) and adjectives (Gough et al., 2013). To solve the requested semantic task in the case of nouns referring to graspable objects, it is most likely that participants relied on the motor representations of potential hand interactions with the object expressed by the verbal label. In this way, the motor system was engaged in two tasks at the same time, that is processing language material and performing a motor response (pressing the button). Hence participants paid a cost as revealed by a slowing down of their responses. It is worth underlining that our findings are not at odds with EEG and MEG studies (for review see Pulvermueller et al., 2009) supporting an early recruitment of the motor system during language processing and possibly a specific role of this system in this function. Thus, they seem to bolster this argument by showing that when the motor system is crucially involved in both a linguistic and a motor task there is a competition for resources. Moreover, our results concerning nouns are not in contrast with studies showing faster motor responses during processing of language material compatible with the direction of movement (Glenberg and Kaschak, 2002; Kaschak and Borreggine, 2008) or the type of prehension (e.g., Tucker and Ellis, 2004) required to give responses (i.e., the so-called Action Compatibility Effect, ACE). Indeed, this facilitation has been interpreted as an outcome that emerges relatively late in the time course of language processing (Taylor and Zwaan, 2008). In fact, the modulation of the motor system during language processing may change over time, moving from an early interference (operating between 100–200 ms after stimulus onset) to a later facilitation (operating later than 200 ms from stimulus presentation). The former effect could be a consequence of the fact that the motor system is a common neural substrate for action performance and language processing, while the latter may reflect priming triggered by the content of language material (for a computational model, see Chersi et al., 2010).

As for photos, it is well-accepted that the visual presentation of a graspable object automatically recruits motor representations of potential actions that the object affords to the observer (Gibson, 1977). We suggest that the recruitment of the motor system during the presentation of photos was relevant and most likely crucial to perform our semantic task, at least for graspable objects. As in the case of nouns, since the motor system was involved both in solving the semantic task and in planning and implementing the motor response, participants were slower when processing graspable objects. Similar findings were reported in a recent paper (Salmon et al., 2014). The authors found slower responses for photos depicting graspable as compared to non-graspable objects during a categorization task. In the present study this interference effect was stronger for photos than for nouns. This difference may be due to the fact that through photos the intrinsic features of objects, relevant for action, are immediately evident and specific (i.e., pertinent to the particular seen object) while through nouns these features are not related to specific objects but rather to a prototype of the class the objects belong to, most likely presented in a decontextualized fashion. It is worth stressing that even within language material it has been shown that the degree of sensorimotor specificity expressed by sentences affects how deeply the motor system is recruited during language processing (Marino et al., 2012).

At odds with a previous paper concerning nouns (Marino et al., 2013) where an interference effect was found only for responses given with the right hand, the present study did not find any difference between responses given by the two hands. In the study of Marino and colleagues, the authors suggested that the differential pattern of interference may be explained by the fact that only the left hemisphere is involved in both the linguistic and motor tasks, with the right one involved in only the motor task. Unfortunately, this explanation cannot account for the present results. We therefore forward that the different results in the two studies may be due to the kind of stimuli used. In fact, while Marino and colleagues used only nouns referring to tools, here we used nouns referring to natural objects. It is well known that tools and natural objects are differently represented in the brain (Boronat et al., 2005; Peeters et al., 2009; Rueschemeyer et al., 2010; Gough et al., 2012; Orban and Rizzolatti, 2012) and in particular, a specific sector of the left inferior parietal lobule is devoted to tool use in humans. It may be argued therefore, that besides the linguistic role of the left hemisphere the different modulation of the two hemispheres in the paper of Marino et al. (2013) is due to the specific role of the left hemisphere in processing tools and in praxic functions (Heilman et al., 1982; De Renzi and Lucchelli, 1988; Buxbaum and Kalénine, 2010).

Taken as a whole, our data support that semantic processing of visually presented graspable objects and nouns referring to the same object category is sub-served by common neural substrates crucially involving the motor system (Ganis et al., 1996; Vandenberghe et al., 1996; Van Doren et al., 2010). A similar modulation of the motor system has been also assessed for visually presented actions and verbs (Aziz-Zadeh et al., 2006; Baumgaertner et al., 2007; De Vega et al., 2014). Recently, Borghi and Riggio (2009) proposed a distinction between stable and temporary affordances of objects, the former being related to features like shape and size, the latter being related to aspects like orientation and position. One plausible explanation for the present findings is that a similar modulation of the motor system during processing of both nouns and photos occurred because, given the task, only stable affordances of objects were coded. In keeping with this explanation, there is evidence that when temporary affordances, such as the position or the orientation, come into account then photos and nouns differently modulate the activity of the motor system (Ferri et al., 2011; Myachykov et al., 2013). An alternative but not mutually exclusive explanation may be related to the kind of stimuli used. As compared to previous studies that in most cases employed tools (or a combination of both tools and natural objects) in the present study we used only natural objects. For this kind of objects it is less clear cut which part of the object can elicit hand actions and it is hard to disentangle between manipulation and function knowledge of objects (Boronat et al., 2005). Indeed information about the position or the orientation of an object may be more relevant when using a hammer rather than when grasping an apple.

# **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at: http://www.frontiersin.org/journal/10.3389/fnhum. 2014.00968/abstract

# **REFERENCES**


*III Giornate Internazionali di Analisi Statistica dei Dati Testuali*, Vol. 1, eds S. Bolasco, L. Lebart, and A. Salem (Roma: Cisu), 103–109.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 19 September 2014; accepted: 13 November 2014; published online: 04 December 2014.*

*Citation: Marino BFM, Sirianni M, Volta RD, Magliocco F, Silipo F, Quattrone A and Buccino G (2014) Viewing photos and reading nouns of natural graspable objects similarly modulate motor responses. Front. Hum. Neurosci. 8:968. doi: 10.3389/fnhum. 2014.00968*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2014 Marino, Sirianni, Volta, Magliocco, Silipo, Quattrone and Buccino. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# **APPENDIX**

**Table A1 | List of the Italian nouns used in Experiment 1 and 2, their English translation, graspability of their referents, lexical frequency (number of occurrence per million in Google search engine—e.g., Marino et al., 2012—and in CoLFIS search engine—Laudanna et al., 1995), length and syllable number.**



**Table A2 | List of the Italian nouns (and their English translation) of the objects depicted in the photographs used in Experiment 1 and 2, their graspability, lexical frequency, length and syllable number.**

# Commentary: Viewing photos and reading nouns of natural graspable objects similarly modulate motor responses

#### Stergios Makris \*

Department of Psychology, Edge Hill University, Ormskirk, UK

Keywords: objects, nouns, affordance, motor cortex, visual perception

#### **A commentary on**

#### **Viewing photos and reading nouns of natural graspable objects similarly modulate motor responses**

by Marino, B. F. M., Sirianni, M., Volta, R. D., Magliocco, F., Silipo, F., Quattrone, A., et al. (2014). Front. Hum. Neurosci. 8:968. doi: 10.3389/fnhum.2014.00968

Marino et al. (2014) in their paper (Frontiers in Human Neuroscience) have tried to investigate how the semantic processing of graspable objects involves an activation of the motor cortex in line with the affordance hypothesis originally proposed by Gibson (1979). They devised a go/non-go behavioral task, during which they presented photos or nouns of natural graspable and nongraspable objects, while for some of the trials the stimuli viewed were scrambled images of the same objects or pseudowords. Participants viewed the stimuli for a period of 150 ms, after which they had to respond to a go or non-go signal (whether the stimuli were real or not) as part of a semantic task. They found that subjects' responses were slower when they were viewing the photos or reading the nouns of graspable objects, as compared to non-graspable ones. The authors explained that this delay in motor responses following the images or nouns of graspable objects is a proof of the motor cortex involvement in the semantic processing of objects that afford a motoric action. Even though these findings are in line with some previous reports about an early activation and involvement of the motor system in language and semantic processes (Pulvermueller et al., 2001, 2005), in this commentary we argue that the stimulus onset asynchrony (SOA) of 150 ms is too early for an affordance effect to occur and thus, we will try to provide a different account of their results and leave some room for further insight on the topic.

There is mounting research evidence suggesting that the simple viewing of objects with action significance can stimulate the motor cortex into generating appropriate motor plans, even in cases that there is no action intention (Tucker and Ellis, 1998; Ellis and Tucker, 2000; Makris et al., 2011, 2013). This is the theory of affordances as originally described by Gibson (1979). Within the affordance literature a key aspect for investigation has been the temporal evolution of the affordance effect. Ellis and Tucker (2000) in a series of behavioral investigations have suggested that the affordance effect is slow and gradually develops 500 ms after the stimulus onset. On the other hand, in previous research with TMS we have proved by means of measures of corticospinal excitability (motor evoked potentials) that the affordance effect is present at 300 ms and rapidly dissipates 500 ms after the stimulus onset (Makris et al., 2011, 2013). Most importantly, in the aforementioned studies we investigated the generation of affordances 150 ms after the subjects were presented with graspable objects, but

#### Edited by:

Arthur M. Jacobs, Freie Universität Berlin, Germany

#### Reviewed by:

Anna M. Borghi, University of Bologna and Institute of Cognitive Sciences and Technologies, Italy

#### \*Correspondence: Stergios Makris,

makriss@edgehill.ac.uk

Received: 05 February 2015 Accepted: 27 May 2015 Published: 09 June 2015

#### Citation:

Makris S (2015) Commentary: Viewing photos and reading nouns of natural graspable objects similarly modulate motor responses. Front. Hum. Neurosci. 9:337. doi: 10.3389/fnhum.2015.00337 we did not find any evidence of involvement of the motor cortex as a result of that. In that sense, the results of Marino et al. (2014) are in contrast with previously reported findings.

Furthermore, Cisek (2006, 2007) has provided a compelling explanation of this delay in the formation of the affordance effect, known as the "affordance competition hypothesis." According to this, in response to attended objects with action significance, multiple competing motor plans are generated across different regions of the motor cortex and through mutual inhibitory connections, a single motor winning act prevails. With this in mind, it is possible that graspable objects suddenly appearing on screen can automatically grab exogenous attention (Yantis and Jonides, 1984) and then for a rapid period after stimulus onset (∼100–150 ms) attention is subsequently withdrawn from the objects in display, leading to a rebalance of the affordance-driven motor plans (see also Makris et al., 2011). This is particularly interesting, as it could provide an alternative explanation for the observed difference in response latencies between graspable and non-graspable objects in the Marino et al. (2014) study. Indeed, it could be that 150 ms after the presentation of the stimuli, exogenous-like attention was withdrawn from the graspable only

# References


objects and not the non-graspable ones. This way, participants would have to re-direct their attention to the graspable objects in order to resolve the semantic task and thus, this process would have some cost in the timing of their responses. Hence, the reported results may not reflect the involvement of the motor system in the semantic processing of graspable stimuli per se, but instead an effect of purely attentional processes. Nevertheless, this is only an alternative proposition to the current findings by Marino et al. (2014) and even so we cannot entirely rule out a relationship between attentional and motor processes (i.e., premotor theory of attention, Rizzolatti et al., 1994).

Overall, it is apparent that the affordance effect remains a compelling topic within cognitive psychology and neuroscience, as it is the need to better understand the underlying visual, attention and motor processes. Theories of a direct or indirect route between visual perception, semantic processing and motor planning may appear contradicting, but in our opinion it could be that they are all providing a valuable insight in the better understanding of human cognition and perception. Hence, it is important for future research to validate or expand upon these insights.

verb processing. Brain Lang. 78, 143–168. doi: 10.1006/brln. 2000.2390


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Makris. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Response: Commentary: Viewing photos and reading nouns of natural graspable objects similarly modulate motor responses

Giovanni Buccino1, 2 \* and Barbara F. M. Marino<sup>3</sup>

<sup>1</sup> Dipartimento di Scienze Mediche e Chirurgiche, Università "Magna Graecia" di Catanzaro, Catanzaro, Italy, <sup>2</sup> IRCCS Neuromed, Pozzilli, Italy, <sup>3</sup> Dipartimento di Psicologia, University Bicocca, Milano, Italy

Keywords: nouns, affordance, objects, embodiment and grounded cognition, semantics

#### **A commentary on**

#### **"Commentary: Viewing photos and reading nouns of natural graspable objects similarly modulate motor responses"**

by Makris, S. (2015). Front. Hum. Neurosci. 9:337. doi: 10.3389/fnhum.2015.00337

In two behavioral experiments, Marino et al. (2014) investigated the modulation of the motor cortex during the semantic processing of graspable and non-graspable objects, presented either as photos or written nouns. They used scrambled images and pseudowords as control stimuli. At 150 ms after stimulus presentation, participants had to respond when the stimulus referred to a real object with their right (experiment 1) or left (experiment 2) index finger, and refrain from responding when a scrambled image or a pseudoword was presented. Participants' responses related to photos or nouns of graspable objects were slower than those related to non-graspable ones, independent of the responding hand. According to the authors, these findings support the notion that the semantic processing of photos and written nouns referring to graspable objects, is due to common neural substrates, crucially involving the motor system. Specifically, they forward that, to solve the requested semantic task, participants relied on the motor representations of potential hand interactions with the object depicted in the photo (affordance as described by Gibson, 1977) or expressed by the verbal label. In this way, the motor system was engaged in two tasks, that is processing stimuli and performing a motor response (pressing the button), at the same time. Participants therefore paid a cost as revealed by the slowing down of their motor responses.

Some previous papers (Tucker and Ellis, 2004; Makris et al., 2011) support the notion that the recruitment of the motor system (affordance effect) during the visual presentation of objects appears later than 150 ms after stimulus presentation. Based on that, in his commentary Makris (2015) proposes an alternative explanation of the experimental findings reported by Marino et al. (2014), namely as due to an attentional effect. The author proposes that graspable objects suddenly appearing on screen can automatically grab exogenous attention (Yantis and Jonides, 1984) and then for a rapid period after stimulus onset (∼100–150 ms) a withdrawal of attention from the objects in display occurs, leading to a rebalance of the affordance-driven motor plans. According to Makris, it could be that 150 ms after the presentation of the stimuli, exogenous-like attention was withdrawn only from the graspable, but not the non-graspable, objects. This way, participants would have to re-direct their attention to the graspable objects in order to resolve the semantic task and this process would have some cost in the timing of their responses. According to the author, this account would fit with a theoretical model known as the affordance competition hypothesis proposed by Cisek (2007).

#### Edited by:

Anna M. Borghi, University of Bologna & Institute of Cognitive Sciences and Technologies (ISTC), Italy

#### Reviewed by:

Andriy Myachykov, Northumbria University, UK Luisa Lugli, University of Bologna, Italy

> \*Correspondence: Giovanni Buccino, buccino@unicz.it

Received: 23 July 2015 Accepted: 10 September 2015 Published: 24 September 2015

#### Citation:

Buccino G and Marino BFM (2015) Response: Commentary: Viewing photos and reading nouns of natural graspable objects similarly modulate motor responses. Front. Hum. Neurosci. 9:524. doi: 10.3389/fnhum.2015.00524

In principle, one cannot rule out the attentional hypothesis, as an alternative explanation to the findings of Marino et al. (2014). However, even admitting a specific role of attention in explaining the data, in the commentary by Makris it is not clear why, at difference with non-graspable objects, the processing of graspable ones would require the withdrawal of attention at about 150 ms after stimulus presentation and the subsequent reallocation of attention to solve the semantic task. It is worth keeping in mind that also non-graspable objects were presented abruptly and therefore they could potentially grab exogenous attention as graspable objects did. Moreover, the affordance competition hypothesis does not seem to support this time course of attention allocation since in this account action selection and specification are parallel and not serial processes. That said, it is worth stressing as Makris himself admits, that it is difficult to disentangle between attentional and motor processes. Based on several studies, one may argue that there is no need to postulate two control mechanisms, one for action and one for attention. Rather, attention derives from the activity of the sensorimotor circuits devoted to interact with objects (e.g., Corbetta et al., 1998; Craighero et al., 1999).

The time course of the recruitment of the motor system during semantic processing of objects and nouns is still a matter of debate. However, there is increasing evidence of an early involvement of the motor system during semantic tasks involving language material. Neurophysiological studies (for review see Pulvermüeller et al., 2009) support a recruitment of the motor system during language processing of words related to action

# References


within the first 200 ms after stimulus onset. In the same time window, behavioral studies have shown that participants give slower hand motor responses when they have to process language material expressing hand actions or hand related objects (as for nouns, see Marino et al., 2013). In a very recent paper (Klepp et al., 2015), by means of magnetoencephalography, it has been assessed that this early slowing down of motor responses is due to a suppression of beta rhythm weaker than that found during the preparation and execution of actual movements. Taken together, these findings may lead to the conclusion that the modulation of the motor system during language processing may change over time, moving from an early interference, operating between 100 and 200 ms after stimulus onset, to a subsequently facilitation, operating later than 200 ms after stimulus presentation (Chersi et al., 2010). As for seen objects, the recruitment of the motor system (affordance effect), has been clearly shown at 200 ms after stimulus presentation (Buccino et al., 2009), that is quite earlier than 300 ms found by Makris et al. (2011). In addition, the findings of Marino et al. (2014) strongly suggest that there is a specific modulation of motor responses during the processing of photos depicting graspable objects already 150 ms after stimulus presentation. This modulation parallels the one occurring during the processing of language material expressing the same object category. Hence the proposal that the neural substrates devoted to processing photos depicting graspable objects and nouns referring to the same object category may be shared and crucially involve the motor system. Future studies should assess at what extent the semantic processing of seen objects and nouns overlap.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Buccino and Marino. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Components of action representations evoked when identifying manipulable objects

# *Daniel N. Bub , Michael E. J. Masson\* and Terry Lin*

*Department of Psychology, University of Victoria, Victoria, BC, Canada*

#### *Edited by:*

*Antonello Pellicano, Rheinisch-Westfälische Technische Hochschule Aachen University, Germany*

#### *Reviewed by:*

*Cristina Iani, University of Modena and Reggio Emilia, Italy Gregory Kroliczak, Adam Mickiewicz University in Poznan, Poland*

#### *\*Correspondence:*

*Michael E. J. Masson, Department of Psychology, University of Victoria, Room A236, Cornett Building, PO Box 1700 STN CSC, Victoria, BC V8W 2Y2, Canada e-mail: mmasson@uvic.ca*

We examined the influence of holding planned hand actions in working memory on the time taken to visually identify objects with handles. Features of the hand actions and position of the object's handle were congruent or incongruent on two dimensions: alignment (left vs. right) and orientation (horizontal vs. vertical). When an object was depicted in an upright view, subjects were slower to name it when its handle was congruent with the planned hand actions on one dimension but incongruent on the other, relative to when the object handle and actions were congruent on both or neither dimension. This pattern is consistent with many other experiments demonstrating that a cost occurs when there is partial feature overlap between a planned action and a perceived target. An opposite pattern of results was obtained when the depicted object appeared in a 90◦ rotated view (e.g., a beer mug on its side), suggesting that the functional goal associated with the object (e.g., drinking from an upright beer mug) was taken into account during object perception and that this knowledge superseded the influence of the action afforded by the depicted view of the object. These results have implications for the relationship between object perception and action representations, and for the mechanisms that support the identification of rotated objects.

**Keywords: action representations, canonical and rotated view, object affordances, object identification, partial feature overlap**

#### **INTRODUCTION**

The functional properties of an object are an essential part of its conceptual representation; we understand what is meant by the phrase "a good pair of scissors" because we know that scissors are typically used for cutting and how reassuring it is when a pair cuts well. More contentious, though, is the role that function plays in the identification of visual objects. Neuroimaging studies have shown that identifying pictures of tools activates motor cortical regions (see Mahon and Caramazza, 2008, for a review), a result that has driven two widely held assumptions. First, it is claimed that to recognize the visual form of an object like a pair of scissors requires knowledge of its proper function. Second, the function of a tool is assumed to be represented in terms of the actual movements we produce and register when we interact with the object. For example, Martin et al. (2000) argued that ". . . information about object function needed to support tool recognition and naming is information about the patterns of visual motion and patterns of motor movements associated with the actual use of the object" (p. 1028).

Both of these claims are contentious. Apraxic patients are impaired in pantomiming the actions associated with a tool, and to a lesser extent, in the movements required to make use of the object itself. Yet they show relatively preserved understanding of the function of tools; for example, patients, despite their apraxia, are able to correctly judge that a scissors and a knife are used for similar purposes (see Garcea and Mahon, 2012, for a review). Clearly, knowing the general function of a tool includes a degree of abstraction beyond the movements associated with its use. There is also evidence that identifying human artifacts can occur purely on the basis of their shape, without regard to their function. Young children acquire the names of many such objects even before they have had the opportunity to learn about their functional properties (Merriman et al., 1993; Landau et al., 1998). Neuropsychological evidence further challenges the view that the ability to name tools depends on functional knowledge. Ochipa et al. (1989) documented the performance of a patient with ideational apraxia who could name tools despite showing severe impairment in tasks that assessed his understanding of their function (e.g., he failed to select a hammer as the correct tool when shown a piece of wood containing a partially embedded nail).

What then are we to make of the undeniable fact that identifying tools is associated with activity in motor cortical regions? Although this result in itself does not necessarily imply a causal role for motor representations in perception (see Mahon and Caramazza, 2008), enough additional evidence has accumulated, some of which we review below, to suggest that motor representations do exert an influence—yet to be adequately defined on the perception of manipulable objects. In what follows, we develop an experimental approach that sheds light on the motor features influencing the perception of handled objects like beer mugs and frying pans. Our research builds on previous work establishing that secondary tasks that require the programming of hand actions have an adverse impact on the ability of normal subjects to identify tools and other graspable objects.

# **ACTIONS PLAY A ROLE IN OBJECT IDENTIFICATION**

Witt et al. (2010) required participants to squeeze a small foam ball with their right or left hand while identifying pictures of tools or animals. Naming was delayed when pictures of tools were displayed with their handles aligned toward the hand carrying out the squeezing task. No comparable effect was obtained for depicted animals presented with their heads oriented toward or away from the responding hand. The authors suggested that squeezing a ball engages motor processes that are also needed to evoke a left or right-handed action associated with grasping the depicted tool. Presumably, these motor representations are causally implicated in the naming task.

More recently Yee et al. (2013) documented the effect of a secondary motor task on the perceptual identification of objects. Participants carried out a three-step sequence of meaningless actions using both hands while concurrently identifying objects associated with a high or low degree of motor experience. A block of trials performed without concurrent motor demands served as a baseline condition. Naming accuracy for objects rated as being frequently touched (e.g., toothbrush) showed greater interference from the motor task than objects (like a bookcase) associated with fewer motor memories. The authors inferred, given these results, that motor information is part of the representation used when identifying manually experienced objects.

An interesting set of methodological issues emerges if we compare the logic of the two studies we have just summarized. The approach favored by Yee et al. (2013) relies on the claim that object concepts in long-term memory are abstracted away from specific instances. The procedure they used generated its effects not because of any degree of similarity between the actions involved in the secondary task and the actions associated with the target objects. Rather, the secondary task presumably demanded motor resources that were also needed for the identification of objects typically associated with a high degree of manipulability. The rival assumption tacitly made by Witt et al. (2010) is that access to the conceptual identity of an object can never be completely separated from its visible form; motor interference depends on the spatial overlap between the left/right hand carrying out the secondary task and a left or right handed grasp evoked by a tool. Indeed, we believe this assumption must surely be valid at some level; the token form of a beer mug (say, rotated with the handle facing upwards) is after all an entry point to the conceptual representation of beer mugs in general. Thus, we are sympathetic to the idea that actions afforded by the handle of an object in a particular orientation play some role in processing its conceptual identity. Nevertheless, it is also true that an object concept is generally founded on a type rather than a specific token identity, consistent with the opposing standpoint taken by Yee et al., As such, actions that are implicated in object perception surely cannot be based entirely on a particular depicted form. How to reconcile these discrepant alternatives?

#### **MOTOR FEATURES IN OBJECT NAMING**

In this section, we describe the logic of our approach to the question we have just posed, which draws on a large body of previous research documenting that a prepared action maintained over a short duration can disrupt performance on an intervening perceptual task (e.g., Hommel et al., 2001; Hommel, 2009). This widely obtained result is taken as support for the claim that action and perception share common representational substrates; a motor task that requires the maintenance of features in working memory will interfere with a perceptual task that invokes the same features. The particular pattern of interference effects is surprising but has nonetheless been repeatedly observed. Performance is impaired only when there is a partial match between the constituents of the working memory task and the perceptual task. A complete match or total mismatch of features has no effect on perception. Hommel (2004) pointed out that this outcome implies not so much a benefit in repeating a feature conjunction as a cost incurred when there are features partially shared between different perceptual-motor events. A single recurring feature in perception will trigger retrieval of a previous event in working memory by spreading activation, and the ensuing conflict, brought about by a mismatching feature or set of features, will hamper stimulus identification and/or response selection (for additional theoretical details, see Stoet and Hommel, 2002; Hommel et al., 2004).

Experiments on motor-visual interference generally incorporate abstract symbols as objects and arbitrary responses as actions, to facilitate parametric variation of elementary features like spatial orientation and position. Nevertheless, given certain assumptions (see below), it is possible to apply the same basic principles underlying the pattern of effects we have just described to the more realistic world of everyday manipulable artifacts and their associated motor representations.

What kind of motor features are evoked by an upright beer mug with its handle on the right? The action corresponding to this depicted view of the object is a right handed, closed grasp, with the wrist oriented vertically (i.e., the ventral and dorsal surfaces of the wrist are vertically perpendicular to the ground). By contrast, a frying pan with the handle on the left requires a lefthanded closed grasp with the wrist oriented horizontally (i.e., the wrist is pronated so that its ventral and dorsal surfaces are parallel to the ground). Thus, we can reasonably conjecture that features such as hand (left vs. right) and wrist orientation (vertical vs. horizontal) would be recruited as part of the motor representations that are implicated in the identification of handled objects. A test of this conjecture, based on motor interference effects generated by a secondary task, is relatively straightforward. We arranged matters so that the constituents of a prepared set of actions maintained in working memory incorporated the above two features, and we examined the impact of this secondary task on the time taken to perceptually identify pictures of handled objects (Bub et al., 2013). Remarkably, our results fully replicated the pattern of interference effects typically obtained with abstract symbols as objects and arbitrary stimulus-response mappings. Object naming latency was slowed when a single motor feature was shared between the prepared action (left or right handed action; vertical or horizontal grasp posture) and the affordance of the target object. Latencies were faster (and accuracy was higher) when the planned action and perceived object shared both or neither of these features. Thus, the manipulability of an object can be decomposed into constituent features that are part of its semantic representation. A particular strength of our methodological approach is that it promises to further clarify the computational role of motor features in the perceptual classification of everyday manipulable objects.

The objects in Bub et al. (2013) were all upright and so, apart from the fact that they varied with respect to the left/right positioning and vertical/horizontal orientation of their handles, each object's depicted view matched its canonical view. We cannot therefore answer the fundamental question we posed earlier: what is the relative contribution of the actions associated with the depicted and canonical form to the identification of a manipulable object? To clarify this issue, we need to distinguish the actions associated with the upright canonical description of an object from those evoked by its depicted form. Imagine a beer mug on its side with the handle facing upwards. The motor features activated as part of the conceptual identity of the object would reference the grasp associated with its upright, canonical form. Thus, a vertical rather than a horizontal wrist orientation should be invoked, while the reverse would be the case for the depicted view. We can determine which of these parameters of the wrist orientation feature is recruited for identification by examining the pattern of interference effects generated by planned actions held in working memory. A motor feature shared between the constituents of working memory and the actions recruited by the object will have an adverse impact on naming performance. In the case of the rotated beer mug, does the shared feature correspond to a vertical wrist orientation (matching the canonical form of the object) or a horizontal wrist orientation (conforming to the depicted view)?

What of the motor feature corresponding to the left/right choice of hand? The canonical description might include the fact that we typically use our dominant hand to lift and use a manipulable object. However, a more complex possibility should be considered. As we have noted, the depicted view of an object is the entry point to knowledge of its identity. Assume that naming an object depends in part on translating the rotated form of an object into a canonical upright representation. An object like a beer mug will evoke a left- or right-handed grasp depending on the location of the handle after rotation. For example, a horizontal beer mug with the mouth or opening on the right will yield a lefthanded grasp when rotated into an upright position. In general, the token form of an object may determine whether the canonical representation activates a left- or right- hand grasp if motor features are consulted as part of the naming process.

To summarize, we conjecture that the speeded naming of manipulable objects (tools and utensils) should recruit the motor features left/right hand, and vertical/horizontal wrist orientation. We will rely on the pattern of interference produced by a secondary working memory task incorporating these features to clarify the nature of the motor representations contributing to performance.

#### **EXPERIMENT 1**

We investigated the influence of action features held in working memory on the identification of pictured objects presented either in their canonical view or rotated 90◦ so that the object's handle was shifted from a horizontal to a vertical orientation, or vice versa. The critical question was whether under this rotation the object would be encoded in its depicted view or in its canonical view and, more particularly, how that encoding would interact with the action representations held in working memory. One possibility is that the relation between the features of the hand actions held in working memory and the depicted features of the object's handle would determine congruency and thereby the pattern of response times for partial feature overlap, complete overlap, and no overlap conditions. Alternatively, congruency might be driven by the relation between the features of the hand actions and the canonical features of the object's handle, not its depicted features. Testing rotated views of the objects allowed us to address this issue.

As an additional test of the nature of the encoding of rotated objects, we included a set of objects that do not have a standard canonical view, inasmuch as they are very often seen and used both in a horizontal and in a vertical position (e.g., hair brush, wrench). For these *acanonical* objects, we anticipated that the influence of working memory load would be determined by the depicted view of the object because there would be no strong canonical view to oppose it.

#### **METHOD**

#### **SUBJECTS**

Thirty students at the University of Victoria participated to earn extra credit in an undergraduate psychology course. The experiments reported here were approved by the University of Victoria Human Research Ethics Committee.

#### **MATERIALS**

Four hand postures, distinct from a simple power grasp1, were selected for use as memory load stimuli. The four postures were: extended forefinger, extended thumb, flat palm, and precision grip with thumb and forefinger. A grayscale digital photograph was taken of a male right hand formed in each of these postures with the wrist oriented horizontally (so the palm of the hand faced downward) and again with the wrist rotated vertically (i.e., the wrist continued to be parallel to the ground, but its dorsal and ventral surfaces were now oriented vertically; see **Figure 1**). Each of these eight photographs was rendered in a left-handed pose by creating a mirror image reflection of the original image.

Twenty-four object types were chosen for use as target objects. All were handled objects that are typically used by applying a power grasp to the object's handle. Eight of the object types had a handle that is vertically oriented when the object is in its canonical position (e.g., beer mug), eight were objects that have a horizontally oriented handle (e.g., frying pan) when in their canonical position, and eight were acanonical objects (often experienced with their handles in either orientation). A list of the names of the 24 object types is given in the Supplementary Material. Four token images of each type were chosen from various internet sites (e.g., four different knives), yielding 96 token images. Each of the

<sup>1</sup>We did not use the two postures that were types of power grasps that had been included in the posture set used by Bub et al. (2013), because they were deemed too similar to the grasp used to hold the target objects. We wished to avoid the possibility that a third attribute, grip posture, might be shared between the target objects and actions held in working memory. The power grasps were replaced by a pointing index finger posture.

96 token images were rendered as grayscale digital images providing a profile view of the object (see **Figure 2**). Two variants of each image were created, one with the handle facing to the right (inviting a right-hand grasp) and one with the handle facing to the left.

A rotated view of the right- and left-hand variant of each token image was created by rotating the image 90◦ such that a canonical object with a vertical handle now had its handle oriented horizontally and positioned on the upper part of the image. For objects with horizontal handles, the chosen 90◦ rotation caused the handle to point downward. For acanonical objects, we arbitrarily deemed images with the handle in a vertical orientation to be upright, and images with a horizontally oriented handle to be rotated. **Figure 2** shows examples of the upright and rotated images for two objects, one whose canonical handle orientation is vertical and the other horizontal. Note that for both the upright and rotated views, a depicted image invites a grasp by one or the other hand. In the case of the canonical view, the handle is positioned to favor one hand. In the rotated view, the preferred hand is determined by the principle of commensurability (Masson et al., 2011), whereby the choice of hand for grasping a rotated object is determined by whether using a particular hand will allow the object to be brought into its upright, functional position with a comfortable wrist rotation (see also Rosenbaum et al., 1990). For example, consider the image of the rotated teapot on the left side of **Figure 2**. Grasping an object oriented that way with the left hand, then rotating the wrist counterclockwise 90◦ would lead to an upright teapot in a comfortable position. Using the right hand to grasp that object, however, would require an awkward and uncomfortable wrist rotation to bring the object to an upright position.

#### **DESIGN**

On each critical trial of the experiment, subjects were presented two hand actions (represented by images of hand postures as in **Figure 1**) as a working memory load. These two actions involved the same hand (right or left) and the same wrist orientation

(horizontal or vertical), but differed in hand posture. The primary manipulation in the experiment was the relationship between the hand and orientation of the two actions in working memory and the right/left alignment and the orientation of the handle of the object to be named on that trial. We use the term alignment to refer to the congruency between the hand actions and the object with respect to the hand used for the actions and the side favored by the handle. For example, actions using the right hand are congruent with an object whose handle is on the right side of the object's image or, in the case of a rotated object, for which a right handed grasp would be commensurate with its function. Orientation refers to the congruency between the wrist orientation of the hand actions in working memory and the orientation of the target object's handle. For example, hand actions with a horizontally oriented wrist posture are congruent with an image of an upright sauce pan.

There were 16 conditions in the experiment, defined by the alignment and orientation of the object's handle and the alignment and orientation of the hand actions that formed the working memory load. Three blocks of 96 critical trials were presented, yielding a total of 288 critical trials. Each of the 96 token images was presented once in each block. Within each block, six objects (two of each class: horizontal, vertical, and acanonical) were randomly assigned to each of the 16 conditions. The assignment of objects to conditions varied across subjects so that each object type was tested equally often in each condition. The specific object image that was presented depended on the condition to which the object was assigned. For example, if an object with a vertical handle when in its upright position were assigned to the condition with a horizontal handle and right alignment, then the rotated image of the object, with its top to the left and its bottom to the right, was used (e.g., the lower right image of the teapot in **Figure 2**). The four hand postures were arranged into six different pairs. Each pair was used with one of the objects in each of the 16 conditions in a block of trials. The order of presentation of the two hand postures within a trial was randomly determined.

#### **PROCEDURE**

All images of hands and objects were scaled to fit within a square extending 14.5◦ of visual angle on each side when viewed from 50 cm. Images were displayed on an LCD monitor controlled by a Macintosh desktop computer. Subjects were tested individually in a quiet room under the supervision of an experimenter who provided instructions and scored responses as they occurred. Subjects wore a headset with a microphone to detect their vocal responses.

In the first phase of the experiment, subjects were familiarized with the set of hand actions and their associated cues. They were given an opportunity to pantomime each combination of hand shape and wrist orientation with each hand in response to the pictured hand cues. Subjects were also given practice at naming the left-facing upright images of each of the object tokens. In the second phase of the experiment, subjects were presented 288 critical trials. On each trial subjects were shown for 1000 ms each of the cues for the two hand actions that constituted the working memory load for that trial, followed by a 1000-ms blank display. The pictured object then appeared and subjects were instructed to name the object as quickly and accurately as possible (see **Figure 3**). Their vocal responses were detected by the microphone on the headset they wore, and the experimenter pressed a key on the computer keyboard to score the accuracy of the response. On a randomly selected 25% of trials, after the vocal response a signal appeared on the monitor indicating that the subject was to pantomime the two hand actions that were held in working memory on that trial. This task ensured that subjects attended to and maintained in memory the hand actions presented on each trial.

## **RESULTS AND DISCUSSION**

#### **REPORT OF HAND ACTIONS**

When reporting the hand actions held in working memory, subjects were scored correct if they reported both actions, regardless of the order in which they were reported. The mean percent correct was 79.3%. This level of performance indicates that the working memory task was a demanding one, but that subjects were able to maintain the assigned actions in most trials (the lowest scoring subject was correct on 70.4% of the trials).

#### **STATISTICAL ANALYSES**

The analyses we report provide both the outcome of a null hypothesis significance test and the corresponding Bayes factor (BF) generated using the BayesFactor package in the open source statistical program R, described by Rouder et al. (2012). The Bayes factor we report for an effect indicates the ratio of the strength of evidence supporting a model of the data that includes all effects in the design relative to a model that excludes only the effect of interest. Larger values of the Bayes factor indicate stronger evidence for the effect.

Naming latencies for correct responses were included in the analyses if they were longer than 200 ms and shorter than 2600 ms. The lower bound was intended to eliminate extraneous activations of the microphone and the upper bound was selected so that no more than 0.5% of the longest response times were removed as outliers (Ulrich and Miller, 1994).

In the analyses, we were interested in congruency between the object to be named and the actions held in working memory with respect to two attributes: hand alignment and wrist orientation. The conditions we used constituted a factorial manipulation of these two types of congruency. For upright object images, congruency was determined in the obvious way (e.g., left-hand actions were congruent with an object pictured with its handle on the left; vertical wrist orientation in hand actions was congruent with an object whose handle is vertically oriented, such as teapot). For rotated object images, congruency of alignment was determined by which hand would be commensurate with grasping the object and comfortably rotating the wrist to bring it to an upright position. Consider, for instance, the sauce pan in the bottom right of **Figure 2**. Its handle would be considered to be aligned with the right hand because a grasp made with that hand could be followed by a 90◦ wrist rotation to bring the pan into a functional position. Congruency of orientation for rotated images was determined by the depicted view of the object. For a rotated beer mug, for example, a horizontal action was deemed congruent.

#### **ACANONICAL OBJECTS**

Analysis of object naming performance was conducted separately for the acanonical objects on one hand, and for the horizontal and vertical objects on the other hand. It was expected that because acanonical objects lacked a typical horizontal/vertical view, they would interact with the actions held in working memory differently than would objects characterized by a typical upright view.

Mean naming latencies for acanonical objects are shown in **Figure 4**, representing conditions defined by object view (horizontal or vertical), congruency of the orientation of the hand actions held in working memory relative to the viewed object (congruent or incongruent), and congruency of the left-right alignment of the hand actions held in working memory and the viewed object (congruent or incongruent). For example, a toothbrush presented in a horizontal orientation with its head on the left and its handle pointing to the right, would be congruent on orientation and alignment with hand actions using the right hand with a horizontally oriented wrist, but incongruent on both dimensions with hand actions using the left hand with a vertically oriented wrist.

A repeated-measures analysis of variance (ANOVA) with object view, orientation, and alignment as factors produced only a main effect of alignment, *F*(1*,* 29) = 10*.*77, *MSE* = 2574, *p <* 0*.*01, *BF* = 4*.*5. For all other effects, *F*s *<* 1 (*BF*s *<* 0.4). As can be seen in **Figure 4**, naming latencies were longer when the hand actions in working memory and the object handle were congruently rather than incongruently aligned (1067 vs. 1045 ms). Note that the lack of an effect of object view is consistent with our assumption that this set of objects is frequently experienced in both horizontal and vertical orientations. The mean naming error rate was 0.6% and across 240 cells of the design (30 subjects × 8 conditions), only 16 had any errors. Therefore, no inferential analysis was applied to the error data.

For acanonical objects, the dimension of orientation congruency did not influence naming time, unlike our previous results (Bub et al., 2013), in which naming time was sensitive to the combination of alignment and orientation congruency. This result suggests that subjects were sensitive not only to the depicted view of the object (as clearly indicated by the effect of alignment), but also to prior knowledge and experience, whereby this set of objects would frequently have been encountered in both horizontal and vertical views. The interference effect associated with congruent hand alignment could be attributed to the binding of that feature with the action plan held in working memory, making it unavailable to processes responsible for identifying the target handled object (Hommel et al., 2001; Hommel, 2004). The unavailability of the orientation feature, which is assumed to be bound to the hand actions in working memory, apparently could be compensated for by knowledge of prior experience with the target object positioned in a manner opposite to the depicted view. As a result, congruency of the orientation feature had no influence on object naming.

#### **OBJECTS WITH A CANONICAL VIEW**

The mean naming error rate was 1.5% and an ANOVA computed with object view and congruency for alignment and orientation found no significant effects.

The mean naming latencies for objects that have a strong, typical view are shown in **Figure 5**. An ANOVA applied to the latency

orientation between the object's handle and the hand actions held in working memory. Error bars are 95% within-subject confidence intervals. data with object view and congruency for alignment and orientation as repeated-measures factors revealed a main effect of object view, *F*(1*,* 29) = 11*.*40, *MSE* = 4476, *p <* 0*.*01, *BF* = 627*.*9, whereby objects were named faster if they were presented in their upright rather than rotated view (1139 vs. 1168 ms). The only other significant effect was the three-way interaction between object view (upright, rotated), alignment congruency, and orientation congruency, *F*(1*,* 29) = 14*.*97, *MSE* = 1768, *p <* 0*.*01, *BF* = 13*.*6. This interaction is consistent with what would be expected if rotated objects were encoded so that action representations associated with their canonical view were evoked, rather than actions implied by their depicted view.

To follow up the three-way interaction, we conducted separate ANOVAs for upright and rotated objects. For upright canonical objects, we had expected to replicate the pattern of congruency effects reported by Bub et al. (2013) and also found in a replication study in our lab (Bai, 2013), whereby shorter naming latencies were obtained when both alignment and orientation were congruent or both were incongruent, relative to when one dimension was congruent and the other incongruent. **Figure 5** indicates that this pattern was only partly replicated, given that a very weak orientation effect was found when alignment was congruent. An ANOVA applied only to upright objects with alignment and orientation congruency as repeated-measures factors found a significant interaction, but the Bayesian analysis provided virtually no support for it, *F*(1*,* 29) = 4*.*58, *MSE* = 1670, *p <* 0*.*05, *BF* = 1*.*1. Neither main effect was significant.

For rotated objects, an ANOVA with alignment and orientation congruency as repeated-measures factors yielded a significant interaction that was also supported by the Bayesian analysis, *F*(1*,* 29) = 5*.*74, *MSE* = 3547, *p <* 0*.*05, *BF* = 4*.*9. In addition, there was a main effect of orientation congruency, *F*(1*,* 29) = 6*.*19, *MSE* = 3*.*074, *p <* 0*.*05, *BF* = 3*.*5, but not of alignment congruency. If rotated objects had been encoded purely on the basis of their depicted view, then we should have seen an interaction between alignment and orientation congruency much like that observed by Bub et al. (2013). Instead, however, the pattern shown in **Figure 5** is more in keeping with the one-feature overlap interference effect that would occur if subjects had encoded the canonical features of objects, rather than their depicted features. That is, response time was particularly long if both alignment and depicted handle orientation were congruent with the hand actions held in working memory. But if we suppose the canonical view of the target object had been encoded, then what is labeled as congruent orientation in **Figure 5** becomes incongruent, and vice versa, so that we now have a pattern that more closely resembles that reported by Bub et al. When alignment was incongruent, however, the effect of orientation congruency was virtually nonexistent. Therefore, neither the upright nor the rotated views produced a pattern of congruency effects that fully matches that obtained by Bub et al. Consequently, we are not yet in a position to draw strong conclusions about how subjects encoded objects presented in a rotated view.

#### **EXPERIMENT 2**

It is possible that the congruency effects found in Experiment 1 for objects that have a typical view were modulated by the inclusion of acanonical objects in the set of target objects. Indeed, Bub et al. (2013) did not include acanonical objects in their experiment when demonstrating the partial overlap interference effect. Therefore, in Experiment 2 we replicated Experiment 1 but excluded acanonical objects. In addition, we examined the response time distributions in Experiment 1 and observed that the partial overlap interference effect was most apparent among the lower two thirds of response times. In a effort to maximize the effect in Experiment 2, we encouraged subjects to make faster responses by providing them with only a brief view of a target object (150 ms) followed by a visual mask.

# **METHOD**

#### **SUBJECTS**

Thirty subjects were recruited from the same source as in Experiment 1, although none had participated in that experiment.

#### **MATERIALS AND DESIGN**

The same images of hand postures and objects were used as in Experiment 1, except that the acanonical objects were excluded. The remaining 64 objects were each presented once in each of four successive blocks, producing a total of 256 critical trials. Within each block, objects and hand actions were again assigned to the same 16 conditions as in Experiment 1 and these assignments varied across subjects so that each object concept was tested equally often in each condition.

#### **PROCEDURE**

The procedure was the same as in Experiment 1, except that on critical trials, the target object was in view for only 150 ms before being replaced by a pattern mask.

## **RESULTS AND DISCUSSION**

Subjects correctly reported the hand actions that were held in working memory on an average of 79.4% of the trials on which they were probed to report them. As in Experiment 1, naming latencies less than 200 ms were excluded from analysis, as well as latencies in excess of 2800 ms. The upper cutoff was set so that fewer than 0.5% of correct trials were excluded. The mean naming latencies are shown in **Figure 6**. An ANOVA with object view (upright vs. rotated) and alignment and orientation congruency as repeated-measures factors indicated that upright objects were named faster than rotated objects (1006 vs. 1025 ms), *F*(1*,* 29) = 8*.*64, *MSE* = 2660, *p <* 0*.*01, *BF* = 30*.*9. There was also a significant interaction between object rotation and orientation congruency, *F*(1*,* 29) = 29*.*08, *MSE* = 1256, *p <* 0*.*01, *BF* = 878*.*7, but this effect was superseded by the significant three-way interaction, *F*(1*,* 29) = 21*.*44, *MSE* = 2309, *p <* 0*.*01, *BF >* 1000. No other factors were significant.

#### **UPRIGHT OBJECTS**

The three-way interaction was examined by computing separate ANOVAs for each object rotation condition with alignment and orientation congruency as repeated-measures factors, as in Experiment 1. For upright objects, there was a main effect of orientation congruency, with longer latencies when the object handle and the hand actions had congruent orientations rather

than incongruent orientations (1015 vs. 996 ms), *F*(1*,* 29) = 6*.*31, *MSE* = 1679, *p <* 0*.*05, *BF* = 2*.*8. But there was also a significant interaction between alignment and orientation congruency, *F*(1*,* 29) = 15*.*96, *MSE* = 1459, *p <* 0*.*01, *BF* = 70*.*5. This interaction generally conforms to the pattern reported by Bub et al. (2013), although the effect of orientation congruency was rather weak when alignment was congruent. This was also the case in Experiment 1.

#### **ROTATED OBJECTS**

For rotated objects, the ANOVA with alignment and orientation congruency as factors yielded a main effect of orientation congruency, although here latencies were shorter in the congruent case (1010 vs. 1041 ms), *F*(1*,* 29) = 16*.*04, *MSE* = 1746, *p <* 0*.*01, *BF* = 82*.*4. Note that if we assume, as suggested above, that subjects encode rotated objects in their canonical view, then the orientation congruency effect can be seen as an interference effect [longer latencies when the encoded (canonical) orientation of the object's handle matches the orientation of hand actions held in working memory], just as was seen with upright objects. The alignment by orientation congruency interaction was also significant, *F*(1*,* 29) = 10*.*43, *MSE* = 2517, *p <* 0*.*01, *BF* = 62*.*4. As in Experiment 1, the pattern of means is similar to what would be expected from the Bub et al. (2013) findings if we assume that rotated objects were encoded in their canonical view and that it was this view that determined congruency with the orientation of hand actions held in working memory. Once again, however, the fit is not perfect because in this case the orientation congruency effect was rather weak when alignment was congruent.

actions held in working memory. Error bars are 95% within-subject

#### **ERROR RATES**

confidence intervals.

Mean error rates are shown in **Figure 7** and it is apparent that congruency effects were similar to those obtained in the latency data. An ANOVA with object view and alignment and orientation congruency as repeated-measures factors found a significant effect of object view, with fewer errors on upright than on rotated objects (1.7 vs. 2.4%), although the effect was not supported by the Bayesian analysis, *F*(1*,* 29) = 6*.*54, *MSE* = 4*.*09, *p <* 0*.*05, *BF* = 1*.*1. There was also a significant three-way interaction, *F*(1*,* 29) = 8*.*18, *MSE* = 11*.*12, *p <* 0*.*01, *BF* = 160*.*1. No other effects were significant. Separate ANOVAs were computed for the upright and rotated conditions with alignment and orientation congruency as factors and the only significant effect from either analysis was the alignment by orientation congruency interaction for rotated objects, *F*(1*,* 29) = 7*.*33, *MSE* = 7*.*88, *p <* 0*.*05, *BF* = 12*.*2. In general, the error data supported the pattern of congruency effects found in the response latency data.

#### **AGGREGATED DATA**

The data from Experiments 1 and 2 for objects that have a preferred view showed a tendency for alignment and orientation congruency effects to follow the partial overlap effect reported by Bub et al. (2013). These results did not, however, fully conform to that pattern. It is possible that by introducing the rotation manipulation we either reduced statistical power relative to the Bub et al. study, or perhaps even modulated the partial overlap effect. With these possibilities in mind, and given that both experiments produced a significant three-way interaction between object view and alignment and orientation congruency, we aggregated the data from the two experiments to yield a more reliable assessment of how well the pattern of that interaction conformed to the partial overlap effect in naming latency. The mean latencies are shown in **Figure 8**. The upright condition shows an approximation to the partial overlap effect, although the orientation congruency effect was weak under congruent alignment, as was seen within each of the two experiments separately. The rotated condition, however, showed a very clear replication of the partial overlap effect, assuming that object view was encoded according to the object's canonical, rather than its depicted view.

An ANOVA that pooled the latency data from both experiments and that included object view and alignment and orientation congruency as repeated-measures factors showed that naming responses were faster when the objects were upright (1072 vs. 1097 ms), *F*(1*,* 59) = 20*.*05, *MSE* = 3554, *p <* 0*.*01, *BF >* 1000. The three-way interaction was also significant, *F*(1*,* 59) = 36*.*47, *MSE* = 2035, *p <* 0*.*01, *BF >* 1000. No other effects were significant. Separate ANOVAs for each object view condition found no main effects but confirmed that the alignment congruency by orientation congruency interaction was significant for upright and for rotated objects (*p*s *<* 0.01, *BF*s *>* 100). These two-way interactions, of course, took opposite forms, suggesting that subjects had encoded rotated objects in their canonical view. Indeed, when we recoded orientation congruency for rotated objects so that it was defined by the objects' canonical rather than depicted view, the resulting ANOVA that included object view, alignment congruency, and orientation congruency, indicated that the significant alignment by orientation interaction (*F*- and *BF*-values were the same as reported above) was not significantly different for the two

rotation conditions (*F <* 1, *BF <* 1). Mean naming latency as a function of alignment and orientation, collapsing across upright and rotated objects (with orientation in the latter case coded for the canonical rather than the depicted view) is shown in **Figure 9**. This pattern of means shows clear evidence for the partial overlap effect.

# **GENERAL DISCUSSION**

It has been widely established that action representations are automatically evoked by manipulable objects, even when such objects are passively viewed. This article concerns the possible contribution, if any, of these representations to perception. We developed a methodology that allows us to analyze the constituents of action invoked when participants engaged in the speeded naming of manipulable objects (see also Bub et al., 2013). Our approach is built on a procedure that previously has been used to examine how motor features in working memory adversely affect a subsequent perceptual or motor task demanding integration of these selfsame features (see Stoet and Hommel, 2002 for an overview). Performance suffers when only one of the features of the planned action overlaps with the sensory-motor features of the target event (e.g., Hommel, 2004; Fournier et al., 2013).

The pattern of interference effects has received the following interpretation. Assume that the contents of working memory include the motor features X and Z bound together into an action plan. Identifying the target object requires features X and Y. Feature X, activated by the perceived target object, primes the same feature held in working memory, leading to its automatic retrieval. However, retrieval of X also brings with it the bound feature Z. The feature Z now competes with feature Y, disrupting

the ability to integrate Y with X as part of the representation of the perceptual target. In contrast, no such interference will occur for objects sharing both or neither of the features constituting the planned action.

Bub et al. (2013) examined the disruptive impact of action features in working memory on the ability to identify manipulable objects. We documented the same partial repetition cost previously obtained in numerous other studies relying on abstract symbols as targets and arbitrary actions as responses. Speeded naming was delayed and less accurate if the target object shared one of the action features in working memory; for example, if the contents of working memory involved the left hand and the palm oriented vertically, then performance was affected for a target object like a beer mug with the handle oriented to the right (wrist orientation matches the feature in working memory but not hand alignment) or a frying pan with the handle oriented to the left (hand alignment matches but not wrist orientation). Performance was faster and more accurate when the target object shared either both (e.g., a beer mug oriented with the handle on the left) or neither (e.g., a frying pan with the handle on the right) of the motor features in working memory.

The approach we have developed allows us to go well beyond previous demonstrations that secondary tasks involving some kind of action selectively disrupt the classification of manipulable objects (Witt et al., 2010; Yee et al., 2013). Our interest concerns the computational role of particular motor features in the identification of visual objects. In the present article we wished to determine whether these features correspond to actions triggered by the depicted form of the object or on a more abstract representation of the object's canonical form. The question is important because to identify an object requires that spatiotemporal information representing the particular token form of an object be mapped onto a more general description in long-term memory reflecting an object type. Accordingly, the role of action representations cannot be separated from the dynamic interplay between token and type descriptions activated during object classification.

### **PERCEIVING ROTATED OBJECTS**

We applied the method developed by Bub et al. (2013) to determine the nature of the action representations evoked by objects rotated 90◦ from their canonical view as well as by objects displayed in their typical upright view. The token or depicted view of a rotated beer mug affords a closed grasp with the wrist oriented horizontally. The object type, based on its canonical form, demands a closed grasp with the wrist oriented vertically. Which of these motor features—wrist horizontal or vertical—is evoked when subjects name the rotated beer mug? The nature of the partial repetition cost unambiguously indicates that a closed grasp triggered when naming a handled object always conforms to the wrist orientation associated with its typical upright view, even when the object has been rotated 90◦. For example, actions in working memory incorporating a vertical wrist orientation interfered with the ability to name a rotated beer mug, despite the fact that the depicted view automatically triggers a horizontal grasp (Masson et al., 2011). This striking outcome allows us to infer that one component—wrist orientation—of the action representation consulted in naming a handled object is based on its canonical rather than its depicted form.

The object's depicted form, however, also exerts an influence on naming performance. The motor feature associated with a left or right handed grasp depends on the location of the opening or mouth of an object like a beer mug; the rotated form with the opening on the left affords a right- rather than a left-handed grasp if the object is returned to its upright (canonical) position. The partial repetition cost induced by the feature left/right hand is thus contingent on the depicted or token form of the object in relation to its canonical form. We access the upright description of beer mug when naming its rotated form but this representation includes a left- or right-handed grasp contingent on the object's initial view. A horizontal beer mug with the opening on the left translates into a beer mug with the handle on the right if rotated by 90◦ into an upright position, generating a right-handed, vertically oriented closed grasp. This action representation plays a role not only in the identification of an upright beer mug (handle on the right), but also in the identification of a horizontal beer mug affording the same grasp when rotated into an upright position.

## **ON THE ROLE OF MOTOR FEATURES IN OBJECT IDENTIFICATION**

A standard result, also observed in the present article, is that naming is slower and/or less accurate for images of objects rotated in the plane than images of upright objects (Jolicoeur, 1985, 1988; Maki, 1986; Jolicoeur and Milliken, 1989; McMullen and Jolicoeur, 1990; McMullen et al., 1995; Murray, 1995, 1997). Convincing evidence has accumulated that the effect of rotation on performance occurs at a fairly late processing stage, and that an object's identity can be established independently of its view or token form. For example, neuropsychological cases have been documented who show accurate recognition of objects presented in different orientations, but severe impairment in assessing their orientation (Turnbull et al., 1995, 1997; Karnath et al., 2000; Harris et al., 2001). Thus, confronted with the image of an inverted dog, such patients, after identifying the animal, may contend that the dog's depicted view is canonical (upright).

Behavioral evidence confirms that establishing an object type or identity does not depend on the orientation of the token or depicted form. For example, Harris et al. (2008) briefly presented masked objects as primes followed by upright objects which were to be named as quickly as possible. The primes were displayed at varying degrees of rotation in the plane from an upright position. The magnitude of the priming effect did not depend on prime orientation, even though naming the same objects was systematically delayed as their orientation deviated from upright (see also Harris and Dux, 2005; Cheung and Bar, 2014, for additional evidence).

Although object identity can be determined independently of orientation, an object's orientation is an important aspect of its episodic representation. Chun (1997) has argued that an object's identity (i.e., its type) must be bound to a spatio-temporal representation (the token form of an object) to enable overt report (see also Harris and Dux, 2005). We conjecture that motor features driven by the depicted form of an object facilitate the binding of type-token descriptions. An upright beer mug with the handle on the left evokes a left-handed, vertical closed grasp, and the motor constituents of this action representation are part of the description that enable individuation of the object for conscious report. In effect, we identify the object as a "left-handed beer mug" because we integrate the object type (beer mug) with a particular token form yielding a left-handed grasp.

The depicted form of a beer mug displayed horizontally with the opening on the right evokes an action that begins with a horizontal left-handed closed grasp and ends with a vertical grasp. This action reflects the dynamic unfolding of a goal-oriented motor representation; a proximal grasp followed by an end-state of the action commensurate with the object's upright position (Masson et al., 2011). A right-handed grasp is not strongly activated by the rotated beer mug, because the proximal action consistent with this particular orientation requires an awkward counterclockwise rotation of the wrist to produce an upright object, a movement at odds with the end-state comfort effect (Zhang and Rosenbaum, 2008).

It is of considerable interest that naming rotated objects implicates a motor feature that reflects the distal goal or end-state of an action plan triggered by the object's token form. For an upright beer mug, the vertical wrist posture is the same for the beginning and end state of the grasp. For a horizontally oriented beer mug, the vertical wrist posture corresponds to the end state of the action triggered by the depicted view, whereas the proximal action involves a horizontal grasp. Because it is a vertical grasp that contributes to naming both upright and rotated depictions of a beer mug, the evidence suggests that the motor representation is based on the distal rather than proximal actions associated with the target object.

#### **IMPLICATIONS FOR APRAXIA**

We conclude by returning to a conundrum posed at the beginning of this article. What is the relationship between naming an object and the actions determined by its form and function? We have conjectured that motor features are recruited as part of the spatiotemporal description of an object enabling conscious report. Motor features should play an increasingly crucial role when it becomes difficult to maintain a distinct episodic representation for a given object type. Under certain conditions, for example, it is hard to identify both instances of a repeated object presented within a 500-ms time window (the well-known repetition-blindness effect). According to Kanwisher (1987), repetition blindness occurs when an abstract description of an object (a *type*) is not encoded as a distinct visual episode (a *token*) because of the spatiotemporal uncertainty created by rapid serial visual presentation.

Interestingly, Harris et al. (2012) have shown that repetition blindness does not occur when the repeated item in a visual stream is the depiction of a manipulable object. In fact, these authors report a repetition advantage for manipulable objects, in contrast to the usual repetition blindness they obtained for nonmanipulable objects. As Harris and colleagues suggest, motor representations associated with manipulable objects may enhance our ability to individuate two instances of the same object type. This possibility gives rise to a prediction concerning the performance of apraxic patients that is worth testing. Such patients can name objects despite impairments in accessing the motor representations associated with their function. How, though, would the performance of apraxic cases differ from age-matched controls, if greater difficulty occurred in establishing the spatiotemporal description of an object for conscious report? Given impaired access to motor features that help individuate an object, there should be no enhancement of the ability to identify two repeated instances of a manipulable object under conditions of rapid serial visual presentation. Unlike neurologically intact participants, then, apraxic individuals should demonstrate repetition blindness for both manipulable and non-manipulable objects.

## **ACKNOWLEDGMENTS**

This research was supported by discovery grants from the Natural Sciences and Engineering Research Council of Canada to D. Bub and to M. Masson, and by National Science Foundation grant #SBE-0542013 to the Temporal Dynamics of Learning Center, an NSF Science of Learning Center. We are grateful to Marnie Jedynak for assistance in conducting the experiments.

# **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at: http://www*.*frontiersin*.*org/journal/10*.*3389/fnhum*.* 2015*.*00042/abstract

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 01 October 2014; accepted: 16 January 2015; published online: 06 February 2015.*

*Citation: Bub DN, Masson MEJ and Lin T (2015) Components of action representations evoked when identifying manipulable objects. Front. Hum. Neurosci. 9:42. doi: 10.3389/fnhum.2015.00042*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2015 Bub, Masson and Lin. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Dissociation between Semantic Representations for Motion and Action Verbs: Evidence from Patients with Left Hemisphere Lesions

Lawrence J. Taylor <sup>1</sup> , Carys Evans <sup>2</sup> \*, Joanna Greer <sup>1</sup> , Carl Senior <sup>3</sup> , Kenny R. Coventry <sup>4</sup> and Magdalena Ietswaart <sup>5</sup>

<sup>1</sup> Department of Psychology, Northumbria University, Newcastle upon Tyne, UK, <sup>2</sup> Department of Psychology, Goldsmiths University of London, London, UK, <sup>3</sup> Department of Psychology, School of Life and Health Sciences, Aston University, Birmingham, UK, <sup>4</sup> School of Psychology, University of East Anglia, Norwich, UK, <sup>5</sup> Department of Psychology, University of Stirling, Stirling, UK

This multiple single case study contrasted left hemisphere stroke patients (N = 6) to healthy age-matched control participants (N = 15) on their understanding of action (e.g., holding, clenching) and motion verbs (e.g., crumbling, flowing). The tasks required participants to correctly identify the matching verb or associated picture. Dissociations on action and motion verb content depending on lesion site were expected. As predicted for verbs containing an action and/or motion content, modified t-tests confirmed selective deficits in processing motion verbs in patients with lesions involving posterior parietal and lateral occipitotemporal cortex. In contrast, deficits in verbs describing motionless actions were found in patients with more anterior lesions sparing posterior parietal and lateral occipitotemporal cortex. These findings support the hypotheses that semantic representations for action and motion are behaviorally and neuro-anatomically dissociable. The findings clarify the differential and critical role of perceptual and motor regions in processing modality-specific semantic knowledge as opposed to a supportive but not necessary role. We contextualize these results within theories from both cognitive psychology and cognitive neuroscience that make claims over the role of sensory and motor information in semantic representation.

Keywords: neuropsychology, left hemisphere, lateral occipitotemporal cortex, affordances, embodied cognition, semantic representation, aphasia

# INTRODUCTION

The motor system is primarily engaged for the execution of actions, but has also been shown to be engaged when familiar actions are observed (e.g., Calvo-Merino et al., 2005), imagined (e.g., Decety, 1996), or even read about (Beilock et al., 2008). For example, reading a sentence describing an action sometimes primes bodily movements matching the referential content (e.g., Glenberg and Kaschak, 2002; Papesh, 2015). Such evidence is frequently taken to support the notion that bodily and action representations are routinely recruited to derive meaning from language (Gallese and Lakoff, 2005; Fischer and Zwaan, 2008). Research over the past decade has demonstrated that language describing familiar actions results in activation of motor systems (e.g., Pulvermuller, 2005; Kemmerer et al., 2008). However, despite the broad and high-profile theoretical claims made in the literature about language understanding and sensorimotor systems, the necessity of such recruitment has not been

#### Edited by:

Ferdinand Binkofski, RWTH Aachen University, Germany

#### Reviewed by:

Jie Yang, Macquarie University, Australia Juliane Klann, SRH Hochschule für Gesundheit Gera, Germany

> \*Correspondence: Carys Evans carysemilyevans@gmail.com

Received: 05 September 2015 Accepted: 17 January 2017 Published: 14 February 2017

#### Citation:

Taylor LJ, Evans C, Greer J, Senior C, Coventry KR and Ietswaart M (2017) Dissociation between Semantic Representations for Motion and Action Verbs: Evidence from Patients with Left Hemisphere Lesions. Front. Hum. Neurosci. 11:35. doi: 10.3389/fnhum.2017.00035 firmly established. For example, the effects found might be merely epiphenomenal or the case may be that "sensory and motor information plays, at best, a supportive but not necessary role in representing concepts" (Mahon and Caramazza, 2008, p. 67). This debate has led others to propose a middle ground—that relying on both "embodied" and "symbolic" mechanisms provides language users with richer and more fault-tolerant representations (Andrews et al., 2009; Dove, 2009; Taylor and Zwaan, 2013). What would clarify this debate however, is evidence to suggest that "embodied" and "symbolic" representations dissociate, and also that varying "perceptual" brain regions may be implicated even within a semantic category. Indeed verbs do not always refer to concrete, dynamic actions; verbs can also refer to events involving movement, mental states, and can state a change. A raindrop might fall to earth and a flower might wilt, resulting in visual motion, but we cannot directly realize such events with our bodies as we might when we hit (a concrete, dynamic action; as described in **Table 1** labeled +Action/+Motion verbs) or hold an object (a motionless action; as described in **Table 1** labeled as the +Action/−Motion Category in our research design).

Brain imaging and behavioral studies alone provide limited information about the relationship between cognitive processes: motor system activation may be a consequence or correlate of comprehension, not a cause (see e.g., David and Senior, 2000 for further debate). Additional persuasive evidence comes from patients and participants with lesions affecting the brain's motor system who show a specific impairment for action knowledge; a trend that has been demonstrated for Motor Neurone Disease, Parkinson's Disease, and stroke (Neininger and Pulvermüller, 2003; Bak et al., 2006; Arevalo et al., 2007; Kalenine et al., 2010). While analogous evidence from healthy participants has been previously demonstrated in the literature with Transcranial Magnetic Stimulation (TMS), the effects found have been inconsistent (see Pulvermüller et al., 2005; Willems et al., 2010). We note here that while some participants with motor lesions do not show such deficits on action verbs (Papeo et al., 2010; Arévalo et al., 2012; Kemmerer et al., 2012; Maieron et al., 2013), none of these studies compare verbs with and without motion components, a contrast investigated as part of this study.


Patients with lesions involving posterior parietal and lateral occipitotemporal cortex are predicted to be impaired on processing words representing Observable events but should perform normally on Motionless action words. In contrast, patients with more anterior lesions sparing posterior parietal and lateral occipitotemporal cortex are predicted to be impaired on processing words representing Motionless actions but should perform normally on processing Observable event words. Impairments on Concrete, dynamic action can arise from either lesion location because the verb refers both to motion and action content. No prediction is made about processing verbs referring to Mental states.

It has been found that visual motion features of verb meanings recruit the posterior parietal area pSTS (for reviews see Gennari, 2012; Watson et al., 2013), but also the middle temporal area of the visual cortex (known in the literature as MT/V5 or Brodmann area 19, noteworthy for its high responsiveness to visually dynamic stimuli and relatively low retinotopy; Grill-Spector and Malach, 2004). We have previously shown MT/V5 to be involved in tasks that merely imply visual motion, such as the perception of static images depicting dynamic motion (e.g., an athlete about to kick a football; Senior et al., 2002) and other studies have revealed that it is also involved during reading tasks that contain the description of motion (e.g., "the car drives toward you"; Rueschemeyer et al., 2010), with MT activation when viewing static images also mediated by the language immediately preceding it (Coventry et al., 2013). Crucially, these studies indicate that visual motion must be strongly implied in order to activate MT/V5. No studies have yet shown this for individual words nor, as noted earlier, have necessary and sufficient conditions for its involvement in the computation of language that describes motion been investigated. Further, previous work examining verbs typically confounds the semantic components of deliberate action and visual motion. Many of these studies use goal-directed actions when examining the recruitment of visual motion areas, and do not disentangle action from motion. Therefore, recruitment of visual motion areas may be contingent upon the verb containing an additional goal-directed action component.

Lateral occipitotemporal cortex (which includes MT) is associated with patterns of motion, motion related artifacts such as tools and depictions of hands (Bracci et al., 2010, 2012) and verbal material referring to actions symbolically (for a review see Lingnau and Downing, 2015). Bedny et al. (2008) are generally cited as having shown that the activation in lateral occipitotemporal cortex associated with verbs is not due to visual motion or motor activity. That fMRI study by Bedny and colleagues contrasted high-motion verbs (concrete dynamic actions such as "jump"), intermediate-motion verbs (changeof-state and bodily function) and low-motion verbs (states), showing that the amount of motion did not modulate activation of the lateral occipitotemporal cortex. However their low-motion verbs were states such as mental states and did not refer to agentive motionless actions such as "holding" or "clutching" which may indeed activate regions much more anterior to lateral occipitotemporal cortex (Kemmerer et al., 2012). Furthermore, high-motion verbs in the study by Bedny and colleagues were confounded with action, while a neater confirmation of motion associated verb activation in lateral occipitotemporal cortex would be verbs involving visual motion but not deliberate action i.e., observable events such as "crumbling" or "flowing." In a later fMRI study by Peelen et al. (2012) showing that lateral occipitotemporal cortex is activated by state verbs (including mental states) and event verbs, the event verb category did not refer to observable events but again included concrete dynamic actions such as walking and running. Unlike previous studies, the current study delineates the action and motion element completely. The behavioral performance of patients who have sustained lesions in the left hemisphere is uniquely placed to

inform our understanding of language processing by addressing this central issue.

Although lesion studies are not suitable to investigate discrete areas such as MT or pSTS, if we can show that defective motion processing is selectively associated with the posterior part of the brain housing MT and pSTS such as Brodmann area 19 or area 39, in contrast to the more anterior brain sparing those regions, we can infer that neuro-anatomically dissociable regions are activated when processing action or motion verbs, and that recruitment of these regions is necessary to derive meaning when processing modality-specific semantic knowledge. A second issue with respect to the possible links between language and recruitment of distinct neural correlates concerns the nature of the tasks used to test these links. "Levels of processing" (Craik and Tulving, 1975) refers to the degree to which a participant recruits semantic knowledge; it constitutes the qualitative difference between, for example, counting the vowels in "sinking" and knowing that "sinking" and "plunging" are more similar than "flowing" and "plunging." Reviews (Taylor and Zwaan, 2009, 2013; Tomasino and Rumiati, 2013) find that the type of language task is a critical factor when determining the recruitment of specific brain regions. For example, semantic decisions ("Is GRASP an action?") affect hand movements while lexical decisions ("Is GRASP a word?") typically do not. This difference in the recruitment of alternative neural networks as a function of task requirements accounts for discrepancies within both behavioral (Lindemann et al., 2006; Sato et al., 2008) and neuroimaging paradigms (Kemmerer et al., 2008; Postle et al., 2008). In each case, a lexical, word-based decision does not result in activation of dissociable processes while a more cognitively demanding semantic task does suggesting that recruitment of neuro-anatomically dissociable regions is only necessary when recruiting semantic representations but not when making lexical decisions that do not rely on semantic information.

In our current design we accounted for these two critical issues by using tasks varying in semantic demand and words that entirely delineate the action and motion element. Firstly, to account for discrepancies in the data regarding recruitment of specific brain regions we included three tasks with different levels of cognitive demand. Our critical Semantic Similarity Judgement Task (SSJT) was expected to indicate any dissociation in action/motion verb processing in patients; as the most cognitively demanding semantic task it was considered most sensitive in identifying these dissociations. An additional Verb-Picture Matching (VPM) task was administered; easier than the SSJT but also reliant on semantic processing it was included to support the SSJT in cases of more severe stroke. Both the SSJT and VPM do not present words in isolation, but instead require comparisons to be made between two verb stimuli. A final Lexical Decision task required classification of a linguistic stimulus as a word, and was expected to rely on inherently more superficial processes that would not require the activation of dissociable processes.

Secondly, we delineated the action and motion element completely (see **Table 1**). As highlighted above verb content varies with some describing action (hitting), some not (desiring) while others describe motion (falling) and others not (holding). In the current fully factorial design, four verb types were used to assess the behavioral and neural independence of action and motion word processing. Verbs contained elements of action and motion (concrete, dynamic actions; "throwing"), action without motion (motionless actions; "holding"), motion without action (observable events; "flowing"), and neither action or motion (mental states; "hoping"). In doing so, the necessity of dissociable and neuro-anatomically separate regions during action and/or motion processing can be wholly explored.

Whilst the current study is not well placed to assess the critical role of the specific brain regions required when processing particular verbs due to diffuse lesion patterns and a sample size that does not allow voxel based lesion analysis, it can certainly confirm the importance of neural correlates. It is predicted that distinctive brain areas are recruited most reliably when a person accesses the relevant semantic dimension. If recruitment of additional brain areas is necessary when representing concepts, then damage to these areas may result in impaired processing of action and/or motion verbs. It is furthermore predicted that the expected dissociations will be evident in the more cognitively demanding semantic tasks but not in a lexical decision task. Finally, although included to maintain a fully factorial design, we do not make predictions about the performance of patients when processing mental state verbs, as these do not include an action or motion element.

# MATERIALS AND METHODS

# Participants and Lesion Location Patients

For this multiple single-case study patients were recruited from UK National Health Hospitals/Stroke rehabilitation units located in the North East of England. Hospital admissions were screened to select patients with CT evidence of a recent ischaemic infarct or haemorrhagic stroke involving the left hemisphere. Anyone with cognitive impairment (identified from hospital screening procedures e.g., Mini Mental State Examination; MMSE), known dementia, or reported substance abuse were excluded. Patients for whom significant comprehension problems were noted in the hospital notes by clinicians or speech and language therapists beyond the acute phase of stroke were not approached because they would not cope with the tasks in this study. At test, language comprehension was further evaluated through use of the Token Task and Mississippi Aphasia Screening Test (MAST) to ensure patients could complete the experimental tasks. These tests are described below in the Screening and Patient Documentation section baed on these. Based on this criteria 25 participants were initially recruited as in-patients however 17 participants could not be followed up after discharge or did not complete all of the experimental tasks of this study.

Finally, based on the radiologist's clinical CT or MRI report we identified patients with lesions implicating either the anterior or posterior portion of the left hemisphere. Using scan images we could reliably classify six out of 8 patients. One patient was excluded because he had lacunar infarct to the left internal capsule that did not fit either anterior or posterior pattern. A second patient (patient CC) had some early signs of left hemisphere low attenuation in an otherwise nonspecific scan not allowing for classification or later lesion analysis. She had furthermore no behavioral deficits indicating a particular lesion site. She was included in the testing nevertheless as an unclassified patient and her normal performance across the experimental tasks is documented in **Table 3**. Thus the individual results of six left hemisphere patients are reported in detail in this study (3 Female, age range 52–75 years, mean 68 years 10 months, SD = 8 years 6 months,). Patients were seen at a mean time of 45.71 days (SD 13.97) post stroke. All were able to provide informed consent.

Details of each patient's lesion as identified in the CT and/or MRI reports are described below. **Table 2** also lists the Brodmann areas implicated in each patient. To determine which Brodmann areas were damaged, each patient's lesions were mapped onto the digital brain image on the basis of the radiologist's report using MRIcron software package (Rorden et al., 2007; http:// www.mccauslandcenter.sc.edu/mricro/mricron/). Scans were normalized (using Clinical Tool box software through SPM; Rorden et al., 2012; http://www.nitrc.org/plugins/mwiki/index. php/clinicaltbx:MainPage) and applied to the Brodmann Atlas included in MRIcron. **Figure 1** includes overlaid scan slides of each patient. On the basis of scan information 3 patients (patients TY, MAS, and SB) were firmly classified as having more anterior lesions sparing the posterior parietal and lateral occipitotemporal regions of interest for motion verbs. Critically, 2 patients (patients FR and JC) had lesions involving the posterior regions of interest for motion verbs. FR had infarcts involving the left internal capsule and an old left parietooccipital lesion. JC also had lesions to the parietooccipital and lateral occipitotemporal cortex. In contrast TY had a frontal infarction that was restricted to inferior frontal and orbitofrontal territory and rostral superior and middle temporal gyrus. SB had a bleed limited to the frontal lobe. Patient MAS's lesion pattern is associated with small vessel disease affecting periventricular white matter, left temporal lobe, and left internal capsule as noted in the clinical report. As such disconnection, potentially affecting the semantic network, is probable. The multiple ill-defined white matter lesions were mostly unsuitable for mapping. However a cortical anterior lesion and small non-cortical white matter posterior lesion were identified. Furthermore, based on her symptoms of motor weakness and expressive aphasia coupled with the implication of more anterior cortical areas (BA 2, 3, 4, 8, and 40) this patient for the purpose of this study was classified as an anterior patient. In relation to the research question this is justified because the lesions in this patient spared posterior parietal and lateral occipitotemporal cortex hypothesized as associated with motion comprehension. One patient (patient DH) had an extensive lesion involving both anterior and posterior parts of the left hemisphere (left frontotemporoparietal and insula) and we therefore would not expect a dissociative pattern of impairments for processing action or motion verbs in this patient. However given that DH's lesion implicated both anterior and posterior cortical areas we felt his behavior was still relevant to the hypotheses.


FIGURE 1 | Overlaid scan slices of each patient applied to a template scan to allow clear visualization of the anatomical landmarks using MRIcron software package (Rorden et al., 2007; http://www.mccauslandcenter.sc.edu/mricro/). Clinical scans could not be obtained for patient SB; the scan for DF was performed too early for the lesion to be accurately localized. Left is right as per neurological convention.

# Healthy Controls

A control group of 15 healthy older adults aged 63–84 years (mean 71 years 8 months, SD 6 years 2 months, 9 female) were recruited from a database of older adults held in the Department of Psychology, Northumbria University. Control participants were right handed (as were patients), and had not sustained any form of stroke or other form of brain damage. The control group received £3.00 for their participation. All procedures were approved by the local Ethics Committee within the Department of Psychology, Northumbria University as well as NHS research ethics.

# Method and Procedure

Verb content varies—some involve action (hitting), some not (desiring) and some involve movement (falling), some not (holding). Because of their versatility, verbs afford firm control over semantic content and linguistic factors while tapping into different, but experimentally predictable, resources (see **Table 1**). The design of the current study allows an investigation of the neural systems to be involved in language comprehension. This pushes for novelty in two ways: By investigating across semantic dimensions and levels of processing.

In line with the depth-contingent processing hypothesis outlined in the introduction, we predict that non-dedicated brain areas are recruited most reliably when a person accesses the relevant semantic dimensions. Hence, anterior lesions will consistently interfere with semantic decisions on verbs describing motionless actions (A+/M−) and posterior lesions will interfere with semantic decisions on verbs describing observable events (A−/M+) only. Crucially, the more cognitively demanding semantic tasks outlined below (Semantic Similarity Judgment Task and Verb-Picture Matching; SSJT and VPM, respectively) do not present words in isolation, but in more meaningful contexts requiring comparisons to be made between stimuli; further, lexical decision merely requires classification of a linguistic stimulus as a word, while the semantic tasks require comparison. Each of these changes enhances the depth of semantic processing. We therefore predict effects in the more cognitively demanding tasks (SSJT and VPM), which rely more heavily on semantic processing, and not in the less cognitively demanding task, which relies on inherently more superficial processes. Further, we expect the SSJT to be more sensitive at identifying dissociations in verb processing (due to recruitment of non-dedicated brain regions) as it is more cognitively

demanding than the VPM. In more severe stroke however, we expect the VPM to add insight into SSJT performance.

# Screening and Patient Documentation

# **Mississippi aphasia screening test (MAST)**

As the participants had suffered damage to the left hemisphere, language and communication skills were assessed using the Mississippi Aphasia Screening Test (MAST; Nakase-Thompson, 2004). The MAST contains nine subtests ranging from 1 to 10 items and provides indices of receptive and expressive aphasia. There was a maximum score of 50 points for each of the receptive and expressive aphasia indices which are noted for each of the patients in **Table 2**.

# **The token test**

The general severity of any receptive aphasia was also assessed using the short version of the token test for language comprehension (De Renzi and Faglioni, 1978). As indicated in **Table 2**, all patients successfully followed commands consisting of at least five stages.

# **Symptoms of apraxia and neglect**

A standard battery of apraxia screening tests was administered to document symptoms of apraxia. These included imitation of hand and finger gestures (Goldenberg, 1996), whereby the patient was required to copy a series of gestures that were demonstrated by the experimenter (pathological score ≤ 17/20 on either task), and pantomime (Goldenberg et al., 2007) and actual use (De Renzi and Lucchelli, 1988) of common objects (pathological score ≤ 43/53 and ≤16/18 for respective tasks); the examiner named the object-use action and patients were marked on the presence or absence of predefined movement features. Based on the overall performance accuracy across all apraxic screening tests, the severity of apraxia was calculated. All patients were no less than 90 percent accurate across the screen except for patient MAS who was 85% accurate. Errors in patient MAS's performance was apparent during the imitation of hand gestures (scoring 17/20) and in the form of body-partas-object errors during object-use pantomime (scoring 31/53). Pathological scores were also noted for FR during the imitation of finger gestures (17/20) and DH during hand gesture imitation (15/20). Remaining patients did not obtain a pathological score during apraxia screening. Visuospatial neglect was assessed using the Apples Test (Bickerton et al., 2011) and is reported in **Table 2**. All the above standard neuropsychological tests were examined within days of the experimental assessment.

# **Object recognition screening task**

Word stimuli were presented in preparation for the experimental session to establish that basic processing of written words and pictures were intact. For this task, participants were presented with a written one-word exemplar (uppercase, Arial font, size 72) and asked to read but not verbalize or attempt to verbalize the presented word. When the participant confirmed they had read the word, they were presented with the pictorial representation of the word amongst three distractors that belonged to the same semantic category. For example, circle (target), rectangle, triangle, and square (distractors). Participants had to identify which one of the four images they believed was a representation of the target word. This procedure was followed for four targets from different semantic categories: an animal (rabbit), fruit (lemon), object (clock), and shape (circle). The pictorial target and distractor stimuli for each semantic category were printed in color onto one A4 laminated sheet. The four exemplars of the aforementioned semantic categories were selected from the Snodgrass and Vanderwart (1980) set of images. None of the patients had difficulty with either of these screening tasks.

# Experimental Tasks

# **Word stimuli used in the lexical task and semantic similarity judgement task (SSJT)**

Common English words (between 4 and 7 letters in length) were selected and the suffix "ing" added to disambiguate all words as verbs. Each word was allocated to one of the four conditions (see **Table 1**). Four independent assessors were provided with all verbs and the operationalized definitions of each condition, and rated whether they agreed (Yes/No response) to each verb/condition pairing. Only the verbs that reached a majority agreement by at least three of the four assessors were retained. A Google search of hits for each verb was used to obtain the frequency of use in the English language. Selected items were matched for letter length, number of syllables, and frequency (details are given in Appendix A).

In addition to the use of independent assessors, we also examined available linguistic resources to extract information regarding imageability and concreteness for individual verbs (Wilson, 1988; Bird et al., 2001), and existing classifications of verbs where relevant (e.g., Levin, 1993). From these resources we constructed a more limited list of verbs for final analysis: the full list and the reduced list are in Appendix (A). The reported analyses are based on the items in bold only. Of course the word lists are supposed to differ in their ratings on some of these dimensions (e.g., a +action verb is clearly more concrete and imageable than a −action verb).

To construct the stimuli for the SSJT—a task successfully implemented in previous research both in neuroimaging and clinical populations (Kemmerer et al., 2008; Fernandino et al., 2013) - each word from the final list, referred to as the "pivot," was matched with a word of similar meaning (target), and a distractor word. Both the target and distractor were taken from the same semantic category as the pivot. Note that distractors are consistently, but only moderately, different from pivots and targets; this requires participants to think carefully about subtleties in the meanings of all three words in order to successfully complete the task. An additional four independent raters confirmed that the target/pivot items were more similar in meaning compared with the distractor/pivot items (see Appendix B for an exhaustive list of pivots, targets, and distractors).

#### **Non-word stimuli used in the lexical task**

A list of 52 non-words was obtained from the ARC Nonword Database (http://www.maccs.mq.edu.au/~nwdb/nwdb. html). These followed the same letter-length criteria as the word stimuli and were converted into verbs as described above. Thirteen non-words were allocated to each of the four conditions, and matched with the corresponding UK English verbs for letter-length and number of syllables. Each non-word was novel with no repetitions across the four categories (see Appendix C).

**Picture stimuli used in the verb-picture matching task (VPM)**

Two pictorial representations of each of the 52 English verbs used for the word stimuli were created. A search on Google Images identified photographic representations of each verb. An additional four independent assessors rated how closely each image represented its associated verb. An image was allocated as the target pictorial representation of each verb if a majority agreement of 1st choice was reached by at least three of the four assessors. The 52 images rated as 2nd choice were retained as distractor images. Each of the 52 target images were randomly paired with a distractor image from the same condition (i.e., the four conditions outlined in **Table 1**).

# Procedure

All participants provided written informed consent and were tested either in hospital/rehabilitation unit, or at their own homes or university premises if they were healthy controls. Testing was completed over two or three sessions depending on how many tasks the participant could complete at each visit. All tasks were administered in a fixed order as below. The computerized tasks were presented to the participants using a Toshiba laptop with a 12 inch screen, and programmed using Eprime2. Participants were asked to identify the target by either stating this verbally or pointing to their choice. The participants' response was recorded by the experimenter using either a left or right mouse click. A 4 trial practice session was administered to ensure the participants understood the task instructions. If necessary this was repeated until the participant demonstrated they fully understood the task requirements. There was no maximum time limit and each set of stimuli was interspersed by a blank screen of no fixed duration to enable the participants to have a rest at any time they needed.

### **Lexical decision task**

The participants were presented with two words on screen; one real word and one non-word. They were asked to identify which was the real word. This task is illustrated in **Figure 2**. Control participants were not assessed on this basic task.

# **Semantic similarity judgement task (SSJT)**

The participants were advised that they would see one word in red colored text (pivot) at the top of the screen. Underneath they would see two words (target and distractor) in black text. They were instructed to choose which one of the two words in black text was most similar to the word in red. Instructions stating "Which of the two words below is most similar to the word above" were also presented on screen below the pivot. The pivot word was presented centrally in the upper third of the screen. The target and distractor words were presented centrally (vertically) and equidistant (horizontally) from the center of the screen (see **Figure 2**). The presentation of the target word on the left/right of the screen was counterbalanced across all trials.

# **Verb-picture matching task (VPM)**

The stimuli consisted of one pivot word (as described in the word stimuli section) and the two pictorial representations (one target and one distractor as described in the picture stimuli section). The pivot word was presented centrally in the upper third of the screen. The target and distractor images were presented centrally (vertically) and equidistant (horizontally) from the center of the screen. As above the participants were advised that they would see one word in black colored text at the top of the screen. Underneath they would see two images. They were instructed to identify which one of the two images was most similar to the word above. Instructions stating "Which picture best matches the following word" were also presented on screen above the pivot. Order of presentation of the target on the left and right of the screen was counterbalanced.

# Data Analysis

The data from 6 patients were included in the analyses. In order to explore the variance in individuals' performance in greater depth, a multiple single-case approach was adopted. The patients' task performance on the experimental tasks was compared to that of the healthy control group using modified t-tests (Crawford and Garthwaite, 2002), a standard statistical analysis which enables significance testing of individual scores compared with a control group. This method has been shown to be robust when comparing single-cases to a small control sample even in instances where such a sample is not normally distributed (Crawford et al., 2006). All patients completed the lexical, SSJT, and VPM tasks and where possible patients were retested on the critical SSJT task to confirm the pattern of results; whilst the

VPM was useful for adding clarity to noisy data in cases of severe stroke, the more cognitively demanding SSJT was believed to be most reliant on the activation of semantic processes when making action/motion decisions. Retest took place 3 months after initial testing on the task (on average across patients retest took place 14 weeks and 3 days after initial testing). It was not possible to retest two of the 6 patients (patient MAS and SB) as they were not reachable after discharge. The scores on SSJT in **Table 3** are those at first testing, and any changes at retest are accounted for in text where available for individual patients.

# RESULTS

Overall, the patients demonstrated dissociable deficits for action or motion verbs depending on lesion location. Inspection of the combined averaged percentage correct from initial and retest of the SSJT task (see individual results for details of duration between test/retest) for each condition identified patients with more anterior lesions sparing posterior parietal and lateral occipitotemporal cortex (TY, MAS, SB) making more errors in the motionless action (+Action/−Motion) condition (t = −3.631, p = 0.001) whilst the patients with lesions involving posterior parietal or lateral occipitotemporal cortex (FR & JC) made significantly more errors in the observable event (−Action/+Motion) condition (t = −3.631, p = 0.001).

To explore a dissociation of semantic representations for action and motion specific verbs, differences in performance on the semantic tasks (SSJT & VPM) were compared between individual patient scores and the normative data from the healthy control participants (see **Table 3**). The performance of patients classed as having anterior lesions are initially discussed followed by those classed as having posterior lesions.

Analysis of the results from initial testing of the Semantic Similarity Judgement Task (SSJT) confirmed that patients with more anterior lesions sparing posterior parietal and lateral occipitotemporal cortex (TY, MAS, SB) showed significantly impaired performance in the motionless action (+Action/−Motion) condition compared to control participants, suggesting a deficit in action comprehension, while performing normally on the observable event (−Action/+Motion) condition. Individual patient performance is as follows:

# Patient TY

# Expect Impaired Processing of Action Verbs **Lesion and deficits**

TY had a frontal infarct implicating BA 47, 11, and 38; presented with aphasia and motor weakness on admission; at test he had no symptoms of expressive or receptive aphasia and no symptoms of visual neglect or apraxia.

# **SSJT**

A robust deficit was observed for processing motionless action (+Action/−Motion) items of the SSJT task at initial and retest (11 weeks, 4 days later) when compared to the control group (both t = −7.746, p < 0.001). TY was significantly impaired in the mental states (−Action/−Motion) category compared to control participants in both SSJT testing sessions (both t = −4.150, p < 0.001). TY performed at ceiling on the observable event (−Action/+Motion) condition at initial testing (t = 0.415, p = 0.342). TY was also unimpaired in the +Action/+Motion condition, performing better than controls on both test and retest sessions in this condition (both t = 2.324, p = 0.018). Of note, at retest TY's performance was impaired in the −Action/+Motion condition (t = −7.746, p < 0.001). This is difficult to interpret, but is not considered indicative of a motion processing impairment given his perfect performance in this condition in the VPM and at initial SSJT test.

#### **VPM**

TY's performance was at ceiling for the two critical conditions (+Action/−Motion and −Action/+Motion) as well as on +Action/+Motion (t = 0.00, p = ns) and comparable to

TABLE 3 | Patient percentage correct for the semantic tasks on the SSJT at initial testing, the VPM, and the Lexical Decision task.


Dark shaded areas in the table highlight the expected pattern of impairments, and light shaded areas highlight the expected dissociating intact performance.\*p < 0.05; \*\*p < 0.001; <sup>1</sup>Patient performance better than control group.

<sup>a</sup>Unclassified lesion (patient scan too early to identify lesion).

<sup>b</sup>More anterior lesions sparing posterior parietal and lateral occipitotemporal cortex.

<sup>c</sup>Lesions involving posterior parietal and/or lateral occipitotemporal cortex.

<sup>d</sup>Widespread left hemisphere lesion including both posterior and more anterior regions of interest.

controls on the mental state (−Action/−Motion) condition (t = −0.645, p = ns).

#### **Interpretation**

Performance at ceiling during the VPM does not allow interpretation, but based on SSJT performance it can be concluded that TY's performance on the initial and retest of the SSJT suggest a robust deficit specific to motionless actions (+Action/−Motion), in keeping with what was predicted on the basis of this patients frontal lobe infarction, sparing posterior parietal and lateral occipitotemporal cortex associated with motion comprehension.

# Patient MAS

# Expect Impaired Processing of Action Verbs

### **Lesion and deficits**

Lesion implicated periventricular white matter, left temporal lobe, left internal capsule (BA 2, 3, 4, 8, 40); presented with aphasia and motor weakness on admission; at test she had no symptoms of neglect but demonstrated expressive aphasia and mild apraxic symptoms.

#### **SSJT**

Compared to controls, MAS showed a distinct impairment in the motionless action (+Action/−Motion) condition: t = −7.746, p < 0.001; performance on remaining verb conditions were comparable to controls (see **Table 3**). Patient MAS' performance was at ceiling on the observable event (−Action/+Motion) condition: t = 0.415, p = 0.342, and mental state (−Action/−Motion) condition: t = 0.415 p = 0.342, and comparable to controls in the concrete, dynamic action (+Action/+Motion) condition: t = 0.194, p = 0.425.

# **VPM**

MAS' performance was at ceiling for the two critical conditions (+Action/−Motion and −Action/+Motion) as well as on the concrete, dynamic action (+Action/+Motion, t = 0.00, p = ns) and comparable to controls on the mental state condition (−Action/−Motion t = −0.645, p = ns).

# **Interpretation**

In conclusion, based on highly selective impairment in the critical motionless action condition of the SSJT task this patient's performance, like the above patient, is in keeping with what was predicted on the basis of this patient's more anterior lesion. Based on her post-stroke behavioral impairments and her lesion data, it is possible that disconnection, potentially affecting the semantic network, has occurred in this patient. Posterior parietal and lateral occipitotemporal cortex associated with motion comprehension are however spared.

# Patient SB

#### Expect Impaired Processing of Action Verbs

### **Lesion and deficits**

SB had a frontal bleed; aphasia was observed on admission, with no symptoms of motor weakness; at test, SB showed no symptoms of aphasia or apraxia but demonstrated left allocentric neglect.

# **SSJT**

SB performed poorly in the critical motionless action (+Action/−Motion) condition (t = −3.873, p = 0.001). Performance in the concrete, dynamic action (+Action/+Motion) condition was also lower than controls (t = −1.936, p = 0.037). Performance was comparable to controls in the observable event (−Action/+Motion) condition (t = 0.415, p = 0.342). There was no difference between SB and the control group's performance in the mental state (−Action/−Motion) condition (t = 0.415, p = 0.342).

# **VPM**

Consistent with the SSJT, SB performed worse than controls in the motionless action (+Action/−Motion) condition (t = −16.460, p < 0.001) and the concrete, dynamic action (+Action/+Motion) condition (t = −21.301-14.254, p < 0.001). Unlike the SSJT, SB was significantly impaired in the mental state (−Action/−Motion) condition (t = −2.259, p = 0.002). Performance was comparable to controls in the observable event (−Action/+Motion) condition (t = 0.00, p = ns).

# **Interpretation**

Although SB was impaired on a number of verb conditions, the dissociation between impaired motionless action (+Action/−Motion) comprehension and intact comprehension of observable events (−Action/+Motion) was clearly evident based on the combined SSJT and VPM performance in this patient. This was predicted based on the frontal bleed sparing posterior parietal and lateral occipitotemporal cortex.

# Patient DH

# Expect Impairment in Processing Either/Both Action/Motion Verbs

#### **Lesion and deficits**

DH suffered a significant stroke leaving him quite impaired; aphasia and right motor weakness were noted on admission and at test DH had severe expressive aphasia, but no visual neglect or apraxia. His clinical scan was performed very early on; too early to reliably localize the lesion. Based on the radiologist's report describing a lesion in the left fronto-temporo-parietal infarct and insula and his disfluent speech indicative of a frontal lesion, DH was classed as both anterior and posterior. It was therefore predicted that this patient would not present a neat dissociation in verb processing performance. This wide-spread damage also seems to be reflected in his non-specific behavior on the experimental tasks.

#### **SSJT**

DH performed poorly across this task on initial test and retest, which may be attributable to the severity of his stroke. At both initial and retest, DH was significantly less accurate across all conditions compared to the control group (all p ≤ 0.037). Initial testing did not reveal a clear pattern of behavior (see **Table 3**); DH showed the most notable deficit in the observable event (−Action/+Motion) condition (t = −11.066, p < 0.001) followed by the concrete, dynamic action (+Action/+Motion) condition (t = −10.651, p < 0.001). At retest and still significantly impaired compared to the controls, DH's performance improved in both the observable event (−Action/+Motion) and concrete, dynamic action (+Action/+Motion), but fared considerably worse in the motionless action (+Action/−Motion) condition.

## **VPM**

Unlike the SSJT, DH's behavior on the less demanding VPM task showed more specific deficits. Compared to controls, DH's performance was significantly poorer in the motionless action (+Action/−Motion) condition (t = −16.460, p < 0.001) as well as on and the observable event (−Action/+Motion) condition (t = −16.460, p = < 0.001). In contrast performance was normal on concrete dynamic action (+Action/+Motion; t = 0.00, p ns) and in the mental state (−Action/−Motion; t = −0.645, p = 0.265) condition.

# **Interpretation**

Although the pattern of results with this patient is somewhat clouded by a general level of impairment (i.e., performing poorly across many conditions on the more demanding SSJT task) it is interesting that this patient on the VMP was impaired only on the two critical experimental conditions, observable events associated with posterior damage and motionless actions associated with more anterior damage, while managing normal performance on the other two conditions of the VPM task, concrete dynamic action and mental states. In conclusion, this patient showed the non-selective pattern of behavior predicted by his lesion involving both areas of interest.

# Patient FR

# Expect Impairment in Processing Motion Verbs **Lesion and deficits**

Lesion implicated the left internal capsule and left parietooccipital region (BA 40, 41 4, 21, 39, 42); aphasia on admission without right motor weakness; at test FR had no symptoms of aphasia, neglect, or apraxia.

### **SSJT**

FR showed poor performance in the critical observable event (−Action/+Motion) condition at initial test (t = −1.936, p = 0.037) and retest (t = −4.150, p < 0.001) 21 weeks 6 days later, suggesting a robust motion deficit (see **Table 3**). Performance on the mental state (−Action/−Motion) condition at initial testing (t = −4.150, p < 0.001) and retest (t = −1.936, p = 0.037) was significantly poorer than controls. Normal performance was however observed in the motionless action (+Action/−Motion; t = 0.242, p = 0.406) and the concrete, dynamic action (+Action/+Motion; t = 0.194, p = 0.425) conditions compared with controls.

### **VPM**

FR's performance was comparable to controls across conditions (all p ≥ 0.265), performing largely at ceiling. This may be indicative of his mild stroke.

# **Interpretation**

A distinct –Action/+Motion deficit with maintained +Action/−Motion and +Action/+Motion performance in the SSJT suggests that FR presented with an isolated deficit in the comprehension of motion verbs in line with a lesion involving posterior parietal cortex.

# Patient JC

# Expect Impairment in Processing Motion Verbs **Lesion and deficits**

Parieto-occipital infarct implicating BA 39, 6, 7, 19, 40; aphasia, right motor weakness and right superior quadrantanopia on admission; at test showed mild expressive aphasia but no symptoms of apraxia.

### **SSJT**

JC demonstrated a reliable motion deficit for observable event (−Action/+Motion) at initial test (t = −6.501, p < 0.001) and retest (t = −4.150, p < 0.001) 11 weeks 4 days later. Impaired performance was also observed at initial test and retest in the concrete dynamic action (+Action/+Motion): both t = −1.936, p = 0.037, and mental state (−Action/−Motion) condition: both t = −1.936, p = 0.037. JC's performance was equivalent to the control participants at both the initial test and retest in the motionless action (+Action/−Motion) condition (both t = 0.242, p = 0.406).

#### **VPM**

Unlike SSJT, JC performed significantly worse in both the motionless action (+Action/−Motion; t = −24.206, p < 0.001) and mental state (−Action/−Motion; t = −3.066, p = 0.004) conditions compared with the control group. Performance was comparable to controls for the dynamic action (+Action/+Motion) and observable event (−Action/+Motion) conditions (both t = 0.00, p = ns).

#### **Interpretation**

Although the contrast between this patient's performance on the SSJT and VPM tasks introduces an element of uncertainty, it is worth noting that performance on the VPM task was not reflected in other tasks. On the basis of the SSJT task performance at both initial test and retest this patient presented with a dissociation between impaired comprehension of motion associated observable events and intact comprehension of motionless actions, in line with this patient's lesion involving both posterior parietal cortex and lateral occipitotemporal cortex.

## **Lexical Decision Task**

As predicted, the pattern of dissociations was evident on the semantic task, but not the lexical processing task. Patients performed worse than the healthy control participants in the semantic tasks and these deficits were selective across the action present/motion present conditions. Conversely, patients performed accurately in the lexical decision tasks and showed hit rates substantially higher compared to hit rates in the semantic tasks, with patients performing at ceiling or making very few errors (see **Table 3**).

To summarize the pattern of dissociations, patients with more anterior lesions sparing posterior parietal cortex and lateral occipitotemporal cortex (TY, MAS, and SB) were consistently poorer on tasks involving verbs describing motionless actions (+Action/−Motion). On the other hand, patients with lesions involving posterior parietal cortex and lateral occipitotemporal cortex (FR, JC) were consistently poorer on tasks involving verbs describing observable events (−Action/+Motion), while patient DH with a large lesion involving both areas of interest did not show dissociate behavior.

# DISCUSSION

In conditions where verbs contained action and/or motion content, patients with lesions involving posterior parietal and lateral occiptotemporal cortex show a selective deficit on semantic decisions regarding verbs that afford motion. Patients with lesions sparing these posterior regions associated with motion processing showed the opposite pattern of selective deficits in action verb processing but intact motion verb processing. The dissociation between action and motion routes to verb understanding is important. In past studies verbs depicting actions have been considered primarily in relation to motor/premotor activations—but actions depict motions as well as actions. For that reason, the variable results found in past studies may partly be a function of two routes to understanding verbs—action and motion. In the patients we have found dissociations between verbs affording motiononly and verbs affording action-only in cognitively demanding semantic tasks. The opposite pattern of results was seen in patients where posterior regions associated with motion were spared: these patients performed poorly on verbs affording actions but not motion while they performed well on verbs affording action but not motion. Whilst in this small sample we cannot perform detailed lesion analyses, the fact that this selectivity is associated with specific anterior/posterior lesion patterns has implications for most assumptions about action verb understanding, indicating multiple routes to comprehension. This would be consistent with recent work on understanding goals and intentions through actions, with evidence that motor/premotor system activation might be one of several routes to action understanding (Eshuis et al., 2009; Gredebäck and Melinder, 2010).

Most broadly, these results contribute to our understanding of language processing as an integrated phenomenon that involves the contribution of knowledge representation from a wide variety of sensorimotor modalities (Barsalou, 1999; Taylor and Zwaan, 2008), converging with the perspective (Binder and Desai, 2011; Yee et al., 2013) that semantic knowledge is distributed across brain areas corresponding to the sensoryfunctional and sensorimotor characteristics of the referent. In this respect, our findings converge with findings from a variety of methodological approaches demonstrating overlapping neural substrates between language and the motor cortex, including transcranial magnetic stimulation (TMS; Buccino et al., 2005; Pulvermüller et al., 2005), magnetoencephalography (MEG; see Hauk et al., 2008 for review), fMRI (Kemmerer et al., 2012), and behavioral studies (see Glenberg et al., 2013 for a review). Our results most closely relate to those of TMS paradigms, as the temporary "artificial lesions" created in healthy participants in a TMS study are reflected in the natural lesions of our sample of participants, allowing us to draw inferences about the substantive contribution these brain areas make to semantic decisions.

All patients in the current study performed at ceiling level on the lexical decision task, which required identification of a real word against a pronounceable and equivalent non-word distractor (e.g., "praying" vs. "pibbling"). This suggests that a lexical decision does not rely on the recruitment of alternative neural networks. The predicted pattern of dissociations was evident however in the more cognitively demanding semantic tasks. The word-based SSJT task, in which required participants to decide whether "praying" was more similar to "wishing" or "judging," was distinctly affected by the different brain lesions that were revealed by the patients studied here. To a large extent results from the picture-based VPM, which required participants to identify a picture for example of a person praying, mirrored those observed in the SSJT for verbs containing an action and/or motion content. Whilst easier than the word-based SSJT but also reliant on semantic processing, the VPM added clarity to poor performance on the SSJT. In particular, patient DH who had suffered a severe stroke, was consistently poor across conditions of the SSJT but only showed poor performance on the critical conditions of the VPM with normal performance on the neutral conditions. Together, performance across the three tasks emphasizes that recruitment of dissociable neural processes is dependent upon task requirement and cognitive demand, which may explain discrepancies found in previous data (Lindemann et al., 2006; Kemmerer et al., 2008; Postle et al., 2008; Sato et al., 2008).

It is worth noting that while the patients show statistically reliable, specific, and robust deficits in the predicted semantic categories, these selective impairments were remarkably subtle and not a reflection of typical aphasia, with receptive performance on the diagnostic screening for aphasia (MAST) near ceiling level (scoring 48 out of 50 or above) for most of our patients. Similarly, all patients performed near ceiling on the lexical decision task, with aggregate accuracy over 95%. These results promote awareness that language deficits resulting from stroke may be subtler than previously imagined, or assumed by current diagnostic material.

At the same time it should be noted that language is usually studied in cognitive psychology laboratories removed from language in the real world. Seeing the word "STOP" on a red sign at a busy traffic intersection is quite different from seeing the word STOP in black text on a white background in an experimental psychology laboratory and as such laboratory based work may lack the ecological validity required to fully understand the cognitive mechanisms that mediate natural language (e.g., Zwaan, 2009). Thus differing aspects of context, motivation, and task may result in drastically different psychological and neurophysiological responses. The choice of language task has serious implications for the identification of language problems. Cognitively demanding semantic tasks are more useful for identifying more distributed neural networks associated with language processing as lexical decisions may not require the recruitment of dissociable brain regions. Further, one of the hallmarks of language is its contextual versatility—from identification of words to conversations requiring extensive inferences and social comprehension. The latter, more semantically rich, contexts are particularly important to tap in neuropsychological testing, as exactly these tasks recruit more distributed neural networks. The current finding that specific parts of the distributed network give rise to selective impairments resonates with an emerging proposal in the cognitive sciences holding that the brain areas and networks associated with an event are a function of context, task, and strategies, not simply constrained within the domain of a particular stimulus (Tomasino and Rumiati, 2013; Bracci et al., 2016). Indeed it emerges that recruitment of several neural networks may be critical to derive meaning from language.

As predicted semantic representations for concrete, dynamic action verbs may be associated with lesions either related to action or motion processing. Indeed, we did not find the selective association with lesion location that we found for motionless events in posterior patients and observable events not associated with bodily action in patients with more anterior lesions. Perhaps more interesting, we did see impairments on processing verbs representing mental states in a number of patients who were not impaired on some of the other verb categories but as predicted without an associated lesion pattern. Although this leaves open the possibility that semantic content regarding motionless and "actionless" mental states is behaviorally and neurally independent from other verbs, this falls outside the remit of the investigation focussed on the independence of action and motion representation and its relationship to posterior parietal and lateral occipitotemporal cortex. Nevertheless, representations for verbs describing mental events in particular are left unresolved, as in previous work by Peelen et al. (2012) for example, where mental state verbs like "she believes" were mixed in with state verbs such as "she is liked," "he lies down" or "she equates." To what extent do verbs referring to mental states rely on visual and motor systems? Existing theories and results on this are particularly conflicted (Gallese and Lakoff, 2005; Rüschemeyer et al., 2007; Postle et al., 2008; Vigliocco et al., 2009; Dove, 2009). With regard to current results, it is worth highlighting that data coming from patients with such mixed lesion patterns do not generate results that are entirely clear cut, as is often the case with neuropsychological research.

A further inherent weakness of the current study—and potentially an area for improvement in future—concerns the selection criteria for items. First, the observable events category contains a small number of lexical items, placing an artificial constraint on the number of verbs possible in the present study. Second, natural confounds exist between verb classes; for example, observable events should inherently have higher imageability and concreteness ratings than mental events. This may also account for poor performance in verbs representing mental states in some patients. During the SSJT, four of the 6 patients performed significantly worse than controls when processing mental state verbs, which was consistent for two of these patients (SB & JC) in the Verb-Picture Matching task. Control participants also showed a drop in performance in the mental state condition of the VPM compared to other conditions. It is likely that the abstractness of these −Action/−Motion verbs, particularly in pictorial form, is generally more difficult to process, resulting in reduced performance in the mental state condition. Nevertheless, we reiterate that performance during mental state decisions cannot be used to evaluate dissociations when processing verbs involving action or motion and therefore do not discredit our other findings in the remaining stimuli. Third, only four raters assessed our categorization—and even they failed to reach a universal consensus on the full list of items. In the present study, then, we faced an inherent tradeoff between statistical power and experimental validity. In future, perhaps more robust selection criteria—for example, including imageability and concreteness ratings for fewer stimuli that enjoy more universal agreement on category - might shift the balance toward improved methodological rigor at the expense of statistical power.

Establishing whether similar effects can be found in healthy participants with artificially-induced "lesions" is critical to demonstrating that these brain regions are in fact essential to action understanding in healthy populations (Taylor and Zwaan, 2009). However the current study is limited by a small sample size preventing the identification of specific non-dedicated cortical regions being determined. Further study would require a larger sample to enable voxel based lesion analyses to pinpoint the critical role of specific brain regions when processing action/motion verbs. The current results must therefore be considered within the larger context of behavioral and neuroscience research (e.g., Lingnau and Downing, 2015). Most immediately the current experimental design and hypotheses lend themselves to replication, both in other patients and in healthy participants who take part in transcranial magnetic stimulation (TMS) protocols in the way we delineated motion and action dimensions completely. Such results would bolster the claims here, showing that they are neither patient centered artifacts nor a bias of stroke victims more broadly. Note, however, that over time patients may well develop alternative routes to understanding—a point that TMS cannot speak to.

Recent advances in imaging analyses using connectivity analysis will afford investigation of the interplay between action and motion processing regions. Such interplay may allow us to explain when +Action/+Motion verbs are preserved or impaired in patients with specific lesions and furthermore reveal potential differential representation of the interesting Mental States verb category.

Neuroimaging work with healthy participants has identified brain activity mapping onto discrete cortical areas for action, motion, contact, and state change (Kemmerer et al., 2008). Previous neuropsychology research has demonstrated a dissociation between action verbs, which tend to be impaired by anterior lesions, and concrete nouns which are impaired by posterior lesions (Neininger and Pulvermüller, 2003). One of the key contributions of the present work is to elucidate the causality behind these effects and to demonstrate a dissociation

within a lexical category. Future work may consider the causality of such activity and build an account of "abstract" concepts, even if this begins with an account of verbs that are not both concrete and have an immediate sensory or bodily referent.

# AUTHOR CONTRIBUTIONS

All authors were involved in the conception if this study. LT, CE, JG, and MI designed the study. CE and JG collected and analyzed the data. LT, CE, JG, and MI collaboratively drafted the manuscript and all authors approved the final version for submission.

# REFERENCES


# ACKNOWLEDGMENTS

This study was funded by Northumbria University. We would like to thank the individuals who took part in the study, the local NHS hospitals and rehabilitation centers in the North East of England where many were recruited, and in particular the responsible clinicians Dr. David Bruce, Dr. Tim Cassidy, Dr. Akif Gani, and Dr. Chris Price and their teams.

# SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnhum. 2017.00035/full#supplementary-material


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Taylor, Evans, Greer, Senior, Coventry and Ietswaart. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Sinistrals are rarely "right": evidence from tool-affordance processing in visual half-field paradigms

Bartosz Michałowski 1, 2 and Gregory Króliczak <sup>1</sup> \*

<sup>1</sup> Action and Cognition Laboratory, Department of Social Sciences, Institute of Psychology, Adam Mickiewicz University in Poznan, Pozna ´ n, Poland, ´ <sup>2</sup> Faculty of English, Adam Mickiewicz University in Poznan, Pozna ´ n, Poland ´

Although current neuroscience and behavioral studies provide substantial understanding of tool representations (e.g., the processing of tool-related affordances) in the human brain, most of this knowledge is limited to right-handed individuals with typical organization of cognitive and manual skills. Therefore, any insights from these lines of research may be of little value in rehabilitation of patients with atypical laterality of praxis and/or hand dominance. To fill this gap, we tested perceptual processing of man-made objects in 18 healthy left-handers who were likely to show greater incidence of right-sided or bilateral (atypical) lateralization of functions. In the two experiments reported here, participants performed a tool vs. non-tool categorization task. In Experiment 1, target and distracter objects were presented for 200 ms in the left (LVF) or right (RVF) visual field, followed by 200 ms masks. In Experiment 2, the centrally presented targets were preceded by masked primes of 35 ms duration, again presented in the LVF or RVF. Based on results from both studies, i.e., response times (RTs) to correctly discriminated stimuli irrespective of their category, participants were divided into two groups showing privileged processing in either left (N = 9) or right (N = 9) visual field. In Experiment 1, only individuals with RVF advantage showed significantly faster categorization of tools in their dominant visual field, whereas those with LVF advantage revealed merely a trend toward such an effect. In Experiment 2, when targets were preceded by identical primes, the "atypical" group showed significantly facilitated categorization of non-tools, whereas the "typical" group demonstrated a trend toward faster categorization of tools. These results indicate that in subjects with atypically organized cognitive skills, tool-related processes are not just mirror reversed. Thus, our outcomes call for particular caution in neurorehabilitation directed at left-handed individuals.

Keywords: laterality, categorization, priming, tools, left-handers, visual half field

# Introduction

In typical right-handed individuals, the processing of information about tools takes place primarily in their left hemispheres (for reviews, see Johnson-Frey, 2004; Lewis, 2006; see also Orban and Caruana, 2014; Vingerhoets, 2014). Interestingly, in the case of tool-related manual skills, the engagement of left-lateralized processes is apparent even when an interaction with a tool is performed with the non-dominant (left) hand (e.g., Johnson-Frey et al., 2005; Króliczak and Frey, 2009). Whether or not the neural underpinning of tool-use skills in left-handed individuals

#### Edited by:

Antonello Pellicano, RWTH Aachen University, Germany

#### Reviewed by:

Giovanni Ottoboni, University of Bologna, Italy Jorge Oliveira, Universidade Lusófona de Humanidades e Tecnologias, Portugal

#### \*Correspondence:

Gregory Króliczak, Laboratorium Badania Działan i ´ Poznania, Instytut Psychologii UAM, ul. Szamarzewskiego 89B, 60-568 Poznan, Poland ´ krolgreg@amu.edu.pl; krol.greg@gmail.com

> Received: 31 December 2014 Paper pending published: 20 February 2015 Accepted: 10 March 2015 Published: 27 March 2015

#### Citation:

Michałowski B and Króliczak G (2015) Sinistrals are rarely "right": evidence from tool-affordance processing in visual half-field paradigms. Front. Hum. Neurosci. 9:166. doi: 10.3389/fnhum.2015.00166 (sinistrals) exhibits the same asymmetry is currently debated (Vingerhoets et al., 2012; Goldenberg, 2013). Surprisingly, this discussion takes place in the absence of systematic research on representations underlying perceptual processing of tools and other man-made objects in this often-discarded (or rather underrepresented in scientific research) population (for a review on this and other topics, see Willems et al., 2014).

Although both neuropsychological (Goldenberg, 2013) and neuroimaging (Vingerhoets et al., 2012) data from sinistrals, as compared to dextrals (right-handers), point to a less asymmetric organization of functions, it is yet to be determined if such an effect is due to a tendency for all left-handers to have their brains more symmetrically organized or due to a rather higher incidence of atypical representation of functions introducing bias in the group data from this population (for a discussion, see Króliczak, 2013a). Indeed, this is quite likely given the evidence showing that up to 30% of left-handed individuals demonstrate atypical—i.e., bilateral or right-sided—organization of cognitive skills such as language (Knecht et al., 2000), praxis, or both (Króliczak et al., 2011; Vingerhoets et al., 2013; see also Meador et al., 1999). If such a pattern was a reflection of a more general organization of functions in their brains, one would predict that left-handers with atypically organized higher-order manual skills would also exhibit atypical laterality of processing underlying the categorization of tools (cf. Ochipa et al., 1989). Testing for this possibility is paramount because, in the long run, it has a clear potential to reveal handedness-independent interrelations of cognitive functions in the brain, whether typical or not.

The easiest and arguably most effective way of addressing this issue is the use of a visual half-field (VHF) paradigm, which is a reliable measure of hemispheric dominance of functions when used properly (Hunter and Brysbaert, 2008; Verma and Brysbaert, 2011; see also: Garcea et al., 2012; Helon and Króliczak, 2014). In the majority of studies that were related to tool processing, however, the issue of typical and atypical representation of this cognitive skill has never been directly addressed (cf. Verma et al., 2013). Notably, one of the first reports to investigate the laterality of tool representations with the use of VHF paradigm was a paper by Verma and Brysbaert (2011), who tested their right-handed participants on a categorization task with bilaterally presented man-made objects (tools, and non-tools). Yet, the sample they used did not allow them to pose a question of typical vs. atypical processing of the tool category. Therefore, in line with previous studies that drew their conclusions only from right-handers (for a review, see: Lewis, 2006), when averaging across tests and participants, the mere effect they observed was some right visual-field (RVF) advantage for the categorization of man-made objects, including tools. A somewhat stronger effect was observed in a study that utilized a different VHF test, i.e., a lateralized masked priming paradigm, by Garcea et al. (2012), in which participants categorized centrally shown pictures of tools or animals preceded by laterally presented identical or scrambled primes. The priming effect they observed only for tools again indicated the RVF advantage for tool categorization. Given that the majority of subjects involved were right-handed, a chance of finding a subset of individuals with atypically represented tool-processing skills was neither high, nor addressed.

In this study, we investigated the processing of tool-related information exclusively in left-handers, a population offering a higher incidence of individuals with atypically lateralized functions (e.g., Króliczak et al., 2011). We wanted to ensure that the to-be-obtained results would specifically concern tools as a unique type of human artifacts. Therefore, the **tool category** for which the object concept is linked not only to the relevant functional properties of that object type but also to a set of invariant, use-related properties or stable affordances (e.g., the type of grip required when manipulating the tool in accordance with its function, Borghi and Riggio, 2009; see also Tucker and Ellis, 2004; Bub et al., 2015; cf. the micro-affordance concept by Ellis and Tucker, 2000) that trigger the relevant representations of manual skills (e.g., Vainio et al., 2008; Bub et al., 2013)—was contrasted with **other man-made objects** (i.e., non-tools), a wider category of human artifacts for which manipulability is no longer that important but some function is still present. Specifically, we tested: (1) whether or not a difference in visual processing of tools vs. other man-made objects would be observed in accuracy and response times (RTs) in two disparate paradigms utilizing VHF presentations, (2) whether or not the potential left-right asymmetry demonstrated in such experiments would be homogenous across left-handers or, conversely, would allow us to divide the group into two different samples showing advantage for one or the other visual field, and (3) whether or not this pattern of performance would be consistent within a group across the selected behavioral tasks.

We hypothesized that a VHF advantage would be present only for the processing of pictures of tools. Specifically, we expected that our left-handed participants would split into two groups, one showing left visual field (LVF) advantage for tool processing, and the other demonstrating the typical, RVF advantage. Finally, we predicted that the processing of non-tools would be unaffected by the side of presentation (Experiment 1), or the side in which the prime appeared (Experiment 2), irrespective of the group.

# Experiments

Although the order of the two experiments described here one with laterally presented targets (in either VHF), and one with laterally presented primes (in either VHF)—was counterbalanced across participants, for simplicity we will nevertheless refer to the presentation of target objects in VHFs as Experiment 1, and to the presentation of primes in VHFs as Experiment 2. Both experiments were run in Action and Cognition Laboratory in the Institute of Psychology at Adam Mickiewicz University in Poznan, Poland. The study was approved by the local Ethics ´ Committee for Research Involving Human Subjects and was carried out in accordance with the principles of the Helsinki 1964 Declaration.

Eighteen healthy left-handed volunteers (undergraduate or postgraduate students, 9 women, mean age = 23.3, SD = 3.7) took part in Experiment 1 and Experiment 2, and both experiments were undertaken with the understanding and written consent of each participant. All subjects had normal or correctedto-normal visual acuity and, as established by the revised version of the Edinburgh Handedness Inventory (Oldfield, 1971; Dragovic, 2004), were strongly left-handed (mean laterality quotient = − 83.9, SD = 22.1).

Before conducting any analyses we examined whether or not there are any atypical cases among our participants based on their responses to all stimuli presented to the left or right visual field. Consequently, two laterality indices (LI<sup>1</sup> for Experiment 1 and LI<sup>2</sup> for Experiment 2) were calculated for each individual in the following way: LI<sup>1</sup> = [(L<sup>1</sup> − R1)/(L<sup>1</sup> + R1)] × 100, where L<sup>1</sup> and R<sup>1</sup> represent RTs for targets (tools and non-tools) presented in the left (L1) or right (R1) VHF, respectively, and LI<sup>2</sup> = [(L<sup>2</sup> − R2)/(L<sup>2</sup> + R2)] × 100, where L<sup>2</sup> and R<sup>2</sup> represent RTs for targets (tools and non-tools) preceded by identity primes presented, again, in the left (L2) or right (R2) VHF. Each individual's LI<sup>1</sup> and LI<sup>2</sup> were then averaged to form a measure of general visual field dominance, LIG[LI<sup>G</sup> = (LI<sup>1</sup> + LI2)/2]. Participants with LI<sup>G</sup> < 0 were classified as representing left visual-field advantage group (LVF-A, N = 9, 5 women), whereas those with LI<sup>G</sup> > 0 were classified as representing right visualfield advantage group (RVF-A, N = 9, 4 women). Despite different directions of the visual field asymmetries, the groups did not differ from each other in terms of the actual strength of these asymmetries [t(16) = 0.29, p = 0.76] as measured in absolute values.

# Experiment 1: Categorization of Target Objects Presented in LVF or RVF

### Methods

The design of Experiment 1 was based on that used by Verma and Brysbaert (2011) with some modifications.

#### Stimuli

The stimuli consisted of 60 line-drawings of familiar man-made objects (30 tools, 30 non-tools; the list of all pictures can be found in the Appendix 1) from the set of 400 pictures used by Cycowicz et al. (1997). They were downloaded from the website of the Cognitive Electrophysiology Laboratory (CEPL) at the New York State Psychiatric Institute and Columbia University Medical Center (http://nyspi.org/cepl/resources) with the consent of one of the authors. Half of the objects from each category (15 tools and 15 non-tools) were rotated so that the long axis of the object was deflected from the vertical by 45◦ , whereas objects from the other half were rotated in the same manner to obtain a deflection of 315◦ . All images were sized to 140 × 140 pixels.

#### Procedure

Before the experiment proper, participants were familiarized with all the stimuli. Images of tools and non-tools were presented in the middle of the screen on a white background. The name of the object was displayed below the picture; the name of the category—above it. Each slide was presented for 3000 ms to ensure proper familiarization with the category of the objects to be shown in the experimental task. Subsequently, a training session of 24 trials was administered, and it involved an equal number of randomly selected pictures from both categories.

Participants were seated in front of the screen at a viewing distance of ∼57 cm. Each trial began with a central fixation cross (sized 1◦ of visual angle) of variable (450, 550, 650, or 750 ms) duration. Next, two images of different objects belonging to the same or different category (tool vs. non-tool) were presented in the left and right visual field (starting at 3◦ of visual angle from the middle of the screen; both images sized 4◦ of visual angle) with a central arrow (sized 1◦ of visual angle) pointing to the left or right. The role of the arrow was to indicate the stimulus to which attention should be paid to. After 200 ms, the images were replaced with black-and-white high-contrast pattern masks for another 200 ms. Similarly to the study by Verma and Brysbaert (2011), the task was to decide (as quickly and accurately as possible) whether the target object was a tool or non-tool. The arrow remained on the screen until the participant responded, but for no longer than 2600 ms after the disappearance of the masks. Participants were asked to respond bimanually with their index fingers when the target was a tool and with their middle fingers when the target was a non-tool. The reaction time, as measured by the first key press, and accuracy of this response were recorded by the software used for stimulus presentation. A 1000-ms blank screen was introduced between the successive trials. The trial structure is depicted in **Figure 1**.

The design was implemented in SuperLab ver. 4.5.2 (Cedrus <sup>R</sup> , San Pedro, CA). The stimuli were presented on a 20 inch CRT monitor with a refresh rate of 85 Hz and a resolution of 1280 × 960. "RB-730" response pad by Cedrus was used for measuring accuracy and RTs. Every participant completed two blocks of randomly presented 240 trials with a 2-min break between the blocks. Each of the 60 stimuli was presented four times in each block: twice in the LVF (with compatible or incompatible distracters) and twice in the RVF (again with compatible or incompatible distracters). Care was taken to ensure that the two images presented in every trial were randomly paired for each participant and depicted different objects.

All the collected data were analyzed with four separate repeated-measures Analyses of Variance (ANOVAs), two for RTs to correctly categorized objects and two for accuracy. In the within-subjects analyses, the factors were target location (LVF, RVF), target category (tool, non-tool), and distracter compatibility (compatible, incompatible). In the mixed analyses, we included an additional, between-subjects factor, i.e., group (LVF-A, RVF-A), in order to account for the fact that each half of our participants demonstrated the opposite overall visual field advantage. The adopted level of significance was alpha = 0.05. The required follow-up tests of simple main effects were Bonferroni corrected (marked Bf-p). For reaction times accompanying a correct categorization of objects, outliers greater than two standard deviations above or below the mean (calculated for each participant in each condition, 4.9% of all trials) were removed. Statistical analyses were carried out using SPSS 21.0 (SPSS Ins., Chicago, IL).

### Results

### Within-Subjects Analyses **Recognition accuracy**

#### We observed a clear trend toward a main effect of target category which just missed the adopted significance level [F(1, 17) = 4.11, p = 0.06, Partial Eta Squared (pη 2 ) = 0.19]. Namely,

participants showed a strong tendency for more accurate categorization of non-tools than tools (average accuracy for nontools = 84%, SE = 2.6% vs. tools = 76.2%, SE = 4.2%). The main effects of target location and distracter compatibility were not significant [target location: F(1, 17) = 0.09, p = 0.76, pη <sup>2</sup> = 0.01; distracter compatibility: F(1, 17) = 0.01, p = 0.92, pη <sup>2</sup> = 0.001]. There was also a trend toward a significant interaction between target category and distracter compatibility [F(1, 17) = 3.76, p = 0.07; <sup>p</sup>η <sup>2</sup> = 0.18], indicating that when a distracter was compatible with the target, non-tools were categorized with greater accuracy than tools (average accuracy for non-tools = 84.7%, SE = 2.6% vs. tools = 75.5%, SE = 4.3%, Bf-p = 0.06). None of the remaining interactions was statistically significant.

### **Response times (RTs) to correctly categorized objects**

Neither target location, nor target category or distracter compatibility had a significant effect on RTs to correctly categorized stimuli [target category: F(1, 17) = 0.62, p = 0.44, <sup>p</sup>η <sup>2</sup> = 0.04; target category: F(1, 17) = 0.001, p = 0.97, <sup>p</sup>η <sup>2</sup> = 0.00; distracter compatibility: F(1, 17) = 2.36, p = 0.14, <sup>p</sup>η <sup>2</sup> = 0.12]. None of the interactions reached the significance threshold. The mean RTs and average accuracy for all the conditions are listed in **Table 1**.

# Between-Subjects (Mixed) Analyses

# **Recognition accuracy**

No significant difference in average accuracy was found between the LVF-A and RVF-A group [t(16) = 0.72, p = 0.48]. In addition to a trend toward a significant main effect of target category [F(1, 16) = 4.27, p = 0.06, <sup>p</sup>η <sup>2</sup> = 0.21] and a trend toward a significant interaction between target category and distracter compatibility [F(1, 17) = 3.95, p = 0.06, <sup>p</sup>η <sup>2</sup> = 0.20], that were both reported above, there was now also a significant interaction between group, target location and distracter compatibility [F(1, 16) = 5.84, p < 0.05, <sup>p</sup>η <sup>2</sup> = 0.27], but none of the post-hoc tests survived the Bonferroni correction.

### **Response times (RTs) to correctly categorized objects**

Again, there was no significant difference in the mean RTs between the LVF-A and RVF-A group [t(16) = 0.28, p = 0.78]. Importantly, we found a significant interaction between group, target location and target category [F(1, 16) = 6.18, p < 0.05, pη <sup>2</sup> = 0.28]. Namely, participants in the RVF-A group showed significantly faster categorization of tools presented in the RVF as compared to the LVF (mean RT in the RVF = 825 ms, SE = 72 ms vs. LVF = 882 ms, SE = 71 ms; Bf-p < 0.01). In contrast, participants in the LVF-A group showed a different pattern: although their responses were faster to tools correctly categorized in the LVF as compared to the RVF (mean RT for the LVF = 835 ms, SE = 71 ms vs. RVF = 866 ms, SE = 72 ms), this tendency did not reach the significance threshold after the Bonferroni correction (Bf-p = 0.13). Neither group showed any significant VHF dominance for non-tool categorization (LVF-A group: Bfp = 1.00; RVF-A group: Bf-p = 1.00). These effects are shown in **Figure 2**.

#### Discussion of Experiment 1

The paradigm used in Experiment 1 provides a unique approach to investigating the laterality of mechanisms involved in the categorization (or even recognition) of stimuli of different kinds. Namely, a required cognitive decision is made on the basis of a target stimulus presented laterally, i.e., appearing exclusively in one of the two VHFs (though accompanied by a non-target on the opposite side), and thus projected only to the contralateral hemisphere. Therefore, any preferential stimulus processing observed

#### TABLE 1 | Targets in VHFs - Experiment 1.


Target location (Left Visual Field, LVF; Right Visual Field, RVF), target category (tool, non-tool), distracter compatibility (compatible, incompatible) with their mean response times (ms), accuracy (%), and their standard errors of the means, for Experiment 1 with targets presented in either LVF or RVF are listed.

in the left or right VHF indicates that the most relevant mechanisms, e.g., here: for the extraction of tool-specific affordances, are predominantly lateralized to the right or left hemisphere, respectively.

In light of the above assumptions, the lack of the main effect of target location (or an interaction between target location and target category) observed both in accuracy and RTs to correctly categorized stimuli in the within-subjects analyses could be regarded as quite surprising. This is no longer the case, however, when one realizes that the left-handed participants we studied clearly represented two disparate groups, each demonstrating visual field advantage on opposite sides. After taking this distinctive attribute into account, i.e., by introducing into our analyses the group factor—which, notably, was independent of the task (or experiment) and stimulus type, we found different patterns of RTs to correctly categorized stimuli.

The right visual field advantage for the categorization of tools observed for RTs in the "typical" (RVF-A) group is consistent with a well-established role of the left hemisphere in encoding and retrieval of visual representations of tools (e.g., Grafton et al., 1997; Perani et al., 1999; Verma and Brysbaert, 2011; Garcea et al., 2012; or tool-use skills, e.g., Helon and Króliczak, 2014; cf. Króliczak, 2013b). A trend toward the LVF advantage observed in the "atypical" (LVF-A) group for tool categorization reveals another important finding, namely that the strength of the involvement of the right-hemisphere mechanisms in processing of human artifacts—and particularly tools—varies substantially across this group of individuals. Indeed, among the subjects with the putative atypical organization of object processing (and perhaps other cognitive skills) there were two participants who despite showing a clear general LVF advantage (irrespective of the task and stimulus kind) did not reveal such an effect for tools. Therefore, it should be emphasized at this point that such a result is not an artifact of the grouping method adopted in our study. A very similar pattern of outcomes has been reported by Verma et al. (2013) in a VHF study on symmetry detection wherein participants with known atypical hemispheric dominance for speech demonstrated greater variability in the studied task, with only about half of them showing LVF advantage for the processing of symmetrical shapes.

The faster categorization of tools observed in the RVF-A group in the dominant VHF and a similar (though much weaker) effect observed in the LVF-A group, as opposed to no comparable effect of any kind for non-tool stimuli, is also consistent with the idea that information about tools, in contrast to other objects (e.g., animals, houses, or graspable shapes with no function), is processed in the brain in a unique way (e.g., Chao et al., 1999; Chao and Martin, 2000; Creem-Regehr and Lee, 2005). In fact, nearly all studies on tasks involving tools in typical (usually right-handed) individuals point to the left hemisphere as the seat of their representations (including their concepts and the relevant manual skills). It is also worth mentioning that our finding of no visual field asymmetry in the accuracy or speed of categorization for non-tools is furthermore in line with numerous neuroimaging and behavioral studies, too (e.g., Biederman and Cooper, 1991; Proverbio et al., 2011; Verma and Brysbaert, 2015). These reports clearly indicate that the representations (or perhaps the mechanisms involved in categorization and/or recognition) of non-manipulable objects are organized more bilaterally. That is, none of the hemispheres seems to be preferentially involved in their encoding and retrieval.

Notably, the lack of preferential involvement of any hemispheres for non-tools did not prevent our participants from being more accurate in their processing (there was at least a clear trend toward greater accuracy in the categorization of non-tools as compared to tools, irrespective of distracter's category). Although this finding may just indicate that our sample was basically more familiar with non-tool objects included in this study (and the presence of compatible distracters seemed to facilitate their categorization even more), this result goes against a hypothesis that a greater expertise with a given category of objects may be accompanied by a more localized and/or lateralized processing (such an argument seems to be tacitly assumed in many studies on tool representations).

But is it really a specific mechanism rather than a more general processing stream that was tackled with the use of the VHF paradigm that we adopted in Experiment 1? Alternatively, can any results obtained with such an approach really tell us anything about the inner organization of the processes that are involved in the task of interest? In our opinion, some light on this issue can be shed by using the laterally-presented objects as primes to the centrally-displayed targets requiring subsequent categorization. This is exactly what has been done in Experiment 2.

# Experiment 2: Categorization of Objects Preceded by Primes Presented in either LVF or RVF

### Methods

The design of Experiment 2 was based on that used by Garcea et al. (2012) with some modifications.

# Stimuli

The stimuli consisted of 60 gray-scaled pictures of familiar manmade objects (30 tools, 30 non-tools, the list of all pictures can be found in Appendix 1). As in Experiment 1, half of the objects from each category (15 tools and 15 non-tools) were rotated so that the long axis of the object was deflected from vertical by 45◦ , whereas objects from the other half were rotated in the same manner to obtain a deflection of 315◦ . Seventy percent of additive noise was overlaid on all the pictures (for a rationale of this manipulation, see Garcea et al., 2012). All images were sized to 174 × 174 pixels.

## Procedure

At the beginning of the experiment participants were familiarized with the stimuli in the same manner as in Experiment 1. They also took part in a training session of 24 trials, which again involved an equal number of randomly selected pictures from both categories.

Each trial began with a central fixation cross (sized 1◦ of visual angle) of variable (450, 550, 650, or 750 ms) duration. Next, a prime (a tool or a non-tool) was presented in the left or right visual field (starting 3◦ of visual angle from the middle of the screen; 5◦ of visual angle in size). In the identity condition, the prime was same as (i.e., identical with) the to-be-seen target, while in the scrambled condition it was a scrambled version of the to-be-seen target. In both conditions, in the opposite visual field, the prime was accompanied by a scrambled version of a different image from the same category (again, starting 3◦ of visual angle from the middle of the screen; 5◦ of visual angle in size). After 35 ms, the prime and the accompanying image were immediately replaced with black-and-white high-contrast pattern masks of the same size for 118 ms. Then, the target image (a tool or a non-tool) was presented centrally and remained on the screen until the participant made a response, but for no longer than for 3000 ms. The task was to decide (as quickly and accurately as possible) whether the target was a tool or non-tool. Similarly to Experiment 1, participants responded bimanually with their index fingers if the target was a tool and with their middle fingers if the target was a non-tool. The time of the first key press and the correctness of the response were recorded. A 1000-ms blank screen was introduced between the successive trials. The trial structure is depicted in **Figure 3**.

The technical equipment and software used was identical to Experiment 1. Every participant completed two blocks of randomly presented 240 trials with ∼2 min break between the blocks. Each of the 60 stimuli was presented four times in each block: twice in the LVF and twice in the RVF, twice in the identity condition and twice in the scrambled condition. Care was taken to ensure that the prime and the accompanying image presented in every trial depicted different objects of the same category (a tool or non-tool), randomly paired for each participant.

Similarly to Experiment 1, the collected data were analyzed with four separate repeated-measures ANOVAs, two for RTs to correctly categorized objects and two for accuracy. The withinsubject factors were prime location (left, right), target category (tool, non-tool), and prime condition (identical, scrambled). The between-subjects factor was group (LVF-A, RVF-A). The adopted

level of significance was alpha = 0.05 and, if necessary, post-hoc tests were Bonferroni corrected (Bf-p). For RTs to correctly categorized objects, outliers greater than two standard deviations above or below the mean were removed (4.8% of all trials).

# Results

#### Within-Subjects Analyses

#### **Recognition accuracy**

There was a significant main effect of target category [F(1, 17) = 11.84, p < 0.01, <sup>p</sup>η <sup>2</sup> = 0.41] such that participants categorized non-tools with greater accuracy than tools (average accuracy for non-tools = 95.9%, SE = 0.9% vs. tools = 88.6%, SE = 1.8%). There was no main effect of prime location [F(1, 17) = 0.54, p = 0.47, <sup>p</sup>η <sup>2</sup> = 0.03] or prime condition [F(1, 17) = 1.66, p = 0.21, <sup>p</sup>η <sup>2</sup> = 0.09]. There was also a significant interaction between target category and prime condition [F(1, 17) = 6.10, p < 0.05, <sup>p</sup>η <sup>2</sup> = 0.26], but the effect of tools being easier to categorize when identical primes instead of scrambled primes were presented just missed the significance threshold (Bf-p = 0.07). No further significant effects were found.

#### **Response times (RTs) to correctly categorized objects**

We found a significant main effect of prime condition [F(1, 17) = 15.55, p = 0.001, <sup>p</sup>η <sup>2</sup> = 0.48], indicating shorter reaction times to targets preceded by identical as opposed to scrambled primes (mean RT for identical primes = 611 ms, SE = 27 ms vs. scrambled primes = 628 ms, SE = 27 ms). There was no main effect of prime location [F(1, 17) = 0.48, p = 0.50, <sup>p</sup>η <sup>2</sup> = 0.03] or target category [F(1, 17) = 0.91, p = 0.35, <sup>p</sup>η <sup>2</sup> = 0.05]. Moreover, interactions between prime location and target category, prime condition and target category, prime location and prime condition, as well as a three-way interaction, were not significant.

# Between-Subjects (Mixed) Analyses **Recognition accuracy**

Participants in the RVF-A group categorized stimuli with greater accuracy than subjects in the LVF-A group [average accuracy in the RVF-A group = 94.2%, SE = 1.0% vs. LVF-A group = 90.3%, SE = 1.4%; t(16) = 2.24, p < 0.05]. We found the previously described significant main effect of target category [F(1, 16) = 14.34, p < 0.01, <sup>p</sup>η <sup>2</sup> = 0.47] and the significant interaction between target category and prime condition [F(1, 16) = 5.90, p < 0.05, <sup>p</sup>η <sup>2</sup> = 0.27], which showed that only in the case of tools, greater accuracy in categorization was associated with identical rather than scrambled primes (average accuracy for identical primes = 89.2%, SE = 1.6% vs. scrambled primes = 88%, SE = 1.6%; Bf-p = 0.05). There was also a significant interaction between group and target category [F(1, 16) = 4.59, p < 0.05, <sup>p</sup>η <sup>2</sup> = 0.22], showing that only the LVF-A group categorized non-tools with greater accuracy than tools (average accuracy for non-tools: 95.9%, SE = 1.3% vs. tools: 84.6%, SE = 2.2%; Bf-p < 0.01). A significant interaction between group and prime condition [F(1, 16) = 5.64, p < 0.05, <sup>p</sup>η <sup>2</sup> = 0.26] moreover indicated that only the LVF-A group categorized stimuli with greater accuracy but only when they were preceded by identical primes, as compared to scrambled primes (average accuracy for identical primes: 90.8%, SE = 1.2% vs. scrambled primes = 89.7%, SE = 1.3%; Bf-p < 0.05).

#### **Response times (RTs) to correctly categorized objects**

LVF-A group and RVF-A group did not differ significantly in the mean RTs [t(16) = 1.09, p = 0.29]. As above, we found a significant main effect of prime condition [F(1, 16) = 14.71, p = 0.001, <sup>p</sup>η <sup>2</sup> = 0.48] such that targets were categorized faster when preceded by identical primes as compared to scrambled primes, and a new significant interaction between prime location and prime condition [F(1, 16) = 4.35, p = 0.05, <sup>p</sup>η 2 = 0.21] such that only in the case of left-sided priming, identical primes led to faster categorization of the subsequent targets, as compared to scrambled primes (mean RT for identical primes = 606 ms, SE = 25 ms vs. scrambled primes = 630 ms, SE = 27 ms; Bf-p < 0.01). However, both these effects should be interpreted with caution, because there was also a significant interaction between group, prime location, and prime condition [F(1, 16) = 6.26, p < 0.05, <sup>p</sup>η <sup>2</sup> = 0.28] which clarified their nature. Namely, the findings were such that only participants in the LVF-A group responded significantly faster when primes presented in their dominant VHF were identical rather than scrambled (mean RT for identical primes = 629 ms, SE = 35 ms vs. scrambled primes = 659 ms, SE = 38 ms; Bf-p < 0.01). In the RVF-A group, this effect missed the significance threshold (mean RT for identical primes = 578 ms, SE = 40 ms vs. scrambled primes = 598 ms, SE = 37 ms; Bf-p = 0.07). Nevertheless, the impact of right-sided identical priming in the RVF-A group was revealed by a planned a priori t-test [t(8) = 2.47, p < 0.05]. These effects are shown in **Figure 4**. Finally, the most important significant interaction was revealed between group, target category, and prime condition [F(1, 16) = 4.47, p = 0.05, pη <sup>2</sup> = 0.22]. Participants who were classified as the LVF-A group responded faster when non-tools were preceded by identical compared to scrambled primes (mean RT for identical primes = 635 ms, SE = 38 ms vs. scrambled primes =657 ms, SE = 36 ms; Bf-p < 0.01). RVF-A group, on the other hand, showed a clear trend toward faster categorization of tools when they were preceded by identical compared to scrambled primes (mean RT for identical primes = 562 ms, SE = 40 ms vs. scrambled primes = 587 ms, SE = 41 ms; Bf-p = 0.06). Indeed, this effect was significant as shown by a planned a priori t-test [t(8) = 2.80, p < 0.05]. These results are shown in **Figure 5**. The mean RTs and average accuracy for all the conditions are listed in **Table 2**.

### Discussion of Experiment 2

Because the task in Experiment 2 involved a centrally presented target stimulus (encoded by both hemispheres), whose processing could potentially be affected by the laterally presented primes, the results obtained with this paradigm may tell us substantially less about the laterality of neural mechanisms involved in the visual categorization of man-made objects, but can potentially tell us much more about the inner organization of the processes that subserve this function.

Despite a very complex pattern of results obtained in this experiment, two patterns of outcomes are clear-cut. No surprisingly, the typical (RVF-A) group responded faster following identical priming coming from its dominant right visual field, and the atypical (LVF-A) group responded faster following identical priming coming from its dominant left visual field. Yet, this was only the case when tools and non-tools were collapsed. In sharp contrast, as the most crucial outcome of our study indicates, whereas the effect of greater facilitation of RTs following identical priming in the typical group was driven primarily by faster reaction times to tools, the greater facilitation of RTs following identical priming in the atypical group was driven primarily (indeed, almost exclusively) by faster reaction times to non-tool targets. It must be clearly emphasized, though, that the latter two effects were now independent of the side in which the priming stimulus occurred.

The results of Experiment 2 therefore suggest that in the case of cognitive mechanisms that are strongly encapsulated (i.e., form a relatively independent module specialized for certain kind of stimulus encoding, Króliczak et al., 2012; cf. Clark, 2009), the presence of the prime in the information-processing stream will affect the subsequent (centrally-categorized) target irrespective

any category (tools and non-tools) preceded by primes presented in the (A) dominant and (B) non-dominant visual fields displayed as a function of the group (representing left or right visual field advantage) and prime type. Regardless of object category, whereas

effect in its dominant (left) visual field, the typical, right visual field advantage group (RVF-A) demonstrated only weak priming in its dominant (right) visual field. Asterisk indicates a difference with Bonferroni-corrected p-value of 0.05 (\*).

FIGURE 5 | Response times to correctly categorized (A) tools and (B) non-tools displayed as a function of the group (representing left or right visual field advantage) and prime type. The atypical, left visual field advantage group demonstrated a strong priming effect only for non-tool

categorization, but now irrespective of the prime side, whereas the typical, right visual field advantage group showed both weaker priming effect and only for the categorization of tools. Asterisks indicate a difference with Bonferroni-corrected p-value of 0.01 (\*\*).

#### TABLE 2 | Primes in VHFs—Experiment 2.


Target location (Left Visual Field, LVF; Right Visual Field, RVF), target category (tool, non-tool), prime condition (identical, scrambled) with their mean response times (ms), accuracy (%), and their standard errors of the means, for Experiment 2 with primes presented in either LVF or RVF are listed.

of priming side and regardless of which hemisphere is involved more in the categorization process itself. This is at least the case in the LVF-A group, where the categorization of non-tools was in fact facilitated (in terms of RTs) by identical primes irrespective of their location. Such a pattern of performance also implies that at least in individuals with atypically lateralized object encoding, and putatively atypical organization of other cognitive skills, (1) the concepts of man-made objects which do not have close affinity to any specific representations of manual dexterity are still organized more symmetrically (see also Experiment 1, e.g., Ishai et al., 1999, 2000; Verma and Brysbaert, 2011), but despite being distributed across the two hemispheres, and somewhat counterintuitively, (2) the critical mechanisms for human artifact categorization seem to be more specialized (encapsulated) for non-tool objects than for manipulable tools (whose usage requires a proper grasp and sequence of hand movements). Indeed, the presence of such a specialized mechanism may be responsible for more accurate categorization of non-tools in this particular group. Conversely, in the RVF-A group, a faster categorization of tools preceded by primes on any side implies a more specialized mechanisms contributing to the processing of this narrower category of objects, which is in line with neuropsychological and neuroimaging evidence from right-handed (most of the time having typical organization of cognitive skills) subjects (for a review, see Frey, 2008).

This study clearly shows that cognitive decisions involving different categories of objects can be easily primed (e.g., Garofeanu et al., 2004; Garcea et al., 2012). Yet, the strength and direction of the effect will depend on object category—will be different for non-tools and tools—and on the mechanisms predominantly involved in their processing. E.g., the priming effect for non-tools may depend more on the overall target shape, whereas for tools, on its afforded action features, i.e., its graspability. Indeed, we expect that the priming effects would be different not only for disparate object categories, but also for the type of task to be performed, including both perceptual and action decisions (e.g., Helbig et al., 2006; McNair and Harris, 2012; Bub et al., 2013; cf. Craighero et al., 1996; Króliczak et al., 2006).

# General Comments

The way the representations of man-made objects are organized and/or lateralized in the human brain, and as a consequence the efficiency with which they are utilized in cognitive processes is likely to depend on—among other factors such as the strength of connections between the object concept and its relevant functional properties, or the distance (the number of levels/nodes/synapses) separating such conceptual and functional knowledge—whether or not a particular type of object affordances is critically linked to the representations of manual skills (e.g., Bub et al., 2008; Pellicano et al., 2010; Proverbio et al., 2011). For example, the chair can be effectively moved closer to the body (or rather legs) with the hands but what it affords has nothing to do with skilled hand movements (thus representing a low degree of manipulability). This is probably why the concepts of tools are special: a reason being the gradually acquired privileged link between the functional knowledge and the knowledge of the relevant movements (i.e., manual dexterity) that comes into play with deft tool use. Such representations and/or links between them are clearly absent in kids who can already name tools but cannot effectively use them, not to mention pantomiming their use (O'Reilly, 1995; Landau et al., 1998).

In right-handers, most of the mechanisms underlying tool categorization and/or tool use abilities, as well as the processes that enable orchestrated interactions of the disparate and often differently localized mechanisms involved in dealing with this subcategory of human artifacts, are lateralized to the left hemisphere. Of course, things may change substantially when a preference for using the left hand gets factored in the build-up of their representations. Hence, in some left-handed individuals tool concepts seem to have greater affinity to the right-hemisphere mechanisms underlying hand dominance, although, as our Experiment 1 shows, in the majority of sinistrals this is not the case (cf. Króliczak et al., 2011; Vingerhoets et al., 2013). If the former happens, though, this does not necessarily entail that the representations of other man-made objects are automatically reorganized, shifted and/or moved to the opposite (left) hemisphere (as clearly demonstrated by Experiment 2). Indeed, the mechanisms invoked during interactions with non-tools may in such cases depend further on the more distributed, bilateral processing, being at the same time less prone to local one-sided injuries.

If we assume that tool concepts form only a unique subset of the category of man-made objects including non-tools, or there is a substantial overlap between the two categories, then a very counterintuitive idea emerges. Indeed, this idea is of paramount importance for the neurocognitive rehabilitation of apraxia (cf. Oliveira and Brito, 2014). Namely, this study suggests that in patients with atypically organized skills the most effective way of alleviating tool-related conceptual and/or motor deficits that would follow right-hemisphere damages might be targeting first their relatively preserved skills to deal with non-tools. After all, as we demonstrated, some of the **processes** involved in the categorization of non-tools (see Experiment 2) are in such individuals organized quite similarly to the **mechanisms** invoked directly during the categorization of tools (see Experiment 1).

This study as a whole convincingly shows that individuals with atypically organized cognitive skills are not just mirror reversed images of typical subjects (cf. Lewis et al., 2006). This is particularly true about the way the representations of tools are encoded and retrieved in the atypical brain. Notably, although an objective method was used here to divide participants into groups (which happened to be equal) with typical and atypical laterality of object categorization, this should not be construed as evidence that 50% of our left-handers demonstrated atypical laterality of tool processing. Depending on how this issue is approached, e.g., based exclusively on Experiment 1 or Experiment 2, only 38.9% or just 33.3% of sinistrals, respectively, demonstrated the atypical left-visual field (right hemisphere) advantage for tool categorization (consistent with Króliczak et al., 2011).

Based on both experiments, there is evidence to indicate that the atypical group seems to possess more refined representations of non-tool objects, despite the involvement of both hemispheres in the processing of such human artifacts. In contrast, individuals with typically organized brains possess more fine-grained representations of tools whereas the non-tool category seems more diffused. Indeed, in our opinion, equivocal effects that were likely obtained while testing left-handers are to blame for the exclusion of sinistrals from scientific research and the lack of interesting reports on their cognitive skills (see also Willems et al., 2014).

# Limitations of the Study

It would be of great interest to test whether or not individuals with atypically organized tool processing would also demonstrate atypical (i.e., bilateral or right-sided) organization of language skills. This could have been easily tested using the VHF paradigm as shown by Hunter and Brysbaert (2008). Based on Króliczak et al. (2011), we expect that no more than 25% of these participants would show atypical language laterality.

In the context of Experiment 1, it would be desirable to include a third type of distracter, i.e., a neutral one, in order to further investigate the possible facilitation or interference effects. In Experiment 2, on the other hand, the inclusion of incongruent primes (i.e., representing objects from the other category) could shed some new light on the efficiency and perhaps the more detailed organization of mechanisms and processes involved in the categorization of man-made artifacts.

# Conclusions

Although dextrals were not included in this project, the results we obtained clearly suggest that dividing study participants based on hand dominance, not to mention the exclusion of sinistrals, makes no sense. A much more reasonable approach would be to group subjects into those representing typical and atypical laterality of cognitive skills. Such a change in the recruitment, inclusion, and assignment process could in fact lead to new and hopefully more adequate models of the organization of functions in the healthy brain, which in turn could generate new approaches to neurocognitive rehabilitation. By the same token, these results also indicate that collapsing across all left-handed individuals in fMRI analyses might not be the most advisable strategy.

# Author Contributions

This project was conceptualized by BM and GK. Data was collected by BM, and analyzed by BM and GK. The manuscript was written by GK and BM.

# References


# Acknowledgments

This work is a part of a greater project supported by the Polish National Science Center (Narodowe Centrum Nauki, NCN) grant Maestro 2011/02/A/HS6/00174 to GK. The equipment used was funded by the Ministry of Science and Higher Education (Ministerstwo Nauki i Szkolnictwa Wyzszego, ˙ MNiSW) grant 6168/IA/128/2012 to GK and NCN grant Maestro 2011/02/A/HS6/00174. During the preparation of this manuscript BM and GK were also supported by the Maestro 2011/02/A/HS6/00174 grant.


representations of observed actions and objects. Cognition 108, 444–465. doi: 10.1016/j.cognition.2008.03.007


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Michałowski and Króliczak. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Appendix 1

# Stimuli Used in Experiment 1

Tools: ax, baseball bat, broom, brush, comb, fork, hammer, iron, key, knife, ladle, lipstick, nail file, net, paintbrush, pen, pliers, rake, rolling pin, ruler, saw, scissors, screwdriver, spoon, stethoscope, syringe, tennis racket, toothbrush, watering can, wrench.

Non-tools: accordion, airplane, anchor, belt, bottle, candle, cigarette, cutting board, dart, envelope, football, glasses, guitar, hanger, kite, ladder, lamp, light bulb, nail, padlock, pipe, plug, roller-skate, screw, shoe, sock, tie, trumpet, violin, watch.

# Stimuli Used in Experiment 2

Tools: bottle opener, brush, cigarette lighter, comb, corkscrew, drill, food mixer, fork, hammer, iron, key, knife, mouse, paint roller, paintbrush, pen, pincers, pliers, punch, razor, saw, scissors, screwdriver, snap-off knife, spatula, spoon, tenderizer, thimble, toothbrush, wrench.

Non-tools: basket, battery, belt, book, bottle, charger, compact disc, cover, dumbbell, extension cord, frame, glasses, glove, hanger, hat, headphones, knight, light bulb, mascot, necklace, padlock, pillow, shoe, sock, spool, suitcase, toilet roll, toothpaste tube, USB flash drive, watch.

# Limb apraxia and the "affordance competition hypothesis"

#### Elisabeth Rounis 1,2\* and Glyn Humphreys <sup>1</sup>

<sup>1</sup> Department of Experimental Psychology, University of Oxford, Oxford, UK, <sup>2</sup> Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK

Limb apraxia, a disorder of higher order motor control, has long been a challenge for clinical assessment and understanding (Leiguarda and Marsden, 2000). The deficits originally described in limb apraxia (Liepmann, 1920) have been classified by the nature of the errors made by the patients leading to, namely, ideational and ideomotor apraxia. The dual stream hypothesis (Goodale and Milner, 1992) has been used to explain these categories: ideational apraxia is thought to relate to a deficit in the concept of a movement (coded in the ventral stream). Patients have difficulty using objects, sequencing actions to interact with them or pantomiming their use. Ideomotor apraxia, on the other hand, is thought to arise from problems in the accurate implementation of movements within the dorsal stream. One of the limitations on understanding apraxia is the failure by the clinical literature to draw on knowledge of the factors determining actions in the environment. Here we emphasize the role of affordance. There is much recent work indicating that our responses to stimuli are strongly influenced by the actions that the objects "afford", based on their physical properties and the intentions of the actor (e.g., Tucker and Ellis, 1998). The concept of affordance, originally suggested by Gibson (1979) has been incorporated in a recent model of interactive behavior that draws from findings in non-human primates, namely the "affordance competition hypothesis" (Cisek, 2007). This postulates that interactive behavior arises by a process of competition between possible actions elicited by the environment. In this paper we argue that "affordance competition" may play a role in apraxia. We review evidence that at least some aspects of apraxia may reflect an abnormal sensitivity to competition when multiple affordances are present (Riddoch et al., 1998) and/or a poor ability to exert cognitive control over this competition when it occurs. This framework suggests a new way of conceptualizing deficits in apraxia which invites further investigations in the field.

Keywords: limb apraxia, ideational apraxia, ideomotor apraxia, affordance competition hypothesis, route to action model

# Introduction

Limb apraxia is a heterogeneous disorder of higher order motor control affecting skilled and learnt actions. It has traditionally been classified by the nature of the errors made by patients and the brain pathways with which these errors are associated (Liepmann, 1920; Leiguarda and Marsden, 2000). Sub-aspects of the disorder have been broadly classified as reflecting

#### Edited by:

Antonello Pellicano, RWTH Aachen University, Germany

#### Reviewed by:

Elisabetta Ambron, Scuola Internazionale Superiore di Studi Avanzati, Italy Dimitrios Kourtis, Ghent University, Belgium

#### \*Correspondence:

Elisabeth Rounis, Clinical Lecturer in Neurology, Nuffield Department of Clinical Neurosciences, Neurology Division, Level 3, West Wing, John Radcliffe Hospital, Headley Way, Headington, Oxford OX3 9DU, UK elisabeth.rounis@ndcn.ox.ac.uk

> Received: 25 May 2015 Accepted: 13 July 2015 Published: 28 July 2015

#### Citation:

Rounis E and Humphreys G (2015) Limb apraxia and the "affordance competition hypothesis". Front. Hum. Neurosci. 9:429. doi: 10.3389/fnhum.2015.00429 impairments of the conceptual representation of actions leading to ideational apraxia, or of the implementation of these concepts, termed ideomotor apraxia, or, if the impairment pertains exclusively to skilled use of finger or hand gestures, to limb kinetic apraxia (Faglioni and Basso, 1985; Leiguarda and Marsden, 2000).

These definitions of apraxia have been strongly debated in the literature (Buxbaum, 2001; Buxbaum et al., 2007) but share a common basis within traditional cognitive theories which view perception, decision-making and actions serially as depicted by ''box-and-arrows'' models, as illustrated in **Figure 1** (Heilman and Rothi, 1997).

Traditional models of apraxia have relied on observational qualitative data and they have typically remained descriptive. Sometimes they contradict observations of patients in real life leading to difficulties defining particular subtypes (e.g., ideational vs. ideomotor, Buxbaum, 2001) or the brain areas involved (e.g., parietal vs. ventral premotor, Pazzaglia et al., 2008; Kalénine et al., 2010).

This article describes a theoretical framework called the ''affordance competition hypothesis'' (Cisek, 2007) which offers an alternative view of apraxia. Our aim is to explore how this proposal could influence our understanding of this complex disorder.

The affordance competition hypothesis derives from ecological psychology and aims to describe real-time interactive behavior in terms of processes that specify potential actions and select between them (Cisek and Kalaska, 2010). According to this hypothesis, processes generating behavior are resolved in parallel, instead of in a serial manner, through competition between currently available opportunities and demands for action (Cisek, 2007).

We will firstly define the affordance competition hypothesis. This is followed by a review of the patient literature to identify examples that support this hypothesis. We finish by exploring predictions from this hypothesis, relevant to different aspects of limb apraxia.

# Part 1: Defining the Affordance Competition Hypothesis

Traditional cognitive theories view the selection and specification of actions in terms of an information processing framework. According to this, perception involves the construction of various levels of internal representations of the world (Biederman, 1972; Marr, 1982) that are used to inform the cognitive system which makes decisions, which in turn can be implemented into action plans by the motor system (Tversky and Kahneman, 1981; Shafir and Tversky, 1995).

Studies reported in recent neurophysiological experiments suggest that this perspective fails to provide a unified account of behavior (Cisek and Kalaska, 2005). There are cases in which functions that should be distinct and appear to involve the same regions, or even the same cells, and others in which functions that should be unified appear distributed throughout the brain. An example of the former is the role of the lateral intraparietal area (LIP). This has been proposed to include control of gaze (Snyder et al., 1997), the representation of space in a body-centered reference frame (Snyder et al., 1998), and the representation of abstract decision variables such as expected utility (Platt and Glimcher, 1999). An example of the latter derives from the neuropsychology literature, and particularly the proposal of the dual stream hypothesis (Goodale and Milner, 1992). This postulates a ventral stream pathway dedicated for object identification distinct from a dorsal stream pathway for the control of action in space, with no account of how the two may integrate to generate real-time behaviors (Schenk and McIntosh, 2010).

An alternative hypothesis for interpreting neural data, which proposes to resolve contradicting results and account for realtime interactive behaviors has been proposed (Cisek, 2007; Cisek and Kalaska, 2010). The ''affordance competition hypothesis'' views interactive behavior as involving simultaneous processes that specify potential motor actions and select between them.

A mathematical model by which the cerebral cortex may implement competition between representations of visuallyguided reaching actions within the dorsal stream is used as an example (Cisek, 2007). In this model attended visual stimuli elicit the generation of motor plans across visuomotor regions. An action is selected to be performed by a process of competition—implemented by a neural mechanism of mutual inhibitory connections influenced by biasing inputs from decision centers.

The concept of affordance, introduced by Gibson (1979), proposes that visual objects and their properties give rise to action representations. For example a handle ''affords'' to be pulled. These action representations depend on contextual demands from the task (Young, 2006).

Fagg and Arbib (1998) have modelled grasp behaviors in a similar way (see **Figure 2**). According to this, action programming can be triggered by sensory cues without the invocation of high-level object recognition processes in the ventral stream. They emphasized the importance of current goals, tasks and internal states of the action system in determining this type of ''action oriented perception'' (Arbib, 1997).

Cognitive psychological studies in humans have described affordances in relation to stimulus-response compatibility effects. In a seminal experiment by Tucker and Ellis (1998), participants had to make a button press response with their right or left hand to indicate the orientation of an object presented on a screen. They observed that, when an object with an elongated handle was presented, responses were faster when the responding hand was compatible with grasping the handle relative to when the hand was in an incompatible position. Tucker and Ellis (1998) proposed that visuo-motor relations between objects and actions activate a motor response for object-use ''automatically'', even when if the response is not required by the task. These compatibility effects could be elicited for different aspects of objects (e.g., the position of a handle, the size of an object) and have been referred to as ''micro-affordances'' (Ellis and Tucker, 2000).

However Cisek's (2007) definition of affordances goes beyond action specification for object interactions. Rather affordances are defined as any opportunity for action provided by the environment. Cisek (2007) proposes that neural activity in the dorsal stream implements a functionally motivated mixture of variables simultaneously as sets of competing sensorimotor loops, rather than serial stages of a representation of objects in space, a representation of motor plans, or cognitive variables such as expected values. His model allows action specification to occur in parallel with action selection.

A number of predictions arise from this model which could be applicable for both abstract and object-directed actions. According to the model, each population in a neural network for action selection is proposed to involve competitive interactions, with biasing influences modulating this competition in different neural regions. Since cortico-cortical connections are bidirectional, any decision which starts to emerge in one region will propagate to other regions. In this way, decisions based on sensory features, which may be salient for action specification (see **Figure 2**), may first appear in parietal cortex and then influence frontal activity. In contrast decisions based on abstract rules may first be expressed in frontal regions and propagate backward to parietal areas. The competition between representations of potential actions is balanced by the accumulation of evidence in favor of a given choice leading to a decision by a process of ''distributed consensus''. These proposals have implications for behaviors observed in patients, which we discuss below.

# Part 2: Examples of Affordance Competition in Patients

There are several examples in support of ''affordance competition'' in the animal literature (Cisek and Kalaska, 2010). In this section we review the evidence for similar processes taking place in humans, and more importantly in patient populations, relating to actions targeted to handled objects, more specific to deficits in limb apraxia. This might allow us to answer the question of whether the affordance competition hypothesis could provide a useful framework for understanding limb apraxia, going beyond previous models in the field (Rothi et al., 1991; Bartolo et al., 2001; Buxbaum et al., 2007).

Riddoch et al. (1998) studied a patient with cortico-basal degeneration who showed strong automatic grasp actions to objects. They explored a task in which the patient had to reach and grasp a cup using the hand that was on the same side of the table as the cup. When the cup was on the left, and its handle oriented to the right, the patient tended not to grasp the object with her left hand (the required response) but rather grasped it with her right hand—the action being cued by the orientation of the object in relation to the patient's preferred hand. Interestingly this grasp action decreased when the cup was inverted, even though the physical positioning of the handle was the same as when the cup was upright. These data suggest that it was the familiar positioning of the cup, in its upright location, that triggered the grasp action to the handle. This pattern of behavior was not observed when patients were asked to point with their left or right hand depending on the location of the cup, suggesting an influence of the intended action on these affordance effects (Hommel, 2000; Linnell et al., 2005; Humphreys and Riddoch, 2007).

In a previous study, Riddoch et al. (1989) described a patient with a modality-specific deficit. This patient showed deficits in pantomiming the use of visually-presented objects only when they were asked to use their right (contralesional) hand. Patient CD had no difficulties in pantomiming actions to objects with his left hand and he was also able to pantomime actions to names using his right hand. However he had a hand- and modalityspecific deficit (right hand, seen objects). The fact that the patient could pantomime actions using his right hand suggests that there was not an ''ideomotor'' problem in effecting right hand actions. Also the fact that he could make actions to seen objects with his left hand indicates that the problem was not an ''ideational'' disorder. To account for the result, Riddoch et al. (1989) proposed that CD had difficulties in selecting the appropriate action with his right hand when multiple affordances were offered by the seen object. That is, there was difficulty in selecting a hand-specific action when multiple actions were evoked for the right hand. Note that, when given the name of an object, multiple affordances would not be invoked, and CD was able to act under those conditions. These results were simulated in an explicit computational model of affordance competition by Yoon et al. (2002).

A further study reported evidence for affordance effects between multiple objects. Humphreys et al. (2000) presented patients showing utilization behavior with multiple objects and asked them to perform an unusual action with two of the items (e.g., ''put the saucer on the cup''). Despite being able to repeat back the instruction, patients made errors by carrying out the familiar action (e.g., they put the cup on the saucer). This was not solely due to the familiarity of the actions offered by the objects. When asked to perform an unfamiliar action that contravened an affordance offered by the physical properties of the stimuli (e.g., ''with the cup stir the pencil''—when they could make a stirring action using a cup over a pencil) patients made errors by carrying out the afforded (novel) action (e.g., stirring the pencil in the cup). Humphreys et al. (2000) proposed that affordances are offered not only by single objects but also by arrays of multiple objects which can afford different actions when used together. The affordance could be based purely on the physical properties of individual objects but also on learned interactions (as in the cup-saucer example above).The presence of multiple affordances in these more complex situations could then contribute to some of the additional symptoms associated with apraxia, such as poor sequencing of behaviors.

These pieces of evidence for both hand-specific and multiobject affordances highlight that, even when we make simple actions to objects, several affordances can be present and evoked separately for each hand and for different object combinations. In utilization behavior there is a difficulty in using task-based constraints to moderate strongly afforded actions. In apraxia there can be a problem in selecting the appropriate action when competition is present, and selection may sometimes be inappropriate leading to (amongst other things) errors in sequencing.

# Part 3: Predictions of Apraxic Deficits Based on the Affordance Competition Hypothesis

Here, we discuss some implications of the affordance competition hypothesis in relation to limb apraxia. We propose a mechanism by which models which posit a direct route to action, distinct from semantic, recognition processes, can be integrated with this framework to reflect the dynamic nature of action selection (Yoon et al., 2002).

# Action Specification and Selection Performed within Similar Networks of Brain Regions

The affordance competition hypothesis suggests that action specification and action selection are performed by the same neural circuits, distributed among a large set of brain regions.

Traditional definitions of limb apraxia have distinguished between ideational apraxia, defined as an incapacity to evoke the action associated with an object (Heilman et al., 1982), from ideomotor apraxia where patients make spatio-temporal errors in performing the appropriate gesture to a task. Ideational apraxia has also been applied to describe patients who make errors in selecting the correct target object when more than one object is present.

A major problem for the field is that the differences between these two forms of apraxia have been difficult to distinguish as few patients show one set of symptoms in isolation from symptoms characteristic of the other disorder (Buxbaum, 2001).

The affordance competition hypothesis would go further in proposing that both types of apraxia are likely to be present to some degree and that one may influence the other dynamically. In this framework ideomotor apraxia may implicate more dorsal networks for action specification, whereas more ventral networks for action selection would be related to ideational apraxia.

This parallels recent findings in the grasp literature, which have challenged the view that reach and grasp components are processed independently (Fattori et al., 2010; Vesia and Davare, 2011). Studies in non-human primates have revealed divisions within the dorsal stream (dorso-dorsal and dorso-ventral) which are thought to provide networks bridging separate functions for reach and grasp behaviors (Rizzolatti and Matelli, 2003; Daprati and Sirigu, 2006).

Similarly, tractography studies are beginning to reveal the detailed anatomical architecture of networks linking dorsal and ventral stream pathways, with direct anatomical connections between inferior parietal and temporal lobes being implicated (Heilman and Watson, 2008; Ramayya et al., 2010).

Here data from lesion mapping studies either implicating ventral premotor or inferior parietal areas in ideomotor apraxia (Haaland et al., 2000; Pazzaglia et al., 2008; Kalénine et al., 2010) may represent different facets of the same syndrome.

# Competition Leading to "Blocking" Effects and the Direct Route to Action Model

Previous neuropsychological studies describe several types of ''blocking effect'' in patients. For example patients with visual apraxia, who have intact object recognition and good gesturing to verbal command, may be poor at gesturing to visually presented objects (Riddoch et al., 1989). Traditional models of apraxia would predict that patients could use an intact semantic route to action (see **Figure 1**). However this example suggests that perceptual information interacts directly with semantic information in selecting the appropriate action to make to an object.

A convergent route model of action selection was proposed by Yoon et al. (2002) to account for this effect. They used an energy minimization network where the response derived from action selection is determined by convergent activation from separate semantic and perceptual representations. This convergent activation pushes the network into a stable state (e.g., a learned output to a given stimulus), which acts as an attractor (Hopfield, 1982). Any initial activation supplied is pushed by the dynamics of the network and by other incoming inputs into a ''basin of attraction''.

In a study by Chainay and Humphreys (2002), this model was able to accurately predict behavior in a number of apraxic deficits. Most notably these authors documented apraxic action errors in a patient with impaired semantic knowledge about objects. Despite this, the use of real objects improved action—without improving semantic identification. Chainay and Humphreys (2002) argued that the sensory/perceptual input directly impinged on action specification, facilitating selection of the motor programme.

Although Yoon et al.'s (2002) model was suited for mechanisms of human action selection, recent evidence from animal neurophysiology studies have revealed that a similar process takes place in non-human primates, for specification of reaching movements (Churchland et al., 2007). The dynamics of large scale neuronal populations were decoded to generate models that account for activity in primary motor cortex (Churchland et al., 2012).

In the affordance competition hypothesis, the dynamic nature of interactions in motor responding is modelled implicitly. Indeed action representations, cued by the environment are likely available within fronto-parietal circuits, akin to the aforementioned ''basins for attraction''. Biasing inputs from basal ganglia or specific cortical areas (e.g., frontal cortex, depending on the task set or parietal cortex, depending on changes in the environment) pushes the network towards a specific action by inhibiting unnecessary or competing ones.

# Subtypes of Apraxia and the Affordance Competition Hypothesis—Different Types of Affordances?

Considering apraxia under this framework reframes it as a set of disorders involving deficits in movement selection at different levels—selection of the overall movement leading to ideational apraxia or selection of specific movement parameters leading to ideomotor apraxia.

In the former case, one would predict deficits in object use arising due to there being problems in ''affordances'' triggering appropriate actions. This may occur because of competition between certain object characteristics which involve perception for actions, or affordances. Ideational apraxia may thus arise due to wrong actions being generated according to errors in affordances. For example, an object may be recognized for its use, yet present the actor with graspable features (affordances) that may be similar to other objects (e.g., grasping a toothbrush may be similar to grasping a knife) and lead to activation of subsequent action representations that are inappropriate for the object at hand. This maladaptive behavior may emerge from affordance triggering incorrect actions within a ''state space'' of action representation (cf. Chainay and Humphreys, 2002). Similarly, although the same object characteristics may generate appropriate affordances, these may in turn trigger several action representations. For example, the same object may be grasped for different uses [e.g., a pen may be grasped to write with or to move it to another location (Daprati and Sirigu, 2006)]. In this situation, patients may be unable to select from these competing actions (or inhibit them) such that an action that may be appropriate for the object but not specific to its use is performed (picking up a pen and toying with it, rather than using it to write).

In the case of ideomotor apraxia, affordance competition would predict that alternative effectors are substituted for performing an action. An example has been described by Bekkering et al. (2005). They replicated results from an original experiment reported by Goldenberg and Hagmann (1997) who used a hand and finger gesture imitation task. Meaningless gesture imitation has been used as a typical test of ideomotor apraxia because it is thought to test a ''direct pathway'' to gesture production that is not reliant on semantic memory or object knowledge. Bekkering et al. suggested that errors can arise in movement selection pertaining to a hierarchy of goals, with more mistakes in action selection for items lower in the action hierarchy (such as the effector used for an action) than those higher in the action hierarchy (such as action goal)—a pattern of errors also found in young children (Bekkering et al., 2000). Here, we suggest a hierarchy of goals based on a conceptual, idea-guided goal and subsequent perceptual-guided movements. Thus, when an action is observed, the action goal is observed rather than the specific movements.

Finally limb kinetic apraxia may arise because of failures in selecting the appropriate gesture or muscle configurations, from a range of possible alternatives, to perform a known and contextually relevant action (Haaland et al., 1999).

These different forms of apraxia, categorized by the affordance competition framework, may also be useful in identifying the neural correlates of the disorder(s). For example, we hypothesize that deficits in selecting appropriate actions corresponding to affordances arise from lesions to parietal cortex whereas deficits in the selection of gestures or finger movements would involve fronto-striatal circuits.

One important implication of our hypotheses is that patients with apraxia may exhibit various forms of response inhibition, due to their failure to resolve affordance competition. Studies have highlighted the importance of subcortical networks (Redgrave et al., 2010) and have identified separate top-down and bottom-up pathways in action selection (Rushworth et al., 2009; Cisek and Kalaska, 2010; Duque et al., 2012). Further studies are required to investigate whether response inhibition deficits, in their own right, contribute to apraxic deficits particularly

# References


in patient populations with basal ganglia disorders in whom apraxia has been documented (Pramstaller and Marsden, 1996; Leiguarda et al., 1997; Leiguarda and Marsden, 2000). These speculations require empirical tests.

# Conclusion

In this review, we present the affordance competition hypothesis and discuss possible implications for limb apraxia. We propose that this framework allows limb apraxia to be defined as a set of disorders in which patients are overwhelmed by the possibilities for action provided in the environment. Viewing behavior as a dynamic process in which action specification and selection occur in parallel allows for several observations to be explained such as the frequent co-existence of ideational and ideomotor deficits which have been debated at length. Moreover the framework introduces the concept of affordances as being a key trigger for action.

We believe that this framework will allow the generation of further studies through testable hypotheses that may help elucidate the complex and poorly understood disorder of apraxia.

# Acknowledgments

This work was supported by a grant from the National Institute of Health Research Clinical Research Facility in Oxford cognitive health (GWH). ER has obtained a Helen Lawson grant from the British Medical Association supporting work on limb apraxia.


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Rounis and Humphreys. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Diagnostics and Training of Affordance Perception in Healthy Young Adults—Implications for Post-Stroke Neurorehabilitation

#### Jennifer Randerath1, 2, 3 \* and Scott H. Frey 3, 4

*<sup>1</sup> Department of Psychology, University of Konstanz, Konstanz, Germany, <sup>2</sup> Lurija Institute for Rehabilitation Science and Health Research, Kliniken Schmieder, Allensbach, Germany, <sup>3</sup> Department of Psychological Sciences, University of Missouri, Columbia, MO, USA, <sup>4</sup> Program in Occupational Therapy, Department of Neurology and Department of Psychological and Brain Sciences, Washington University School of Medicine, St. Louis, MO, USA*

Affordance perception is critical to adaptive behavior. It comprises the ability to evaluate whether the environment and the actor's capabilities enable particular actions. It remains unclear how brain damage and its behavioral sequela impact this ability. Two affordance based judgment tasks were applied in healthy young adults that were adapted for prospective diagnostic purposes in patients. In addition to the commonly analyzed error-rate we included response times and accuracy measures based on a detection theory approach. Moreover, a manipulation was added intended to determine the effectiveness of feedback-based learning. We further applied control tasks that consider whether errors in affordance perception can be explained by errors in perception. Participants responded yes or no to decide prospectively if a given setting would afford a particular action. In study1, 27 participants judged whether their hand would fit through a given aperture (adapted from Ishak et al., 2008). In study2, 19 participants judged whether objects are reachable [adapted from Gabbard et al. (2005)]. For both studies two sessions were administered. In the first session all participants solved the judgment-task without executing the action. In the second session (feedback manipulation), half of the participants were allowed to first judge and then perform the task for each trial (reach forward and touch the object, or fitting the hand into the aperture). Judgments were slowest and errors most frequent for openings or distances close to the individual's actual physical limits. With more extreme settings accuracy increased and responses became faster. Importantly, we found an advantageous effect of feedback on performance in both tasks suggesting that affordance perception is rapidly trainable. Further, the aperture task demonstrated that feedback experienced with one hand can transfer to the other. This may have important implications for rehabilitation.

Keywords: affordance perception, perception-action, training, detection theory, transfer

#### Edited by:

*Ferdinand Binkofski, RWTH Aachen University, Germany*

#### Reviewed by:

*Katrin Sakreida, RWTH Aachen University, Germany Markus Martin, University Medical Center Freiburg, Germany*

> \*Correspondence: *Jennifer Randerath*

*j\_randerath@hotmail.com*

Received: *29 March 2015* Accepted: *30 November 2015* Published: *06 January 2016*

#### Citation:

*Randerath J and Frey SH (2016) Diagnostics and Training of Affordance Perception in Healthy Young Adults—Implications for Post-Stroke Neurorehabilitation. Front. Hum. Neurosci. 9:674. doi: 10.3389/fnhum.2015.00674*

# INTRODUCTION

Affordance perception comprises the perception of action opportunities, including processing the properties of the environment as well as one's own capabilities (Gibson, 1979). The theory of affordances points out the close relationship between perception and action. When navigating through our environment and interacting with tools and objects it is necessary to prospectively adapt our movement plan based on what we perceive. Appropriate affordance perception supports us in determining what actions we can and will execute. On the other hand it also helps determining what actions to avoid, when the environment or our bodily capabilities do not provide the appropriate conditions. Despite the tendencies to overestimate or underestimate abilities for certain tasks, healthy young adults are perfectly able to perform appropriate decisions such as when reaching for objects (Carello et al., 1989; Gabbard et al., 2005, 2006), passing between obstacles (Wagman and Malek, 2008; Higuchi et al., 2009) or fitting the hand into an aperture (Ishak et al., 2008). Few studies investigated the effects of training and exposure to actions on affordance based judgments. The results thus far are promising in that learning and improvements in these tasks have been demonstrated for healthy young adults. After training and exposure they show quick adaptation to new constraints (Mark and Vogele, 1987; Mark et al., 1990; Weast et al., 2011).

The function affordance perception is critical to adaptive behavior. Major misjudgments of action opportunities could lead to precarious situations, including such mishaps as limb injuries and falls. While slight misjudgments in healthy young adults may not breach safety boundaries, this may be different in patients with brain damage. When seated, healthy subjects for example typically overestimate what they can reach (Carello et al., 1989; Mark et al., 1997; Gabbard et al., 2011). Yet, they seem to adequately adapt their estimation criterion within their safety boundaries. For reachability judgments Gabbard et al. (2007) for example found less overestimation to be apparent in a standing vs. seated condition. The authors attribute this more conservative estimation in the standing position to greater perceived postural demands.

It is feasible that brain damage and resulting lost functions may affect adequate affordance perception, and in patients the tendency to be out in their estimation could be magnified dramatically. There is some evidence that the likelihood for falls may increase. For example, errors in perceiving postural limits by estimating the maximum reach of the non-affected side of hemiplegic patients correlates with high risk for falling (Takatori et al., 2009). However, neither the incidence for potentially undiagnosed deficient affordance perception after stroke nor the potential underlying mechanisms are thus far enlightened. It is therefore important to provide tools for diagnostics and training in affordance perception for patients with brain damage.

Notably, the picture of impaired affordance perception may be complex, since brain damage could affect this ability on diverse levels. First, brain damage can change bodily capabilities, for example by causing hemiplegia. These new body constraints have to be taken into account when planning and executing actions. Interestingly, some (Johnson, 2000; Johnson et al., 2002), but not all (Buxbaum et al., 2005) stroke patients retain the ability to plan movements that have become impossible due to hemiplegia with remarkable accuracy. This has a potential downside, as it could lead to attempting now impossible actions and precipitate costly errors including failed actions, unstable postures and falls. Second, impaired cognitive functions may correlate with the ability to adequately perceive affordances. Left brain damage due to stroke can lead to problems in action planning (Rushworth et al., 1998; Buxbaum et al., 2005; Sunderland et al., 2011), which typically is attributed to malfunctions of a left lateralized praxis network. The associated cognitive motor disorder summarizing resulting problems like selecting or producing inappropriate actions is called limb apraxia (Goldenberg, 2013). One underlying mechanism may be the impaired integration of information into an action plan, that includes processed information about environmental properties and own body parts (Frey, 2007; Randerath et al., 2009, 2011). These are essential aspects of the concept of affordance perception, thus limb apraxia may correlate with disturbances therein. Further, right brain damage may lead to visuo-spatial neglect and impair the perception of spatial properties in the contralesional hemispace (Karnath et al., 2011). Interestingly, several studies demonstrated that actual reaching or grasping even in the contralesional field is similar to that in controls or patients without neglect (Himmelbach and Karnath, 2003; Mcintosh et al., 2004; Harvey and Rossit, 2012). Yet, while these studies indicate that neglect patients perform relatively better in action tasks, their severe visuospatial impairments may affect prospective judgments about action opportunities. Both, new constraints as well as cognitive disabilities may affect affordance perception in stroke patients.

Attempts to study possible disruptive effects of impairments after stroke on prospectively judging action opportunities are scarce. We here present a paradigm applied in healthy young adults, which measures the ability to judge action opportunities. With the future goal to evaluate preserved or impaired affordance perception in the stroke-patient population, our approach takes known challenges such as aphasia, neglect and hemiparesis into account by using simple instructions, limited number of trials (doable within 30 min) and factoring in difficulties with attention to the contralesional hemi-space as well as the use of only one hand. Moreover, the paradigm includes the possibility of training and thus potentially improving behavior.

In addition to typical accuracy percentages we used a detection theory approach to analyze our data. When affordance based judgments are required, subjects' decision making may be influenced by a number of different factors including their perceptual sensitivity and their response biases. Participants may decide whether an action is possible by comparing observations with a criterion. We therefore calculated subjects' discriminability, response bias, and diagnostic accuracy with the help of detection theory (Green and Swets, 1966; Fox, 2004; Macmillan and Creelman, 2004). Further, we tested for positive effects of task exposure on affordance based judgments, and we explored whether feedback presented to one side of the body may be transferred to the other side.

In two studies we evaluated response time (RT) and accuracy measures for separate tasks (study1: aperture-task; study2: reachtask) without and with feedback. Each study required healthy young adults to make judgments about body-object relationships. For both studies two sessions were administered. In the first session participants judged (yes or no) whether for a given setting a particular action is possible (study1, aperture-task: fitting the hand into the aperture; study2, reach-task: touching the object). In the second session, half of the participants were allowed to first judge and then actually perform the task for each trial (Experimental-group), and the other half (Controlgroup) once more merely judged whether the given setting allows the particular action. In study1 the task was to judge whether the hands can fit through an aperture. In study2 we looked at how well seated people determine if an object is within their reach while bending forward is allowed. Hence, the latter task introduces more degrees of freedom.

Participants were confronted with a fixed set of increments based on the individual's capabilities (maximum reach or smallest possible aperture to fit in) measured at the beginning of each session<sup>1</sup> . With this approach it has been shown that for certain increments reachability judgments vary within individuals. Healthy young adults have no problems judging items that are located further away from their actual reach limit, but for items positioned very close to the actual reach limit error-rates are close to chance (Gabbard et al., 2007). In the current study, we predicted that judgments are slower and less accurate when decisions have to be made for settings that are closest to the actual physical limits compared to more extreme settings, independent from the used paradigm (opening width or object distance). Significant increases in accuracy due to feedback were expected for the Experimental-group only in the second compared to the first session.

We further expected affordance perception to engage a complex network of components involved in motor cognition. Perception of environmental properties (such as size or distance) is one of these facets and may correlate with the ability to judge action opportunities. In order to determine the relationship we added a size adjustment-task to the aperture paradigm and a depth perception task to the reachability paradigm.

# STUDY 1: APERTURE TASK

# Method

#### Participants

Twenty seven individuals (14 female; mean age = 21 ± 3.7 years) from the University of Oregon participated in the twosession study. All participants were right-handed according to the Edinburgh Handedness Inventory (Oldfield, 1971), had normal or corrected-to-normal vision (at least 30 ft/9 m), and were naïve to the specific goals of the study. Participants provided informed consent in accordance with the local IRB and the Declaration of Helsinki. The study took approximately 2 h to complete (45–60 min per session). Participants received financial or study credit compensation. Next to the experimental tasks (approximately 30 min) they completed the consent-form, a handedness questionnaire, a vision test and received studydebriefing. Half (N = 14) of the group were assigned to an Experimental-group receiving feedback in the second session, the other half served as Control-group. Participants were randomly assigned.

### Materials and Procedures

#### **Material**

The aperture apparatus was custom made for this project and mounted on a height adjustable table (**Figure 1**). The width and height of a centrally placed rectangular opening was manipulated manually by the experimenter with knobs on the back of the device. The participants' hands (left = LH, right = RH) served as stimuli for two tasks—a size perception and an affordance based judgment task (Aperture task). Participants wore platogoggles throughout the experiment to control for visual feedback and to allow for response time measurements (Translucent Technologies Inc.).

#### **Measurements**

Each session started out with measuring the width of the two hands in the aperture by closing the opening tightly around the flat hands with fingers closely spaced. For the measuring procedures the goggles were turned opaque to avoid visual feedback. The hands were measured at the widest part which was defined as the horizontal distance from the outer side of the pinky to the outer side of the thumb, the measurement was taken at the transition of the proximal phalanges and metacarpal bones. During the sessions the vertical opening size was always set to the thickness of the individual's hand to be judged upon.

FIGURE 1 | Setting for the aperture task. Participants were seated centrally in front of the apparatus, with the eyes being on the level of the opening and their tip of the nose being positioned vertically in line with the edge of a dowel fastened below the table (about 45.5 cm from the apparatus). Manual adjustments of the aperture were possible using a belt mechanism in the back of the apparatus; dimension specifications allowed for measuring millimeter increments. Participants wore plato-goggles (here: opaque) throughout the experiment to control for visual feedback and to allow for RT-measurements. Goggles were opened for the start of each trial. The rightwards shifted positioning of the response box reflects an affordance judgment block for active right hand judgments.

<sup>1</sup>While all increments here are treated as if fitting exactly into yes/no-response– categories it needs to be noted that actual capability can sometimes vary minimally across repeated trials. This applies to those settings that are one increment lower or higher than the actual capability measure.

Frontiers in Human Neuroscience | www.frontiersin.org January 2016 | Volume 9 | Article 674 |

# **Affordance based judgments**

In the so called Aperture task participants had to determine whether they would be able to fit the hand through a given aperture. The horizontal width of the opening was varied using fixed negative and positive increments according to each hand's size (opening: −1.6, −0.8, −0.4, −0.2, ±0, +0.2, +0.4, +0.8, and +1.6 cm). Participants solved 36 (4×9 openings) plus 4 filler trials per hand, resulting in a total of 80 trials. The 0-trials reflected the measurement, i.e., the smallest possible size of the opening when the hand could fit into the aperture. To avoid an imbalance toward more frequent yes-trials, we added filler trials for which the correct answer would be "No" (smaller than −1.6 cm).

It was blocked whether judgments were made for the left or right hand (group A/B: LRLR or RLRL). Blocked presentation was used in anticipation of applying this paradigm in stroke patients with potential hemiparesis, who are not able to change the hand's position frequently. Furthermore, stroke patients may be restricted to use their unaffected hand for indicating their judgments with a button press response. Thus, in the current study our healthy participants were assigned to two groups: Half of the group indicated their responses via button press with their left hand, the other half with their right hand. That way we could analyze whether hand-dominance may play a role when indicating the response. Participants always started with a block of judgments for the assigned button pressing hand, then they judged the other hand (while indicating the response with the assigned hand). Here we name the hand that always indicates button responses the active hand. The other hand is called the passive hand. Stimuli were positioned onto a mark. If the active hand served as stimulus, the response-box was set on top of the stimulus-mark,—if the passive hand was judged the response-box with the active hand was moved 8 cm toward the outer edge of the apparatus to allow for the passive hand to be positioned on the stimulus-mark. The stimulusmark was slightly shifted from the aperture's midline (3.5 cm) to avoid direct alignment strategies. In anticipation of applying this paradigm in stroke patients with potential neglect, the shift was yoked with the group either toward left hemispace (group A, left hand active) or right hemispace (group B, right hand active). Before each block participants were reminded about what hand they next had to base their judgments on. Each block started with 2 demonstration trials, presenting an extreme small or wide aperture. Summarized, participants solved 4 blocks judging whether one specified hand could fit into a given opening, two blocks of judgments were made for the assigned active hand (first, third) and two for the passive hand (second, fourth).

#### **Feedback session**

In order to see whether participants' judgment would profit from experience, in a second session the Experimental-group (E) was instructed to try to fit their assigned active hand into the opening with vision being provided. Feedback was automatically delivered in case of a successful fit through: participants touched a to handlength distance adjusted back-board, that triggered a bell. After experiencing one hand in the aperture (45+5 filler trials), subjects had to solve a judge-only block for the passive hand (36 + 4 filler trials). The Control-group had to solve all trials without the exposure, but with judgment only.

Please note, as described earlier for half of the total group the button pressing active hand was the dominant right hand and for the other half the assigned active hand was the non-dominant left hand. As described above we assigned an active hand because we wanted to test a paradigm suitable for unilateral stroke patients with hemiparesis of their left or right arm. We further divided the group according to condition into the experimental group that received feedback and the control group that did not receive feedback. In the experimental group only the active hand received feedback (again because patients would not be able to use their paretic arm). For half of the experimental group the active hand was the non-dominant hand. Thus, half of the experimental group received feedback for their dominant right hand and the other half of the experimental group experienced their non-dominant left hand in the aperture.

#### **Size-estimation task**

In the size-estimation task we assessed the ability for horizontal size perception. Subjects had to decide when a gradually adjusted opening width had the same size as the widest part of the hand (say stop with the allowance to correct). Horizontal startopenings were varied: In half of the 8 trials the horizontal width was gradually decreased starting from a 20 cm opening, in the other half openings were gradually increased starting from 0 cm. The vertical width was kept constant during the experimental conditions (set to individual's hand height). Left and right hand were presented in a fixed randomized order.

Half of the participants started with the control task first, half started with the affordance based judgments.

### Data-Analyses

The data-analysis was divided into two sections: section A. assessed the influence of different variables on overall judgment accuracy and response times (RT), and section B. used a detection theory approach.

#### **ANOVA**

Judgment accuracy (%) and RT (ms). First we ran a control analysis to test potential confounding effects of group assignment and gender. An ANOVA with the variables group (Experimental/Control) and gender (male/female) was applied for the affordance based judgment task accomplished in the first session.

The greatest interest was in analyzing effects of feedback and opening. However, which hand had to be judged may additionally influence RT and judgment accuracy and potentially it also may affect how quickly feedback is integrated. Judgment accuracy for example could be modulated by whether the hand to be judged had to press the button or remained passive or whether in our right hand dominant sample the hand to be judged was left or right hand. We therefore ran two separate analyses taking these two variables of hand into account. Hand dominance, opening and/or feedback were fed into a repeated measures ANOVA with between subjects variable group (Experimental and Control) and within subjects variables hand (left/right), opening (−1.6, −0.8, −0.4, −0.2, ±0, +0.2, +0.4, +0.8, and +1.6 cm)

and session (1 and 2). Further, a repeated measurements ANOVA was computed with between subjects variable group (Experimental and Control) and within subjects variables hand (active/passive), opening (−1.6, −0.8, −0.4, −0.2, ±0, +0.2, +0.4, +0.8, and +1.6 cm) and session (1 and 2).

Analyzing size perception (cm). To see whether hand dominance, start-opening and/or gender played a role a repeated measures ANOVA was computed with between subjects variable gender (male/female) and within subject variables hand (left/right) and start-opening (0/20).

The correlation between size-perception and accuracy was analyzed (Pearson).

# **Detection theory approach**

To analyze response tendencies we calculated subjects' discriminability, response bias and diagnostic accuracy with the help of detection theory (Green and Swets, 1966; Fox, 2004; Macmillan and Creelman, 2004).

The discriminability index d′ . The discriminability index is a measure of the subjects' perceptual sensitivity that is independent of the criterion. The more sensitive the participant is at discriminating between reachable and non-reachable targets, the larger the d′ value will be. Its calculation is described below:

> d ′ <sup>=</sup> Z(Hit Rate) <sup>−</sup> Z(False Alarm Rate).

False Alarm Rate = (No. of False Alarms)/(No. of Actual

Negative Cases)

Hit Rate = (No. of Hits)/(No. of Actual Positive Cases)

Please note, to correct for Hit and False Alarm (FA) Rates of 0 (z would become infinite), we used the following standard correction (Macmillan and Creelman, 2004, p. 8):

If Hit Rate <sup>=</sup> <sup>1</sup> : Hit Rate <sup>=</sup> <sup>1</sup> <sup>−</sup> <sup>1</sup>/(2∗No. of Actual Positive Cases).

If FA Rate <sup>=</sup> <sup>0</sup> : FA Rate <sup>=</sup> <sup>1</sup>/(2∗No. of Actual Negative Cases). If FA Rate <sup>=</sup> <sup>1</sup> : FA Rate <sup>=</sup> <sup>1</sup> <sup>−</sup> <sup>1</sup>/(2∗No. of Actual Negative Cases).

If Hit Rate <sup>=</sup> <sup>0</sup> : Hit Rate <sup>=</sup> <sup>1</sup>/(2∗No. of ActualPositive Cases).

Response bias. The participant's strategy is revealed by the sign of the response bias c. When c is negative the participant is liberal (i.e., responds Yes more often than the ideal observer). When c is positive the participant is conservative (i.e., responds No more often than the ideal observer). Its calculation is as follows:

> <sup>c</sup> = −0.5<sup>∗</sup> [Z(Hit Rate) + Z(FA Rate)].

ROC curves. Diagnostic accuracy can be demonstrated by Receiver Operating Characteristic (ROC) curves. ROC curves represent a graphic description of how the Hit Rate of an observer changes as a function of changes in the False Alarm (FA) Rate. The Area Under the Curve (AUC) reflects perceptual accuracy by combining sensitivity and specificity into a single value. Plots representing perfect discrimination pass through the coordinates 0 and 1. These indicate 100% Sensitivity (Hit Rate, sensitivity) and Specificity (FA Rate, 1-specificity) and represent an AUC value of 1. According to an arbitrary guideline (based on a suggestion by Swets, 1988), one may distinguish between noninformative (<0.5), less accurate (0.5–0.7), moderately accurate (0.7–0.9), highly accurate (0.9) and perfect ratings (1).

# Results and Discussion Study1

#### Affordance Based Judgments

First we tested for confounding effects of group assignment (Experimental/Control) and gender (male/female) on affordance based judgments in the first session. This ruled out effects of both factors in the baseline accuracy- and RT-data [F(1,23) < 2.75, p > 0.11].

In order to evaluate the paradigm, the major goal was to test effects of group, opening, and session. Furthermore, to see whether hand dominance plays a role or whether there is an effect of hand indicating vs. not indicating the response two separate repeated measurements ANOVAs were computed with between subjects variable group (Experimental vs. Control) and within subjects variables hand [(left vs. right) or (active vs. passive)], opening (−1.6, −0.8, −0.4, −0.2, ±0, +0.2, +0.4, +0.8, and +1.6 cm) and session (1 vs. 2). For a detailed overview, please see Supplements (Supplementary Table 1 for descriptive statistics and Supplementary Table 2 for F- and p-values).

As expected, openings closer to the actual hand-size were judged more poorly and slower compared to more extreme openings (**Figure 2A**). The tests of within subjects contrasts show this main effect of opening to be quadratic [accuracy: F(1, 25) = 461.20, p < 0.001; RT: F(1, 25) = 30.62, p < 0.001].

The predicted judgment improvement after feedback is confirmed. The interaction session\*group demonstrated an increase in accuracy in the second compared to the first session only for the Experimental-group [Bf-p = 0.025; t(13) = −2.83, p = 0.014], but not for the Control-group [t(12) = 1.66, p = 0.126; see **Figure 2**]. Further, it was found that in the second session the Control-group initiated faster responses compared to the Experimental-group [t(24.8) = −2.87, p = 0.008]. The interaction with opening revealed that the RT advantage in the Controlgroup is specific to trials ranging close to the actual hand-fit for which decisions according to the accuracy results appear to be more difficult [opening<sup>∗</sup> session<sup>∗</sup> group interaction: Bf-p = 0.006; −0.2: t(19.0) = 3.24, p = 0.004; 0: t(22.6) = 4.09, p < 0.001; +0.2: 0: t(21.8) = 3.06, p = 0.006; +0.4: t(24.6) = 3.35, p = 0.003]. A possible explanation for these group-specific differences in the second session is the formation of a stable criterion during the first session. Only the Control-group was able to use a stabilized criterion in the second session, which may have enabled faster RT compared to the first session [t(12) = 3.48, p = 0.005]. In contrast the Experimental-group had to reset the individual's criterion to integrate the feedback information and develop a new response strategy. This group therefore shows a similar response initiation in both sessions [t(13) = −0.68, p = 0.509]. Importantly, no differences between groups were found for the first baseline

FIGURE 2 | Accuracy and RT in the aperture task. (A) Displays accuracy (left) and RT (right) distributions across openings per group and session. The graphs show the main effect of opening with judgment accuracy being lower (left) and judgments being slower (right) for those openings that are close to the physical constraints of the hands' fit into the aperture (0 cm) compared to more extreme deviations (e.g. ±1.6 cm). The Experimental-group significantly increased judgment performance in session 2, which is most obvious for openings close to the hands'constraints. For trials ranging close to the constraints (openings: −0.2 to +0.4) the Control-group initiated responses faster in the second session than participants who received feedback (Experimental-group). Error-bars represent 95% confidence intervals. (B) Displays accuracy and response times per session, group and hand. Participants indicated responses with one designated hand (active hand), the other hand is called the passive hand. Left: In the second session the Experimental-group showed overall increased accuracy compared to the Control-group. Within the Experimental-group responses are significantly more accurate for the passive hand, which was judged subsequent to the learning experience with the active hand. The accuracy data implies that the improved criterion was transferred to the following passive hand trials. Right: Slower RTs for judgments when experiencing the active hand in the opening may reflect a reevaluation taking place during the feedback trials. Generally, faster responses in the second session within the Control-group suggest that this group based decisions on a judgment criterion settled in the first session. Error-bars represent 95% confidence intervals.

session in RT or accuracy [Bf-p = 0.025; RT: t(22.7) = 0.25, p = 0.806; accuracy: t(17.7) = −1.71, p = 0.111].

The affordance based judgment task is thought to be solved by a higher order motor cognition capacity. We therefore were interested to see transfer effects induced from the hand experiencing feedback (active hand) toward the hand that was not exposed to the actual constraints of the aperture (passive hand). When distinguishing between active and passive hand an interaction between hand, session and group occurred for affordance judgment accuracy (**Figure 2B**). Post-hoc analyses demonstrated that within the Experimental-group responses in the second session are more accurate for the passive compared to the active hand [Bf-p = 0.004; t(13) = −4.49, p = 0.001]. These results are at first sight puzzling, but can be explained by the way the experimental session is set up. Only the active hand is engaged in feedback trials. When judging for the active hand the participant is in the learning phase, whereas judgments for the passive hand are made in a subsequent block. Greater accuracy for the passive hand suggests that learning has been transferred.

In line with this, there is a tendency of the Experimentalgroup to respond slower with the active hand during the feedback trials compared to the following passive hand trials [Bf-p = 0.004; t(13) = −2.82, p = 0.015]. The RT-delay for the active hand experiencing feedback in the Experimental-group can be

explained by enhanced computational demands. After indication of the response, the participant is required to gently try to fit the hand into the opening in order to obtain feedback. RT may have been influenced by planning the subsequent action of the active hand toward the opening and participants had to update their evaluations according to feedback.

Against expectations there was no main effect of handdominance with respect to RT, instead an effect of hand dominance was found for accuracy (**Figure 3**). Surprisingly, judgments for the non-dominant left (M = 79.4%, SD = 8.3) compared to right hands (M = 76.8%, SD = 7.3) were significantly more accurate [F(1.0,25.0) = 5.83, p = 0.023].

Although, the interaction of hand\*opening did not reach significance, there was a trend for subjects to respond "no" more often for the right compared to the left hand when it actually could fit through (increments: +0.2, + 0.4 cm). One may argue that judgments that involve actions with the right hand may be based on a larger hand representation. Notably, as will be shown hereafter, left and right hand did not differ significantly in size, and size-estimations for the two hands did not deviate. A conceivable explanation is that these judgment-errors for the right hand are not based on errors in size-estimation but based on the perception of extended action boundaries for the dominant hand. This has been suggested before based on evidence that subjects perceive themselves being able to grasp bigger objects with their right compared to their left hand (Linkenauger et al., 2009, 2011).

### Size Estimations

The average hand size was 9.7 cm (SD = 0.9), for male participants this was 9.9 cm (SD = 0.9), females' hands were 9.5 cm on average (SD = 0.8). Left (M = 9.6, SD = 0.9) and right (M = 9.7, SD = 0.9) hand sizes did vary within some participants but on a group level these did not differ significantly [t(26) = 1.40, p = 0.174].

Size-estimations were defined by the difference of the estimated minus actual hand-size in centimeter (cm). On average, the size of the hands was overestimated (M = 1.05, SD = 1.0; Min = −0.61, Max = 3.35).

An ANOVA with gender (male/female), hand (left/right), and start-opening (0/20) was run. There was no significant effect of gender [F(1, 25) = 0.30, p = 0.590]. Size-estimations for left and right hands did not differ [F(1, 25) = 0.02, p = 0.879].

Despite that subjects were encouraged to adjust their first attempt whenever they felt it to be necessary, their estimations differed between the closing vs. opening adjustment. Hand-sizeestimation for gradual outward-adjustments starting from 0 cm (M = 0.9, SD = 0.9) were significantly better than estimations for inward-adjustments starting from 20 cm [M = 1.2, SD = 1.0, F(1, 25) = 13.85, p = 0.001; see **Figure 4A**]. This could potentially be attributed to firstly subjects overestimating their hand-size and secondly to their demand for safety. Participants may imagine their hands in the opening and respond with stop relatively fast for the closing aperture to ensure their hand's safety.

### Correlations

We calculated correlations between actual average hand-size, size-estimations, accuracy-judgments, and RT for the aperture paradigm. For the size-estimation we first multiplied all negative differences with −1. The actual hand-size itself did not correlate significantly with accuracy (r = 0.27, p = 0.170) nor sizeestimation (r = −0.34, p = 0.085).

None of the variables correlated significantly with RT (p > 0.523).

We found a significant correlation between the mean difference for size-estimations and accuracy in affordance judgments (Pearson: r = −0.66, p < 0.001). The more deficient the size-estimation the lower the affordance judgment accuracy (see **Figure 4B**). Interestingly, in accordance with size estimations being off by about 1 cm on average (SD = 1), the entire sample appears to achieve 100% accuracy between 0.8 and 1.6 cm for affordance based judgments.

### Detection Theory Approach

Our main interest was in analyzing effects of feedback. The detection analysis confirms improvement in judgments for the passive hand as a result of preceding feedback experienced with the active hand in the Experimental-group only. The effect is shown through pairwise comparisons between sessions per group (Bf-p = 0.025). In the first session the mean criterion is close to zero (M = 0.05, SD = 0.68, MD = 0.15) indicating no extreme bias. However, there is a tendency of participants to erroneously say "no," judging that their hand cannot fit into the aperture when it actually could. After feedback, a significant increase in Sensitivity [d′ : t(13) = −3.95, p = 0.002] and accuracy [AUC: t(13) = −8.36, p < 0.001] is achieved for the passive hand in the Experimental-group. A major contribution to this improvement is the rise in the Hit Rate (equals a reduction of miss rate), which however, on its own does fail to reach statistical

significance (**Figure 5**). Please see Supplementary Table 3 for further descriptive statistics and t- and p-values.

Some of our results are not in line with previous reports analyzing affordance based judgments when fitting one's own hand into an aperture (Ishak et al., 2008). We replicated that participants scaled their motor decisions to their body dimensions. However, while we find a close to ideal criterion with a tendency toward a rather conservative approach, Ishak et al. found that participants wedged their hands into apertures that were one centimeter smaller than their actual fit. Further, Ishak et al. did not find an effect of hand dominance, but in our study judgments for the right hand were significantly worse, reflecting a more conservative approach than with the left hand. The differential findings can be explained by differences in the approach. In the study by Ishak et al. affordance perception was measured in the rate of attempts to fit the hand through a diamond formed aperture. The authors point out that their experimental situation included low penalty for errors. Instead, participants were rewarded with candies when successfully reaching through the aperture to grasp the incentive.

# STUDY 2: REACHABILITY TASK

# Method

# Participants

Nineteen individuals (12 female; mean age = 23 ± 2.6 years) from the University of Missouri participated in the two-session study. All participants were right-handed according to the Edinburgh Handedness Inventory (Oldfield, 1971), had normal or aid corrected-to-normal vision (at least 30 ft/9 m), and were naïve to the specific goals of the study. Participants provided informed consent in accordance with the local IRB and the Declaration of Helsinki. Participants solved two sessions. The study took approximately 2 h to complete (45–60 min per session), and participants received financial or study credit compensation. Next to the experimental tasks (approximately 30 min), participants completed the consent-form, a handedness questionnaire, a vision test and received study-debriefing. We randomly assigned half (N = 10) of the participants to an Experimental-group receiving feedback in the second session, the other half served as Control-group.

#### Materials and Procedures

#### **Material**

The custom made reaching apparatus consisted of a height adjustable table with three tracks mounted onto it as well as three rectangular objects with sensors (**Figure 6**). The objects could be manually moved within the tracks. On each track one object was presented. Object-distances were manipulated manually with the help of mounted measurement-tapes. The table height was adjusted to each participant's solar plexus. Participants were seated 25 cm away from the table on a rigid chair wearing a seatbelt. Participants wore plato-goggles throughout the experiment to control for visual feedback and to allow for RT measurements.

#### **Measurements**

Each session started out with measuring the maximum reach of one assigned side (left, right). Without vision participants had to push each object with their index-finger along the track as far as possible while bending forward was allowed but the bottom needed to stay seated. This was repeated 3 times and the maximum value was used for further settings. The seatbelt and table-edge prevented participants from falling, in case they would lose their equilibrium while reaching forward. For the measuring procedures the goggles were turned opaque to avoid visual feedback.

#### **Affordance based judgments**

In the affordance perception task participants had to judge whether a presented object was within reach. Reachability judgments were made for one assigned side of the body, the same side the subject pressed the response buttons with. The distance of the objects was varied using fixed negative and positive increments according to the maximum reachability

sessions (1, 2) for the active and the passive hand respectively. In (B) the ROC curves for individual participants are displayed (Control-group C: gray lines; Experimental-group E: black lines) for each session. 1-Specificity reflects the False Alarm Rate, and the Sensitivity indicates the Hit Rate. The reference line is at chance level (Ref. Line: light gray dots). The area under the curve is a measure of accuracy, which is perfect when FA = 0 and Hit = 1 (in the upper left corner). Accuracy significantly improved after feedback in session 2 for the subjects that were assigned to the Experimental compared to the Control-group.

FIGURE 6 | Setting for judging reachability. Participants were seated centrally in front of the height adjustable reaching apparatus, 25 cm from the table edge. The table's height was set underneath breast-level. Distance-measurements were possible using measurement-tapes mounted at the end of each track. Dimension specifications allowed for measuring millimeter increments. Participants wore plato-goggles (here: opaque) throughout the experiment to control for visual feedback and to allow for RT measurements. Goggles were opened at the beginning of each trial. The positioning of the response box reflects an affordance judgment block for the left side.

(distance: −16, −8, −4, −2, ±0, +2, +4, +8, and +16 cm). Per session participants solved 54 (2 blocks × 3 tracks × 9 distances) plus 6 filler trials, resulting in a total of 60 trials. The 0 trials reflected the measurement, i.e., the individual's maximum reachability. To avoid an imbalance toward more frequent Yestrials, we added filler trials for which the correct answer would be "No" (further than +16 cm).

In anticipation of applying this paradigm in stroke patients with potential hemiparesis, who are not able to use one arm, the task was solved for one side only. Participants in the current study indicated the response with the same side that they were to judge, e.g., if reaching ability was judged for the right arm, button presses were executed with the right hand and vice versa. Half of the group always indicated their responses via button press with their left hand, the other half with their right hand. The response box was positioned between two tracks, either left or right from the center depending on what side the participant had to judge for and press the buttons with. In anticipation of applying this paradigm in stroke patients with potential neglect, before the plato-goggles opened it was verbally communicated on which track the next object would be presented (left, right, and middle). This allowed orientation toward the correct side. The experiment started with 2 demonstration trials, presenting an extreme close or far distance on the left and right track.

In total subjects solved two sessions with each two blocks of 30 trials judging whether a given object was within reach (session1: block 1 and 2; session2: block 3 and 4).

#### **Feedback session**

In the second session all participants started out with an introductory block of 36 trials. In order to see whether participant's judgment would profit from experience, for this introductory block the Experimental-group (E) was instructed to first indicate their response as soon as the goggles opened and then try and reach toward each presented object. Feedback was automatically delivered in case of a successful touch of the sensor that was registered via the Superlab software and triggered a sound. The other half of the sample had to solve the same trials without the exposure, but with judgment only. After this introductory block (which was not included in data-analysis), all participants solved two more blocks with 30 trials each judging whether a presented object was within reach.

# **Depth-perception task**

In the depth-perception task subjects had to decide when a gradually adjusted object on the track was aligned with a rigid object next to the track (say stop with the allowance to correct). Start-positions for the movable object on the track were set either about 8 cm before or behind the rigid object. Thus, for alignment, the object on the track was either gradually moved toward or away from the rigid object until the participant said stop. To cover the range of distances on each track the rigid object was presented for two distances: +16 and −16 cm from actual maximum reach. This results in a total of 12 trials (3 tracks × 2 start-positions × 2 distances). Trials were presented in a fixed randomized order.

Half of the participants started with the Depth-Perception task first, half started with the affordance judgments.

## Data-Analyses

Similar to study 1 the data-analysis is divided into two sections: section A. assesses the influence of different variables on overall judgment accuracy (%) and response times (RT), and B. using a detection theory approach. Below we describe the RT and accuracy variables assessed for the reachability paradigm.

#### **ANOVA**

Judgment accuracy (%) and RT (ms). To test for confounding effects of group assignment and gender in the affordance perception task an ANOVA with the variables group (Experimental/Control) and gender (male/female) was run for the first session. Our main interest was to assess the effects of distance and feedback. In addition we analyzed potential effects of hand dominance and visual field (track). To allow for full cells two repeated measurements ANOVAs were computed. The first ANOVA included the between subjects variables hand (left/right) and group (Experimental and Control) and within subjects variables track (left/right/middle) and session (first vs. second). The second ANOVA included the between subjects variable group and within variables distance (−16, −8, −4, −2, ±0, +2, +4,+8, and +16 cm) and session (1, 2).

Depth perception (cm). To see whether track, start-distance, and/or gender plays a role a repeated measurements ANOVA was computed with between subjects variable gender (male/female) and within subject variables track (left/right/middle) and startposition (+8/−8). Furthermore, the correlation between depthperception and accuracy was analyzed (Pearson).

#### **Detection theory approach**

The used detection theory approach is identical to study1.

# Results and Discussion Study2

To test effects of group assignment and gender for the affordance perception task in the first session an ANOVA with the variables group (Experimental/Control) and gender (male/female) was run for accuracy- and RT-data, that ruled out effects of both factors [F(1,15) < 1.55, p > 0.23].

We ran two repeated measures ANOVAs to test effects of the variables group, track, hand, distance, and feedback on RT and accuracy. Please see Supplementary Table 4 for a full list of F- and p-values. We proposed that distances close to the actual reach are hardest to judge and that feedback will improve behavior.

We found a main effect of session [F(1.0,17.0) = 27.01, p = 0.006] and a quadratic effect of distance [F(1.5,25.3) = 27.01, p < 0.001] for accuracy as well as an interaction between the two variables [F(2.5,42.1) <sup>=</sup> 3.32, <sup>p</sup> <sup>=</sup> 0.037]. The group<sup>∗</sup> session interaction did not reach significance. For RT, there was a main effect of session [F(1.0,17.0) = 11.88, p = 0.003] and distance [F(2.6,44.9) = 11.08, p < 0.001]. Paired testing for all distances or the interaction would result in too many comparisons, therefore we here refer to the descriptive images **(Figure 7)**. As predicted accuracy appeared lowest and RTs delayed for distances that were slightly further away than actual maximum reachability. Moreover, reachability judgments were most accurate and quickest for distances close to the participant, replicating previous findings (Gabbard et al., 2007). The main effect of session demonstrates a general improvement in both groups. Participants improved and judged faster over the course of testing. In the first session judgments (accuracy: M = 74.8, SD = 5.2; RT: M = 961.4, SD = 382.0) were less accurate and slower compared to the second session (accuracy: M = 81.5, SD = 8.8; RT: M = 761.4, SD = 259.9). General improvement could be explained by repeated exposure to the task. Further, it may be possible that being exposed to the first session led to more conscious reaching in everyday life activities, which in turn may have led to uncontrolled feedback effects. These possibilities underline the necessity of including a control-group. However, the general trend toward improvement in both groups, as shown by the main effect, may have masked the role of feedback in the second session. To clarify this point we post-hoc compared the sessions within groups. In accordance with our hypothesis, only within the Experimental-group accuracy improved significantly in the second session, and despite the trend toward the same

Randerath and Frey Diagnostics and Training of Affordance Perception

direction significance was not reached within the Control-group (**Table 1**).

The second ANOVA included between subjects variables hand (left/right) and group (Experimental and Control) and within subjects variables track (left/right/middle) and session (first vs. second). The main effect of session is described above. Between subjects there was no effect of left vs. right hand assignments, therefore hand dominance does not seem to make a significant difference for reachability judgments. There was no effect of track for accuracy values. Although, participants were cued about where the object was presented before the goggles opened, RTs were significantly faster for objects presented on the middle track (M = 801.7, SD = 270.8) compared to objects presented on the left (M = 871.7, SD = 285.2) or right tracks [M = 899.4, SD = 332.6, t(18) > 4.37, p < 0.001]. Except for the directional cues there was no particular instruction for how to position the head or eyes. Hence, this delay of responses toward the side could be due to shifting or calibrating the field of view into an optimal position for judging the laterally presented objects and/or due to an attentional shift from midline (Posner et al., 1980; Remington, 1980).

#### Depth Perception

In the depth perception task subjects had to say stop as soon as the object moving along the track was aligned with an object positioned next to the track. The dependent variable was measured as the difference between the two objects resulting from the subject's verbally indicated adjustment. On average participants were able to make depth-perception adjustments with 1-mm accuracy (SD = 0.1, Min = −0.1, Max = 0.4).

A repeated measures ANOVA was run with within subjects factors track (left, right, and middle) and presented distance of the fixed object (+16 cm or −16 cm from the participant's maximum reach). The analysis revealed an interaction between distance and track [F(1.9,33.6) = 10.90, p < 0.001]. Pairwise comparison demonstrated that only for the middle track judgments were significantly worse for objects presented faraway (M = 0.29 cm, SD = 0.27) compared to those nearby [M = − 0.02 cm, SD = 0.14, Bf = 0.005, t(18) = 4.44, p < 0.001; see **Figure 8**]. The miss-estimation for the far-away objects is consistent with an underestimation of depth (Saunders and Backus, 2006).

#### Correlations

We calculated correlations between actual average maximum reach, depth-estimations, arm-length estimation and accuracyjudgments, and RT for the reach paradigm. For the estimationvalues we first multiplied all negative differences with −1. Participants' arm-length was measured from the shoulder to the index finger-tip. The maximum reach has been measured from the table-end and therefore lower values reflect further reaches and cause the correlation with arm-length to be negative. At the end of the study participants were asked to estimate their arm-length. Most subjects overestimated their arm-length with a mean error of 4.8 cm, but it needs to be pointed out that participants' responses were very variable (SD = 7.7, Min = −7.8, Max = 22.0 cm).

TABLE 1 | Mean accuracy values (%) for each group compared post-hoc between sessions.


FIGURE 8 | Depth perception. Subjects were able to judge whether two objects were aligned with less than 0.5 cm misestimating. For the middle track an effect of distance occurred, with participants' estimation being significantly worse for objects presented further away. Error-bars represent 95% confidence intervals.

The correlations confirm that the longer the participants' arms are the further they can actually reach (r = −0.715, p = 0.001). Against our expectation none of these perceptual measures correlated with accuracy or RT for reachability judgments,—this includes depth perception.

#### Detection Theory Approach

Importantly, the detection analysis confirmed improvements in affordance based judgments due to feedback as demonstrated by pairwise comparisons between sessions per group (Bf-p = 0.025). In the Control-group none of the variables of the detection theory approach demonstrated differences between sessions. In the Experimental-group the FA Rate significantly dropped in the second session [t(9) = 2.75, p = 0.022], as can best be seen in **Figure 9A**. Hit Rates did not differ between sessions. Accordingly, in session 2 the sensitivity value d′ [t(9) = − 3.83, p = 0.004] and the AUC [t(9) = −3.56, p = 0.006; **Figure 9B**] increased significantly for the Experimental-group demonstrating an improvement in accuracy after feedback. In line with these results the Experimental-group revealed a trend of the criterion changing over time from a liberal (M = −0.42, sd = 0.67) toward an ideal strategy (M = −0.09, sd = 0.44). In the Control-group the mean criterion remained at the same

session. Accuracy improves significantly after feedback in the second session.

liberal level in session 1 (M = −0.62, sd = 0.35) and 2 (M = − 0.61, sd = 0.70). Please see Supplementary Table 5 for further descriptive statistics and t- and p-values.

# GENERAL DISCUSSION

Affordance perception encompasses determining action opportunities for a given setting. It comprises a dynamic integrating process of cognitive components involved in perception and action necessary to gauge and update the relationship between relevant body constraints and environmental properties. Thus, affordance perception resembles a multifaceted construct, probably engaging a complex neural network of components involved in motor cognition. Stroke may affect appropriate affordance based judgments on different levels, due to changed body constraints or impaired cognitive functions. Our goal was to test an affordance based judgment paradigm in healthy young adults with the perspective to be applied in stroke patients. We used simple instructions, a limited number of trials (doable within 30 min), and took into account difficulties with attention to the contralateral hemi-space as well as the use of only one hand. To specify setting dependent estimations we studied two tasks. With two independent samples of young healthy adults, we investigated the ability to judge the fit of one's hand in an aperture (study1) and the ability to judge whether objects are within reach (study2) based on the same approach.

Overall, we confirmed that settings close to the actual measure were judged more poorly and slower compared to more extreme settings, replicating prior results (Gabbard et al., 2007). In accordance with prior studies participants overestimated their reaching capacities (Carello et al., 1989; Mark et al., 1997; Gabbard et al., 2007). However, when participants judged whether their hand can fit into an aperture the current results demonstrated a rather conservative approach.

In order to study potential accuracy-improvements in affordance based judgments, in a second session we analyzed effects of task exposure and feedback. In line with our predictions and findings of previous affordance perception related studies (Mark and Vogele, 1987; Mark et al., 1990; Weast et al., 2011), we here demonstrate that subjects' accuracy improved. This was most obvious for more error-prone settings close to the bodily constraints.

Furthermore, according to the affordance theory, perception of environmental properties (such as size, or depth perception) is a fundamental component. Therefore, we also analyzed correlations of these abilities with the capability to explicitly judge action opportunities. As predicted size-estimations correlated with the accuracy in determining whether the hand can fit into an aperture. However, performance in the depth perception task did not correlate with reachability judgments.

### **Affordance perception strongly depends on the setting and task at hand.**

Due to the between subjects design it needs to be noted that comparisons between the two affordance based judgment tasks need to be interpreted with some caution. However, we strived to use similar measurement approaches for both the reaching and the aperture task.

The present data suggests that the mechanisms involved in affordance perception strongly depends on the task. In study2 participants overestimated their reachability while being seated in accordance with prior reachability studies (Carello et al., 1989; Mark et al., 1997; Gabbard et al., 2007). In contrast study1 shows rather conservative response tendencies when subjects decided whether their hand fits into an aperture,—despite the fact that we used a similar setting and measurement approach for either task. Note however, Ishak et al. (2008) rewarded their participants with candies when they were able to reach through diamond shaped apertures and found a very liberal response tendency. Thus, even when solving a similar task, the response criterion may vary depending on the setting, including the risks and benefits that participants may attribute as a consequence to their behavior.

#### **The ability to perceive relevant perceptual properties only partly explains performance in affordance based judgments. Its role for affordance perception may depend on the degrees of freedom in a task.**

The ability to perceive and estimate hand-size was quite accurate in our sample. For judging whether a hand can fit into an aperture, size perception seems to be a strong determinant. Study2 demonstrated that participants were very good at perceiving depth, but very variable when estimating their armlength. In contrast to our aperture study, these measurements of perceiving environmental and bodily properties did not correlate with affordance based judgment accuracy or RT. This is in line with other studies indicating that different processes are involved in visual depth perception and visually directed action tasks (e.g., Loomis et al., 1992). Still one can assume that preserved depth perception is a prerequisite for the here described affordance perception task. A possible explanation for the weak correlation between perceiving environmental properties and affordance based judgments could be the number and impact weight of single properties defining certain action opportunities. In contrast to the reachability task, affordance perception for the aperture task does not require taking as many properties into account. For example, when deciding upon the hand's fit, the movement component of bending forward to the target should not have much impact, instead the judgment is predominantly based on information about the size of the hand and the opening. More generally the hand may be used as a stable perceptual metric for scaling objects that afford actions,—an argument that recently has been similarly formulated by Linkenauger et al. (2014). In their study Linkenauger et al. magnified the size of different body parts and objects to the same degree and demonstrated that subjects perceived their hand as less magnified than other body parts or objects. However, hand size perception cannot explain the entire construct of affordance perception when judging whether a hand can fit into an aperture. Against expectations, the analysis reveals that affordance based judgments for the non-dominant left hand were significantly more accurate compared to judgments for the dominant right hand. Interestingly, although size-estimations correlated with accuracy-judgments, this left vs. right hand accuracy-judgment difference was not found for size-estimations, suggesting that only partly overlapping mechanisms are used to solve the two tasks.

### **Participants integrate newly acquired knowledge.**

Further, differences between the two tasks occur in the second session. When deciding whether the hand can fit into an aperture the Control-group produced faster RTs during the second session compared to the first session and compared to the Experimental-group. This may indicate that in the second session the Control-group retrieved a represented evaluation strategy that was developed during the first session. Whereas feedback during session 2 requires the Experimental group to integrate newly acquired knowledge.

In contrast to the aperture task, in study2 general exposure to the task seemed to improve accuracy in reachability judgments and led to quicker responses in both the Control and Experimental-groups. It is feasible that participants integrated knowledge acquired from exposure to daily life reaching between sessions, therefore leading to a trend toward improvement in the Control-group as well. Thus, the argument about the Control-group using a stabilized criterion can only be made for the aperture task. This is in line with the idea of the hand-size being commonly used as a stable criterion when judging action opportunities,—unless an update by direct experience is provided. New knowledge can be integrated into the representation as demonstrated by the performance increase of the Experimental-group in the second session. For the reachability task instead it seems that participants cannot revert to such a stable criterion. This may be explained by higher degrees of freedom in this task.

Despite of these differences between the two settings, signal detection analysis underlines an important similarity, namely that feedback plays a significant role in improving affordance based decisions. Feedback in the Experimental-group had a significantly advantageous effect on judgment accuracy, which is most obvious for more error-prone settings close to the bodily constraints, i.e., actual hand-fit or maximum reachability. In the aperture task, the Experimental-group judged best for the passive hand, following the learning trials with the active hand, demonstrating transfer toward the untrained hand.

#### **The results have implications for research in patients with brain damage.**

One major novelty of the study is provided by the method which was developed for future assessment of affordance perception in patients with brain damage. Thus, an important gain from this study are the paradigms themselves. As described in the introduction, stroke for example may endanger appropriate affordance perception on different levels,—by immensely changed bodily capabilities after damage to motor relevant brain areas, by impaired insight into the disorder as well as by problems with action planning or problems with perceiving object and spatial properties.

Aside from the proposed diagnostic value, we also demonstrated that the paradigm has the potential for training applications. In line with previous studies (e.g., Weast et al., 2011) we demonstrated that feedback can change perception of affordances. With two studies each testing a unique affordance perception task we show for the first time that training of affordance perception can improve detection measures independent of the prior trend of response tendencies. On the one hand our studies illustrated that feedback can lead to changes from a previously liberal to a rather ideal strategy when judging reachability in a seated position. Or in case of conservative tendencies for judging the fit of ones hand in an aperture, it can lead to an improvement in sensitivity measures. The results have important implications for neurorehabilitation of patients with unilateral brain damage and new bodily constraints. For one it is good news that updates and improvements of judgments seem to be possible even for healthy participants with quite accurate judgments. However, the transfer toward the untrained limbs as demonstrated in the aperture task needs to be regarded with caution. First, further studies need to test for generalization of this statement toward other tasks. Second, if the same abilities are attributed to both sides of the body, then learning with the

# REFERENCES


intact hand may be problematic for patients with asymmetrical motor functions. Patients with residual motor functions for example may have to be trained regularly with both hands in these types of tasks in order to adequately adapt affordance based judgments to the asymmetrical motor functions while also taking into account possible motor-rehabilitation progress.

While we successfully tested our paradigm in healthy young adults it needs to be noted that a subsequent patient study will require an age matched healthy control-group. For one, body capabilities change while growing older, and secondly cognitive skills may decline. Aging appears to affect affordance perception, as demonstrated by several studies investigating reachability (Gabbard et al., 2011; Gabbard and Cordova, 2013). Alarmingly, falls are reported to correlate with reduced reaching capabilities in elderly adults (Butler et al., 2011).

To our knowledge, this project presents the first approach to assess affordance perception accounting for the challenges of working with a stroke patient population. We introduce two tasks that clearly have daily life relevance. Subpopulations with particular difficulties may be identified and it could be tested whether controlled feedback leads to a reduction of erroneous judgments. We conclude that future applications of the paradigm should include a patient population and a healthy age matched control-group to diagnose potential difficulties with affordance perception after brain injury, as well as the effects of training.

# FUNDING

This work was funded by the James S McDonnell Foundation Collaborative Activities Award (grant no. 220020190) to SHF and a EU FP7 Marie Curie Zukunftskolleg Incoming Fellowship Programme, University of Konstanz (grant no. 291784) awarded to JR.

# ACKNOWLEDGMENTS

We would like to thank Kim Martin and Ronny Wolfram for their assistance with data collection. We declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

# SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fnhum. 2015.00674

Neuropsychologia 43, 917–929. doi: 10.1016/j.neuropsychologia.2004. 09.006


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Randerath and Frey. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Impaired Communication Between the Dorsal and Ventral Stream: Indications from Apraxia

Carys Evans <sup>1</sup> \*, Martin G. Edwards <sup>2</sup> , Lawrence J. Taylor <sup>1</sup> and Magdalena Ietswaart <sup>3</sup>

<sup>1</sup> Faculty of Health and Life Sciences, Department of Psychology, Northumbria University, Newcastle upon Tyne, UK,<sup>2</sup> Institute of Research in the Psychological Sciences, Université catholique de Louvain, Louvain-le-Neuve, Belgium, <sup>3</sup> Psychology, School of Natural Sciences, University of Stirling, Stirling, UK

Patients with apraxia perform poorly when demonstrating how an object is used, particularly when pantomiming the action. However, these patients are able to accurately identify, and to pick up and move objects, demonstrating intact ventral and dorsal stream visuomotor processing. Appropriate object manipulation for skilled use is thought to rely on integration of known and visible object properties associated with "ventro-dorsal" stream neural processes. In apraxia, it has been suggested that stored object knowledge from the ventral stream may be less readily available to incorporate into the action plan, leading to an over-reliance on the objects' visual affordances in object-directed motor behavior. The current study examined grasping performance in left hemisphere stroke patients with (N = 3) and without (N = 9) apraxia, and in age-matched healthy control participants (N = 14), where participants repeatedly grasped novel cylindrical objects of varying weight distribution. Across two conditions, object weight distribution was indicated by either a memory-associated cue (object color) or visual-spatial cue (visible dot over the weighted end). Participants were required to incorporate object-weight associations to effectively grasp and balance each object. Control groups appropriately adjusted their grasp according to each object's weight distribution across each condition, whereas throughout the task two of the three apraxic patients performed poorly on both the memory-associated and visual-spatial cue conditions. A third apraxic patient seemed to compensate for these difficulties but still performed differently to control groups. Patients with apraxia performed normally on the neutral control condition when grasping the evenly weighted version. The pattern of behavior in apraxic patients suggests impaired integration of visible and known object properties attributed to the ventro-dorsal stream: in learning to grasp the weighted object accurately, apraxic patients applied neither pure knowledge-based information (the memory-associated condition) nor higher-level information given in the visual-spatial cue condition. Disruption to ventro-dorsal stream predicts that apraxic patients will have difficulty learning to manipulate new objects on the basis of information other than low-level visual cues such as shape and size.

#### Edited by:

Ferdinand Binkofski, RWTH Aachen University, Germany

#### Reviewed by:

Annalisa Setti, University College Cork, Ireland Marc Himmelbach, University Hospital Tuebingen, Germany

#### \*Correspondence:

Carys Evans carysemilyevans@gmail.com

Received: 05 September 2015 Accepted: 11 January 2016 Published: 01 February 2016

#### Citation:

Evans C, Edwards MG, Taylor LJ and Ietswaart M (2016) Impaired Communication Between the Dorsal and Ventral Stream: Indications from Apraxia. Front. Hum. Neurosci. 10:8. doi: 10.3389/fnhum.2016.00008

Keywords: apraxia, visual affordance, ventro-dorsal stream, visual pathways model, grasping

# INTRODUCTION

Apraxia is a high-level movement disorder that commonly occurs after lesions to the left frontoparietal motor network. In addition to impaired gesture imitation, apraxia is recognized by performance errors when demonstrating how objects are used (Goldenberg, 1995; Buxbaum, 2001). Although these errors are most apparent when pantomiming the use of objects, with a marked improvement during actual object-use, both pantomime and actual use can be affected (De Renzi and Lucchelli, 1988; Buxbaum and Saffran, 2002; Sunderland and Shinner, 2007; Goldenberg, 2009). Skilful manipulation of objects requires the integration of stored information about the object's typical use and action processes enabling the object to be grasped appropriately based on the object's visual affordances and spatial location. In the case of apraxia, it is believed that this integrative process is disturbed. However it is currently not clear whether these deficits affect apraxic patients' ability to learn to manipulate new objects.

Close examination of object knowledge in apraxic patients confirms that performance errors cannot be attributed to impaired ventral (vision-for-perception) or dorsal (vision-foraction) streams of the visual pathways model (Goodale and Milner, 1992; Milner and Goodale, 2006). Apraxic patients can identify visually presented objects (Daprati and Sirigu, 2006) and order familiar objects in weight order (Dawson et al., 2010; Li et al., 2011). These patients also use structural properties to appropriately reach and grasp familiar objects, infer the use of novel objects based on their affordances, and apply appropriate grip force using recent sensorimotor feedback (Gordon et al., 1993; Sirigu et al., 1995; Goldenberg and Hagmann, 1998; Ietswaart et al., 2006; Frey, 2007; Hermsdörfer et al., 2011; Randerath et al., 2011; Sunderland et al., 2013; Eidenmüller et al., 2014). However, patients with apraxia produce incorrect hand postures attributed to functional use of objects and disturbed anticipatory grip force control for familiar objects (Buxbaum et al., 2003). These results confirm that different mechanisms of the visual pathways model are important depending on the goal of the motor act and support recent evidence suggesting that a ''ventro-dorsal'' sub-stream of the traditional dorsal pathway may be necessary when processing sensorimotor information based on long-term action representations of how objects are functionally used (Buxbaum and Kalénine, 2010; Binkofski and Buxbaum, 2013). It could be that this sub-stream may be implicated in apraxia.

Unlike the dorsal pathway that extends bilaterally from occipital to superior parietal and dorsal pre-motor areas, the ventro-dorsal sub-stream is left lateralized, projecting medially from occipital to left inferior parietal lobe (IPL) and ventral pre-motor regions. Through a mutual connection with the ventral stream via the left IPL, perceptual information can be incorporated into action plans (Rizzolatti and Matelli, 2003; Buxbaum and Kalénine, 2010; Rizzolatti et al., 2011; Binkofski and Buxbaum, 2013; Vingerhoets, 2014) enabling objects to be grasped for use by applying stored knowledge of how objects are functionally manipulated to the physical properties of the objects presented (Frey, 2007; Almeida et al., 2013; Garcea and Mahon, 2014). In support of object-use errors observed in apraxia, there is an established relationship between apraxic symptoms and damage to regions implicated in the ventro-dorsal stream, in particular inferior parietal regions that suggest this pathway may indeed be disrupted (Haaland et al., 2000; Buxbaum, 2001; Buxbaum et al., 2006, 2007; Frey, 2007; Goldenberg, 2009; Garcea and Mahon, 2014). The subsequent failure to effectively access and implement information from the ventral stream into the action plan results in an over-reliance on the intact dorsal stream. Consequently, objects are manipulated based on what is visually afforded irrespective of the goal of the action (Randerath et al., 2011).

That said, apraxic patients have shown equivalent performance to controls when making memory-driven reach and grasp movements also reliant on the integration of ventral and dorsal processes (Ietswaart et al., 2001; Dawson et al., 2010). Although these findings suggest that apraxic patients can successfully utilize stored representations, it remains possible that the visuo-motor transformation involved in simple reach and grasp movements may not be difficult enough to place sufficient demand on high-level perceptual processes. The proposal of ventro-dorsal disturbance in apraxia has also been argued to place too much importance on different components of object knowledge; in particular, retrieval of knowledge of an objects prototypical use that is dependent on previous experience, which cannot account for apraxic errors during novel object-use (Goldenberg and Hagmann, 1998; Goldenberg, 2014). Yet such knowledge retrieval furthermore assumes that skilled object-use relies on the retrieval of information from ''storehouses'' as opposed to the convergence of short- and long-term visual representations depending on the goal of the motor act.

While the research outlined suggests apraxic patients have difficulties accessing and incorporating stored knowledge of actions related to skilled use of familiar objects, it remains unclear how these patients learn to manipulate new objects. Of the few studies that have assessed this issue, Barde et al. (2007) trained patients to match novel gestures to novel object pictures that were high or low afforded by associated objects. Apraxic patients demonstrated a greater ability to correctly match gestures to object shape for the high than low afforded gestures during action recognition, but were consistently poor compared to controls during action production regardless of affordance. This may be due to the use of two-dimensional objects during training reducing the affordance bias during action production. Retrieval of the appropriate action associated with the object may also have been more difficult when the goal was simply to produce the correct action, as there is no clear feedback as to whether the action goal was achieved in a comparable manner to appropriately grasping an object to fulfil a function.

The current study explored the impact of affordance on object manipulation by requiring participants to repeatedly lift and balance novel objects of differing weight distribution. Over two conditions, the weight distribution of different cylindrical objects was indicated using different object-weight associations, either by a symbolic memory-association between the color of the object and its weight distribution or by a visual-spatial cue of a ''dot'' over the weighted end of the object. Change in object manipulation over repeated lifts determined whether apraxic patients successfully used object knowledge obtained through experience to inform their grasp, or whether they continually relied on the visual cues to guide action.

Specifically, this study examined participants' point of grasp along the object depending on weight distribution. When grasping unbalanced objects, healthy adults intuitively choose a grasp close to the center of mass in order to minimize the energy required by grip force to compensate for load torque (Salimi et al., 2003; Duemmler et al., 2008; Endo et al., 2011). This is said to be estimated visually prior to initial object grasping, which is reflected in accurate grasping of unfamiliar objects for the first time (Lederman and Wing, 2003) or when asked to visually point to the center of mass (Baud-Bovy and Soechting, 2001; Duemmler et al., 2008). Action execution was used throughout the study rather than perceptual task learning. This enabled apraxic patients to get strong visual feedback as to whether the action goal of balancing each object had been achieved during each trial. It was anticipated that apraxic patients would show greater performance accuracy when the object afforded the correct gesture with increased contextual information provided (akin to findings by Barde et al. (2007) in the recognition task).

During the memory-associated condition, when each object's weight distribution was indicated symbolically by the color of the object, apraxic patients were expected to be impaired. Due to the symmetrical shape of the object, apraxic patients were expected to be biased towards more central grasp points and require a greater number of trials to accurately balance the object. In the visual-spatial cue condition, when the center of mass is indicated by a ''dot'' over the weighted end, apraxic patients may benefit from this meaningful visible cue over time to prompt a more accurate grasp-point over each trial. An alternative prediction was that apraxic patients might continue to use low-level affordance cues of object structure to indicate weight distribution, resulting in more central grasps rather than to the left or right of the object. Inappropriate manipulation of memory-associated and visual-spatial cued objects would confirm that apraxics over-rely on visual information processed by the dorsal visual stream due to ventral, stored knowledge, being unsuccessfully incorporated into the action plan via the ventro-dorsal sub-stream. Such behavior would add insight into what information apraxic patients can effectively utilize during goal directed action.

# METHODS

# Participants

Twenty-seven right-handed participants were recruited, 13 of which had suffered a stroke (Mage 68 ± 14, 8 male) within 27 months (Mmonths 15 ± 10) and 14 age-matched healthy control participants (Mage 70 ± 9, 5 male). In the patient group, and at the time of testing, three patients displayed symptoms of apraxia and 10 patients did not show signs of apraxia. The ethics committee within Northumbria University's Department of Psychology and a local NHS ethics committee approved the project.

On the basis of CT, MRI scans and clinical notes, patients who had a brain hemeorrhage or an infarct involving the left hemisphere were recruited from rehabilitation centers and National Health Hospitals within the North East of England. Patients presented with degrees of aphasia, right-sided weakness, or sensory loss. **Table 1** describes each patient's lesion and the Brodmann areas implicated. Lesions were mapped using MRIcron software package (Rorden et al., 2007) <sup>1</sup> based on the radiologist's MRI and/or CT clinical scans of each patient. The areas of damage for each patient were mapped using MRIcron software package; lesions were determined based on the radiologist's scan reports and the digital brain image. Scans were then normalized to a common stereotaxic space using Clinical Tool box software through SPM and applied to the Brodmann Atlas included in MRIcron (Rorden et al., 2012) 2 . Lesions for the three apraxic patients are visually documented in **Figure 1**.

The presence of apraxia was classified on the basis of abnormal performance in one or more of the apraxia screening tools assessing gesture imitation and familiar objectuse (pantomime and actual use). Further test batteries and clinical notes were used to exclude any patient presenting with global cognitive deficits or known dementia, severe receptive aphasia or failure to follow one-stage commands (according to the language comprehension token test by De Renzi and Faglioni, 1978), or significant signs of visuospatial neglect (according to the Apples Test by Bickerton et al., 2011). One non-apraxic patient was later excluded (FR) as he was diagnosed with early onset of vascular dementia. Patient details are described in **Table 2** and apraxia screening performance in **Table 3**.

Healthy age-matched control participants did not have a history of brain damage or stroke. These participants were recruited from the Psychology Department's participant database and were given monetary compensation for their time.

# Materials

### Apraxia Screening

## **Gesture imitation of hand and finger postures (Goldenberg, 1996)**

The experimenter demonstrated different hand postures relative to the head and finger postures irrespective of the hands position in relation to the body. Gestures were performed ''like a mirror''; the experimenter sat opposite the patient, performing each posture with their right hand to be imitated by the patients' left hand after the demonstration had ended. Successful imitation of each gesture on the first trial was awarded two points; one point was given if the patient was successful after a further demonstration; zero points if the gesture was not imitated correctly. A total score of 20 could be achieved by imitating 10 gestures of each kind.

<sup>1</sup>http://www.mccauslandcenter.sc.edu/mricro/mricron/

<sup>2</sup>http://www.mricro.com/clinical-toolbox/

**Pantomime of object use (based on Goldenberg et al., 2007)** Participants were required to demonstrate the use of 19 objects. The experimenter presented a drawn image of each object (taken from Cycowicz et al., 1997) and named the action to be pantomimed. Points were given for the presence of predefined movement features (Goldenberg et al., 2007 details these). With exception to demonstrating the use of scissors, body-part-as-object errors were marked as incorrect. A total of 53 points could be obtained, with less than 43 measured as pathological.

## **Actual object use (based on De Renzi and Lucchelli, 1988)**

Participants were given the same verbal description of the action to be demonstrated as in the pantomime task. Eighteen of the pantomimed objects were presented; one point was given if used correctly and zero if incorrect. The incorrect use of two or more objects was considered pathological.

# Object Grasping Task **Object stimuli**

Five cardboard cylinder tubes (length: 24.5 cm, diameter: 3.7 cm) were used, each containing a 17 g weight (length: 2 cm, diameter: 1.5 cm) in one or both ends. The five cylindrical objects comprised of two experimental conditions: ''memoryassociated'' and ''visual-spatial cue'', and one screening condition: ''neutral-control''. The ''memory-associated'' condition consisted of one green and one blue cylinder; when presented to the participant, the green object was weighted on the left, whereas the blue object was weighted on the right. Participants were required to remember the color-weight associations when lifting the object without a visual cue indicating weight distribution on either end of the cylinder. The visual-spatial cue condition consisted of two gray objects that were unevenly weighted, containing a weight in either the left or right end of the object. The heavier end of each object was marked with a red ''dot'' (1 cm diameter), which acted as a visual cue of the weight distribution when acting upon the object. Finally, the neutral-control condition consisted of one gray object that was evenly weighted with one weight in each end of the cylinder. This screened for any confounds such as visuospatial neglect or comprehension issues that would impact task performance. In addition to the main objects, two white practice cylinders were used when giving task instructions: one evenly-weighted (length: 42 cm, diameter: 1.5 cm) and one unevenly-weighted object (length: 46, diameter 1.7 cm, 34 g weight on the right side). The practice cylinders did not resemble test objects in size and weight to minimize priming effects of grasping these objects prior to the main experiment.

A horizontal bar (length: 30 cm, diameter: 0.5 cm) was positioned perpendicular to the participant, 35 cm in front of the participant and 24 cm above the table. Both the experimenter and participant used the bar to indicate the extent to which the object was balanced. For the duration of testing a video camera was placed behind the horizontal bar and recorded each trial. A schematic representation of the experimental setup can be seen in **Figure 2**.

# Procedure

Each participant was seated at the workspace where the objects were presented. Using the horizontal bar as a guide, participants were instructed to lift and balance each object using


#### TABLE 1 | Description of each apraxic (top) and non-apraxic (bottom) patient's lesion as described in the radiologist's CT and/or MRI reports and when mapped onto the Brodmann atlas.

Note: F, Female; M, Male; Y, Yes; N, No; L, Left; R, Right; ACA, Anterior Cerebral Artery; MCA, Middle Cerebral Artery. Brodmann areas ascribed to the IPL, inferior parietal lobe (areas 39 and 40) are indicated in bold.

TABLE 2 | Screening performance of patient groups, including apraxics (top) and non-apraxics (bottom); includes FR who was excluded due to early onset vascular dementia.


Note: F, Female; M, Male; Y, Yes; N, No; L, Left; R, Right; n.t, not tested.

a pincer grip with the index and thumb of their left hand. After the object was lifted to the horizontal bar, participants returned the object to the table and removed their hand from it before another trial began. It was emphasized that if the object was imbalanced, they should not compensate by tightly pinching the object or rotating their wrist during or at the end of each lift. Task instructions were demonstrated using the evenly weighted practice cylinder. Participants were then requested to practice the task procedure using the same cylinder. Once participants successfully completed the movement they were presented the unevenly weighted practice cylinder and repeated the process. After it was evident that participants understood the procedure, the main task was started. During the main task, to ensure each participant had

the same experience with the object, they were asked to lift and balance each object five times before being presented the next object. In each block, objects were presented in a random order. Overall, there were five testing blocks in which participants saw each object once; including each individual trial, participants lifted each object 25 times, totalling 125 trials. The video camera recorded participants completing each trial.

# Data Analysis

Task performance across each condition was initially compared between each control group (healthy and non-apraxics) using a two-way mixed model ANOVA exploring OBJECT (memoryassociated; visual-spatial cue; neutral-control) × GROUP


#### TABLE 3 | Apraxia screening performance and error types in apraxics (top) and non-apraxics (bottom).

Note: Types of performance error were given the following acronyms: gesture imitation: perseveration (p); hand misorientation (hm): misorientation of the hand relative to the face; finger extension (fe): incorrect fingers extended from hand; spatial misorientation (sm): hand misorientation relative to the experimenter, e.g., back of hand instead of palm facing. Object use: action addition (aa): miscellaneous actions not interpretable as a step in the task, e.g., waving; action omission (ao): failed to perform any recognisable action; step omission (so): failed to complete some parts of the movement, e.g., rotating hand when squeezing a lemon; body-part-as-object (bpo): e.g., brush teeth with finger; semantic substitution (ss): e.g., stir with fork; grasp misestimation (gm): incorrect grasp size/type for object, e.g., pincer grip for cup; spatial misestimation (sm): incorrect relationship between object relative to body or another (reference) object.

(Healthy vs. Non-apraxic controls) to rule out differences across control groups. Each apraxic patient was then compared to the control groups separately using modified t-tests recommended when estimating the abnormality of an individual patient's score against a control sample that is modest in size (Crawford and Garthwaite, 2002; Crawford et al., 2010). In order to purely assess whether object-weight associations were learnt as opposed to participants relying on semantic labelling (e.g., green is left weighted) to complete the task, object-weight associations were not explicitly described to the participants during the study. This also accommodated for any language deficits. Instead, learning of object-weight associations was determined by assessing participants' change in performance accuracy over trials (TC) and change in performance accuracy over blocks (BC). The former would indicate whether apraxic patients' performance improved with repeated lifts of the same object and the latter would confirm whether apraxic patients applied what they had learned in previous blocks when each object was reintroduced. The points at which the object was grasped were used as a guide to evaluate grasp behavior.

Firstly, in order to analyze the video footage, photo snapshots were created when participants were at the maximal point of object lift. From each snapshot, the ''point of grasp'' was measured based on the midpoint position of the index finger along the object (from right to left). Grasps were considered accurate depending on whether the object was successfully balanced and an appropriate point of grasp was applied to compensate for the objects weight distribution. This ensured that participants were accurate due to adjusting their grasp-point along the object, as opposed to applying greater grip force or by rotating their wrist during each lift. If the location of an individual's grasp was greater than two standard deviations from the ''optimum'' point of grasp (OP) to compensate for weight distribution, it was marked as inaccurate. The optimum point of grasp was measured for each object based on healthy control participants mean point of grasp for the fifth trial across all blocks.

#### Accuracy Change Over Trials (TC)

Grasp accuracy was compared between Trial 1 and Trial 5 across blocks. Performance change across trials would indicate whether apraxic patients' performance improved with repeated grasps of the same object. To compare performance, accuracy was first weighted; accurate grasps in early trials (e.g., Trial 1) received a greater weighting compared to accurate grasps in later trials (e.g., Trial 5). This reflected the extent to which performance was driven by trial-and-error or learning each objects weight distribution. Inaccurate grasps were given a negative score: fewer points were deducted when grasps were inaccurate in early trials and greater points deducted when performing inaccurately in later trials. These reflected the extent to which participants failed to adapt their grasp based on each objects' weight distribution with repeated grasps of the same object (see **Table 4** for weighted scores). As a greater score could be achieved in Trial 1 compared to Trial 5, these scores were then calculated as proportions of the maximum score achievable in that trial, across all 5 blocks. For example, in Trial 1 an accurate grasp scores 5 points, over 5 blocks a maximum score of 25 can be achieved, whereas for Trial 5 an accurate grasp scores 1 point, over 5 blocks a maximum score of 5 can be achieved. Once participants' scores in Trial 1 and Trial 5 were transformed into proportions, accuracy in Trial 5 was deducted from Trial 1 (as outlined in the equation below). Based on this calculation, a greater negative score signifies improved accuracy across trials, a positive score signifies reduced or consistently poor performance across trials, and a score of zero indicates that the participant achieved the highest accuracy across trials.

Accuracy change (TC) = (block 1–5 average scoretrial 1/maximum scoretrial 1) − (block 1–5 average scoretrial 5/maximum scoretrial 5).

#### Accuracy Change Over Blocks (BC)

Using the same calculation, performance across blocks was assessed by comparing the average accuracy across trials between Block 1 and Block 5. Performance change across blocks would

TABLE 4 | Weighted scores for analyses of accuracy change over Trial and Block.


confirm whether apraxic patients applied what they had learned in previous blocks when each object was reintroduced. As with trial data, performance across blocks was weighted using positive and negative scores. In early blocks, participants received greater points for accurate grasps and fewer points were deducted for inaccurate grasps, whereas in later blocks participants received fewer points for accurate grasps and more points were deducted for inaccurate grasps. Scores were transformed into proportions of the maximum score before accuracy in Block 5 was deducted from accuracy in Block 1.

Notably during testing, non-apraxic patients BH and JS completed only four testing blocks due to experiencing fatigue when lifting the objects several times. The same calculation applied to the final block was instead applied to Block 4 for these patients.

# RESULTS

In order to confirm whether apraxic patients utilized memoryassociations or visual-spatial cues regarding weight distribution when balancing each object, performance change across trials and across blocks were assessed. Points of grasp for each object were used as a guide to evaluate grasp behavior.

# Accuracy Change Across Trials (TC)

#### Healthy Controls vs. Non-Apraxics

An initial two-way mixed model ANOVA exploring OBJECT (memory-associated; visual-spatial cue; neutral-control) × GROUP ruled out differences in performance change across Trials in healthy and non-apraxic controls. Non-significant main effects confirmed that performance was comparable across control groups (GROUP: F(1,21) = 0.139, p = 0.713, η 2 <sup>p</sup> = 0.007) and between objects (OBJECT: F(1.357,28.504) = 3.583, p = 0.058, η 2 <sup>p</sup> = 0.145). However, a significant interaction OBJECT × GROUP (F(1.357,28.504) = 8.479, p = 0.004, η 2 <sup>p</sup> = 0.288) was identified. Independent samples t-test did not reveal significant differences in performance for all conditions (p > 0.05) except the neutral-control condition (t(21) = 2.353, p = 0.028). Nonapraxics showed greater improvement in task performance from Trial 1–5 (TC = −0.333 ± 0.280) on the evenly weighted object compared to healthy controls whose performance reduced (TC = 0.257 ± 0.714). Notably, differences easily arise on the evenly-weighted neutral-control object, because the point scoring system works with difference from the mean and standard deviation on this condition in normal performance is very small (and differences are therefore of limited interest).

Despite variances in performance change for the neutralcontrol object, healthy and non-apraxic controls consistently grasped the object close to the optimum grasp-point (OP = 13.18 cm). Examining grasp-point behavior of controls across all three conditions, both groups initially grasped closer to the center of each object in Trial 1, but by Trial 5 were ≤1.32 cm from the optimum grasp-point for each object. Observing individual scores for performance change over trials (TC) confirms that each control participant appropriately adapted their grasp-point over repeated lifts to account for the weight distribution of each object. Of note, non-apraxic control participant JS did not perform as efficiently as the other non-apraxic patients in the memory-associated and visualspatial cue conditions. However, she was still markedly more accurate than AH and GW. Patient JS also performed at ceiling during the language comprehension test and apraxia screening indicating that her performance was not applicable to poor comprehension or apraxia. Instead, her performance may be more attributable to her age; JS was the oldest participant (91) and testing had to be terminated after the fourth test block as she became fatigued. Together, these findings indicate that healthy and non-apraxic controls effectively utilize both memoryassociated and visual-spatial cued information to improve performance when repeatedly lifting each object (see **Table 5** for performance change over trials, **Table 6** for participants' average points of grasp, and **Figure 3** for accuracy change across trials).

#### Patient AH

Single case t-tests confirmed that when grasping memoryassociated objects, patient AH was significantly worse than healthy (p < 0.001, t = 17.100) and non-apraxic controls (p = 0.001, t = 4.775) with at least a minimum of 99.93% of controls falling below AH's score. During the visual-spatial cue condition, patient AH also performed significantly worse than both healthy controls (p < 0.001, t = 13.363) and non-apraxics (p = 0.007, t = 3.160) with at least a minimum of 99.33% of controls falling below AH's score. For both memory-associated and visual-spatial cue conditions, AH's accuracy was consistently poor (TC ≥ 2.52) whereas control groups generally improved performance across trials (TC from 0.045 to −0.274).

Observing the average grasp-points for both the memoryassociated and visual spatial cue conditions, patient AH maintained a point of grasp towards the center of each object (from 11.10 cm to 13.45 cm). These grasps were at least 4.8 cm from the optimum grasp-point to compensate for weight distribution of each object. Unlike control groups, patient AH did not adjust her grasp towards the weighted end of across trials.

As this patient did not adjust her grasp away from the midpoint, when grasping the neutral-control object AH's performance change was comparable to both healthy controls (p = 0.367, t = −0.348; an estimated 36.68% falling below AH's score) and non-apraxics (p = 0.271 t = 1.128; an estimated 85.40% falling below AH's score). AH's use of midpoint grasps confirms that her symptoms of right-sided visual neglect identified in the cancellation task did not affect grasp performance.

#### Patient GW

Performance of patient GW mirrored that of patient AH. Performance change over trials was worse than healthy and nonapraxic controls when grasping unevenly weighted objects in both the memory-associated and visual-spatial cue conditions: for all comparisons p ≤ 0.001, with at least an estimated 99.93% of controls falling below GW's score. Patient GW was consistently unsuccessful in balancing these objects (TC = 4.8 for each), with average points of grasp ranging from 13.46 cm to 14.76 cm across all four objects, and at least 5.18 cm from the optimum grasp-point. Overall, GW's average grasp was consistently close to or slightly to the left of each object's center regardless of their weight distribution, with minimal variance in grasp-points across conditions. However when grasping the neutral-control object, GW's performance was comparable to both healthy (p = 0.367;

TABLE 5 | Performance change over trials (TC) and blocks (BC) in non-apraxic (top) and apraxic (bottom) patients.


Note: M, mean.

an estimated 36.68% falling below GW's score) and non-apraxic controls (p = 0.146; an estimated 85.40% falling below GW's score). Patient GW's average grasp-points were close to the optimum point of grasp.

## Patient JA

Apraxic patient JA's performance change across trials was comparable to both healthy and non-apraxic controls for the memory-associated and neutral-control conditions (p > 0.05; an estimated 25.65% to 61.96% of controls falling below JA's score). During the visual-spatial cue condition, although JA was comparable to non-apraxics (p = 0.349, t = 0.402; an estimated 65.10% of controls falling below JA's score), performance change was significantly different to healthy controls (p = 0.005, t = 3.032; an estimated 99.52% of controls falling below JA's score). It was evident in this condition that JA did not greatly improve grasp accuracy between Trial 1–5 (TC = 0.360) and continued to make errors by the final trial. Although JA achieved largely normal performance on this measure of accuracy change across trials, her qualitative behavior did not look normal. She was slow and deliberate in her reach movements, apparently in an attempt to compensate for her difficulty performing this task. This prompted a closer look at grasp-point and grasp-point variance, in an attempt to quantify her unusual behavior in performing the task. Average grasp-points in Trial 1 and 5 suggests JA typically reorients her grasp towards the weighted end of the object, grasping ≤1.31 cm from the optimum grasp-point. When grasping the right-weighted object, JA deviated to a more extreme rightward grasp; average grasp-point was 4.20 cm further right than the optimum point (6.29 cm) by Trial 5, whereas grasppoints of healthy controls were less than half a centimetre from the optimum point. Observing the grasp-points of JA in relation to the optimum grasp point to compensate for object weight distribution, her point of grasp was further from the optimum point in Trial 5 compared to Trial 1 in the visualspatial cue condition for both the left and right weighted objects, showing that she continues to adapt her grasp-point even if they were more accurate in previous trials. Similarly, patient JA's grasps are much more varied suggesting that she does not confidently learn the object-weight associations but may continue to exercise a trial-and-error procedure throughout.

Statistically this behavior was not so much apparent in the average grasp-point variance itself but in the standard deviation of her grasp-point variance. On the average grasppoint variance JA showed marginally significant differences on the memory associated condition (M = 20.69 cm) compared to healthy controls (M = 12.78 cm, p = 0.057, t = 1.691; an estimated 94.26% falling below JA's score) and non-apraxic controls (M = 12.92 cm, p = 0.055, t = 1.798; an estimated 94.50% falling below JA's score) controls. In the visual-spatial cue condition JA's grasp-point variance was not different from control participants (healthy controls: p = 0.435, t = 0.168; non-apraxics: p = 0.453, t = 0.122). But critically JA did differ in both conditions on the standard deviation of her grasp-point variance. On the memory associated condition

 6 | Point of grasp (cm). Top: Trial 1 and 5 across blocks, including the overall average point of grasp and standard deviation across every trial for each object. Bottom: Block 1 and 5 across trials, includingaverage point of grasp and standard deviation across every block for each object.

 the

TABLE


JA's variance standard deviation at 20.20 cm was significantly larger than healthy controls (M = 4.52 cm, p = 0.018, t = 2.333; an estimated 98.18% falling below JA's score), and non-apraxics (M = 4.10 cm, p = 0.001, t = 4.504; an estimated 99.9% falling below JA's score). This is similarly evidenced by the standard deviation of patient JA's grasp-point variance in the visual-spatial cue condition. JA's grasp-point variance standard deviation at 19.74 cm was significantly greater than healthy controls (M = 6.28 cm, p = 0.02, t = 2.279; an estimated 97.99% falling below JA's score), and nonapraxic participants (M = 5.23 cm, p = 0.014, t = 2.667; an estimated 98.58% falling below JA's score). Of course on the neutral-control condition neither JA's grasp-point variance (M = 2.92 cm) nor the standard deviation of patient JA's grasp-point variance (M = 5.80 cm) was different from healthy controls (both not significantly different to JA at M = 9.22 cm and M = 6.27 cm subsequently) or non-apraxics (both not significantly different to JA at M = 11.29 cm and M = 3.37 cm subsequently).

# Accuracy Change Across Blocks (BC)

#### Healthy Controls vs. Non-Apraxics

Non-significant main effects and interactions from the twoway mixed model ANOVA confirmed that performance change across Blocks was comparable between control groups: OBJECT, F(1.288,27.045) = 0.986, p = 0.381, η 2 <sup>p</sup> = 0.045, GROUP F(1,21) = 0.385, p = 0.542, η 2 <sup>p</sup> = 0.018, OBJECT × GROUP F(1.288,27.045) = 0.264, p = 0.671, η 2 <sup>p</sup> = 0.012. Both healthy and non-apraxic controls adjusted their point of grasp across blocks depending on the weight distribution of each object; individual scores for performance change over blocks confirms that all healthy and non-apraxic control participants successfully adapted their grasp-point to accommodate for the weight distribution when the objects were reintroduced in later blocks (see **Table 5** for performance change over trials, **Table 6** for average grasp-points and **Figure 3** for accuracy change across blocks); grasps were ≤1.32 cm from the optimum grasp-point by the final block. Accuracy was also maintained across blocks (BC ranged from 0.094 to 0.583).

#### Patient AH

Accuracy change was worse than both healthy and non-apraxic controls during the memory-associated and visual-spatial cue conditions (for all comparisons p < 0.05, with at least an estimated 99.65% of controls falling below AH's score). Patient AH's score for accuracy change across blocks (BC ≥ 3.24) was indicative of consistently inaccurate object grasps compared to both control groups (BC ≤ 0.583). Average grasp-points confirm that AH did not adjust her grasp according to the weight distribution of each object but maintained a more central grasp; across both Block 1 and Block 5, AH's grasp-point ranged between 11.50 and 13.45 cm, at least 5.20 cm from the optimum point of grasp. This suggested that AH failed to utilize stored knowledge of weight distribution when the object was reintroduced.

As before, patient AH's performance change was comparable to healthy (p = 0.344, t = −0.411; an estimate of 34.38% of controls falling below AH's score) and non-apraxic controls (p = 0.339, t = −0.430; an estimate of 33.94% of controls falling below AH's score) when grasping the neutral-control object. Patient AH's accuracy was consistently high (BC = 0) and maintained a central grasp-point within 1.48 cm from the optimum point of grasp.

#### Patient GW

Similarly, during the memory-associated and visual-spatial cue conditions patient GW performed worse than healthy controls and non-apraxics; for all comparisons p < 0.05, with at least an estimated 96.76% of controls falling below GW's score. Patient GW grasped each object centrally at least 5.18 cm from the optimum grasp-point resulting in a consistently poor accuracy change across blocks (BC ≥ 4.20).

Mirroring patient AH, when grasping the neutral-control object, GW's performance change was equivalent to healthy (p = 0.344, t = −0.411) and non-apraxic controls (p = 0.339, t = −0.430). Patient GW maintained a central point of grasp within 1.77 cm from the optimum grasp-point confirming that grasps were consistently accurate across blocks (BC = 0).

#### Patient JA

Across all three conditions (memory-associated/visual-spatial cue/neutral-control) patient JA's performance change was comparable to controls (p > 0.05; an estimated 12.60% to 67.27% of controls falling below JA's score)**.** However, as discussed when examining grasp-point behavior across trials, patient JA makes slow and deliberate movements as if she struggles with the task, evident in a sub-analysis showing abnormal grasp-point variance across trials. The same sub-analysis is also applied here to show that JA exercises a trial-and-error procedure until the final experimental block. When grasping the left weighted object in the memory-associated condition and the right weighted object in the visual-spatial cue condition, grasp-points moved further away from the optimum point of grasp to compensate for weight distribution in Block 5 compared to Block 1 (**Table 6**). Additionally, the average point of grasp of the left weighted visual-spatial cue condition in Block 1 was on the opposite side of the object from the optimum grasp-point indicating that she did not utilize the dot cue to indicate weight distribution. Therefore, although performance change appears comparable to control groups, patient JA's grasp behavior demonstrates performance deficits that differentiate her from control groups and may be indicative of more subtle deficits in the integration of visible and known object properties.

# DISCUSSION

To assess whether apraxic patients successfully integrate stored knowledge of objects into action plans, participants were required to learn different weight distributions when lifting and balancing objects using a pincer grip. Over two conditions, each objects' weight distribution was indicated by either a memoryassociated cue (object color) or visual-spatial cue (visible dot over the weighted end). If apraxic patients fail to incorporate stored information into their grasp, we expected that patients might disregard the location of the objects' center of mass and instead over-rely on visual information, resulting in more centrally oriented grasps based on object structure. The experiment was designed to examine whether patients could learn to grasp the weighted objects accurately when given a meaningful visualspatial cue indicating the object weight distribution, which would result in increasingly accurate grasps over time if this higher-level information was successfully integrated.

Performance change across trials (TC) and across blocks (BC) in the neutral-control screening condition confirmed that all apraxic patients (AH, GW, and JA) successfully grasped and balanced the evenly weighted object, eliminating the possibility any confounds such as hemispatial neglect or impaired task comprehension might be impacting their performance in the experimental conditions. Comparable to healthy and nonapraxic controls, during consecutive grasps of the neutral-control object (TC) and when grasping the object as it was reintroduced in later blocks (BC), apraxic patients' central grasp-points remained close to the optimum point of grasp to compensate for weight distribution. Accurate grasping performance during the neutral-control condition indicates that apraxic patients can successfully manipulate objects when the weight distribution is indicated by the objects' structure (symmetrical cylinder).

Although patient JA's performance change was within the normal range (see below for a discussion of JA's pattern of results) during a majority of the memory-associated and visualspatial cue conditions, patients AH and GW failed to update their grasp-point when the objects were unevenly weighted in both conditions. For both the memory-associated and visualspatial cue conditions, patient AH and GW maintained a central grasp-point during recurrent trials with the same object (TC) or when the objects were reintroduced in later blocks (BC). Failure to compensate for load torque by reorienting grasps towards the center of mass suggests that these apraxic patients failed to integrate acquired knowledge regarding objects into action plans. Inaccurate grasp-points persisting into the final test block was particularly representative of this. Paired with unimpaired behavior in the neutral-control condition, grasp performance of patients AH and GW suggests an over-reliance on the structural properties afforded by the object. Maintained central grasp-points in the memory-associated and visual-spatial cue conditions perhaps indicate that AH and GW continually referred to structural properties afforded by the object to guide their grasp behavior and did not benefit from either a meaningful visual-spatial cue or symbolic cue of weight distribution.

Patient AH and GW's performance is compatible with previous research indicating that in addition to impaired perception of skilled object-use (Buxbaum and Saffran, 2002; Buxbaum et al., 2003; Myung et al., 2010), apraxic patients frequently choose inappropriate non-functional grasps (Randerath et al., 2009, 2010; Sunderland et al., 2011) or demonstrate impaired grip force for familiar objects (Gordon et al., 1993; Dawson et al., 2010; Hermsdörfer et al., 2011; Eidenmüller et al., 2014). The performance of patient AH and GW across all three conditions support the proposal that the ventro-dorsal stream is compromised in these patients, resulting in impaired performance when grasping asymmetrically weighted objects. Confirmation that the impairment lies at the ventro-dorsal level comes from the fact that processing of object structure remains intact. Therefore ventro-dorsal disruption appears to impair skilled use of familiar objects, but also when learning to manipulate novel objects.

Interestingly, both patients AH and GW did not appear to benefit at all from the ''dot'' cue in the visual-spatial cue condition, and there was no evidence of learning. In healthy populations when an object is asymmetrically weighted, grasppoints typically migrate towards the weighted end, particularly when visual cues indicate where the center of mass is located (Endo et al., 2011). Apraxics use of familiar objects also improves from pantomime to actual-use with increased affordance or contextual cues (De Renzi and Lucchelli, 1988; Buxbaum and Saffran, 2002; Sunderland and Shinner, 2007; Goldenberg, 2009; Randerath et al., 2011). Although apraxic patients would not use the visual-spatial cue as effectively as control participants, it was hypothesized that the presence of increased visual information in the form of a visible dot over the weighted end might prompt more appropriate grasps in later trials or when the object was reintroduced.

It is possible that a visual cue, such as a dot, is not ecologically meaningful and subsequently requires more explicit learning. This differs from implicit visual geometric cues of shape and size that are ecologically meaningful (Gentile, 2000; Salimi et al., 2003). Consequently the explicit learning of a visual dot-weight association may also be reliant on higher order perceptual processes to conceptualize the meaning of the dot cue. If this is the case, comparable performance in the memoryassociated and visual-spatial cue conditions may be due to both requiring integration of stored and visible information via the ventro-dorsal stream. Therefore, it is reasonable that apraxic patients AH and GW did not benefit from the highlevel visual cue. Studies showing improved apraxic performance with increased contextual information may be attributed to an increased presence of low-level affordance cues regarding the objects' size and structure. Yet, it remains that apraxic patients may be able to register and utilize these memory-associated and visual-spatial cues but that low-level affordance cues are more dominant. According to the affordance competition hypothesis (Cisek, 2007), potential motor actions are generated simultaneously and selected on the basis of the action goal. Therefore, if object affordances compete for selection, the more symbolic memory-associated or visual-spatial cues may be overpowered by more salient low-level cues of object structure. Although it is not certain why these apraxic patients did not benefit from the visual-spatial cue, this observation is interesting when trying to understand what information, be it visual or symbolic, individuals use when manipulating objects to achieve action goals. If apraxic patients are more reliant on low-level affordance cues, this could have a substantial impact on their ability to learn to use new objects or appropriately use familiar objects when these cues are ambiguous. However, as very few studies have assessed learning of skilled movement in apraxia this can only be speculated, and emphasizes the need to explore learning in apraxia to determine the types of cues these patients can successfully utilize to inform their grasp.

Additionally, it was somewhat surprising that patients' AH and GW did not benefit from short-term sensorimotor feedback to improve grasp performance during subsequent trials within a block (TC). Attributed to the bilateral dorsal stream, rapidly decaying sensorimotor memory is formed and updated with repeated grasps of the same object (Bursztyn and Flanagan, 2008; Buxbaum and Kalénine, 2010). Apraxic patients apply appropriate fingertip force when repeatedly lifting novel objects, suggesting sensorimotor memories can be formed and applied (Gordon et al., 1993; Ietswaart et al., 2001; Dawson et al., 2010; Hermsdörfer et al., 2011; Li et al., 2011; Randerath et al., 2011; Eidenmüller et al., 2014). However, more central grasp-points remained fairly constant between the first and last trial in the current study. AH and GW may fail to update their-grasp points with repeated lifts due to visible structural information and short-term sensorimotor feedback being in conflict; object shape suggests a central weight distribution whereas sensorimotor feedback indicates it is either to the left or the right of the object. In grip force studies, the novel objects were typically symmetrical with a central weight distribution; the shape of the novel object corroborates sensorimotor feedback of object weight, resulting in improved fingertip force with repeated lifts (for examples see Gordon et al., 1993; Dawson et al., 2010; Li et al., 2011). Consequently it is argued that failure to use short-term sensorimotor feedback by patient AH and GW is not because this process is disrupted, but that the design of the current task causes an impediment between visual and sensorimotor information leading to low-level visual affordance cues to be favored. Taken together, the performance of patient AH and GW in memory-associated and visual-spatial cue conditions confirms that they fail to incorporate stored knowledge into action plans even in the presence of certain visible cues.

Interestingly, patient JA's performance change was comparable to control groups in all conditions, except when compared to healthy controls during repeated grasps (TC) of the visual-spatial cue objects. However, further analyses of grasp-point indicate that patient JA did indeed struggle to apply knowledge-based information or visual-spatial cues in learning to grasp the weighted objects. Exploring JA's behavior when grasping visual-spatial cued objects, a positive score for accuracy change over trials indicates that JA continued to make errors to the final trial. Although these errors were only minor in contrast to patient AH and GW who consistently failed to adjust their grasp-point according to weight distribution, when examining individual participants' performance change none of the non-apraxic patients or healthy controls failed to adapt their grasp-point over repeated lifts (TC) and when the objects were reintroduced (BC). Therefore it is possible that apraxic patient JA used compensatory mechanisms to improve performance. Patient JA's variable grasp behavior also suggests that she may be maintaining a trial-and-error procedure throughout the experiment. In particular, when grasping specific objects within the memory-associated and visual-spatial cue conditions, patient JA's grasp-point deviated further from the optimum point of grasp to compensate for object weight distribution in later trials and when the objects were reintroduced, whereas control participants grasps moved closer to the optimum grasp-point. Likewise, patient JA's point of grasp was grossly variable from Block 1–5; JA adjusted her grasp-point by almost 20 cm in both the memory-associated and visual-spatial cue conditions. This behavior seemed to demonstrate a more subtle manifestation of the deficit in the integration of visible and known object properties that results in more changeable grasp accuracy.

These subtle effects in JA were in line with the behavior she displayed. JA, a young and highly motivated patient, performed the task slowly and deliberately. She appeared more aware of her deficit than the other patients. Perhaps this due to the fact that she was aware of her apraxic symptoms that included actual objectuse (evident in standard apraxia screening). If this is the case, JA is more likely to compensate for her impairment resulting in improved grasping performance compared to the other apraxic patients. Although patient AH has a similar lesion to JA, she inevitably will have been less aware of her apraxic symptoms that did not include actual object-use. Likewise, GW demonstrated more severe apraxic errors across the screening tasks and may be less able to effectively compensate for his impairment. No compensative strategies in performance of the experimental task were apparent in AH or GW who performed the task very quickly, immediately reaching for the object at the start of each trial and rapidly lifting each object before returning it to the table. In contrast, JA showed awareness of difficulty with the task, commenting on completion that she tried to apply strategies: she said that when the object was placed in the testing area, she observed whether one end of the object landed on the table first as a potential clue to its weight distribution. Although the availability of such cues were avoided through careful placement of each object, it may be beneficial to occlude participants' view when objects are placed on the table. However, it was felt that the presence of each object during testing ensured that participants were aware that each object reintroduced in later blocks was the same as those seen previously. Finally, the less gross errors of patient JA on the grasping task compared to AH and GW cannot be attributed to better comprehension, as JA scored the least in the language comprehension test. Likewise, JA did not suffer from milder apraxic symptoms; as described, patient GW demonstrated the more severe apraxic symptoms whereas JA's apraxic behavior was comparable to AH.

Rather than ventro-dorsal processing remaining intact in patient JA, it is believed that through her careful performance, she managed to assemble compensatory strategies, even when weight distribution was afforded by a high-level visual-spatial cue. Appropriate performance when behavior is delayed in apraxic patients suggests that stored knowledge is maintained, but difficult to access. As described, accurate memory-driven reach and grasp performance is observed when apraxic patients pick up basic blocks based on simple size and distance information (Ietswaart et al., 2001). Myung et al. (2010) also confirmed that during semantic judgements, apraxic patients also showed greater fixations on object pictures that were manipulation-related to the target word (e.g., ''typewriter'' and ''piano'') when the manipulation relationship was not task relevant; the fixation position was comparable to their nonapraxic control group but the effect emerged later, again indicating that stored representations are preserved but not easily accessible. The magnitude of delayed activation of manipulation related action information in apraxia is predicted by poorer object-use pantomime performance and the extent to which inferior parietal and posterior temporal regions were compromised (Lee et al., 2014). Therefore, the extended delay between reach and grasp movements used by JA in her slow and deliberate performance (compared to patient AH and GW who initiated grasps immediately) may have enabled her to incorporate stored knowledge into action plans. Further, the variable nature of her points of grasp along each object may be indicative of when her compensatory strategies were less effective. This may also indicate why JA continued to make grasping errors by the final trial when grasping the visual-spatial cued objects.

Although the design of the current study delayed reachto-grasp action between trials by requiring participants to return their hand to the table before beginning another grasp movement, the duration of this delay was not controlled. Further investigation is required to confirm whether delay between reaching and grasping can reduce performance errors when balancing novel objects. It is probable that such compensatory strategies may rely on critical brain structures being intact; JA presented with frontal lesions that implicate white matter whilst parietal regions remain undamaged (as was the case in AH). In contrast, GW's lesion implicates temporal and parietal regions of the left hemisphere suggesting that the critical juncture between the ventral and dorsal pathways may be compromised (Rizzolatti and Matelli, 2003; Buxbaum and Kalénine, 2010; Rizzolatti et al., 2011; Binkofski and Buxbaum, 2013; Vingerhoets, 2014). This corresponds with patient GW's markedly poor performance across all apraxic tests. Based on research showing a strong association between impaired objectuse and temporal and parietal damage (Goldenberg, 2009; Vingerhoets, 2014), impaired use of memory-associated and visual-spatial cued information is expected in this patient.

In conclusion, apraxia was associated with a disrupted ability to utilize memory-associated or visual-spatial cued information indicating weight distribution. Specifically, patient AH and GW failed to successfully incorporate memoryassociated information where weight distribution was indicated by the objects color, and visual-spatial cued information in the form of a dot cue over the objects weighted. Grasps were inaccurate during repeated lifts and when the objects were reintroduced. A third apraxic patient (JA) seemed to compensate for these difficulties but still showed performance errors that may be attributable to a more subtle impairment. These results indicate that apraxia impairs the ability to utilize meaningful visual-spatial cue or symbolic memory-associated cues when grasping objects to achieve specific action goals. Crucially, the abnormal grasping behavior in these apraxic patients suggests that integration of visible and known object properties attributed to the ventro-dorsal stream is impaired. Not only does disruption to ventro-dorsal processing impair use of familiar objects, but

# REFERENCES


also these results would predict that apraxia is associated with difficulty learning to manipulate new objects.

# AUTHOR CONTRIBUTIONS

CE, conception and design of the research task; acquisition, analysis and interpretation of the data; drafting the manuscript and final editing. MGE, contribution to the conception of the task, critically revising and editing the manuscript, and final approval of the manuscript. LJT, contribution to the conception of the task, final approval of the manuscript version to be published. MI, substantial contribution to the conception and design of the research task, interpretation of the data and critically revising and editing the manuscript.

# ACKNOWLEDGMENTS

This study was funded by Northumbria University. We would like to thank the individuals who took part in the study, the local NHS hospitals and rehabilitation centres in the North East of England where many were recruited, and in particular the responsible clinicians Dr. David Bruce, Dr. Tim Cassidy, Dr. Akif Gani, and Dr. Chris Price and their teams.


evidence from eyetracking and semantic judgment tasks. Brain Lang. 112, 101–112. doi: 10.1016/j.bandl.2009.12.003


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Evans, Edwards, Taylor and Ietswaart. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# The Representation of Objects in Apraxia: From Action Execution to Error Awareness

Loredana Canzano1,2 , Michele Scandola1,3 , Valeria Gobbetto<sup>3</sup> , Giuseppe Moretto<sup>4</sup> , Daniela D'Imperio2,3 and Valentina Moro<sup>3</sup> \*

1 IRCCS Santa Lucia Foundation, Rome, Italy, <sup>2</sup> Department of Psychology, La Sapienza University, Rome, Italy, <sup>3</sup> NPSY-Lab.Vr, Department of Philosophy, Education and Psychology, University of Verona, Verona, Italy, <sup>4</sup> UOC Neurology A, Azienda Ospedaliera Universitaria Integrata, Verona, Italy

Apraxia is a well-known syndrome characterized by the sufferer's inability to perform routine gestures. In an attempt to understand the syndrome better, various different theories have been developed and a number of classifications of different subtypes have been proposed. In this article review, we will address these theories with a specific focus on how the use of objects helps us to better understand upper limb apraxia. With this aim, we will consider transitive vs. intransitive action dissociation as well as less frequent types of apraxia involving objects, i.e., constructive apraxia and magnetic apraxia. Pantomime and the imitation of objects in use are also considered with a view to dissociating the various different components involved in upper limb apraxia. Finally, we discuss the evidence relating to action recognition and awareness of errors in the execution of actions. Various different components concerning the use of objects emerge from our analysis and the results show that knowledge of an object and sensorymotor representations are supported by other functions such as spatial and body representations, executive functions and monitoring systems.

#### Edited by:

Ferdinand Binkofski, RWTH Aachen University, Germany

#### Reviewed by:

Stefan Heim, RWTH Aachen University, Germany Juliane Klann, University Hospital, RWTH Aachen University, Germany

> \*Correspondence: Valentina Moro valentina.moro@univr.it

Received: 28 July 2015 Accepted: 23 January 2016 Published: 10 February 2016

#### Citation:

Canzano L, Scandola M, Gobbetto V, Moretto G, D'Imperio D and Moro V (2016) The Representation of Objects in Apraxia: From Action Execution to Error Awareness. Front. Hum. Neurosci. 10:39. doi: 10.3389/fnhum.2016.00039 Keywords: objects in apraxia, action recognition, imitation and pantomime, error awareness

# APRAXIA: A MULTIFACETED AND COMPLEX SYNDROME

The term Apraxia covers a wide spectrum of disorders, all referring to motor cognition and the inability to perform actions that have been previously learned and/or were possible before the onset of the syndrome. These deficits cannot be explained by elementary motor or sensory deficits and are not due to language comprehension disorders (Zadikoff and Lang, 2005). Apraxia is usually the result of left frontal and parietal lesions (prevalence ranging from 28 to 57%, Donkervoort et al., 2000), although in some cases apraxia following right brain damage has been reported (prevalence ranging from 0 to 34%, Donkervoort et al., 2000). In addition, lesions involving the corpus callosum cause unilateral left apraxia. Thus, the left hemisphere appears to be dominant in processing actions (Petreska et al., 2007).

Apraxia is characterized by an automatic—voluntary dissociation (De Renzi et al., 1982). In other words, patients can execute spontaneous gestures when the environmental context induces their involuntary/automatic response (e.g., waving their hand to say goodbye when they are going away) but they are not able to intentionally execute the same action out-of-context or when asked to do so by an experimenter. For this reason apraxia is considered as a disorder of the voluntary and aware ability to perform gestures (Wolpe et al., 2014).

Steinthal first introduced the term Apraxia (literally = without action) in 1871 to describe the difficulty that certain patients had when they tried to execute an action which involved an object or a tool. He suggested that the deficit depends on disorders in the relationship between the patients' movements and their abilities to manipulate objects (Steinthal, 1871, 1881). Since then, various different forms of apraxia have been described, some which involve objects, others which do not.

Liepmann (1920) proposed a classification of the different subtypes of apraxia with the aim of identifying the various motor and cognitive aspects. He identified three different subtypes. A person who is able to name familiar tools and objects but is almost totally unable to use them correctly suffers from Ideational apraxia. In this case, the person has lost the ability to conceptually organize intended actions. Ideo-kinetic (or ideo-motor) apraxia is a disorder affecting the production component of the praxis system, resulting from an apparent dissociation between the idea of an action and its execution. This also involves an inability to pantomime actions or mimic an action with an object or tool (without actually holding the object in question). Finally, limb-kinetic apraxia refers to a loss of dexterity or deftness, characterized by hesitations and a disrupted smoothness in movements (Liepmann, 1920; cited in Goldenberg, 2013).

The cognitive nature of apraxic deficits was also discussed by Geschwind who suggested that apraxia does not extend to novel or meaningless movements, but exclusively concerns learned motor skills: ''the hemisphere dominant for handedness is a storehouse of the learning involved in the acquisition of motor skills'' (Geschwind and Damasio, 1985, p. 191). When this storehouse, localized in the lower left parietal area, is damaged or disconnected from verbal and visual commands or from the premotor cortex (Heilman et al., 1982; Petreska et al., 2007), patients are apraxic. However, this hypothesis was not exhaustive. Indeed, evidence of deficits in the imitation of novel, meaningless gestures (with meaningful actions spared) has led to the identification of a new subtype of apraxia, the visuo-imitative apraxia (Goldenberg and Hagmann, 1997). In this case, patients do not present with a general defect affecting imitation but suffer from a specific deficit in the imitation of meaningless gestures. This dissociation has been explained by the Dual-Route Model (Gonzalez Rothi et al., 1991) that suggests the existence of two streams involved in the production and imitation of actions. With the direct route (or non-lexical route), the gesture is produced by means of a direct translation of visual input into motor outputs. This permits the imitation of both novel, meaningless gestures and significant and familiar actions. The alternative, semantic route (or lexical route) needs lexical, semantic memory and is exclusively useful for familiar and meaningful gestures (Gonzalez Rothi et al., 1991). Thus, an interruption in the direct route does not affect meaningful actions, but it does cause a specific disorder affecting the selective imitation of new and meaningless gestures (visuo-imitative apraxia). The Dual-Route Model was revised by Cubelli et al. (2000) who added a system specifically devoted to the direct transcoding of visual input into motor programs (the ''visuo-motor conversion mechanism'') and a system for short-term representation of the whole action (the ''gestural buffer''; **Figure 1**).

Taking all these approaches to apraxia into account, we may consider that both sensory–motor and cognitive components play a role in the execution of gestures (Buxbaum et al., 2014; Goldenberg, 2014; Osiurak and Le Gall, 2015). Using an object or pantomiming the use of an object without a model certainly requires the recruitment of cognitive functions such as knowledge of the object and its function and/or the context in which it is usually employed. Nevertheless, these need to be integrated with motor and sensory functions (Goldenberg, 2013, see below). Body representations may also play a crucial role. A strong connection between gestures and the body has been demonstrated in apraxia. Goldenberg (1995) have shown that knowledge of one's own body is necessary in gesture imitation. Indeed, patients who are impaired when executing gestures involving their own body are also impaired when reproducing the same gestures using a manikin. This indicates a close link between body representations and action planning. Disorders in imaging and planning the functional relationship between body parts and objects are also suggested by the typology of the errors which apraxic patients frequently commit when they pantomime transitive actions: they often use their hand as if this was the object or a part of the object (Body part as object). Finally, the existence of effector-specific forms of apraxia suggests a relationship between body representations and gesture making disorders. In fact, various different types of apraxia have been described involving the face (upper/lower face apraxia, oral apraxia, orofacial apraxia, apraxia of speech), the eyes (eyelid apraxia, ocular apraxia, gaze apraxia), the limbs (hand apraxia, finger apraxia, apraxic agraphia, dressing apraxia, magnetic apraxia), the legs (leg apraxia, gait apraxia) and the trunk (axial apraxia; Petreska et al., 2007). These subtypes correspond to at least partially different lesion sites. For example, while an impairment in the imitation of hand gestures is associated with left inferior parietal lesions, impairments in the imitation of finger gestures may follow both right and left pre-central and inferior frontal lesions. Disorders in lower face movements are a consequence of damage to the left ventral precentral frontal gyrus, while deficits in upper face movements may follow both left and right sided lesions (Goldenberg, 2013). In this article, the term apraxia will refer to limb apraxia unless otherwise specified. In general, limb apraxia is more frequent after left as compared to right hemisphere brain damage. In addition, left hemisphere lesions usually cause bilateral signs of apraxia, while damage to the right hemisphere only affects the left hand (Petreska et al., 2007).

The role of body representations in the execution of gestures was also suggested by Buxbaum et al. (2000) in their revision of Rothi's model. The authors emphasized the importance of spatial components, in particular the need for updates regarding the reciprocal spatial dynamic positions of the body parts in relation to an object while an action is being executed. This stage of an action is between the lexical and nonlexical route and subserves both meaningful and meaningless actions.

In this context, apraxia appears to be a complex, multifaceted syndrome. In addition to specific knowledge (of an object, its function and its relative context) and sensory-motor abilities (the planning and execution of actions), other elements such as body and space representations may affect the execution of a gesture. As such, understanding the role of objects in apraxia may help us to achieve a better understanding of the nature of apraxia. Indeed, deficits related to the use of objects have been recognized as the main symptom of the syndrome since it was first identified (Steinthal, 1871) and it is this disorder that patients complain about most. Moreover, research has recently shown that the shape and position of objects can activate motor responses in healthy people (affordance, Gibson, 1979; Ellis and Tucker, 2000) and that body-object interaction may involve specific non-semantic types of knowledge (e.g., memory of movements, knowledge regarding manipulation, mechanical problem solving, action monitoring and error awareness).

The role of objects in action execution and error recognition is the topic of this article. We will start by describing some types of apraxia and the main dissociation between deficits involving the use of objects (i.e., transitive actions) and those which do not involve objects (i.e., intransitive actions). We will also briefly introduce two specific subtypes of apraxia that in some way involve the incorrect use of objects: constructional apraxia (Critchley, 1953) and magnetic apraxia (Denny-Brown, 1958). Deficits in action execution will then be analyzed with reference to the three tasks usually administered in the assessment of apraxia: the use of objects and the imitation and pantomiming of actions. Finally, the potential effects of apraxia on the recognition of actions and the role of objects in the detection of errors in the execution of actions will be discussed.

# OBJECTS IN APRAXIA

The distinction between transitive and intransitive gestures is based on whether or not an action involves the use of an object. The transitive/intransitive dissociation has been documented in several case studies reporting gesture-specific forms of apraxia (Rapcsak et al., 1993; Dumont et al., 1999).

A transitive gesture is tool-based (e.g., hammering in a nail) and it is in some way shaped by the nature of the object and by any knowledge possessed regarding its functions or potential uses. Indeed, if an object is actually present, the action may take a third route, in addition to the two previously mentioned routes in Rothi et al.'s (1985) model, the ''selective tactile route''. There is evidence that this third route may be specific and crucial for actions involving objects, which may potentially be driven by tactile information inherent to objects (Graham et al., 1999). The existence of this additional route seems to be confirmed by evidence collected from a patient who was impaired when responding to verbal or visual commands which requested him to perform certain gestures, but who was able to execute an action when he took hold of a tool (Buxbaum et al., 2000). In spite of this ''third route'', accuracy in transitive gestures is usually reported to be lower than in intransitive gestures (Haaland and Flaherty, 1984; Gonzalez Rothi et al., 1988; Schnider et al., 1997; Haaland et al., 2000), although apraxic patients are often impaired in both transitive and intransitive actions. Nevertheless, it has been suggested that symptoms affecting both hands only affect transitive gestures, while disorders in intransitive actions usually only involve the contralesional hand (Watson et al., 1986; Binkofski et al., 2001).

A problematic aspect concerning the classification of transitive actions concerns the distinction between single step and multiple step gestures. Some authors (Heilman and Rothi, 1993; Raymer and Ochipa, 1997) have suggested a distinction between the difficulty experienced when using a tool or object (conceptual apraxia) and the inability to execute multistep actions (ideational apraxia). Of course, both of these involve the idea of an action, but in the case of ideational apraxia, errors might be due to other than sensory-motor errors (e.g., step omissions or perseverance). For this reason, the definition relating to Action Disorganization would seem to be more appropriate (Schwartz et al., 1995; Humphreys and Forde, 1998). Action Disorganization refers to cases where habitual actions are performed perfectly but disturbances arise when an action requires a preformed plan in accordance with a specific goal (Poeck, 1983; Schwartz et al., 1993, 1995; Goldenberg et al., 2007). In this way, we can see it is possible to distinguish between ideational components and more executive aspects.

An important issue in the debate on transitive actions concerns the source of the knowledge which is necessary for the appropriate use of an object. Two types of knowledge are considered to be necessary: knowledge regarding the features relating to a particular tool or object (Functional Knowledge) and knowledge of the action required for that object and of how to organize the individual motor sequences involved in that action (Manipulation Knowledge; Roy and Square, 1995).

Functional knowledge of tools lies in the semantic memory (Goldenberg and Randerath, 2015) and associates various types of tools with their purpose and the actions they can be used for. When a tool has several possible uses, functional knowledge is used to weigh these up based on their relative frequency and familiarity. The prototypical use invariably predominates (Goldenberg, 2013).

Manipulation knowledge refers to the (modality specific) motor representations that underlie the use of familiar tools and objects. This corresponds to the ''engrams'' or ''movement memory'' (Heilman and Rothi, 1993) that are thought to contain the features of gestures (i.e., muscular and joint actions, hand postures) which are invariant and critical when one needs to distinguish between one gesture and another (Buxbaum, 2001). However, each action requires adaptations of its invariant features in order to deal with changes in environmental conditions (e.g., the position, shape or size of an object). These engrams cannot therefore be rigid and stable. Goldenberg (2013) suggests that this specific manipulation knowledge is only necessary for the special, expert use of a tool (e.g., using a hammer for sculpting, playing a violin) but not for conventional tools. In everyday activities, manipulation knowledge would be replaced by the interaction between general functional knowledge and mechanical problem solving processes (Goldenberg, 2013, p. 125). Mechanical problem solving involves the ability to infer its function from the structure of an object (Goldenberg and Hagmann, 1998). It refers to general rules in the context of mechanical interactions with objects rather than to the functional properties of an individual object. These rules are based on the general principles of physics and mechanics that people acquire over the course of their lives (''folk physics'', Povinelli et al., 2000) and they apply to concrete constellations of tools and objects. As familiar and novel objects share a similar repertoire of functionally significant parts and properties (e.g., a handle, a blade) and since familiar applications of tools obey the same physical regularities, mechanical problem solving allows the accommodation of new objects and assists in the identification of alternative ways of using familiar objects (e.g., a coin used as a screwdriver). Deficits in functional knowledge lead to the defective use of common tools, while disorders in mechanical problem solving impact unusual, alternative uses of familiar objects and novel tools (Goldenberg, 2013). Of course, in both situations components relating to knowledge about an object, sensorymotor information and spatial and body representations are involved. However, while people exclusively rely on previously learned contents when using common tools, mechanical problem solving (which is necessary for novel actions) requires the integration of these components in a totally new way or in a way that is only partially similar to previously used methods. This may explain the fact that some patients can perform habitual actions but are totally unable to use unusual objects.

We can thus understand that when people perform new, unusual actions, the mechanical problem solving they resort to is based on information provided by the object they wish to use. When people identify an object, they activate exploratory movements to upload its tactile properties (Loeb and Fishel, 2014). ''Perception is not something that happens to us or in us: it is something we do'' (Noe, 2004, p. 1). An elegant exemplification of this affirmation was made by Gibson (1979) who coined the term ''affordance'', that is, the implicit effect of the association of an object with the various actions and functions that it allows. Affordance depends on the setting between the physical properties of the body and the physical features of the environment (Warren, 1984; Adolph and Berger, 2006). Ellis and Tucker (2000), in fact, proposed the term ''micro-affordances'' to refer to the activation of action components appropriate for interacting with objects. The fact that body representations are necessary has been demonstrated in studies indicating that adults judge affordances with respect to intrinsic information about their bodies (Warren, 1984; Mark, 1987; Warren and Whang, 1987; Mark et al., 1990). However, it is not only the perception of affordance that guides an action: perception and actions are in a continuous feedback loop (Patla, 1998; Adolph and Berger, 2006; Franchak et al., 2010). In addition to the pragmatic process, which includes an analysis of the various different affordances and potential translations into action, higher order visual areas provide a perceptually based parallel semantic description of the object (Jeannerod et al., 1995; Ellis and Tucker, 2000; Maranesi et al., 2014).

Thus, using tools and grasping objects (with a configuration of the hand in accordance with the object) are highly specialized behaviors in primates (Jeannerod et al., 1995; Macfarlane and Graziano, 2009; Maranesi et al., 2014) indicating that they are able to reinterpret the physical world as a series of abstract features (Penn et al., 2008). An inability to use tools may thus reflect damage to the ''stored representational system of gestures'' (Buxbaum, 2001). This system supports representations regarding a tool (the Functional Knowledge), its associations and the purpose of any actions performed with it (the Manipulation Knowledge) or, as Luria (1978) suggested, it may be the result of deficits in executive planning (i.e., Dysexecutive syndrome). Finally, some authors attribute difficulty in using tools to a specific problem with technical reasoning (Gagnepain, 1990; Le Gall, 1998; Osiurak et al., 2009, 2010), including difficulties in identifying and unifying the technical means relevant for a given technical end (Jarry et al., 2013).

Taken as a whole, these complementary analyses of the various processes involved in transitive actions make it possible to identify a further component, a sort of implicit, nonverbal, practical/technical reasoning which may or may not be dissociable from executive functions. Although Mechanical Problem Solving is based on all the other components (the visual and tactile perception of objects and environments, motricity, spatial and body representations), it is probably crucial to understanding apraxia (Goldenberg, 2013). For example, it may explain two well-known dissociations: the automatic/voluntary association and the know/unknown action dissociation. Ignoring these aspects often leads to an underestimation of any diagnosis of apraxia in patient reports and the onset of symptoms is only reported after the patient has been discharged from hospital.

We also wish to put forward a hypothesis suggesting errors linked to the various components of the execution of an action may be differently associated with Functional Knowledge or Mechanical Problem Solving.

Several classifications of apraxic errors in the use of objects have been suggested (De Renzi and Lucchelli, 1988; Humphreys and Forde, 1998; Schwartz et al., 1998; Goldenberg et al., 2001; Rumiati et al., 2001; Petreska et al., 2007). Among these, errors due to disorders in Functional Knowledge may be Perplexity, Conduits d'approche, Omission and Misuse (involving content, substitutive, augmentative, fragmentary and associative errors, Petreska et al., 2007).

Patients who show Perplexity seem to have no idea what they can do with an object: ''The patient looked hesitatingly at the objects, picked up one of them, turned it over, put it down, then tried with another object, giving unmistakable signs of not knowing what to do'' (De Renzi and Lucchelli, 1988, p. 1177). Sometimes patients seem to try various different actions in order to progressively reach the right one (e.g., when trying to use a toothbrush, the patient starts hitting his/her cheek, reaches his/her mouth and is finally able to brush his/her teeth). These Conduits d'approches are very similar to those of aphasic patients when speaking. Omissions may be present in multistep actions such as when patients forget ''to carry out an action necessary for completing the sequence, for example, the stamp was not moistened'' (De Renzi and Lucchelli, 1988, p. 1177) and this leads to incomplete executions. In the case of Misuse, the object is used in a conceptually inappropriate way or is used as if it was another object (Parapraxic errors). Here the patient not only does not have any idea of what to do, but seems not to realize his/her difficulty when using the object incorrectly. Other errors indicating object misuse are the replacement of one movement with another that shares one or more similar features (Associative errors), the fragmentation of gestures or the production of inappropriate steps (Augmentative errors).

The disorders which are linked to Mechanical Problem Solving seem to be Clumsiness, Mislocation, Sequence errors and Perseveration. Clumsiness refers to when an action appears to be conceptually correct for the tool but is ''carried out in an awkward and ineffectual way, because of poor control of skilled hand movements'' (De Renzi and Lucchelli, 1988, p. 1177). Mislocation is when an object is used in an appropriate way but in a non-appropriate place. Spatial misorientation of an object or of an object with respect to the body is also sometimes considered to be the same type of error. When the Sequence is incorrect, part of an action is executed without the previous step having been completed (e.g., the envelope is sealed before the letter is placed inside it). Finally, Perseveration refers to a situation where a patient continues to repeat part of an action without any apparent aim or he/she is unable to stop executing one step in order to execute the next.

Although these errors are much more frequent in left damaged apraxic patients, it is worth noting that very similar errors may be also present in non apraxic patients. For example, Mislocation and errors in Trajectory are frequent in right hemisphere damaged people (in particular in the presence of spatial neglect) and Perseveration and Frequency errors are a typical index of frontal damage. Although the most part of right hemisphere damaged patients' errors are usually considered due to spatial and more general attentive disorders (Goldenberg, 2013), only in depth qualitative investigation will enable a better understanding of the various different expressions of action errors.

# CONSTRUCTIONAL APRAXIA

Constructional apraxia was defined by Benton as ''the impairment in combinatory or organizing activity in which details must be clearly perceived and in which the relationship among the component parts of the entity must be apprehended'' (Benton, 1967). Although constructional apraxia is usually assessed by means of drawing or copying tasks, this also impacts the patient's ability to put together the components of an object (e.g., a coffee machine or a food mixer) with consequences affecting everyday activities. The main cause of this syndrome seems to involve a disorder in Mechanical Problem Solving. Nevertheless, other action components may impact on constructional abilities. Critchley (1953, p. 191) described this form of apraxia in these terms: ''The defects which characterize constructional apraxia essentially involve those movements which are directly concerned with space per se, i.e., manipulation of the three dimensions of space, and particularly the translation of an object from one spatial dimension into another''. In fact, lesions in both the right and left hemisphere may produce constructional apraxia, although the symptoms are qualitatively different. After left hemisphere lesions, errors regard the comprehension of the function of an object or its parts, the sequence required to put together the various parts and the organization of that sequence. Copies of drawings respects the appropriate distance to the model and the global orientation and outlines, although the drawing appears impoverished by lack of or simplification of details. In contrast, in the case of right hemisphere lesion, patients mainly commit spatial errors regarding the positioning of the individual parts of an object and their reciprocal relations. Copies of drawings are badly placed and sometimes too close to the model or overlapping (''closing in'') with a distortion of the horizontal and vertical axes (Goldenberg, 2013). When spatial neglect is present, the parts of the picture in the contra-lesional space are totally omitted and the global structure is broken.

# MAGNETIC APRAXIA

The compulsive tactile exploration and object grasping which often occurs in the contra-lesional hand after left or right frontal lobe damage is called Magnetic Apraxia (Denny-Brown, 1958; Moro et al., 2015). In this condition, the mere visual presence of an object near the hand (or touching the hand) triggers groping movements as well as grasping. In spite of the fact that these movements appear to be goal directed, they are totally involuntary and the patient is not able to inhibit the behavior of the hand. Magnetic Apraxia is often associated with grasping, an inability to release the grip (Forced grasping response) and groping (i.e., movements toward a stimulus based on the mere proximity of the stimulus and not triggered by tactile stimulation). In addition, utilization behavior (i.e., involuntary and inappropriate use of objects) and the compulsive involuntary manipulation of tools may be present. Finally, when magnetic apraxia is a symptom of the Anarchic Hand syndrome, it may be associated with Intermanual conflict (i.e., the hand movements interfere with non-anarchic actions) and Diagonistic dyspraxia (i.e., uncontrolled crosspurpose actions of the Anarchic Hand are triggered by voluntary activities of the non-Anarchic Hand; Moro et al., 2015).

These involuntary movements may lead one to object that Magnetic Apraxia is not strictly a form of apraxia. In fact this is not a disorder affecting the voluntary and aware ability to make gestures (Wolpe et al., 2014). Nevertheless, alterations in objectbody (i.e., hand) interactions are the main symptom of Magnetic Apraxia associated with an inability to inhibit involuntary actions and the exacerbation of automatic responses. The result is a dysfunctional use of objects.

# IMITATION AND PANTOMIME

Despite the fact that early descriptions of limb apraxia mainly concerned the difficulty that patients experienced in the use of objects, assessments of apraxic symptoms are usually carried out by means of imitation and pantomime tasks (for the object use task, see De Renzi and Lucchelli, 1988). In imitation tasks, subjects are asked to reproduce the actions executed by the examiner, while in pantomime tasks they are requested to make specific gestures on verbal or kinaesthetic command or after an object is presented (but with the pantomime being performed without the object).

The differences between these two types of tasks are crucial if we wish to understand the nature of apraxia and the potential role of objects (Goldenberg, 2013). In fact, in a seminal model of apraxia, Roy and Hall (1992) proposed distinguishing between two sequentially arranged phases in gesture production. In the first phase, a mental image of the action is created using the longterm memory (with the involvement of the Semantic Route of Gonzalez Rothi et al., 1991). This is typical for pantomime tasks but not necessary for imitation tasks. In the second phase, the image is converted into motor response programs (in addition to all the components previously discussed). In imitation tasks, only this second phase is necessary since an image of the action is provided by the examiner who executes the gesture that patients have to imitate (the Direct Route, following Gonzalez Rothi et al., 1991). From this perspective, a deficit in imitation would always be associated with a disorder in pantomime due to the deficit affecting the second phase of the process which is common to both the tasks.

The fact that a process of action goal recall is possibly also involved in imitation tasks belies Roy and Hall's theory. According to the Theory of Goal-Directed Imitation (GOADI; Wohlschläger and Bekkering, 2002), an imitator does not necessarily need to imitate the observed movement but can use the model as a cue to select pre-existing motor programs. In this case, when the gesture is executed, the motor program does or does not match the movement of the model, but the main goal of the action is achieved properly. The central principle of GOADI is that the selected goals elicit the motor program with which they are most strongly associated even though these motor programs may not necessarily lead to matching movements (Wohlschläger et al., 2003). In this way, the existence of two routes sustaining imitation is postulated, one of which relies on existing motor programs and the other that bypasses them. Only familiar actions can use pre-existing motor programs, while new unfamiliar gestures replicate the motor programs showed by the model. As a result, unfamiliar actions may be more similar to the model than familiar ones. Gravenhorst and Walter (2009) advance the idea that an interference effect of familiarity is modulated by perception, and that perception is in turn modulated by habitual style. Dissociations in the ability to imitate familiar and unfamiliar gestures are thus possible.

To sum up, both pantomime and imitation can be performed when action goal and motor memory recall come into play. Nevertheless, while imitation can be also performed without these elements, pantomime can not. Probably for this reason, it has been suggested that the most sensitive test in order to assess motor memory and action goal recall in apraxia is the pantomime on verbal command task since this provides the least cues and is almost entirely dependent on stored learned movement representations (visuo-kinesthetic movement engrams or praxicons, Mozaz et al., 2002). Seeing or holding a tool, as well as observing an examiner perform a pantomime, may provide a patient with cues, and if the movement representation is only partially degraded, these cues may obscure the diagnosis (Mozaz et al., 2002). In fact, as previously discussed, it has been suggested that not only visual, but also tactile feedback about the shape, weight and other properties of an object or tool may have a role in eliciting correct actions. In particular, De Renzi et al. (1982) showed that there are no differences in pantomime on verbal command with the tool in sight, while patients improve in the condition when, although blindfolded, they execute the action with the tool in their hand. Nevertheless, more recent evidence questions the possibility that tactile feedback per se is sufficient to evoke motor programs of correct tool use, and suggests that the facilitation is rather induced by the provision of additional information on the structural and functional features of the real use of the tool (Goldenberg, 2013) and all the possibilities that the environment offers (''affordance'', Gibson, 1979). The ''creation of pantomimes requires transformation of knowledge about the tool and its manipulation into emptyhanded gestures that communicate the identity of the tool and the manner of its manipulation to other persons'' (Goldenberg, 2013, p. 155).

A qualitative analysis of the errors which occur in the pantomime of transitive gestures is particularly interesting as it provides evidence of a distortion in the body-object relationship. In fact, gestures may be correct in terms of the identification of the tool and the action, but patients fail because they use a body part as if it was the object (Body part as object, Goodglass and Kaplan, 1963) or demonstrate the shape of the object rather than pantomime its use (Goldenberg, 2013).

In their analysis of errors, Buxbaum et al. (2000) consider four components of gesture imitation and pantomime. These are: (i) Hand Posture/Grasp—i.e., when ''the hand posture/grasp is unrecognizable, flagrantly incorrect or transiently correct; (ii) Arm Posture/Trajectory—i.e., when ''the arm posture and/or the trajectory/shape of the movement are flagrantly incorrect or only transiently correct''; (iii) Amplitude of movement—i.e., when ''the size of the movement is clearly too large or too small, or the size is only transiently correct''; and (iv) Timing/Frequency of movement—i.e., when ''the speed of the movement is flagrantly too fast or slow and/or the number of cycles is flagrantly too few or many'' (Buxbaum et al., 2007, p. 423; see also Buxbaum et al., 2000, 2005; Moro et al., 2008, 2015).

With the exception of one report (Belanger et al., 1996), many studies have reported that patients with apraxia are more impaired when performing transitive pantomimes (e.g., using a knife to cut bread) than intransitive gestures (e.g., waving goodbye; Haaland and Flaherty, 1984; Gonzalez Rothi et al., 1988; Roy et al., 1991; Schnider et al., 1997; Foundas et al., 1999; Haaland et al., 2000). Similar results have also been found in healthy people (Mozaz et al., 2002; Carmo and Rumiati, 2009). Although it is possible that the movements associated with transitive pantomimes are more complex than those involved in intransitive gestures, differences in the frequency with which these gestures are performed may also have a role. When people observe other people or when they want to communicate with a nonverbal message, they activate representations of intransitive gestures. In contrast, people primarily use transitive postures when they use tools or objects. A request to perform a transitive pantomime is thus less natural than a request to make an intransitive gesture (Mozaz et al., 2002).

At least partially different neural correlates have been reported between pantomime and imitation. The most common impairment after LBD involves both pantomime and imitation in both transitive and intransitive gestures, with more deficits for transitive than intransitive actions (Goodglass and Kaplan, 1983; Roy et al., 1993; Almeida et al., 2002; Stamenova et al., 2010). Conversely, selective deficits in imitation have been more frequently found after LBD for intransitive gestures. Disorders in imitation of of transitive gestures have been shown also after RBD, in both acute and chronic patients (Stamenova et al., 2010). Of course, given the role of the right hemisphere in spatial functions, these selective deficits may represent a secondary effect of deficits affecting the processing of visuo-spatial information or the translation of the spatial component of a movement into action (Roy, 1996).

Some authors have suggested that the left hemisphere may control transitive gestures while both hemispheres may be involved in the control of intransitive gestures (Haaland and Flaherty, 1984; Mozaz et al., 2002; Buxbaum et al., 2007). Nevertheless, more recent neuroimaging studies indicate that both transitive and intransitive gesture execution activates a common left hemisphere network involving frontal, parietal and temporal regions (Króliczak and Frey, 2009). This does not exclude a participation of the right hemisphere in the qualitative features of gestures. In fact, in spite of the dominance of the left hemisphere, the same studies have shown bilateral activation during preparation for pantomime performance (Króliczak and Frey, 2009) and during observation of actions (Grèzes and Decety, 2001). This involvement of both hemispheres in the control of movement may explain the rare cases of apraxia after right hemisphere damage.

This analysis of studies which have specifically addressed components of object use, pantomime and imitation has provided evidence supporting the idea that using objects represents a complex function involving the integration of multiple components. A preliminary (but probably not exhaustive) representation of these components is shown in **Figure 2**.

# ACTION RECOGNITION

The relationship between action execution and recognition has for some time been a matter of debate due to the inconsistent results from clinical and neuropsychological studies. The first clinical report indicating that patients with focal lesions could also have deficits in gesture recognition came at the same time as the first description of apraxia (Finkelnburg, 1870). A century later, in the 1980's, a few pioneering studies on patients with limb apraxia reported an association between the inability to perform gestures and to understand their meaning and left parietal lesions (Heilman et al., 1982; Rothi et al., 1985; Watson et al., 1986). Since then, many studies have reported a co-occurrence of the two disorders (Duffy et al., 1975; Gainotti and Lemmo, 1976; Ferro et al., 1983; Duffy and Watkins, 1984; Rothi et al., 1985; Varney and Damasio, 1987; Wang and Goodglass, 1992; Bell, 1994; Buxbaum et al., 2005; Pazzaglia et al., 2008b) with the result that some authors describe ideomotor apraxia in these terms: ''These patients typically have no difficulty with object recognition, are deficient in performing skilled actions with objects, and even more tellingly, are impaired in recognizing object-related actions. The impact of ideomotor apraxia clearly extends beyond laboratory tasks. IMA patients make more errors with implements while eating than subjects without apraxia, and gesture recognition and tool manipulation knowledge are strongly significant, independent predictors of sequencing errors in multistep naturalistic action'' (Buxbaum and Kalénine, 2010, p. 203).

This very close link between the perceptual and the motor components of actions finds its neuronal correlates in the discovery of bimodal neurons and in particular of the mirror system (Gallese et al., 1996; Fogassi et al., 2005) where neurons are activated during both action execution and observation (Avenanti et al., 2007; Aglioti et al., 2008; Candidi et al., 2008; Sacheli et al., 2013; Tidoni et al., 2013; Urgesi et al., 2014). Results from neuropsychology, neuroimaging and electrophysiological studies based on the effects of temporary virtual lesions induced by repetitive transcranial magnetic stimulation demonstrated that this system and in particular the inferior frontal cortex is crucial for action understanding (Pobric and Hamilton, 2006), pure visual discrimination of actions (Urgesi et al., 2007; Moro et al., 2008) and imitation (Heiser et al., 2003).

Nevertheless, other studies involving a comparatively large sample of LBD and RBD patients, reported that those with left parietal and frontal lesions were impaired in gesture execution but failed to show any relationship between action execution and comprehension (Halsband et al., 2001; Negri et al., 2007). Some evidence was found regarding gesture recognition and the time course of a pathological process (acute vs. chronic; Ferro et al., 1983) and regarding the type and complexity of the gesture (Gainotti and Lemmo, 1976; Buxbaum et al., 2005). In addition, a number of neuropsychological single-case analyses report that the ability to imitate pantomimes is not necessary in order to be able to recognize object-associated pantomimes and the ability to use objects is not necessary in order to be able to recognize objects (for a review, see Negri et al., 2007). On the basis of these inconsistent results, it has been argued that motor production processes associated with object use are involved but not necessary for successful action or object recognition (Negri et al., 2007).

The idea of a complex multi-componential network involved in action recognition is also supported by some studies on lesions. Recognition deficits have been found to be correlated with both the left inferior parietal lobule (Buxbaum et al., 2005; Tessari et al., 2007) and the opercular and triangularis portions of the left inferior frontal gyrus (Pazzaglia et al., 2008b). Perception of different types of gesture may engage partially different networks. For example, Villarreal et al. (2008) pointed out that the right pre-supplementary motor area (pre-SMA), and bilaterally the posterior superior temporal cortex, the posterior parietal cortex, occipito-temporal regions and visual cortices are involved in the recognition of different types of gesture. This suggests that selective disruptions in different parts of the circuits may lead to distinct clinical deficits. Finally, Pazzaglia et al. (2008a) report neuropsychological evidence suggesting a close link between impairments in producing actions and impairments in recognizing the sounds of actions. The authors recruited two groups of patients (and a group of non-apraxic patients as the control), with bucco-facial and limb apraxia respectively. The first group was differentially impaired in imitating actions involving the mouth, while the other group (with limb apraxia) was differentially impaired in

imitating actions performed with the hand or limb (e.g., using scissors). In a sound-picture matching task, the patients with (selective) bucco-facial apraxia failed to recognize mouth-related actions (e.g., slurping soup). In contrast, patients with (selective) limb apraxia were differentially impaired in sound-picture matching of limb-related actions. Both groups performed well in non-human related environmental sounds (e.g., an airplane flying).

Taken together, these results suggest that the perception, recognition, representation and execution of actions are heavily interactive processes in which various different features of the action (goal, meaning, kinematics, spatial organization, monitoring, etc.) co-operate. In this light, it might be simplistic to consider that a single lesional locus is responsible for all possible types of gesture recognition deficits and more in depth analyses are necessary.

Very recently a new aspect concerning the monitoring of action and the awareness of action-error has been investigated. Action and error monitoring are processes which have been well studied in the fields of psychology and neuroscience (for previous reviews, see e.g., Bush et al., 2000; Falkenstein et al., 2000; Taylor et al., 2007; Ullsperger et al., 2010). Specific but widespread brain areas are involved: the anterior insula, the anterior cingulate, the supplementary motor area, the thalamus, the brainstem, and the parietal lobe (Harsay et al., 2012). Electrophysiological and functional MRI studies have shown that our errors are processed as errors by the brain even if we are unaware of making them (Nieuwenhuis et al., 2001; Endrass et al., 2005, 2007; O'Connell et al., 2007, 2009; Pavone et al., 2009; Shalgi et al., 2009; Dhar et al., 2011; Hughes and Yeung, 2011). In particular, the anterior-cingulate region has been found to be associated with the generation of an electrophysiological pattern, Error-Related Negativity (ERN; Dehaene et al., 1994; Brázdil et al., 2002; Debener et al., 2005; Hester et al., 2005; Klein et al., 2007) that does not reveal any differences between aware and unaware errors (see also Stemmer et al., 2004). Nevertheless, awareness of errors is associated with larger bilateral activation of the prefrontal and parietal regions (Hester et al., 2005) or with left anterior insula activity (Klein et al., 2007).

When patients are able to identify and judge errors made by other people, but cannot recognize their own errors, they are considered to be affected by Anosognosia. Anosognosia can be defined as the impaired ability to recognize the presence of deficits in sensory, perceptual, motor, affective, or cognitive functioning or to appreciate their severity (Babinski, 1914; for a review, see Prigatano, 2010).

Among the various different types of anosognosia, the one which has been most investigated and is the most involved in action recognition is Anosognosia for Hemiplegia. In this condition, patients declare that they are able to execute actions with their paralyzed hand, to walk and to have an unrealistic degree of autonomy in daily life activities (Vocat et al., 2010; Moro et al., 2011). It has been suggested that this syndrome results from a combination of cognitive and sensorimotor dysfunctions, including impairments in the action monitoring system and in the detection of any mismatch between intention and outcome (Gandola et al., 2014; Preston and Newport, 2014). In fact, when forced to recognize their errors, at least some anosognosic patients improve their awareness (Fotopoulou et al., 2009; Besharati et al., 2015; Moro et al., 2015).

Although usually reported after right hemisphere damage, anosognosia for hemiplegia may also occur after left hemisphere lesion (Della Sala et al., 2009). So, the question is now whether a deficit in awareness may (or may not) exist in patients affected by apraxia and if so, how to distinguish it from a disorder in error monitoring.

In a recent study carried out by our group (Canzano et al., 2014), the first evidence for anosognosia in patients suffering from bucco-facial apraxia was found. Awareness deficits were considered to be present when patients showed that they were able to correctly evaluate the actions and errors made by other people but scored their own incorrect actions as being correct. This happened both in on-line judgement (i.e., at the moment of execution) and in off-line judgement, when patients watched themselves executing actions in a previously recorded video clip. In fact, in contrast with the ameliorative effects described in patients affected by Anosognosia for Hemiplegia (Fotopoulou et al., 2009; Besharati et al., 2015), self-observation by means of the video did not seem to impact the patient's awareness of apraxic deficits. Previous studies have demonstrated deficits in action recognition in apraxic patients (Duffy et al., 1975; Gainotti and Lemmo, 1976; Ferro et al., 1983; Duffy and Watkins, 1984; Rothi et al., 1985; Varney and Damasio, 1987; Wang and Goodglass, 1992; Bell, 1994; Buxbaum et al., 2005; Pazzaglia et al., 2008a,b; Buxbaum and Kalénine, 2010).

# REFERENCES


Nevertheless, this type of deficit was excluded in Canzano et al.'s (2014) study as a result of the patients' spared ability to judge the actions of other people. The impact of these results is of course to date limited due to the low number of patients studied. Moreover, the results are currently limited to bucco-facial apraxia and would need to be verified for limb apraxia.

However, the evidence of a co-occurrence of deficits involving gesture execution and error recognition or awareness indicates that a specific system of action monitoring is involved in action. Although the results are only preliminary, since the potential experimental and clinical implications are significant, action recognition and error monitoring require in the future more in depth investigation.

# CONCLUSION

In this review, we have taken into account a good deal of recent evidence on the subject of the interaction between objects, body parts and the environment and have thus been able to use our findings to emphasize the important role that tools and objects play in the perception, understanding and production of actions. We suggest that the use of objects is the result of a multifaceted process where multiple components are involved. These include not only knowledge about an object and sensorymotor representations, but also spatial and body representations and executive functions. In addition, a specific system devoted to the monitoring of actions is probably necessary in order to check performance. Many questions remain unresolved, such as the role of the right hemisphere in apraxia and the importance of action monitoring system in awareness of errors. These issues need further in depth investigation in order to understand their potential impact in the definition of new models of motor controls and in the devising of new rehabilitative techniques.

# AUTHOR CONTRIBUTIONS

LC and VM contributed to the concept of the article and prepared the first draft of the manuscript. All authors discussed, reviewed and contributed to the published version of the manuscript. MS contributed in preparing figures.

# ACKNOWLEDGMENTS

This work was supported by the Italian Ministry of Health (and RF-2010-2312912) to VM and by IRCSS Santa Lucia Foundation to LC.


with and without awareness: a high-density electrical mapping study. Eur. J. Neurosci. 25, 2571–2579. doi: 10.1111/j.1460-9568.2007.05477.x


V. Gallese, and J. Benjamins, (Amsterdam/Philadelphia: John Benjamins), 101–114.


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Canzano, Scandola, Gobbetto, Moretto, D'Imperio and Moro. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.