# **THE COGNITIVE AND NEURAL BASES OF HUMAN TOOL USE**

**Topic Editors François Osiurak and Cristina Massen**

#### *FRONTIERS COPYRIGHT STATEMENT*

© Copyright 2007-2015 Frontiers Media SA. All rights reserved.

All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.

The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.

Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.

Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.

As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.

All copyright, and all rights therein, are protected by national and international copyright laws.

The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.

Cover image provided by Ibbl sarl, Lausanne CH

**ISSN** 1664-8714 **ISBN** 978-2-88919-426-1 **DOI** 10.3389/978-2-88919-426-1

# *ABOUT FRONTIERS*

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

# *FRONTIERS JOURNAL SERIES*

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing.

All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

# *DEDICATION TO QUALITY*

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view.

By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

# *WHAT ARE FRONTIERS RESEARCH TOPICS?*

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area!

Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# **THE COGNITIVE AND NEURAL BASES OF HUMAN TOOL USE**

# Topic Editors:

**François Osiurak,** Université de Lyon, France and Institut Universitaire de France, France **Cristina Massen,** Leibniz Research Centre for Working Environment and Human Factors, Germany; Bonn-Rhein-Sieg University of Applied Sciences, Germany

Humans are not unique in using tools. But human tool use differs from that known to occur in nonhumans in being very frequent, spontaneous, and diversified. So a fundamental issue is, what are the cognitive and neural bases of human tool use?

This Research Topic of Frontiers provides a venue for leading researchers in the field of tool use to present original research papers, integrative reviews or theoretical articles that further our understanding of this topic.

Articles address a wide range of issues including, for instance, the nature of the underlying representations (e.g., conceptual, sensorimotor), the mechanisms supporting the incorporation of tools into body schema, the link between imitation and tool use, or the evolutionary origins of human tool use.

Articles are included from experimental psychology, neuropsychology, neuroimaging, neurophysiology, developmental psychology, ethology, comparative psychology, and ergonomics. The goal of this Research Topic of Frontiers is to provide a state-of-the-art view of the field.

# Table of Contents



# The cognitive and neural bases of human tool use

# *François Osiurak1,2\* and Cristina Massen3,4*

*<sup>1</sup> Laboratoire d'Etude des Mécanismes Cognitifs, Université Lyon 2, Lyon, France*

*<sup>2</sup> Institut Universitaire de France, Paris, France*

*<sup>3</sup> Leibniz Research Centre for Working Environment and Human Factors, Dortmund, Germany*

*<sup>4</sup> Bonn-Rhein-Sieg University of Applied Sciences, Rheinbach, Germany*

*\*Correspondence: francois.osiurak@univ-lyon2.fr*

#### *Edited and reviewed by:*

*Bernhard Hommel, Leiden University, Netherlands*

#### **Keywords: tool use, action, motor control, neural bases, cognitive bases**

It is a euphemism to say that humans use tools. Humans possess a vast repertoire of tools they use every day. In fact, as language or bipedal locomotion, tool use is a hallmark of humans. Tool use has also been often viewed as an important step during evolution (van Schaik et al., 1999) or even as a marker of the evolution of human intelligence (Wynn, 1985). So a fundamental issue is, what are the cognitive and neural bases of human tool use? The present series of papers in this special topic represents the newest additions to that research topic.

Central to that topic is the issue of the nature of the representations underlying tool use. Most of our understanding has come from the study of brain-damaged patients with tool use disorders, also called apraxia of tool use. When asked to light a candle, for example, those patients can light the candle correctly but then put it to the mouth in an attempt to smoke it. Such observations have led traditional cognitive models of apraxia to assume that tool use is supported by sensorimotor knowledge about tool manipulation (e.g., Rothi et al., 1991; Buxbaum, 2001). Consistent with this, Gainotti (2013) reviews a series of neuropsychological and neuroimaging studies indicating that perceptual, motor, and encyclopedic sources of knowledge have different weights in the construction of the different object categories (i.e., living things, tools) that are stored within the brain. This sensorymotor hypothesis assumes that manipulation knowledge stored within inferior fronto-parietal areas is critical to tool use skills. This link is also suggested by van Elk (2014), who conducted an fMRI study wherein participants had to predict the subsequent use of a presented tool. His results indicate that the left inferior parietal lobe might store hand-posture representations that can be used for planning tool-directed actions as well as for predicting other's actions.

Contrary to the traditional cognitive models of apraxia, a growing body of literature suggests that the left inferior parietal lobe might rather support technical reasoning, namely, the ability to reason about physical object properties (Goldenberg and Spatt, 2009; Osiurak et al., 2009, 2010, 2013; Goldenberg, 2013; Osiurak, 2014). Support for the technical reasoning hypothesis comes from findings demonstrating a strong association in left brain-damaged patients between the ability to use familiar tools and the ability to use novel tools to solve mechanical problems (for reviews, see Goldenberg, 2013; Osiurak, 2014). Four review articles of this special issue also provide evidence in line with the technical reasoning hypothesis. Bienkiewicz et al. (2014), Orban and Caruana (2014), and Vingerhoets (2014) emphasize that the ability to understand mechanical actions might be the specificity of the anterior portions of the inferior parietal lobe (particularly the supramarginal gyrus) while the posterior parietal cortex might be involved in the planning of the grasping and reaching components of both tool-use and non-tool-use actions. In the same vein, by reviewing studies investigating tool use disorders in left brain-damaged patients over the last 30 years, Baumard et al. (2014) suggest that the loss of mechanical knowledge might be the core deficit in left brain-damaged patients with apraxia of tool use.

Two experimental articles also address the issue of the involvement of mechanical vs. manipulation knowledge in tool use. First, Parry et al. (2014) examine both functional dynamics (i.e., the understanding of the mechanical actions involved in the task) and joint contribution profiles of participants with different levels of expertise in a primordial percussive task (i.e., production of stone flakes using the Oldowan method). Their results show that when people learn a tool use activity what they learn is the functional dynamics rather than any particular movement *per se*. Second, Müsseler et al. (2014) asked participants to use lever tools or to imagine using them in order to explore the role played in response generation by the spatial compatibility relationships between stimulus (S; at which the effect points of the lever aims at), responding hand (R) and effect point of the lever (E). They observed that the most prominent compatibility effects were for RE compatibility, corroborating the idea that even in tool use planning is influenced not only by the spatial relationship between stimulus and response, but also by the intended action effects. Similar results are reported by Rieger et al. (2014), who had participants perform circling movements with a stylus (movement) and presented distorted visual feedback of the movements on a screen (visual effect). When participants had to synchronize the visual feedback dot with a second, rotating stimulus on the screen (stimulus), strong compatibility effects emerged for the relationship between the hand movement (response) and the visual effect of this movement on the screen.

As Fagard et al. (2014) state, the development of tool use in human infants has received little interest until recently. For example, an unresolved issue is whether tool use appears through sudden insight or emerges progressively through familiarization with experience. Fagard et al. (2014) address this issue by conducting a longitudinal study on five infants from age 12 to 20 Osiurakand Massen Cognition and tool use

months. Children have to use a rake-like tool to reach toys presented out of reach. Their results indicate that it is only between 16 and 20 months that the infants suddenly start to intentionally try to bring the toy closer with the tool. For them, this sudden success at about 18 months might correspond to the coming together of a variety of capacities, such as the development of means-end behavior.

Tools are also specific because they modify our perception of the world. For instance, it is known that using a tool can alter space perception in that far stimuli become processed as if they were nearer (Maravita and Iriki, 2004; Witt et al., 2005; Osiurak et al., 2012). Likewise, body representations can be modified when using a tool so that the tool is incorporated and becomes part of our body (Iriki et al., 1996; Cardinalli et al., 2009). An interesting issue, however, is whether these modifications only occur after the real use of tools or can also appear in a tool-use imagery condition. Baccarini et al. (2014) provide a positive answer to this issue by showing that tool-use imagery is sufficient to affect the representation of the user's arm.

In line with the view of common representations for perception, imagery, and action, Kelly and Wheaton (2013) investigate the understanding of tool-use actions viewed from different perspectives and conclude that perception and understanding is facilitated when tool-use actions are viewed from an egocentric (as opposed to allocentric) perspective.

Finally, two theoretical papers also contribute to this special topic on broader issues. In line with the extended mind view, Borghi et al. (2013) suggest that words can be conceived as quasiexternal devices (or tools) that extend our cognition. For example, words function like tools because they also enlarge the bodily space of action and, as a result, modify our sense of body. Baber et al. (2014) propose the notion of distributed cognition to account that tool use is not only based on internal representations (e.g., manipulation knowledge or mechanical knowledge) but also external representations such as the location of tools within the workspace.

In sum, this special issue includes a series of articles from neuropsychology, neuroimaging, experimental psychology, developmental psychology, and ergonomics that provide very interesting findings and open new issues for future research on the topic. Let's hope that we possess the good tools to solve them!

# **ACKNOWLEDGMENTS**

This work was supported by grants from ANR (Agence Nationale pour la Recherche; Project "Dèmences et Utilisation d'Outils/Dementia and Tool Use," N◦ANR 2011 MALZ 006 03; François Osiurak; Project "Cognition et économie liée à l'outil/Cognition and tool-use economy" ECOTOOL, François Osiurak), and was performed within the framework of the LABEX CORTEX (ANR-11-LABX-0042) of Universitè de Lyon (François Osiurak), within the program "Investissements d'Avenir" (ANR-11- IDEX-0007) operated by the French National Research Agency (ANR).

#### **REFERENCES**

Baber, C., Parekh, M., and Cengiz, T. G. (2014). Tool use as distributed cognition: how tools help, hinder and define manual skill. *Front. Psychol.* 5:116. doi: 10.3389/fpsyg.2014.00116


Vingerhoets, G. (2014). Contribution of the posterior parietal cortex in reaching, grasping, and using objects and tools. *Front. Psychol.* 5:151. doi: 10.3389/fpsyg.2014.00151

Witt, J. K., Proffitt, D. R., and Epstein, W. (2005). Tool use affects perceived distance, but only when you intend to use it. *J. Exp. Psychol. Hum. Percept. Perform.* 31, 880–888. doi: 10.1037/0096-1523.31.5.880

Wynn, T. (1985). Piaget, stone tools and the evolution of human intelligence. *World Archaeol.* 17, 32–43. doi: 10.1080/00438243.1985.9979948

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 11 September 2014; accepted: 12 September 2014; published online: 06 October 2014.*

*Citation: Osiurak F and Massen C (2014) The cognitive and neural bases of human tool use. Front. Psychol. 5:1107. doi: 10.3389/fpsyg.2014.01107*

*This article was submitted to Cognition, a section of the journal Frontiers in Psychology.*

*Copyright © 2014 Osiurak and Massen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

**REVIEW ARTICLE** published: 10 October 2013 doi: 10.3389/fpsyg.2013.00727

# Controversies over the mechanisms underlying the crucial role of the left fronto-parietal areas in the representation of tools

# *Guido Gainotti 1,2 \**

*<sup>1</sup> Department of Neurosciences, Center for Neuropsychological Research, Policlinico 'A. Gemelli', Università Cattolica del Sacro Cuore, Rome, Italy <sup>2</sup> Department of Clinical and Behavioral Neurology, Istituti di Ricovero e Cura a Carattere Scientifico Fondazione Santa Lucia, Rome, Italy*

#### *Edited by:*

*Cristina Massen, Leibniz Research Centre for Working Environment and Human Factors, Germany*

#### *Reviewed by:*

*Lucilla Cardinali, Western University, Canada Randerath Jennifer, University of*

#### *Missouri, USA \*Correspondence:*

*Guido Gainotti, Department of Neurosciences, Center for Neuropsychological Research, Policlinico 'A. Gemelli', Università Cattolica del Sacro Cuore, Largo A. Gemelli 8, 00168 Rome, Italy e-mail: gainotti@ rm.unicatt.it*

Anatomo-clinical and neuroimaging data show that the left fronto-parietal areas play an important role in representing tools. As manipulation is an important source of knowledge about tools, it has been assumed that motor activity explains the link between tool knowledge and the left fronto-parietal areas. However, controversies exist over the exact mechanisms underlying this relationship. According to a strong version of the "embodied cognition theory," activation of a tool concept necessarily involves re-enactment of the corresponding kind of action. Impairment of the ability to use tools should, therefore, lead to impairment of tool knowledge. Both the "domains of knowledge hypothesis" and the "sensory-motor model of conceptual knowledge" refute the strong version of the "embodied cognition hypothesis" but acknowledge that manipulation and other action schemata play an important role in our knowledge of tools. The basic difference between these two models is that the former is based on an innate model and the latter holds that the brain's organization of categories is experience dependent. Data supporting and arguing against each of these models are briefly reviewed. In particular, the following lines of research, which argue against the innate nature of the brain's categorical organization, are discussed: (1) the observation that in patients with category-specific disorders the semantic impairment does not respect the boundaries between biological entities and artifact items; (2) data showing that experience-driven neuroplasticity in musicians is not confined to alterations of perceptual and motor maps but also leads to the establishment of higherlevel semantic representations for musical instruments; (3) results of experiments using previously unfamiliar materials showing that the history of our sensory-motor experience with an object significantly affects its neural representation.

**Keywords: tools representation, left fronto-parietal areas, embodied cognition theory, domains of knowledge hypothesis, innate theories, sensory-motor experiences, sensory-motor model of conceptual knowledge**

"fpsyg-04-00727" — 2013/10/8 — 21:07 — page 1 — #1

# **INTRODUCTION**

Tools constitute a very important and very specific category of objects, which includes the man-made artifacts that have driven the transition from the prevalence of biological, innate factors to the prevalence of cultural determinants in the development of the human mind (Vaesen, 2012; Lefebvre, 2013). For this reason, the problem of specific aspects of the cognitive and neural bases of human tool use must be viewed in the context of the wider problem of the cognitive and neural bases of categories in general. In the l980s, Warrington and colleagues laid the groundwork for a contemporary approach to this problem. In an influential series of papers, these authors showed that different brain lesions can provoke different kinds of category-specific disorders which selectively affect action names/verbs (Baxter and Warrington, 1985; McCarthy and Warrington, 1985), biological entities (Warrington and Shallice, 1984; McCarthy and Warrington, 1991), and manmade artifacts (Warrington and McCarthy, 1983, 1987). However, to explain these category-specific disorders they did not claim that various categories of knowledge are separately represented at the brain level. Rather, they proposed a general principle (i.e., the "differential weighting hypothesis"), which, on one side, acknowledges that concepts are based on the convergence of different perceptual and motor information in specific cortical areas but, on the other side, stresses the different weight that various sources of knowledge can have in the acquisition of different conceptual categories. According to this principle, category-specific semantic disorders result from disruption of the brain structures underlying perceptual, motor and language-related sources of knowledge, which have a critical role in organizing the corresponding categories. Within this context, category-specific disorders for verbs are considered due to disruption of the semantic aspects of actions and the dissociation (in objects) between living beings and artifacts is considered the consequence of the different weight of visualperceptual and functional attributes in the construction of living and artifact categories (the "sensory-functional theory"). More specifically, this model suggests that the brain networks which are damaged in category-specific semantic disorders for animal and plant-life items have a critical role in processing the high-level visual attributes which allow distinguishing members of the "biological" categories. On the other hand, the cortical areas that are disrupted in patients with a prevalent defect for tools and other artifacts are involved in processing the functional and manipulative functions on which the knowledge of artifacts is based (Gainotti, 2000, 2005; Buxbaum and Saffran, 2002; Buxbaum et al., 2007).

Warrington and colleagues' positions were at variance with cognitive models (e.g., Pylyshyn, 1973; Fodor, 1975; Caramazza et al., 1990; Patterson and Hodges, 2000) that proposed the existence of a unitary, abstract and amodal semantic system. The latter was accessed by the highest levels of the various perceptual modalities (i.e., "structural descriptions"), which include a complete perceptual specification of objects prior to their meaningful recognition. According to these cognitive models, there should be no trace of the various sensory-motor modalities beyond the level of the corresponding "structural descriptions," because the format of semantic representations should be symbolic, abstract, amodal and propositional. On the other hand, some reviews of the anatomical correlates of category-specific disorders for living beings and artifacts were consistent with Warrington and colleagues' interpretations: first, those by Saffran and Schwartz (1994) and by Gainotti et al. (1995) and, subsequently, in a more detailed manner, by Gainotti (2000) and Capitani et al. (2003). Indeed, all these reviews showed that in patients with categoryspecific semantic disorders for living beings lesions bilaterally affect the anterior parts of the temporal lobes, where the ventral stream of visual processing terminates (Ungerleider and Mishkin, 1982; Mishkin et al., 1984; Goodale et al., 1991); but in patients with impaired knowledge of tools and other artifacts, the lesions encroach upon the inferior parts of the left frontal and parietal lobes, which process action and somato-sensory data. The importance of the inferior parts of the left frontal and parietal lobes in tool representation was supported by the anatomo-clinical data of Buxbaum et al. (2000) and Buxbaum and Saffran (2002) and the functional magnetic resonance imaging (fMRI) experiments conducted by Kellenbach et al. (2003) and Boronat et al. (2005). The first authors showed that the expression "functional attributes," which should characterize artifacts, includes heterogeneous components. They, indeed, distinguished the function of an object from its manipulation and suggested that because "manipulation" is related to a sensory-motor activity it might be the component most tightly linked to the "differential weighting hypothesis." Kellenbach et al. (2003) and Boronat et al. (2005) confirmed this hypothesis by asking normal subjects in two fMRI studies to make judgments about actions and functions associated with manipulable and non-manipulable objects. Both studies showed that the left inferior frontal and parietal areas responded more strongly to actions (vs functions) and to manipulable (vs non-manipulable) objects. Therefore, these results confirmed that brain regions specialized for sensory-motor functions have a critical role in the representation of tools and other manmade objects. This, obviously, does not mean that tools are represented only in action linked left fronto-parietal cortical areas, because some authors (e.g., Lewis, 2006 and Frey, 2007) have rightly noted that this system must interact, within a "tool use network," with an other more general system, involved in conceptualizing, planning, and accessing knowledge associated with tool use. According to Frey (2007), this interaction should involve, on one hand, sensorymotor knowledge, represented within the dorsal stream of visual processing (Goodale et al., 1991) and, on the other hand, semantic knowledge, represented, at least in part, within the ventral stream. An alternative model, advanced by proponents of the "Semantic Hub" hypothesis (e.g., Patterson et al., 2007; Lambon Ralph and Patterson, 2008), assumes that this more general semantic network should be bilaterally located in the anterior portions of the temporal lobes, which are atrophic in Semantic dementia (SD). Hodges et al. (2000) have indeed, shown that in patients with SD naming, semantic knowledge and use of tools can be markedly impaired. The present review, however, will not dwell on this problem, because it will be focused on the specific issue of the mechanisms underlying the crucial role of the left fronto-parietal areas in the representation of tools and not on the general problem of the cortical network underlying tools representation. As a matter of fact, according to the "Semantic Hub" hypothesis, a bilateral atrophy of the anterior temporal lobes should provoke a semantic impairment, more or less equally affecting all kinds of concepts. No specific interaction should, therefore, be predicted between the "semantic hub" and the specific left fronto-parietal cortical representation of the tools category.

Controversies over the mechanisms underlying the crucial role of the left fronto-parietal areas in the representation of tools can be viewed, from a very general point of view, as a "tool" specific version of some of the oldest controversies in neuroscience: that between nature and nurture as well as that between localization principle, emphasizing the specificity and modularity of the brain on the one hand and holistic views, stressing unified, global functions and Gestalt phenomena on the other hand (see also Edelman, 1993 and Tononi et al., 1998). The present review highlights strength and weaknesses of three accounts which have tried to explain the relationships between tool knowledge, manipulation and left frontal and parietal areas, following the lines of thought illustrated in this introduction.

# **MODELS ADVANCED TO ACCOUNT FOR THE RELATIONSHIP BETWEEN MANIPULATION AND TOOL KNOWLEDGE**

Different theoretical models have been advanced to explain the relationship between manipulation and tool knowledge and the role played by the left ventral frontal and parietal areas. One of these interpretations is based on a strong version of the "embodied cognition hypothesis" (Barsalou, 1999; Barsalou et al., 2003; Gallese and Lakoff, 2005) and maintains that the conceptual processing of tools necessarily involves the retrieval or simulation of the movements associated with tool usage. According to these views, motor programs are run in the course of object recognition and are necessary to ground conceptual knowledge of objects. One prediction that can be made on the basis of this hypothesis is that loss or impairment of motor programs concerning the use of tools should be associated with disruption of the corresponding conceptual tool knowledge. This "strong version," however, is not the only account of the embodied cognition theory, because a weaker version, simply stressing the importance of manipulation and other action schemata in our knowledge of tools, is definitely accepted by most authors and because some representatives of the

"fpsyg-04-00727" — 2013/10/8 — 21:07 — page 2 — #2

embodied cognition theory (e.g., Barsalou, 2008), seem to separate themselves from the rigid view that has been just described.

Other theoretical models therefore acknowledge that motor programs associated with tool use have an important role in the construction of tool representation, but deny that a necessary and sufficient relationship exists between the re-enactment of these sensory-motor processes and tool knowledge.

One of these models is the "domains of knowledge" hypothesis, proposed by Caramazza (1998); Caramazza and Shelton (1998), Capitani et al. (2003) and Caramazza and Mahon (2003). This model acknowledges that conceptual knowledge is organized in categories at the brain level and holds that innate (rather than experience-dependent) factors subsume this categorical organization. It also assumes that natural selection produced specialized and therefore dissociable neural circuits for animals, "fruit and vegetables," tools and "conspecifics," because these categories have an important and specific role in human survival (Caramazza and Mahon, 2003). One development of this model, called "the distributed domain-specific hypothesis" (Mahon and Caramazza, 2011), argues that innately determined patterns of connectivity mediate the integration of information criticalfor the organization of each domain of knowledge.

A different theoretical model, which acknowledges the important (but not exclusive) role of specific motor programs in the construction of tool representation, is the "sensory-motor model of conceptual knowledge" (Saffran and Schwartz, 1994; Gainotti et al., 1995; Chao et al., 1999; Chao and Martin, 2000; Gainotti, 2000, 2005; Martin et al., 2000; Martin and Chao, 2001; Martin, 2007). This model holds that various perceptual, motor and encyclopedic sources of knowledge have different weights in the construction of different living and artifact categories and attributes their role to experience-dependent (rather than innate) factors. Data supporting and contrasting each of these models will be briefly discussed in the following sections of this review.

# **DATA SUPPORTING AND CONTRASTING THE "STRONG" VERSION OF THE "EMBODIED COGNITION THEORY"**

As a general rule, data supporting the "strong" version of the "embodied cognition theory" come from functional neuroimaging experiments, whereas data weakening or undermining this theory come from the field of brain pathology. Several authors have documented that the left inferior frontal and parietal areas are selectively activated when subjects perform tasks with tool stimuli but not with non-manipulable objects (see Grèzes and Decety, 2001; Martin and Chao, 2001 and Caramazza and Mahon, 2006 for reviews). Other authors (e.g., Hauk et al., 2004; Buccino et al., 2005; Tettamanti et al., 2005; Kemmerer et al., 2008; Pulvermuller et al., 2009;Arévalo et al., 2012) have shown that a more fine grained relationship exists between actions performed with tool stimuli and activation of specific frontal and parietal areas. Indeed, when normal subjects are presented with stimuli involving actions that refer to specific body parts (such as objects associated with the use of the hand, mouth or foot), activation prevails in the corresponding somatotopically organized cortical areas. Therefore, the activation peak for each effector corresponds with the somatotopic organization of the motor homunculus, which was first described by Penfield and Boldrey (1958).

Critics of these findings (e.g., Fischer and Zwaan, 2008; Arévalo et al., 2012) noted that: (a) functional neuroimaging evidence showing that motor programs participate in verbal and non-verbal tool knowledge does not imply that these action schemata are necessary or sufficient to support tool processing; (b) the somatotopical distribution of activations observed in the fMRI studies is not always clear and a good match for all three effectors across tasks is rarely reported (Fernandino and Iacoboni, 2010; Kemmerer and Gonzalez-Castillo, 2010).

Furthermore, lesion data, which are more relevant to clarify whether the activated areas are necessary for grounding tool conceptual knowledge or simply participate in their processing, have provided results inconsistent with the strong version of the "embodied cognition theory." Thus, research on patients with apraxia, whose performance is impaired when imitating observed actions, using objects or pantomiming their use from visual presentation, has shown that the ability to use objects may be differentially impaired relative to naming objects or knowing their function (Buxbaum et al., 2000; Buxbaum and Saffran, 2002; Rosci et al., 2003; Negri et al., 2007). Garcea et al. (2013) reported the detailed study of a patient with a large left hemisphere lesion whose object knowledge was relatively spared in spite of a severe motor (action production) defect and impaired conceptual knowledge of actions. Arévalo et al. (2012) presented left hemisphere stroke patients with pictures and words representing objects and actions typically associated with use of the hand, mouth and foot. They correlated results obtained on these tasks with data obtained from voxel-based lesion-symptom mapping analyses, but found no support for a correlation between body parts involved in the use of objects and somatotopically organized locus of damage. Taken together, the few studies that have used lesion data to test predictions deriving from the "embodied cognition theory" have provided data inconsistent with this theory. In keeping with these conclusions, based on the comparison between results of functional neuroimaging experiments and of anatomo-clinical studies, are also more general considerations. If we take into account, for instance, the automatic perception of object affordances (namely the fact that the action representations of an object can be automatically activated from its view) we must acknowledged that this automaticity is not consistent with the attainment of tool identity from action representations (Creem-Regehr and Lee, 2005). Contrasting opinions exist, however, on this subject. Some authors (e.g., Bub et al., 2008) suggest that that activation of motor representations depends on a form of attentional orienting to the object. Other authors (e.g., Randerath et al., 2013) propose that there are limitations to the automatic perception of affordances, because factors such as tool use context, and type of task play an influential role. Finally, it must be noted that the "strong version" of the embodied cognition theory entails a none or all mechanism, which render the model rather implausible.

### **DATA SUPPORTING "INNATE" AND "EXPERIENCE-DEPENDENT" INTERPRETATIONS OF THE RELATIONSHIPS BETWEEN MANIPULATION AND TOOL KNOWLEDGE**

"fpsyg-04-00727" — 2013/10/8 — 21:07 — page 3 — #3

Both the "domains of knowledge hypothesis" and the "sensorymotor model of conceptual knowledge" refute the strong version of the "embodied cognition hypothesis" but acknowledge that

manipulation and other action schemata have an important role in our knowledge of tools. The basic difference between these two models is that the "domains of knowledge hypothesis" is based on an innatist model and the "sensory-motor model of conceptual knowledge" maintains that the brain's organization of categories is experience-dependent.

In fact, the "domains of knowledge" hypothesis' holds that the brain is really organized by categories and that this organization results from innately determined patterns of connectivity which mediate the integration of information critical for each category. On the contrary, the "sensory-motor model of conceptual knowledge" holds that the categorical organization of the brain is only apparent because each category results from the convergence of different sources of knowledge whose organization is not innate but experience-dependent. According to the first model, which was labeled "the distributed domain-specific hypothesis" by Mahon and Caramazza (2009), a domain-specific neural system is a network of brain regions in which each region processes a different type of sensory, motor, affective or conceptual information about the same category of objects. Furthermore, the computations that must be performed on items in the same category are sufficiently specific to merit a specialized process. For instance, there is a strong need to integrate motor-relevant information with visual information for tools and other artifacts; this need is less strong for animals and faces. In a similar manner, there is a strong need to integrate affective information, biological motion processing and visual form information for animals and conspecifics; this need is less strong for tools and other artifacts. Thus, supporters of the "distributed domain-specific hypothesis" propose that specialization for faces in the lateral fusiform area of the ventral visual stream occurs because this region of the brain is connected with the amygdale and the superior temporal sulcus, which are importantfor the extraction of socially relevant information. By contrast, specialization for tools and manipulable objects is driven by connectivity between the inferior frontal and parietal cortex, which subserve object manipulation and regions of the medial fusiform gyrus, which are involved in tools visual processing. Data supporting the innate nature of these patterns of connectivity come from work indicating that congenitally blind subjects show activation for words (presented in Braille) in the same regions of the ventral stream activated by visually presented words in sighted individuals (Buchel et al., 1998). Furthermore, Mahon et al. (2009) showed that the same medial-to-lateral bias in category preferences for artifacts vs animals which is present in the ventral surface of the temporo-occipital cortex in sighted individuals is also present in congenitally blind subjects. Mahon and Caramazza (2011) suggested that if visual experience is unnecessary for the emergence of category-specificity in the ventral stream, innate connectivity between regions of the ventral stream and other regions of the brain could drive category-specificity.

Nevertheless, some data argue against the "domains of knowledge hypothesis" and the innate nature of the brain's categorical organization. Among these, we can include the following clinical and experimental data:

(a) The observation that in patients with category-specific disorders the semantic impairment does not respect the boundaries between living/biological entities and non-living/artifact items.

"fpsyg-04-00727" — 2013/10/8 — 21:07 — page 4 — #4

In particular, Warrington and Shallice (1984); Warrington and McCarthy (1987), Basso et al. (1988); Silveri and Gainotti (1988); Damasio (1990), Hillis and Caramazza (1991); Sacchett and Humphreys (1992), Breedin et al. (1994); Farah et al. (1996), Forde et al. (1997), Dixon et al. (2000) and Masullo et al. (2012) showed that the representation of "body parts" tends to be disrupted in association with that of artifacts, and the representation of "musical instruments" tends to be disrupted in association with that of biological entities. For two reasons, this observation is consistent with the "sensory-functional theory" and inconsistent with the "domains of knowledge hypothesis." On one side, we observe here a systematic breakdown across categories. On the other side, musical instruments (which are not recognized by their function but by their different shape and acoustic features) are more similar to "living" items than to other artifacts from the viewpoint of their sources of knowledge, whereas body parts are identified on the basis of the somato-sensory and action-related information, which also has a critical role in the recognition of tools and other artifacts (Gainotti et al., 2009).

(b) Still within the category of musical instruments (but shifting from the contrast between the disruption of real categories and the disruption of representations based on the same sources of knowledge to the"innate vs experience-dependent"opposition), interesting data supporting the experience-dependent interpretation were recently reported by Hoenig et al. (2011). These authors, starting from the premise that professional musicians constitute a very good model for understanding experience-dependent plasticity in the human brain, wondered whether this neuroplasticity might extend beyond basic perceptual and motor functions and shape the semantic representation of musical instruments. Using fMRI, they showed that in musicians (but not in musical laypersons) conceptual processing of visually presented musical instruments activates the auditory association cortex encompassing the right posterior superior temporal gyrus, which is also recruited in the auditory perception of real sounds. Therefore, experience-driven neuroplasticity in musicians is not confined to alterations of perceptual and motor maps but also leads to the establishment of higher-level semantic representations for musical instruments.

(c) The role of prior motor experience in the cortical representation of objects was also addressed by Creem-Regehr et al. (2007); Kiefer et al. (2007), Weisberg et al. (2007) and Bellebaum et al. (2013). As the history of previous sensory-motor experience with familiar objects cannot be controlled, these authors tried to use previously unfamiliar material and submitted their subjects to different types of extensive training with these objects. In Kiefer et al.'s (2007) study, the plasticity of conceptual representations was assessed by training subjects with novel objects under different training conditions. In one class of stimuli, object categorization was based on a detail feature of the novel objects, affording a particular action. During training, participants were asked either to make an action pantomime toward the detail feature or simply to pay attention to it, by pointing to it with their index finger. Only in the pantomime group an early activation was found in the frontal areas, whereas in the pointing training group this effect was absent. These results show that action information contributes to conceptual processing, depending on the specific learning experience, and suggest that conceptual representations are established by the learning-based formation of cell assemblies in different cortical areas.

Creem-Regehr et al. (2007) investigated, by means of fMRI, the influence of action knowledge associated with viewing, grasping, and using novel graspable objects. Participants were trained on complex actions associated with novel objects ("tools") and had experience manipulating other visually similar novel objects ("shapes"). The largest differences between "tools" and "shapes" were found in using, in which greater effect sizes were observed for tools versus shapes in the left inferior parietal lobule (IPL), the pre-supplementary motor cortex (pre-SMA) and, marginally, in the left ventral premotor cortex (VPM). These results suggest that representations of tools are constructed on the basis of complex action schemata, which recruit processes related to graspability, action plans and use of objects.

Weisberg et al. (2007) usedfMRI in subjects who should visually match pictures of novel objects before and after extensive training dealing with the use of these objects to perform specific tool-like tasks. After training, neural activity emerged in regions associated with the motion (left middle temporal gyrus) and manipulation (left intraparietal sulcus and premotor cortex) of common tools, showing that experience of direct interaction with previously unfamiliar objects led to new neural object representations in the same cortical areas underlying the neural representation of tools. Finally, Bellebaum et al. (2013) studied with fMRI the impact of different types of object-related sensorimotor experiences on the neural representations of novel objects, contrasting the manipulation training (MTO) with the visual training (VTO) and the absence of training (NTO). The post-training activity in the left inferior/middle frontal gyrus and the left posterior IPL was higher for MTO than for VTO and NTO suggesting that manipulation experience specifically yields higher activities in regions of the fronto-parietal cortex.

(d) The final point, which argues against the hypothesis that an innate connectivity pattern may subsume the categorical organization of the human brain, is that handedness, rather than hemispheric language lateralization, seems to account for the special role played by the left ventral frontal and parietal areas in tool knowledge. In the introductory part of this review, I mentioned that in category-specific semantic disordersfor living beings lesions affect the anterior parts of the temporal lobes bilaterally (where highly processed visual data are integrated with other sensory modalities). Differently, in patients with impaired knowledge of tools and other artifacts, lesions encroach upon the inferior parts of the left frontal and parietal lobes, which process action and somatosensory data. A theory stressing the innate aspects of brain organization and a theory stressing the importance of experience-dependent factors should make opposite predictions about the relationships among lateralization of the tool-related fronto-parietal activation, language lateralization and handedness. Innate theories should predict that strongly left-handed subjects will continue to show left fronto-parietal activation because of the same genetic factors which subsume the left hemisphere specialization for language (Annett, 2000; Corballis, 2009). Experiential theories should predict right fronto-parietal activation resulting from the execution of movements with the left side of the body. Two recent studies were conducted by Lewis et al. (2006) and Willems et al. (2010) in strong right- and left-handers to evaluate the role played by asymmetries in motor experience and the left dominance for language on the lateralization of tool representation. In the first study, Lewis et al. (2006) compared the pattern of cortical activation evoked by hand-manipulated tool sounds and animal vocalizations and found that tool sounds preferentially evoke activity in high-level motor-related cortical regions of the hemisphere opposite to the dominant hand. In the second study, Willems et al. (2010) used fMRI to compare premotor activity associated with understanding action verbs (strictly related to tool use) and showed that right-handers preferentially activated the left premotor cortex and left-handers, the right premotor areas. Therefore, in both studies and in agreement with the positions defended by the theory stressing the importance of experience-dependent factors, the laterality of cortical regions activated by high-level action and tool use was related to the side of the body involved in actions and not to left-hemisphere dominance for language. Note, however, that in a paper recently published by Goldenberg (2013) on "apraxia in left-handers" there were three aphasic patients with pervasive apraxia caused by left-sided lesions, who showed a dissociation of apraxia from handedness. Conversely there were also three patients with pervasive apraxia caused by right brain lesions without aphasia, who showed a dissociation of apraxia from aphasia. The implications of these data for the problem at issue requires clarifications.

# **CONCLUSION**

"fpsyg-04-00727" — 2013/10/8 — 21:07 — page 5 — #5

Taken together, results of the present review suggest that neither a strong version of the "embodied cognition theory" nor an "innate" categorical organization of conceptual knowledge can account for: (a) the important role of manipulation and other action schemata in our knowledge of tools and (b) the links between tool knowledge and the inferior fronto-parietal areas. On the other hand, the "sensorimotor model of semantic knowledge" can explain data obtained in brain-damaged patients (showing that tool knowledge can be spared after disruption of the motor processes engaged in tool use) and data stressing the role of prior motor experience in the construction of the cortical representation of objects. Furthermore, the sensorimotor model of semantic knowledge is supported by the results of studies that assessed the weight of various sources of knowledge in the construction of biological and artifact categories in normal subjects. These studies used either feature verification tasks (e.g., Vigliocco et al., 2004; McRae et al., 2005) or Likert scales (e.g., Gainotti et al.,2009,2012; Hoffman and Lambon Ralph, 2013) to evaluate the weight that different"sources of knowledge" could have in the construction of different semantic categories. Regardless of the methodology used in these investigations, results showed that visual information is considered the dominant source of knowledge across categories, but the second most important sources of information are different in biological and artifact categories. In fact, they consist of other perceptual data for the living categories and actions and somato-sensory data for tools and the other artifact categories. Therefore, vision, actions and somato-sensory information have a major role in the representation of tools and other artifacts, whereas visual and other perceptual input have a dominant role in the representation of

animals and other living things. The fact that normal subjects have considered both vision and action-related information as important sources of knowledge about tools and other artifacts supports: (a) the crucial role of the left fronto-parietal areas, subsuming transitive actions, in the representation of tools; (b) the thesis of authors (e.g., Creem-Regehr and Lee, 2005; Buxbaum et al., 2007; Frey, 2007) who have claimed that both the dorsal and the ventral stream must play a role in the representation of tools.

It would certainly be desirable, at the end of this survey, to predict (if we assume that a "strong version" of the embodied cognition hypothesis is untenable), which are the future directions of research that could more strongly support the "domains of knowledge" or the "sensory-motor model of

# **REFERENCES**


*Cogn. Brain Res.* 23, 361–373. doi: 10.1016/j.cogbrainres.2004.11.001


conceptual knowledge" hypothesis. However, a definite choice between "innatistic" and "experience dependent" mechanisms can hardly be made, because both mechanisms certainly intervene in the cognitive development. Coming back to the part of this survey, in which I claimed that tool-related research cannot be considered apart from investigations concerning in general the brain categorical organization, I think that, in any case, it should be important to more clearly assess: (a) if in category-specific disorders the semantic impairment respects or not the boundaries between biological entities and artifacts; (b) what is the role of the patient's familiarity with disrupted and spared categories, to see if this variable can strongly influences the observed patterns of categorical semantic impairment.

category-specific deficits? A critical review of the clinical evidence. *Cogn. Neuropsychol.* 20, 213–261. doi: 10.1080/02643290244000266


"fpsyg-04-00727" — 2013/10/8 — 21:07 — page 6 — #6


P. (1991). A neurological dissociation between perceiving objects and grasping them. *Nature* 349, 154–156. doi: 10.1038/349154a0


in human sensory-motor areas. *J. Cogn. Neurosci.* 19, 525–542. doi: 10.1162/jocn.2007.19.3.525


"fpsyg-04-00727" — 2013/10/8 — 21:07 — page 7 — #7

agnosia. *Brain Cogn.* 80, 155–159. doi: 10.1016/j.bandc.2012.06.002


role of tool use context and task complexity. *Cortex* 49, 1679–1693. doi: 10.1016/j.cortex.2012.08.016


M., Scifo, P., et al. (2005). Listening to action-related sentences activates fronto-parietal motor circuits. *J. Cogn. Neurosci.* 17, 273–281. doi: 10.1162/089892905 3124965


**Conflict of Interest Statement:** The author declares that the research was

"fpsyg-04-00727" — 2013/10/8 — 21:07 — page 8 — #8

conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 01 August 2013; accepted: 20 September 2013; published online: 10 October 2013.*

*Citation: Gainotti G (2013) Controversies over the mechanisms underlying the crucial role of the left fronto-parietal areas in the representation of tools. Front. Psychol. 4:727. doi: 10.3389/fpsyg.2013.00727*

*This article was submitted to Cognition, a section of the journal Frontiers in Psychology.*

*Copyright © 2013 Gainotti. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# The left inferior parietal lobe represents stored hand-postures for object use and action prediction

# *Michiel van Elk\**

*Department of Psychology, University of Amsterdam, Amsterdam, Netherlands*

#### *Edited by:*

*François Osiurak, Université de Lyon, France*

#### *Reviewed by:*

*Fausto Caruana, Italian Institute of Technology, Italy François Osiurak, Université de Lyon, France*

#### *\*Correspondence:*

*Michiel van Elk, Department of Psychology, University of Amsterdam, Weesperplein 4, 1018 XA Amsterdam, Netherlands e-mail: m.vanelk@uva.nl*

Action semantics enables us to plan actions with objects and to predict others' object-directed actions as well. Previous studies have suggested that action semantics are represented in a fronto-parietal action network that has also been implicated to play a role in action observation. In the present fMRI study it was investigated how activity within this network changes as a function of the predictability of an action involving multiple objects and requiring the use of action semantics. Participants performed an action prediction task in which they were required to anticipate the use of a centrally presented object that could be moved to an associated target object (e.g., hammer—nail). The availability of actor information (i.e., presenting a hand grasping the central object) and the number of possible target objects (i.e., 0, 1, or 2 target objects) were independently manipulated, resulting in different levels of predictability. It was found that making an action prediction based on actor information resulted in an increased activation in the extrastriate body area (EBA) and the fronto-parietal action observation network (AON). Predicting actions involving a target object resulted in increased activation in the bilateral IPL and frontal motor areas. Within the AON, activity in the left inferior parietal lobe (IPL) and the left premotor cortex (PMC) increased as a function of the level of action predictability. Together these findings suggest that the left IPL represents stored hand-postures that can be used for planning object-directed actions and for predicting other's actions as well.

**Keywords: fMRI, objects, action prediction, action semantics, inferior parietal lobe**

# **INTRODUCTION**

Imagine yourself sitting in a restaurant at a romantic dinner with your partner. If your partner would lift a bottle of wine you would likely infer that he wants to pour you a glass of wine. Upon offering your glass, you expect him to pour wine and to subsequently put the bottle back in the wine cooler. You would be quite surprised if your partner would pour wine in the wine cooler instead. As this example illustrates, many of our everyday actions rely on the use of action semantic knowledge about objects, specifying what to do with and how to use objects (van Elk et al., 2013). Action semantics can be used to guide our own actions involving objects (e.g., we brush our teeth, pour coffee or write a letter) and to predict other's object-directed actions as well (e.g., seeing some grasping a wine bottle allows one to infer the subsequent goal of the action).

Neuropsychological studies have provided important insight in the neural organization of action semantics. For instance, studies with left-brain damaged patients have indicated that these patients exhibit strong impairments in the ability to use objects (often specifically following damage to the left inferior parietal lobe (IPL); cf. Buxbaum, 2001; Buxbaum and Saffran, 2002; Goldenberg, 2009; Osiurak et al., 2011) and that they may no longer be able to apply the correct hand posture to an object (e.g., inserting the wrong fingers in a pair of scissors; Sirigu et al., 1995). Based on these findings it has been suggested that the IPL stores the motor programs required for successful hand-object interaction and that ideomotor apraxia is characterized by an impairment in accessing manipulation knowledge about objects (i.e., knowing how to apply a correct hand posture for interacting with objects; cf. Heilman et al., 1982).

Behavioral studies and neuroimaging studies have underlined the importance of motor-related knowledge for successful object interaction. Several behavioral studies have shown for instance that the mere observation of objects automatically results in the activation of the motor programs associated with using these objects (Klatzky et al., 1989; Ellis and Tucker, 2000; Tucker and Ellis, 2001; Bub et al., 2008). For instance, participants were faster to respond to object pictures when using a grip that was congruent with the size of the object that was presented (e.g., faster responding to the presentation of car-keys when making a precision grip; Ellis and Tucker, 2000). Neuroimaging studies have shown that the observation of manipulable objects and the retrieval of manipulation knowledge about objects is associated with activation in motor-related regions, such as the premotor cortex (PMC), the supplementary motor area (SMA) and the inferior parietal lobe (IPL; Chao and Martin, 2000; Okada et al., 2000; Grezes and Decety, 2002; Creem-Regehr and Lee, 2005; Noppeney et al., 2005). In single-cell studies a strong specificity for hand-shape in relation to the manipulation of objects has been found in the monkey homolog of the IPL (Sakata et al., 1995; Murata et al., 2000). Furthermore, neuroimaging studies in humans have also shown that the IPL is selectively involved in the visuomotor transformations required for successful grasping and interacting with an object (Culham et al., 2003; Grol et al., 2007; Cohen et al., 2009). Accordingly it has been proposed that the activation in parietal areas in response to object observation reflects the automatic coding of hand-object interactions and that action semantics are stored in motor-related brain regions (Beauchamp and Martin, 2007; Barsalou, 2008; van Elk et al., 2013).

As the example with the wine bottle illustrates, in addition to using semantic knowledge for guiding our own actions, we use action semantics to predict others' actions as well (van Elk et al., 2008; Springer and Prinz, 2010). The last decade, many studies have shown that the observation of others' actions recruits the action observation network (AON), consisting of the PMC, the SPL and IPL, the inferior frontal gyrus (IFG), and the extrastriate body area (EBA) (see: Caspers et al., 2010 EBA; for a meta-analysis of studies on action observation). Activity in the AON increases as a function of the familiarity of the action (Calvo-Merino et al., 2005; Vingerhoets, 2008; Cross et al., 2009), indicating an important role for action experience in shaping the associations between executed and observed movements (Heyes, 2010). It has also been shown that the AON is more strongly activated for the observation of object-directed actions compared to intransitive actions (Buccino et al., 2001; Koski et al., 2002; Aziz-Zadeh et al., 2006; Caspers et al., 2010). For instance, single cell studies in monkeys have shown that neurons in the ventral PMC selectively responded to object-directed actions, even when the final phase of the action was occluded (Umilta et al., 2001). Furthermore, it has been found that neurons in the parietal lobe and PMC responded differentially depending on the final outcome of the action (Fogassi et al., 2005; Umilta et al., 2008). In an fMRI study in humans it has been found that activation in the AON in response to the observation of grasping actions varies as a function of the to-be-performed goal (Iacoboni et al., 2005). Based on these findings it has been suggested that within the AON actions are represented primarily in terms of the goal or outcome of the observed action (Iacoboni et al., 2005; van Elk et al., 2008; Newman-Norlund et al., 2010). Furthermore, it has been proposed that the AON may support action prediction by enabling observers to infer the goal of an observed action through the recruitment of similar mechanisms as involved in planning an action oneself (Blakemore and Decety, 2001; Wilson and Knoblich, 2005; Kilner et al., 2007). According to the "predictive coding account of action observation," information about observed actions is used to minimize the prediction error at different levels in the action hierarchy, which allows one to infer the most likely goal or outcome of the action (Kilner et al., 2007). In support of this account, it has been found for instance that motorrelated areas are activated during action prediction tasks (Kilner et al., 2004; Aglioti et al., 2008) and that TMS-induced disruption of the AON impairs action prediction (Stadler et al., 2012; Avenanti et al., 2013).

However, most studies on action prediction have focused on relatively simple actions and on the role of low-level kinematic cues in action understanding and prediction (Schubotz, 2007; Stadler et al., 2012; Avenanti et al., 2013; Zimmermann et al., 2013). In contrast, in daily life we often rely on semantic knowledge about objects in order to fine-tune our predictions about others' action. Behavioral studies have shown that action prediction is modulated as a function of both contextual, kinematic and object information (Stapel et al., 2012) and that semantic information can affect action prediction (Springer and Prinz, 2010). Action semantics may facilitate action prediction, by enabling the observer to use prior information to constrain the number of possible inferences about an observed action (e.g., an object is associated with only a limited set of possible goals) and by disambiguating the observed kinematics within the context of the objects involved (e.g., grasping a wine bottle when two glasses are empty entails a different prediction than when the two glasses are full). Whereas previous studies on action observation have compared transitive to intransitive actions (Buccino et al., 2001; Koski et al., 2002; Aziz-Zadeh et al., 2006; Caspers et al., 2010), it is not known whereas activation in the AON is modulated as a function of the predictability based on action semantic information. For instance, observing someone grasping a full bottle of wine is more predictable in a context in which both glasses are empty, but less predictable in a context where both glasses are full (cf. Newman-Norlund et al., 2013). Accordingly, the aim of the present fMRI study was to investigate how activation in the AON is modulated as a function of the predictability of an action involving multiple objects that require the use of action semantics.

In this fMRI study an action prediction task was used in which participants were required to predict the subsequent use of a centrally presented object, that was presented in association with two flanker objects (see **Figure 1** for example stimuli). By manipulating the number of possible target objects the predictability of the action could be manipulated. For instance, a wine bottle presented with two unrelated distractor objects (e.g., two other bottles) resulted in an action of low predictability, whereas a wine bottle presented with a target object (e.g., a wine glass) resulted in an action of high predictability. In addition, the availability of actor information was manipulated, by including trials with or without a hand grasping the central object. In this way, it could be investigated whether using semantics for predicting imagined and observed actions recruit comparable neural mechanisms. Neuroimaging studies have suggested that comparable brain areas (i.e., the IPL and the PMC) are involved in the retrieval of action semantics (van Elk et al., 2013), in motor imagery (Zacks, 2008) and in action observation (Caspers et al., 2010). However, a direct comparison between the brain areas involved in using action semantics for motor imagery and for action prediction has not been made. In line with the "predictive coding account of action observation" (Kilner et al., 2007), it was expected that the use of semantics for predicting observed actions relies on similar neural mechanisms as involved in using semantics to guide our own (imagined) actions as well. Accordingly, in the present study a direct comparison was made between trials in which participants were asked to imagine planning an object-directed action and trials in which participants were asked to predict observed object-directed actions.

Based on previous neuropsychological and neuroimaging studies, the following predictions were made. First, it was expected that the observation of an action (i.e., comparing trials with and without an action cue) should be associated with increased activation in the AON, consisting of the dorsal

or 2 Target Objects/0 Distractor Objects (upper row of figure). Pictures were presented without an action cue (left side of figure) or with an

incongruent with respect to the target object in the picture (see right side of figure).

premotor cortex (dPMC), SPL and IPL, the IFG, and the EBA (see: Caspers et al., 2010 for meta-analysis of studies on action observation). Second, it was expected that comparing trials in which a target object was presented compared to trials in which no target object was presented, would require the retrieval of stored hand-object postures, which should become apparent in a stronger activation of the left IPL (Caspers et al., 2006). Third, by using a conjunction analysis it could be directly investigated if there is an overlap between the brain areas involved in action observation and in the retrieval of action semantics for imagined actions (Kilner et al., 2007). It was expected that the use of action semantics for motor imagery and action observation should converge in two core regions of the fronto-parietal motor network, notably the IPL and the PMC (Zacks, 2008; van Elk et al., 2013).

# **MATERIALS AND METHODS**

**PARTICIPANTS**

In total 20 people participated in the fMRI study (12 men, mean age = 23.0 years, *SD* = 2*.*4 years) after giving informed written consent according to institutional guidelines (Ethics Committee, University of Amsterdam, The Netherlands) for payment of 10 C/h. All participants were right-handed as assessed through subject self-report and had normal or corrected-tonormal vision. One participant made more than 50% errors on trials in which only 1 target object was presented and this subject was excluded from all analyses.

# **ACTION PREDICTION TASK**

During the experiment participants observed pictures representing three objects positioned on a table next to each other (see **Figure 1**). Participants were instructed to predict whether the *central object* would be moved to the left, to the right or to neither side, by pressing one of three buttons on a button box with their right hand (left, middle, or right button). Participants were instructed that predictions should be based on the type of objects that were presented in the picture and/or the action information presented by the actor grasping the central object.

As stimuli I used standardized pictures (750 × 500 pixels) representing a central object with respectively, 0, 1, or 2 target objects and 2, 1, or 0 distractor objects at either side (see **Figure 1**). A *target object* was defined as an object that would yield a meaningful action sequence in combination with the central object. A *distractor object* was defined as an object that was semantically related to the central object but that could not be used in a meaningful action sequence with the central object. For instance, a wine bottle can be used in combination with a wine glass to pour wine or in combination with a wine cooler to cool wine. However, a wine bottle cannot be combined in a meaningful action sequence with a beer bottle or a sports drinking bottle.

In half of all pictures an action cue was presented, representing a hand grasping the upper or lower side of the central object. Each grasp type (upper vs. lower side) was associated with using a different target object. For instance, grasping the wine bottle at the lower side affords pouring wine in a wineglass, whereas grasping the wine bottle at the upper side affords putting the wine bottle in the wine cooler. Thus, I created pictures according to a 3 (# of Target Objects: 0, 1, 2) × 2 (No Action Cue vs. Action Cue) design. I selected 10 different central objects that were associated with two different target objects and that were paired with two different distractor objects (see **Table 1**). Different pictures were created for all possible combinations of the location of the target objects (left vs. right), the location of the distractor objects (left



vs. right), and the action cue (No Cue, Cue-Up vs. Cue-Down). In the "Action Cue—1 Target Object" condition the grip type represented by the action cue could be congruent or incongruent with respect to the target object presented in the picture (see right side of **Figure 1**). For instance, grasping a bottle opener at the upper side would be congruent in combination with a wine bottle (i.e., affording the use of this object), but would be incongruent in combination with a beer bottle (i.e., grasping the tool in this way does not allow opening the beer bottle). In the analyses described below, trials were collapsed across both congruent and incongruent conditions, because at a neural level, comparison of incongruent with congruent trials did not yield significant differences using FWE correction for multiple comparisons.

Participants engaged in 60 practice trials outside the scanner and 8 practice trials in the fMRI environment. During the fMRI experiment, participants conducted two sessions of 160 trials that were separated by a short break (*<*2 min). Participants stayed inside the scanner during the break. Within each session trials were divided in four blocks of 40 trials, with rest breaks between blocks. Trials were presented in a pseudo-randomized order, such that each session contained the same number of trials for each condition.

Each trial began with the presentation of a fixation cross, followed by the presentation of a picture representing the different objects to which the participant responded by pressing one of three buttons on the response box. The picture was always presented for a duration of 3 s and participants were instructed to respond within this interval, before the next trial would be presented. Next, a fixation cross appeared and the next trial was initiated after a jittered interval of 2.5–4.5 s. During the scanning sessions eye movements were recorded using an MR-compatible eye tracker (Eyelink 1000; SR Research Ltd., Ontario, Canada). Due to technical issues, we did not collect eye movement data from two participants during the fMRI task.

#### **EBA LOCALIZER TASK**

A functional localizer was used to localize the EBA, using a standardized stimulus set consisting of 20 pictures of human bodies and 20 pictures of chairs (http://pages.bangor.ac.uk/∼ pss811/page7/page7.html). These stimuli were presented using a blocked design with a presentation of 300 ms per stimulus followed by a 450 ms blank screen and with 20 stimuli per block.

### **ANALYSIS OF BEHAVIORAL DATA**

Analysis of the behavioral data focused on the error rates and reaction times (RTs) obtained during the action prediction task in the fMRI experiment for the different experimental categories. For the analysis of the RTs incorrect trials and trials in which the RTs exceeded the subject's mean by more than two standard deviations were excluded from analysis. Behavioral data was analyzed using a repeated measures ANOVA with the factors Action Cue (No Cue vs. Cue) and # of Target Objects (0, 1, or 2 Target Objects). Effects that exceeded *F*-values corresponding to *p*-values *<* 0.05 were considered significant.

### **EYE MOVEMENT DATA**

The eye movement data were analyzed using Matlab and analysis focused on the time window from stimulus onset until the subject made a response. For each subject and each experimental condition (i.e., No Cue vs. Cue; 0, 1, or 2 Target Objects) the number of saccades, the amplitude of saccadic eye movements, the onset of the first saccade following stimulus onset, the number of fixations and the number of blinks were calculated. The averaged eye movement data was analyzed by using a repeated measures ANOVA with the factors Action Cue (No Cue vs. Cue) and # of Targets (0, 1, or 2 Targets). Effects that exceeded *F*-values corresponding to *p*-values *<* 0.05 were considered significant.

### **IMAGE DATA ACQUISITION**

The fMRI data were acquired on a 3T scanner (Achieva, Philips) in a single scanning session consisting of two runs. During each run 540 T2-weighted echoplanar images were acquired (time repetition [*TR*]/time echo [*TE*] = 2000/28 ms; voxel size 3 × 3 × 3 mm). Anatomical images were acquired with a T1 weighted sagittal scan of the whole brain before the functional runs (*TR/TE* = 8*.*2*/*3*.*8 ms, voxel size 1 × 1 × 1 mm). The head of each participant was carefully constrained using foam padding and subjects were instructed to move as little as possible.

#### **IMAGING DATA ANALYSIS**

Statistical analyses were conducted using SPM8 software (Wellcome Department of Cognitive Neurology, London, UK). Preprocessing steps involved spatial realignment (Friston et al., 1995), correction for motion and differences in slice acquisition time, spatial normalization and smoothing with an isotropic Gaussian kernel of 8 mm full-width at half-maximum. Anatomical normalization to MNI space was performed by coregistration of the functional images with the anatomical T1 scan (Ashburner and Friston, 1999).

First-level fMRI analyses were performed for each individual subject in the context of the General Linear Model (Friston et al., 1996). The fMRI time series for both sessions was fitted in one statistical model, with six regressors of interest and their temporal derivatives according to the six possible combinations of Action Cue (No Cue vs. Cue) and # of Target Objects (0, 1, or 2). Each trial was modeled by constructing a square-wave function with the duration that corresponded to the reaction time of that trial. Regressors of no interest included: incorrect and missed responses and the presentation of a fixation cross. Residual head movementrelated effects were modeled by including Volterra expansions of the six rigid- body motion parameters (Lund et al., 2005). To control for potential confounding effects of eye movements, hrfconvolved metrics of eye movements (i.e., number of saccades, length of saccades, and number of eye blinks) were included as additional regressors of no interest.

After estimation, beta values were taken to the second level for random effects analysis (Friston et al., 1999). Contrasts were thresholded at *p <* 0*.*05 using familywise error (FWE) correction for multiple comparisons at the voxel level. An anatomical representation of significant clusters was obtained by superimposing the structural parametric maps on a standard MNI template. Brodmann areas (BAs) were assigned based on the SPM anatomy toolbox (Eickhoff et al., 2005). Analysis focused on the main effects of Action Cue, # of Target Objects and the overlap between Action Cue and # of Target Objects.

#### **RESULTS**

#### **BEHAVIORAL RESULTS**

**Table 2** presents the RTs and the error rates for the different experimental conditions. A speed-accuracy trade-off was observed, reflected in relatively more errors and faster RTs for the "Action Cue—2 Target Objects" condition compared to the "Action Cue—1 Target Object condition." To control for the speed-accuracy trade-off, for the analysis of the behavioral data, the inverse efficiency was calculated by dividing the RTs by the proportion of correct responses (Townsend and Ashby, 1978).

As can be seen in **Figure 2**, response times were faster for trials in which no action cue was presented [1318 ± 52 ms; (mean ± SE)] compared to trials in which an action cue was present [1382 <sup>±</sup> 47 ms], *<sup>F</sup>*(1*,* 18) <sup>=</sup> <sup>31</sup>*.*5, *<sup>p</sup> <sup>&</sup>lt;* <sup>0</sup>*.*001, <sup>η</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>*.*64. RTs increased with an increasing number of target objects, [0 Target Objects: 1239 ± 52 ms; 1 Target Object: 1356 ± 45 ms; 2 Target Objects: 1456 <sup>±</sup> 53 ms], *<sup>F</sup>*(2*,* 36) <sup>=</sup> <sup>91</sup>*.*6, *<sup>p</sup> <sup>&</sup>lt;* <sup>0</sup>*.*001, <sup>η</sup><sup>2</sup> <sup>=</sup>

**Table 2 | Error rates and reaction times according to the different experimental conditions.**


*Standard errors are between brackets.*

0*.*84. The interaction between Action Cue and # of Target Objects was not significant, *F*(2*,* 36) = 2*.*1, p = 0.14. There was no significant difference between trials in which the action cue was congruent (1388 ± 47 ms) or incongruent (1425 ± 41 ms) with respect to the target object and in all subsequent analyses, data was collapsed over both incongruent and congruent stimuli.

#### **EYE MOVEMENT DATA**

The eye movement data is represented in **Table 3**. A comparable statistical pattern was observed for the number of saccades, the amplitude of saccades and the number of fixations, which was reflected in (1) a main effect of Action Cue: more eye movements and fixations were made for the action cue compared to the no action cue condition, (2) a main effect of Target Object: more eye movements and fixations were made with an increasing number of target objects and (3) an interaction between Action Cue and Target Object: for the 0 and 1 target object conditions the number of eye movements and fixations increased when an action cue was presented, but for the 2 target object conditions the number of eye movements and fixations did not differ depending on whether an action cue was present. The statistical results for the eye movement data are summarized in **Table 4**.

#### **EFFECTS OF ACTION CUE**

Comparing trials in which participants made a prediction about an upcoming action based on the observation of an action cue compared to no action cue (Action Cue *>* No Action Cue) revealed increased activation in the AON, consisting of the left Middle Temporal Gyrus (MTG), the right Inferior Temporal Gyrus (ITG), the IPL bilaterally and the left dPMC (see **Figure 3A** and **Table 5**). The cluster in the MTG falls within the 30–50% probability range of BA 36 (Eickhoff et al., 2005) and overlaps with the EBA as identified by the functional localizer data (peak activation for contrast Body *>* Chair at *x* = 48, *y* = −64, *z* = 4 and *x* = −45, *y* = −67, *z* = 7). The activity increases in the left IPL were found to be within the 30–80% probability range of BA

**Table 3 | Eye movement data according to the different experimental conditions.**


*Standard errors are between brackets.*

**Table 4 | ANOVA results for the analysis of the eye movement data.**


40 and extended to the left supramarginal gyrus (SMG). The right IPL cluster was found to be within the 60–100% probability range of BA2 and extended to the right SMG. The activation in the left dPMC was found to be within the 10–40% probability range of BA6. The reverse contrast (No Action Cue *>* Action Cue) did not reveal significant activations when using the FWE-correction for multiple comparisons.

#### **EFFECTS OF THE # OF TARGET OBJECTS**

Comparing trials in which a target object was presented compared to trials in which no target object was presented (2 Target Objects and 1 Target Object *>* 0 Target Objects) revealed activation in the IPL bilaterally, the right superior parietal lobe (SPL), the dPMC and the left IFG (see **Figure 3B** and **Table 5**). The left IPL falls within the 30–60% probability range of area hIP1 and the right IPL falls within the 20–40% probability range of area hIP2 (Caspers et al., 2006). The activation in the SPL was within the 20–30% probability range of BA 7A. The activation in the dPMC was within the 0–30% probability range of area BA6. Activation in the left IFG was found to be within the 10–30% probability range of BA 45 and overlapped with the pars triangularis. No increased activation was observed for the reverse contrast (0 Target Objects *>* 1 Target Object and 2 Target Objects).

#### **OVERLAP BETWEEN ACTION CUE AND # OF TARGET OBJECTS**

To investigate whether areas within the AON were differentially activated as a function of the predictability of the action, a conjunction analysis was conducted ("Action Cue *>* No Action Cue" and "2 Target Objects and 1 Target Object *>* 0 Target Objects"). As can be seen in **Figure 3**, activity within the AON increased as a function of the presence of a target object in the left IPL and the PMC. When applying a more lenient statistical threshold for the AON mask (*p <* 0*.*001, uncorrected), an additional cluster was observed in the right IPL (see **Table 6**).

#### **EFFECTS OF ACTION CUE CONGRUENCY**

In all analyses reported, for the "Action Cue—1 Target Object condition" the data was collapsed over congruent and incongruent action cues. Directly comparing the effect of action cue congruency did not reveal significant differences in brain activation between congruent and incongruent action cues. Excluding trials in which the action cue was incongruent with respect to the target object also did not change the pattern of results that were reported above. These findings warrant the fact that in the reported analyses the data was collapsed over both congruent and incongruent action cues.

# **DISCUSSION**

The present study investigated how action semantics facilitates the prediction of imagined and observed actions and which neural mechanisms are involved. Participants performed an action prediction task in which they were required to anticipate the use of a centrally presented object that could be moved to an associated target object. At a behavioral level it was found that action prediction was modulated as a function of the predictability of the action (i.e., the number of target objects involved) and the availability of actor information (i.e., whether a hand could be observed grasping the central object). At a neural level it was found that predicting actions that involved a target object resulted in increased activation in the bilateral IPL and frontal motor areas. The presentation of an action cue was associated with increased activation in the EBA and the fronto-parietal AON. Within the AON, activity in the left IPL and the left PMC increased as a function of the level of action predictability. These findings indicate that the retrieval of action semantics for imagined object use and action prediction rely on comparable neural mechanisms, in line with the predictive coding framework of action observation (Kilner et al., 2007).

In this study participants were required to predict actions with objects that could be used in multiple ways and that could be associated with different action goals. It was found that RTs increased as a function of the presence of a target object, likely reflecting that more action semantic information needed to be retrieved to predict the upcoming goal of actions involving multiple objects (van Elk et al., 2012). Predicting actions involving a target object was associated with increased activation in the left IPL and in frontal motor areas. Neuroimaging studies have shown that this region is selectively involved in the observation of human handobject interactions (Johnson-Frey et al., 2005; Peeters et al., 2009, 2013; Valyear et al., 2012) and in the planning of object-directed actions (Culham et al., 2003; Valyear et al., 2007; Gallivan et al.,

2013). The increased activation in the left IPL for making a prediction about an action involving a target object likely reflects a motor simulation process, in which participants imagined grasping the central object to derive at the most likely action in the given context (Wolpert and Kawato, 1998; Buxbaum et al., 2005). This interpretation is in line with neuroimaging studies on motor imagery, indicating that activity in the IPL increases when participants are required to imagine more complex movements (de Lange et al., 2005, 2006; Zacks, 2008).

The finding of the involvement of the left IPL in predicting the use of object-directed actions is in line with neuropsychological studies with apraxic patients, suggesting that the left IPL is a critical region for storing hand postures required for the interaction with objects (Heilman et al., 1982; Heilman and Rothi, 1993; Buxbaum, 2001; Buxbaum and Saffran, 2002). Recently, an alternative account of the deficits observed in tool use following damage to the left IPL has been proposed, according to which apraxic patients are primarily characterized by impairments in technical reasoning (Osiurak et al., 2009, 2011; Osiurak and Lesourd, 2014). On this account, apraxic patients have difficulties with technical reasoning about abstract physical properties of objects and specifically in identifying the technical means to achieve a specific technical end (for a similar view, i.e., the "mechanical problem solving" account, see: Goldenberg, 2009). This view is supported by the finding that apraxic patients showed an impaired performance on a problem solving test involving the selection and use of novel objects (Goldenberg and Hagmann, 1998) and furthermore impairments in the use of novel tools are often accompanied by an impaired use of well-known objects as well (Osiurak et al., 2009; Jarry et al., 2013). The implication of the technical reasoning account is that in many cases, the successful use of objects does not rely on stored semantic or motor


**Table 5 | Brain regions associated with increased activity during prediction of actions based on action cues compared to no action cues (upper part of table).**

*Brain regions associated with increased activity during prediction of actions with an increased number of target objects (lower part of table). p < 0.05, FWE-corrected.*

**Table 6 | Brain regions associated with increased activity during prediction of actions based on action cues compared to no action cues (upper part of table).**


*Brain regions associated with increased activity during prediction of actions with an increased number of target objects (lower part of table). p < 0.05, FWE-corrected.*

representations, but requires applying mechanical or technical knowledge instead (i.e., knowledge about abstract mechanical principles, such as "lifting" or "screwing"; cf. Osiurak et al., 2009, 2013). This view provides an important alternative account of the available neuropsychological data and has implications for the supposed role of the left IPL in object use as well, indicating that this region may play a critical role in mechanical or technical reasoning in relation to the use of objects.

The availability of actor information was manipulated by including trials in which a hand could be observed grasping the central object and trials in which no hand was presented. The observed grasp type (i.e., whether the central object was grasped at the upper or lower side) could be used to disambiguate the upcoming action, *only* when two target objects were presented (e.g., a wine bottle in association with a wine glass and a wine cooler). When only one or no target object was presented at all, the action prediction could be based solely on the basis of the objects involved (e.g., a wine bottle in association with a wine glass). Closer inspection of the behavioral data indicates that when two target objects were presented, actor information indeed facilitated the disambiguation of the upcoming action, resulting in faster RTs (and less eye movements) but at the expense of more errors (i.e., a speed-accuracy trade-off was observed). In contrast, when only one or no target objects were presented, participants responded faster when no action cue was presented, but they made more errors. Correcting for this speed-accuracy trade-off, by using the inverse efficiency instead (Townsend and Ashby, 1978), indicated that response times increased when an action cue was presented, irrespective of the number of target objects. This finding indicates that participants automatically processed the actor information—even though in some cases it was irrelevant likely because their focus of attention was initially on the central object and action cues were always centrally presented (Duncan, 1984).

The observation of an action cue was associated with increased activation in the EBA, the IPL, and the dPMC. These areas are commonly referred to as the (AON; Caspers et al., 2010) that is typically found activated during the observation of others' actions. In the present study activation in the AON was observed by using an action prediction task, in which participants were required to anticipate an upcoming action. The finding that the AON is involved in action prediction is in line with previous studies, indicating the central role of the AON in action prediction tasks as well (Kilner et al., 2004; Aglioti et al., 2008; Stadler et al., 2012; Avenanti et al., 2013).

An important question is to what extent the action cue may have been perceived primarily as a hand grasping an object, or as a spatial cue indicating the relevant side of the object instead (i.e., up or down). This question has been addressed extensively in research on imitation that is characterized by a similar discussion to what extent effects of observed actions are driven by the biological properties of the stimulus or rather reflect spatial compatibility effects (Heyes, 2011). Several studies indicate that spatial compatibility can be dissociated from imitative compatibility effects, suggesting a special role for the processing of observed biological stimuli (Brass et al., 2000; Catmur and Heyes, 2011). This notion is further supported by the present fMRI data, indicating that the observation of an action cue did not only result in activation of brain areas associated with the processing of spatial information (i.e., the superior parietal lobe and the dPMC; Crammond and Kalaska, 1994; Iacoboni et al., 1996; Koski et al., 2005), but in the activation of brain areas involved in the perception of biological stimuli as well, such as the EBA (Chan et al., 2004; Downing et al., 2006).

The activation of the AON in response to an action cue may be partly driven by the stimuli in which either a hand was visible or not (Downing et al., 2001), resulting in the automatic activation of the corresponding motor programs used for grasping objects (Buccino et al., 2001; Brass and Heyes, 2005). Furthermore, in the present study static images depicting a human hand were used as stimuli rather than dynamic stimuli depicting biological motion. By using static images it was ensured that participants would predict the upcoming action solely based on the objects presented in the picture and the initial grasping location of the hand, rather than the dynamic cues associated with hand movements. Previous studies on action observation have shown that the observation of static action images results in reliable activation of the AON (Johnson-Frey et al., 2003; de Lange et al., 2008) and also in this study the AON was found activated for pictures representing a hand compared to no hand. It could be argued that the use of static compared to dynamic images may have resulted in an induced process of motor imagery, in which the participant imagines completing the observed action. Previous studies have indicated that motor imagery also activates similar brain regions as observed in action observation, such as the IPL and the PMC (Zacks, 2008; Caruana et al., 2014), and the stronger activation of these areas in the present study may be partly related to a more complex motor imagery processed (de Lange et al., 2005, 2006; Zacks, 2008). This suggestion is also supported by the reaction time data, indicating that participants responded slower when they were presented with an action cue, likely because the integration of an observed action cue required additional processing time. However, it should be noted that in the present study, participants were *always*required to predict actions, either by imagining the use of visually presented objects, or by imagining how an actor would use the objects presented. Thus, the underlying process of action prediction may be functionally equivalent for trials in which an action cue was presented and trials in which no action cue was presented, such that participants always relied on using an internal forward model to infer the most likely outcome of the action (Wolpert and Flanagan, 2001). When no action cue was presented, participants may have directly engaged in a process of motor imagery (Zacks, 2008; Caruana et al., 2014), whereas in the case of an action cue the observed action first needed to be matched unto one's motor repertoire, as implied by the AON literature (Kilner et al., 2007).

Interestingly, it was found that activation within the AON varied as a function of the presence of a target object and the predictability of the action. That is, in the left IPL and the left PMC activity increased when a target object was presented, indicating that these regions support the use of semantic information for understanding and predicting observed actions. The overlap in activation in the left IPL and the left PMC for the independent effects of target objects and action cue, may indicate that upcoming actions are predicted, either through a process of motor imagery (when no action cue is presented) or by matching the observed action to stored hand postures for object use (when an action cue is presented). The finding that the activation of the AON is modulated not only as a function of the low-level kinematic features of the observed action, but also by the involvement of semantics for action is in line with the view that the AON represents higher-level aspects of observed actions as well, such as the correctness or meaningfulness of an action (Koelewijn et al., 2008; Newman-Norlund et al., 2010, 2013; Stapel et al., 2010). The present study extends these findings, by indicating a stronger involvement of the AON for unpredictable actions that require the use of action semantics. Furthermore, the finding that similar areas are involved in using semantics for imagined actions and in action observation, is in line with the "predictive coding account of action observation" (Kilner et al., 2007), according to which predicting other's actions relies on similar neural mechanisms as involved in the planning of our own actions. In sum, the present study indicates that the left IPL and PMC represent stored hand-postures that can be used for planning object-directed actions and for predicting other's actions as well.

# **ACKNOWLEDGMENTS**

This research was supported by a VENI grant no. 016.135.135 from the Netherlands Organization for Scientific Research (NWO).

# **REFERENCES**


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 12 March 2014; accepted: 31 March 2014; published online: 23 April 2014. Citation: van Elk M (2014) The left inferior parietal lobe represents stored handpostures for object use and action prediction. Front. Psychol. 5:333. doi: 10.3389/fpsyg. 2014.00333*

*This article was submitted to Cognition, a section of the journal Frontiers in Psychology.*

*Copyright © 2014 van Elk. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# The tool in the brain: apraxia in ADL. Behavioral and neurological correlates of apraxia in daily living

# *Marta M. N. Bie ´nkiewicz1\*, Marie-Luise Brandi 1,2 , Georg Goldenberg3 , Charmayne M. L. Hughes1 and Joachim Hermsdörfer <sup>1</sup>*

*<sup>1</sup> Lehrstuhl für Bewegungswissenschaft, Technische Universität München, München, Germany*

*<sup>2</sup> Graduate School of Systemic Neurosciences, Ludwig-Maximilians-Universität München, München, Germany*

*<sup>3</sup> Klinik für Neuropsychologie, Städtisches Klinikum München, München, Germany*

#### *Edited by:*

*François Osiurak, Université de Lyon, France*

#### *Reviewed by:*

*Ferdinand Binkofski, University of Luebeck, Germany Didier Le Gall, Laboratoire de Psychologie des Pays de la Loire (EA 4638), France*

#### *\*Correspondence:*

*Marta M. N. Bie ´nkiewicz, Lehrstuhl für Bewegungswissenschaft, Technische Universität München, Georg-Brauchle-Ring 60-62, München, Germany*

*e-mail: marta.bienkiewicz@tum.de*

Humans differ from other animals in the way they can skilfully and precisely operate or invent tools to facilitate their everyday life. Tools have dominated our home, travel and work environment, becoming an integral step for our motor skills development. What happens when the part of the brain responsible for tool use is damaged in our adult life due to a cerebrovascular accident? How does daily life change when we lose the previously mastered ability to make use of the objects around us? How do patients suffering from compromised tool use cope with food preparation, personal hygiene, grooming, housework, or use of home appliances? In this literature review we present a state of the art for single and multiple tool use research, with a focus on the impact that apraxia (impaired ability to perform tool-based actions) and action disorganization syndrome (ADS; impaired ability to carry out multi-step actions) have on activities of daily living (ADL). Firstly, we summarize the behavioral studies investigating the impact of apraxia and other comorbidity syndromes, such as neglect or visual extinction, on ADL. We discuss the hallmarks of the compromised tool use in terms of the sequencing of action steps, conceptual errors committed, spatial motor control, and temporal organization of the movement. In addition, we present an up-to-date overview of the neuroimaging and lesion analyses studies that provide an insight into neural correlates of tool use in the human brain and functional changes in the neural organization following a stroke, in the context of ADL. Finally we discuss the current practice in neurorehabilitation of ADL in apraxia and ADS aiming at increasing patients' independence.

**Keywords: apraxia, action disorganization syndrome (ADS), activities of daily living (ADL), tool use, cerebrovascular accident (CVA), quality of life, stroke patients**

# **INTRODUCTION**

Left brain damage caused by ischemic or hemorrhagic stroke is the most frequent neurological correlate of apraxia (Goldenberg, 2013). However, features of apraxic behavior can be also observed in numerous other neurodegenerative disorders (such as Parkinson's disease, Alzheimer's disease or posterior cortical atrophy; Bohlhalter and Osiurak, 2013) or occur as an effect of anoxia (Sirigu et al., 1995) and herpes encephalitis (Sirigu et al., 1991). Apraxic behavior in tool use is primarily attributed to the impaired or lost access to the tool related knowledge, concepts of use and problem solving (Goldenberg, 2013). Patients frequently show compromised ability to carry on everyday activities and often show action disturbances leading to safety hazards after dismissal from hospital units (Hanna-Pladdy et al., 2003). Such slips might involve attempts to use a knife in a wrong orientation to cut a slice of bread, bite a toothbrush instead of applying a brushing movement inside the mouth, toy with boiled water or tear the teabag to make a cup of tea. Problems with sequential tasks, concepts of use and smooth execution on the spatiotemporal level cannot be attributed to the deficit of function on the ipsilesional hand of patients. Patients are not able to perform the task even

with the contralesional limb which might have preserved motor functionality.

The purpose of this review is to present a comprehensive summary of the research investigating apraxia syndrome following a cerebrovascular accident (CVA) and its influence on independence during activities of daily living (ADL). First, we provide a systematic overview of the behavioral research investigating impact of apraxia on three basic areas of object and action related abilities: sequencing of action, tool and gesture knowledge and spatiotemporal features of the movement, in the context of basic needs of independence. A particular focus is placed on research investigating the influence of those functions on ADL such as food preparation, personal hygiene, grooming and use of household appliances, or housework tools. The second part of this review is dedicated to the cut-edge neuroimaging research, demonstrating how multi-faceted the neural basis of tool use and ADL is as well as the current state of the art.

### **DEFINITION OF APRAXIA**

The most commonly used definition of apraxia was coined by Rothi and Heilman (1997) which states: "Apraxia is a neurological disorder of skilled movement that is not explained by deficits of elemental motor or sensory system." In other words, apraxia is considered as being independent from other stroke comorbidity symptoms such as hemiplegia (loss of proprioception and motor control over limb on one side) or visual deficits such as hemianopia or neglect. However, as discussed in the penultimate section of this review, comorbidity symptoms occurring as a consequence of CVA contribute to overall ADL in a substantial manner and might even be difficult to disentangle with apraxic features. Until recently, a vast number of clinicians and researchers used the original postulation by Hugo Liepmann (a German pioneer in apraxia research) and distinguished three separate types of apraxia: ideational, ideo-kinetic (or ideomotor), and limb-kinetic (Goldenberg, 2003, 2013). Ideational apraxia refers to an inability to use familiar tools that were previously handled in an effective and purposeful manner; choosing the right object for a required action goal and carrying out multistep naturalistic action (Goldenberg, 2013). The second category, namely ideo-kinetic apraxia, described compromised ability to pantomime actions; mimicking tool use without holding object, and/or difficulty with gesture production. In the literature, gesture production is usually divided into transitive and intransitive acts. Transitive gestures relate to object use, showing how one would use an object, whereas intransitive gestures refer to non-tool related movements, such as waving goodbye or giving someone the thumbs up. Thus, patients were reported to be unable to produce gestures that would mirror the relevant semantic representation they wished to convey (Hogrefe et al., 2012). Interestingly, even if apraxic patients attempt to operate the tool in a goal-directed fashion, they might do it in a spatiotemporally erratic manner (Poizner et al., 1995; Hermsdörfer et al., 1999; Laimgruber et al., 2005; Randerath et al., 2010). These errors are reminiscent of "limb-kinetic apraxia," which was introduced to describe hesitation and disrupted smoothness of the movement when operating tools (both multiple and single) or disruptions of fine and precise movements, but affects only the limb opposite to the lesion (Heilman et al., 2000). To summarize, the main cognitive domains affected by apraxia comprise of the use of tools (multiple and single) and gesture production.

# **DISAMBIGUATION AND COMMON GROUND BETWEEN APRAXIA AND ACTION DISORGANIZATION SYNDROME**

As previously mentioned, apraxia, since the work of Hugo Liepmann, is usually linked to left brain damage (Goldenberg, 2013). Original descriptions (i.e., by Pick) of ideational apraxia were inclusive of disturbances in multi-step action performance (Goldenberg, 2013). A plethora of research demonstrates that patients suffering from right brain lesions also show disruption in terms of naturalistic action organization, referred to as action disorganization syndrome (ADS; Schwartz et al., 1999; Forde et al., 2004; Hartmann et al., 2005). ADS is a term used to describe compromised ability to sequence fixed chains of actions in an appropriate manner in relation to any naturalistic action (Humphreys and Forde, 1998). However, the differentiation between ADS and apraxia (especially ideational) is disputed. Therefore apraxia and ADS can be described under the umbrella term"apraxia and action disorganization syndrome" (AADS; Humphreys and Forde, 1998).

Therefore in this review we incorporate studies investigating ADS, especially since patients with left brain damage also show difficulties with sequencing of action subtasks (Goldenberg, 2013). Probably the most puzzling element in the investigation of AADS is the lack of consistent evidence as to which brain lesions are related to the designated action problems.

# **EPIDEMIOLOGY**

The epidemiology of AADS was most recently reported by Bickerton et al. (2012). Approximately 46% of patients, who suffered from a first CVA were identified as symptomatic of AADS (within 6 weeks from CVA, 231 participants) based on the neuropsychological assessment (Birmingham Cognitive Screen). The criterion was impairment on one of four praxis tasks: pantomime, tool use during multi-step actions, gesture recognition or imitation. Furthermore, in the same study around 52% of those patients have shown persistent signs of AADS that did not diminish with the course of neurorehabilitation (24% of the initial sample). Previous reports, which solely focused on left hemisphere stroke survivors, estimated a rate of ideo-kinetic apraxia occurrence at approximately 30% (De Renzi, 1989). Donkervoort et al. (2001) had found that around 28% of all CVA survivors in the Dutch rehabilitation centers and 37% of nursing homes, show persistent signs of apraxia and therefore compromised ADL independence. In a later study, Donkervoort et al. (2006) stated that 88% of patients diagnosed clinically, in the acute stage with features of apraxia, were still apraxic 20 weeks post first measurement (100 days after CVA). Importantly, greater improvement over the course of rehabilitation was observed in patients that initially have had more severe deficits, whereas those with mild impairments tended to improve to a (clinically) less significant extent (measured with Barthel Index; Mahoney, 1965). Donkervoort et al. (2006) concluded that apraxia is a persistent impairment and has a negative effect on ADL. In a similar vein, Smania et al. (2006) demonstrated that apraxia is negatively correlated with the ADL independence, based on responses from patients and caregivers. On the contrary, De Renzi (1986) reported that in natural setting apraxic features are less salient due to the contextual cueing. In other words, if a patient in the hospital or lab setting has a difficulty with a simple gesture production, the same individual might still be able to perform the gesture whenever prompted by the environment (for example, to wave goodbye). Environmental information therefore has the potential to provide additional cues to promote selection of an appropriate motor program (Hermsdörfer et al., 2006). Although there is a lot of theoretical evidence supporting this view, there is no scientific ground yet to support this stance.

# **USE OF ADL SCALES IN AADS**

Several scales are commonly used by the clinical professionals for the assessment of ADL independence in neurological patients. Such scales are usually based on self-report or questionnaire (Barthel Index of ADL or Bristol ADL Scale; Mahoney, 1965; Bucks et al., 1996) or observation of action performed during clinical assessment (e.g., E-ADL, TULIA, NE-ADL; Gladman et al., 1993; Graessel et al., 2009; Vanbellingen et al., 2010). Those assessment tools are used not only to aid the clinical diagnosis of patients' impairments, but also, if not primarily, to monitor efficacy of

interventions to foster independence in cohort studies or clinical trials for example. Therefore the application of those scales in the clinical setting is common. Moreover, some studies have attempted to predict the speed and extent of patients' recovery based on the overall score. For instance, Barthel Index scores measured within the approximately 3 weeks of CVA were found to be accurate predictors of compromised ADL independence in 6 months post CVA (Nakao et al., 2010). Similarly, a recent study by Bickerton et al. (2012) has noted a correlation between a multi-step action task execution and Barthel Index. Nonetheless, the assessment scales and neuropsychological batteries do not capture fully the apraxic problems patients might encounter during their daily life. Hence, relevant behavioral studies were selected for the purpose of this review to shed a light on the spectrum of difficulties that can be observed in those patients during ADL.

### **BEHAVIORAL STUDIES**

Most of the behavioral studies investigating apraxia following CVA focus on behavioral data with qualitative error categorization (Foundas et al., 1995; Schwartz et al., 1999). As such, the most predominant methodology includes video recordings of patients' performance and then arbitrary classification of action errors. Setting aside the original descriptions and attempts to classify apraxia, for the purpose of this review, we can distinguish three major dimensions of action performance where apraxic features can be identified. The first one refers to sequencing problems during ADL and links to the description of ADS, compromised ability to perform subsequent actions in the correct temporal order with spatial constraints, in order to achieve an action goal (pack a lunchbox; Humphreys and Forde, 1998). For example, if one attempts to make a cup of tea, common error would involve putting cold water, not previously heated in a kettle, straight into the mug (omission error). The second area that will be discussed in this review refers to conceptual errors that might lead to the selection of the inappropriate motor plan. For example, with reference to the previously used example of tea making, one can use coffee grains instead of tea bags (substitution error; Goldenberg, 2013). In a similar fashion, communicative gestures might be misused or misunderstood. Finally, other mistakes might occur on the spatiotemporal dimension, even if the right tool is selected for action. The handling of the tool might not be adequate in terms of movement orientation, applied speed of the movement or grasp (Laimgruber et al., 2005; Randerath et al., 2010). For example, an apraxic individual might be unable to open the kettle lid during an attempt to make a cup of tea.

#### **SEQUENCING PROBLEMS**

Daily activities rarely rely on single tool actions which require only one tool-object interaction. The majority of the actions we perform involve multistep actions leading to an action goal. The achievement of the action is comprised of the different action subgoals, constituting to chains of different activities (Goldenberg, 2013). To perform a coherent action (i.e., make a sandwich), different steps need to be organized within certain constraints of time and space (Goldenberg et al., 2001). For example, even if the individual action step is performed in a correct manner, the temporal position in the sequence chain might be out of place, in effect, leading to failure in achieving the action goal. Referring again to making a cup of tea, a person might put the kettle on, having not previously put the water inside or using another example: brush their teeth having not put the toothpaste on. Usually those errors refer to the temporal organization of the action sequence, but are not related to the personal context of actions. The overall execution of specific sequences during ADL varies interpersonally and relies on personal abilities and preferences (Land, 2006; Goldenberg, 2013; Hughes et al., 2013). Therefore, the scientific investigation of ADLs is inherently burdened with a high level of complexity of analysis and must permit a certain level of homogeneity between examined subjects. For example, healthy adults might perform an action of making a cup of tea in a variety of ways and preferences (i.e., time of the tea bag being dipped in the mug, number of sugar cubes inside) with some other sequences being constant (i.e., heating the water in a kettle before pouring it in the mug), in order to achieve an action goal (make a cup of tea). Hence certain sequences are always fixed, whereas others show a high level of inter-subject variability (Hughes et al., 2013). If the error occurs in the fixed chain of sequences, it leads to the failure to achieve the task goal and is not recoverable until the next attempt (pouring cold water into the mug with teabag inside). If however the error occurs in the "not-fixed" chain of activities, it might be recoverable.

The most frequent sequencing error in terms of action performance is the omission error, which refers to omitting a step before another one (Schwartz et al., 1999). For example, a patient might put a piece of paper into an arch file before using the hole-punch. In addition, more general sequence errors are when the patients perform something in the wrong order. Such an instance would be putting or adding an extra sequence or ingredient (addition) that is not needed or that is repeated (perseveration error; Rumiati et al., 2001). In another scenario, a subtask might be performed too early in the chain of sequences (anticipation error). An example of sequence addition error would be folding a piece of paper before putting it into the arch file in a document filing task. Another type of addition, based on the use of additional objects or ingredients (in food related tasks) would be (using the previously mentioned example) putting a piece of scotch tape on the top of the paper. In sum, CVA subjects might engage in sparse subtasks that are not relevant in the context of achieving the action goal. In the same task, a perseveration error would describe repetition of the previously accomplished subtask, such as making more punch holes than necessary. There is a plethora of research that has attempted to capture the most common error occurrences in naturalistic action performance with different types of error patterns. However, the results show some incongruence between the terminology used and the classification of errors (see Goldenberg, 2013, Chap. 9, for review on this issue). Previously mentioned omission errors reach an approximate ratio of 40–50% for all action errors (Schwartz et al., 1999; Bickerton et al., 2007). Importantly, the tendency to skip a step that is necessary for achieving the action goal seems to be linked to the level of familiarity with the object. Novel object, which are not familiar to patients seem to elicit the highest number of those errors (Bickerton et al., 2007). Other authors also point out the prevalence of these types of action errors, but they use different terminology to describe it, namely sequence error

(De Renzi and Lucchelli, 1988) or action anticipation (Rumiati et al., 2001). **Table 1** presents an overview of research describing the sequencing errors related to the ADL in stroke survivors studies.

As reported in **Table 1**, there is a substantial body of research attempting to capture problems with sequencing of ADL in CVA patients. Different classifications are proposed by many research groups, but not all of them fit to every ADL, due to the variation in the fixed or not fixed action chains. However, most authors agree that problems with the organization of particular subtasks should be referred to as sequence errors, with subclasses, such as addition errors or anticipation, or without (De Renzi and Lucchelli, 1988; Schwartz et al., 1999; Rumiati et al., 2001; Goldenberg, 2013). In the seminal study by Foundas et al. (1995) conducted on 10 patients with unilateral left hemisphere CVA no error classification was used. Authors observed the lunchtime behavior (via video taping) on the hospital ward and divided the overall meal organization into three phases: preparatory, eating and clean up. Only 20% of CVA patients proceeded with all three phases of the meal and only 40% demonstrated preparatory behavior. In comparison to all healthy age-matched controls engaged in preparation of the meal, and 80% in the

clean-up phase. In addition, patients used fewer tools (cutlery) than controls and shown different pattern of food consumption (consuming one ingredient in a sequential fashion or drink a glass of refreshment at once) in comparison to controls (who preferred to mix different ingredients and take small sips of drink).

### **CONCEPTS OF USE AND GESTURE KNOWLEDGE**

On the cognitive level, the knowledge about concepts of use can be referred to as both functional knowledge (Sirigu et al., 1995) and the ability for mechanical problem solving (Goldenberg and Hagmann, 1998; Osiurak et al., 2009). Functional knowledge specifies the typical purpose, recipients, and manner of using distinct types of tools (Sirigu et al., 1991; Hodges et al., 2000; Rumiati et al., 2001). This type of expertise embraces global motor concepts, inclusive of the recipient of the action, relevant manipulation, and tool selection for the desired action goal (Goldenberg, 2013). For example, a hammer can be used to put a nail into a block of wood through forceful strokes. The knowledge necessary to achieve this goal includes: choosing the right tool from the toolbox (hammer); knowing how to position the nail in the block of wood and knowing what


*LBD – left brain damage, RBD – right brain damage.*

movement to apply. There is, however, controversy whether the kinematics of actions and the formation of adequate hand postures are stored in a separate compartment of semantic memory as "manipulation knowledge" or are derived from structural properties of tools by mechanical problem solving (Goldenberg and Spatt, 2009; Osiurak et al., 2009; Kalénine et al., 2010). Patients with loss of functional tool use knowledge may be able to infer the function of the object from their structure (Goldenberg, 2009). In the modern type of devices however, such as technically advanced coffee machines with capsules, patients are not able to deduce (use mechanical problem solving) how to operate the device based on its physical structure. Therefore those types of the devices (such as tablets or smart TV) might be almost impossible to operate for apraxic individuals (Hartmann et al., 2005).

In principle, ADL can be divided into multiple tool use and single tool use actions (Goldenberg, 1996, 2013). For example, making a cup of tea would be an example of complex and multiple tool based action. On the contrary, fixing a loose screw would be based on single tool use, namely a screwdriver. One of the common errors noted in the literature is mislocation or misplacing of the tool (De Renzi and Lucchelli, 1988; Schwartz et al., 1999) or spatial error as described by Humphreys and Forde (1998). De Renzi and Lucchelli (1988) tested 20 patients in the tool use and pantomime paradigm. Among other errors, author's differentiated mislocation as appropriate action carried out in the spatially inadequate place. For instance, patients were able to strike a match, but tried to lit the wrong side of the candle. Misuse of the tool has also been identified by De Renzi and Lucchelli (1988) and Rumiati et al. (2001). Misuse can be defined as use of object in conceptually inappropriate way, i.e., rubbing candle onto the table, or handling object by the wrong end (De Renzi and Lucchelli, 1988). All of the error classifications mentioned refer to the impaired ability to handle the tool in a relevant manner (i.e., also include uncomfortable grasp of the tool). Other research also reports wrong object selection (Humphreys and Forde, 1998; Goldenberg, 2009) or object substitution (Schwartz et al., 1999). Humphreys and Forde (1998) tested two patients with features of AADS on ten ADL tasks (see **Table 1**). In the tea making task, one of the patient demonstrated repetitive errors of pouring milk into the teapot rather the mug. Authors referred to it as semantic error, specific for object selection. Schwartz et al. (1999) tested 30 patients with right hemisphere lesions following CVA on three ADL tasks (making a toast, wrapping present, and packing lunchbox). Object substitution was defined as correct movement performed with wrong object, i.e., putting a slice of bread on a hot plate instead inside the toaster. In addition, misestimation errors, i.e., too little or too much of one ingredient, were introduced in studies looking into food related behavior (Foundas et al., 1995). For example, patients were reported to put too little food on their plate and fork during daily lunchtime behavior or making a toast (Foundas et al., 1995; Schwartz et al., 1999). Importantly, the differences within classification of the errors are arbitrary and do not have a consequence on the overall understanding of the difficulties patients exhibit with ADL. Patients might choose the wrong tool for an action, for example, picking up a screwdriver

to connect two sheets of paper together. In many occasions the difficulties with access to the adequate motor concepts do not manifest themselves directly but are observable as perplexity or toying behavior. Those errors are not explicitly categorized separately by all researchers (e.g., Schwartz et al., 1999). Perplexity refers to pauses in movement, or inefficient manipulation. For example, the patient might pick up objects and then set them back on the work surface and cease further trials to accomplish the task goal. Toying, on the other hand describes handling an object in a non-purposeful fashion. One measure that can capture those behaviors, aside from video scoring of conceptual errors committed by patient, is movement time for the task completion.

# **SPATIOTEMPORAL FEATURES OF APRAXIA**

A seminal study by Foundas et al. (1995) on meal preparation, has revealed that left brain damaged patients were less successful in the overall organization of the preparation of meals and that the "correct tool actions" measure significantly correlated with the apraxia score (Florida Apraxia Battery, Rothi et al., 1992). The overall time difference between patients (slightly prolonged) and healthy controls was however not statistically significant. Spatiotemporal errors of movement execution have been documented mostly during pantomime of tool us but have also been found during real tool use (Hermsdörfer et al., 2006). Spatiotemporal errors in the task performance can have a discrete demonstration when the individual is performing an action in a kinematically incongruent manner, which might or might not be observable with the naked eye even for a non-expert viewer. Poizner et al. (1995) and Clark et al. (1994) have demonstrated that apraxic patients with left brain damage suffer from impaired joint coordination and imprecise plane of motion, along with trajectory shape in a bread slicing task. In addition, impaired coupling between the hand speed and trajectory shape was identified. However, it remains open whether these kinematic irregularities reflect deficits of motor-coordination directly or are due to slow and hesitating movement execution due to conceptual problems with planning the action (Hermsdörfer et al., 2006). In other words, impaired movement on the spatiotemporal dimension might be a reflection of compromised movement planning, but not be a feature of limb apraxia. In a seminal study by Laimgruber et al. (2005) left brain damaged patients were found to demonstrate a prolonged adjustment phase before grasping a glass of water, whereas right brain damaged patients showed a decreased velocity of the movement. Speed deficits were also found in the sawing tasks in left brain patients in comparison to age-matched controls (Hermsdörfer et al., 2006). Other variables such as prolonged reaction times and reduced amplitude of the movement were reported for the hammering and scooping movement actions in left brain damaged patients (Hermsdörfer et al., 2006, 2012). Deficits of spatiotemporal aspects of movement execution may be directly or indirectly related to apraxia as indicated above, but also may be indirectly related to spatial deficits such as neglect or they may also be independent consequences of damage to the motor-dominant hemisphere (Hermsdörfer et al., 2012). Randerath et al. (2010) has found that left brain damage patients show impairment in the grasping movements during single tool use. In comparison to

healthy age-matched controls, patients demonstrated significantly higher percentage of non-functional grasps of the tools' handles. The impaired grasp was predominately followed by erratic demonstration of the tool use. In the real life scenario, those spatiotemporal deficits might result in mishandling of the object, leading to safety hazards, or frustration (Hanna-Pladdy et al., 2003). In the next section we will present an overview of the neural underpinnings of ADL and apraxia, which will shed more light on the complex organization of human tool use.

# **THE NEURAL BASIS OF ADL**

This section of the review is organized in a similar fashion to the behavioral part, with division of the studies to the sequencing of subgoals of ADL, then conceptual understanding and finally spatiotemporal features of ADL. To provide an insight into the neural correlates of ADL and apraxia, we present neuroimaging studies with healthy subjects followed by lesion analyses with apraxic patients.

# **HEALTHY ADULT STUDIES**

We aim to discuss the neural basis of ADL by including functional neuroimaging studies on viewing, understanding, imagining, pantomiming and executing ADL and single tool use in healthy adults. Furthermore only studies on sequencing actions, tool knowledge and the spatiotemporal features of actions with tools are summarized and visualized here. For the visualization of the neural correlates of these three aspects of ADL, we used the GingerALE toolbox (Eickhoff et al., 2009, 2012) for conducting a meta-analysis. The relevant peak coordinates (in Talairach space) from whole brain analysis were entered separately for the three aspects of ADL. The main aim of this analysis was to provide a descriptive visualization of the activation patterns found in the relevant studies. Therefore a relatively low threshold (*p* < 0.05 FDR corrected) was used to create the ALE images (Laird et al., 2005). The toolbox Mango (Designed and developed by Jack L. Lancaster and Michael J. Martinez) was used to map these thresholded ALE images of all three categories on a rendered brain and to locate the visualized brain areas.

# **ACTION SEQUENCING**

As described previously, patients suffering from AADS show difficulties with sequencing multi-step actions and single tool use. The neural underpinnings of action sequencing in ADL are not yet fully understood. Only a few studies have so far investigated brain regions relevant for sequencing sub-actions of ADL. The most seminal studies in the area were conducted by Schubotz et al. (2012) and Zacks et al. (2001). In these studies subjects had to watch videos depicting different ADL with multiple sequences (for example washing the dishes or ironing a shirt) and had to detect the time borders when each of the sub-actions had commenced. In addition, Weiss et al. (2006) has analyzed the processing of errors in the sequential structure of ADL. Here, subjects had to watch videos of ADL including, for example, pouring a glass of water and drinking it, lighting a candle with matches or affixing a stamp on a letter. These videos were either correct or included errors in the order of sub-actions, which the subjects had to detect. In summary the brain areas relevant for processing the separation and ordering

of sequences in ADL cover areas of the frontal, parietal, temporal and occipital cortex. More precisely, these areas were pinpointed to the inferior and middle frontal gyrus, angular gyrus and adjacent precuneus, middle temporal gyrus, fusiform gyrus, and middle occipital gyrus of the left hemisphere. Additional clusters can be seen in the right middle frontal gyrus, middle occipital gyrus, precuneus, inferior and superior temporal gyrus, and fusiform gyrus. The ALE image depicting results from those studies is shown in the **Figure 1** in red.

# **CONCEPTUAL KNOWLEDGE OF TOOL USE**

To get an overview of the neural basis of the conceptual knowledge in the context of ADL and single tool use, we summarized studies investigating how the knowledge of tools and their function is coded in the brain. We included studies comparing correct versus incorrect use of a tool dependent on the context (Mizelle and Wheaton, 2010; Wurm et al., 2012) and studies comparing tool actions of familiar compared to unfamiliar tools (Menz et al., 2010). Exemplary stimuli used in these studies were videos showing actions like punching holes in paper (Wurm et al., 2012) or images and animations of using a hammer (Menz et al., 2010; Mizelle and Wheaton, 2010). In addition, two other studies were included (Manthey et al., 2003; Hoeren et al., 2013), which evaluated both the conceptual understanding of ADL and also the processing of the spatial organization of actions separately. The latter aspect will be discussed in the next paragraph. In the study of Manthey et al. (2003) subjects had to watch videos with ADL and detect object related errors (for example: pour coffee in a glass instead of a cup), or movement errors in the viewed actions (for example: open a bicycle lock but holding the key transverse to the lock). In the Hoeren et al.'s (2013) study subjects were asked to decide, if the object used in an action fits to the context (for example: a cake lifter is used for cake not for a steak in a pan), or if the hand position is correct to perform the known action with the object. In all studies subjects had to show a conceptual understanding of ADL to perform the different tasks. More specifically, the participants had to know the purpose of the actions they saw and the function of the tool used in the actions. Findings from these studies have demonstrated that understanding and tool use function in ADL recruits a wide (mostly left lateralized) network covering frontal, parietal, temporal and occipital centers. Main activation sites were reported on the left hemisphere in the frontal cortex and include inferior, middle and superior frontal gyrus; in the parietal cortex clusters covering anterior to posterior part of the intraparietal area, angular and supramarginal gyrus, and superior parietal lobule activations were reported. Activations in the middle and superior occipital gyrus were found in the occipital cortex. In the temporal lobule, activation patters mainly covered the posterior part of the middle and inferior temporal gyrus and the fusiform gyrus. In the right hemisphere, activation was pinpointed to the middle, superior and inferior frontal gyrus in the superior parietal lobule and anterior part of the intraparietal area, as well as in middle temporal, inferior occipital, and fusiform gyrus. The activation in the right hemisphere is partly homologous to the left areas, but the overall activation pattern comprises less brain areas. A summary of brain network recruitment reported in the mentioned studies is shown in **Figure 1** in blue.

**FIGURE 1 | ALE images for studies focusing on action sequencing in red, conceptual understanding of ADL in blue, and spatial orientation of ADL in green; Overlays are depicted in purple (blue + red), light blue**

**(blue + green), and white (all three).** Images are produced with the GingerALE toolbox (Eickhoff et al., 2009) and have a threshold of *p* < 0.05 with FDR correction.

# **SPATIOTEMPORAL ORGANIZATION OF MOVEMENTS**

As mentioned in the previous sections of this review, the third component of ADL (following the sequencing of the actions and conceptual knowledge) concerns the tool manipulation necessary to achieve the intended goal and incorporates spatiotemporal features of the movement. This includes grasping the tool in a correct way and moving it accordingly across space. Functional imaging studies have analyzed the brain areas relevant for selecting the correct grip for tool usage during ADL (Valyear and Culham, 2010; Vingerhoets et al., 2010; Hoeren et al., 2013) or the spatial organization of the movement (Manthey et al., 2003; Weiss et al., 2006; Yoon et al., 2012). The neural correlates of this component are more bilateral and mainly include parietal, frontal and occipital areas of both hemispheres. These include the superior and inferior parietal regions, the area close to the posterior part of the intraparietal area and the parieto-occipital sulcus (parieto-occipital junction), premotor cortex and the middle occipital gyrus in both hemispheres. In addition, studies mentioned above have found that the ventral premotor area is relevant in the right hemisphere and the anterior insula in the left. In general, it can be mentioned that most clusters relevant for grip selection and the spatial monitoring of tool use mainly cover regions related to the dorso-dorsal pathway as described by Binkofski and Buxbaum (2012).

# **SUMMARY OF THE FUNCTIONAL IMAGINING HEALTHY ADULTS SECTION**

Investigation of main cortical activation sites of all three aspects of ADL yields the involvement of a wide neural network including frontal, parietal and temporal centers (**Figure 1**). Overlaps were found between the different maps for regions processing conceptual and spatial information of tool use and ADL including frontal clusters in the dorsal and ventral premotor areas, in the anterior cingulate cortex, in the parietal lobe along the intraparietal area, the superior parietal lobule, the supramarginal gyrus, around the parieto-occipital sulcus and in the inferior temporal gyrus of the temporal lobe of the left hemisphere. We have found less overlaps in the right hemisphere, which comprise parts of the parietal lobule, precentral gyrus and inferior temporal gyrus. In addition, we report a partial congruency between clusters from sequencing studies and studies focusing on knowledge of tool use. These are associated with activation in the dorsal premotor area, posterior part of the intraparietal area, middle occipital gyrus and fusiform gyrus of the left hemisphere. In summary, ADL and single tool use are complex tasks with multiple aspects to be processed which recruit wide brain networks. Importantly it has to be stated that the neural bases of the three aspects discussed here cannot be clearly separated in actual tool use but need to be integrated to perform ADL. Evidence supporting the importance of the mentioned network is also provided by studies focusing on the neuronal basis

of actual tool manipulation, which covered more general or other aspects of tool use (Hermsdörfer et al., 2007; Imazu et al., 2007; Gallivan et al., 2013). In addition, studies on pantomime of tool use also support the present findings (Moll et al., 2000; Inoue et al., 2001; Johnson-Frey et al., 2005; Króliczak and Frey, 2009; Vingerhoets et al., 2011).

# **LESION ANALYSIS IN PATIENT WITH BRAIN DAMAGE**

Another method that sheds light on the neuroanatomical correlates of tool use is a lesion symptom analysis in CVA patients. In those studies, behavioral measures are correlated with lesion sites to create statistical brain maps showing the location of lesions closely linked to a behavioral deficit. Compared to the studies with healthy subjects, studies including lesion analysis focusing on executing or recognizing ADL are relatively rare (Pazzaglia et al., 2008; Goldenberg and Spatt, 2009; Randerath et al., 2010; Hermsdörfer et al., 2013; Kalénine et al., 2013). Therefore, a differentiation of action sequencing, conceptual understanding and spatiotemporal aspects of tool use, to the same extent as in healthy subjects or purely behavioral clinical studies, is limited. Hence, we aim to concentrate on studies including tasks testing performance of actual tool use and understanding or recognition of goal directed actions (Pazzaglia et al., 2008; Goldenberg and Spatt, 2009; Hermsdörfer et al., 2013; Kalénine et al., 2013). Additional information is given on the neuronal correlates of tool grasping next to tool usage (Randerath et al., 2010) and to increase the scope on the neural basis of sequencing ADL in patients, a study focussing on the sequencing of pantomime tool use (Weiss et al., 2008) will also be mentioned here.

In a study by Goldenberg and Spatt (2009), 38 patients with left sided brain lesions, were tested to assess possible deficits in functional knowledge of tools and objects, mechanical problem solving (which was tested with the use of novel tools), and additionally the selection and usage of common tools. Impairments in these tasks were related to two major lesions sites, one around the middle frontal gyrus reaching to the inferior frontal gyrus, which was related to deficits in all three tasks, and a second lesion site in the parietal cortex, reaching from the supramarginal gyrus through the inferior parietal lobule to the superior parietal cortex. The second lesion site mainly impaired the selection and use of common and novel tools. After looking at a subset of patients with deficits in the functional knowledge of tools (but not in mechanical problem solving) Goldenberg and Spatt (2009) found an association of this selective impairment to lesions in the middle temporal gyrus.

The relation of performance in tool use and lesions of patients with left sided brain damage was also analyzed by Hermsdörfer et al. (2013). Next to pantomime and imitation tasks, the correct performance of real tool use was measured and put in relation to the patients' lesions. In this study, low performance was also associated with parts of the inferior frontal gyrus including pars opercularis, triangularis, and insula.

As well as these two studies, which analyzed actual tool use, there are other studies focussing more on the understanding or recognition of actions. Kalénine et al. (2013) distinguish two parts of goal directed actions: action means and action outcome. The first component – dealing with "what" has to be done to achieve a goal (spatiotemporal features of the tool use) and the latter one –

representing the actual outcome of the action (conceptual knowledge). Patients with left sided brain lesions, were asked to evaluate if two actions they saw in a video, were the same or different. These videos differed either in their action means or outcome. The performance of this detection task was combined with information from the patients' lesions, demonstrating a specific relation between lesions in the inferior parietal lobe with action means but not outcome. This underlines previously mentioned findings, stating the relevance of the inferior parietal lobe in processing the knowledge of what has to be done with a certain object or tool to achieve a goal.

The recognition of action related sounds and the execution of these actions was analyzed in a study from Pazzaglia et al. (2008) linking to the conceptual knowledge of tool use. Sounds of buccofacial-related or limb-related actions known from daily life had to be recognized by the patients and also executed. The lesion analysis revealed that impairment of action recognition and execution of buccofacial-related sound was mainly correlated with lesions in the inferior frontal gyrus and insula. Impaired limb-related action recognition and execution on the other side was associated with lesions in the inferior parietal lobe, supramarginal gyrus, angular gyrus, and also the inferior frontal gyrus. A stronger involvement of tool related parietal regions in limb-related action recognition, compared to buccofacial-related actions can be due to the fact that limb-related action sounds and executed actions included more tool actions, than the other condition.

Another lesion analysis including the analysis of actual tool use in patients with left sided brain damage was performed by Randerath et al. (2010). Patients had to grasp a tool and demonstrate its use for various tools with handles oriented toward or away from their body. In this study, the type of grasp (functional or non-functional) and the correct demonstration of tool use were evaluated and correlated with patients' lesions. The main findings related an impaired grip of tools to the lesions in the parietooccipital junction, the angular gyrus, and especially in the inferior frontal gyrus, in particular the pars orbitalis, opercularis and triangularis. An incorrect demonstration of tool use on the other side was most closely linked to lesions in the supramarginal gyrus of the inferior parietal cortex and the gyrus postcentralis. An overlap between impaired grip and incorrect demonstration of tool use was found mainly in the inferior parietal cortex. As discussed by the authors, these findings are in line with the assumptions that the function specific manipulation of tools is mainly processed in the ventro-dorsal part of the dorsal stream (Rizzolatti and Matelli, 2003; for review see Binkofski and Buxbaum, 2012). According to this theory, reaching and grasping movements are related to dorso-dorsal regions like the superior parietal lobe, caudal parts of the intraparietal sulcus, parietal-occipital sulcus and the adjacent parietal-occipital junction (Karnath and Perenin, 2005; Prado et al., 2005). The findings of Randerath et al. (2010) underline the relevance of the parietal-occipital junction for correct grasping, especially for using tools.

To our knowledge, so far, only one study has performed a lesion analysis including sequencing of actions of daily living. In a study of Weiss et al. (2008) patients had to detect sequential and spatial errors in object related actions with or without the object. The main focus of the lesion analysis in this study was sequential error detection in actions without an object (pantomime of action). This analysis revealed that patients with severe problems in recognizing the correct timing sequence of an action had a common lesion site in the left angular gyrus of the parietal lobe.

In summary, the impairment in the recognition or performance of ADL including tool use was reported by many studies to be related to frontal lesions, especially the inferior frontal gyrus, inferior parietal lesions including supramarginal and angular gyrus and the neighboring parieto-occipital junction and lesions in the middle temporal gyrus. An overview of the affected regions and the associated tasks which were impaired, after lesions in these areas, is shown in **Figure 2**. Further evidence of the relevance of these brain regions in apraxia can be derived from lesion analyses focusing on pantomime of tool use. Again the ability to recognize pantomime of daily actions (Kalénine et al., 2010) or the execution of it (Buxbaum and Saffran, 2002; Buxbaum et al., 2005; Goldenberg et al., 2007; Hermsdörfer et al., 2013; Manuel et al., 2013) is strongly related to the already described lesion sites.

Considering the functional imaging studies on tool use and actions of daily living of healthy adults, we see a substantial overlap with the results of the lesion studies. For action sequencing, both imaging studies and lesion analysis show that the left angular gyrus plays a critical role. The conceptual understanding of tool use in ADL, on the other hand, comprises a larger network with core centers in the inferior frontal gyrus, the inferior parietal lobe and middle temporal gyrus. The neuronal processes of the spatiotemporal organization of actions in both healthy adults and

also in patients were related to the posterior part of the parietal lobe including the angular gyrus, the parieto-occipital junction and the inferior frontal gyrus.

# **COMORBIDITY SYMPTOMS**

As mentioned before, AADS syndrome might be enhanced by other comorbidity syndromes following a stroke (Goldenberg, 2013). The research that attempts to link different types of errors to other deficits that are co-morbid to apraxia in the CVA patients is partially unfruitful. One of the problems is that it is difficult to untwine which of the symptoms contribute the most to the difficulty with task execution. Around 30% of ischemic stroke survivors suffer from cognitive impairments apart from the motor disability, affecting speech ability, vision, memory and attention (Katz et al., 1999). For example, Walker et al. (2012) has demonstrated that dressing problems in the right brain damaged patients are mostly attributed to visuospatial deficits. In a similar vein, other studies have reported that visuospatial neglect (impairment of spatial attention) is a stable predictor for the functional outcome of the rehabilitation in the post hospitalization period (Denes et al., 1982; Edmans and Lincoln, 1991; Katz et al., 1999; Jehkonen et al., 2000). Other symptoms, such as hemiparesis, amnesia, visual construction problems and language deficits were reported to lack predictive power (Jehkonen et al., 2000). Importantly, this was contested by research conducted by Wade and Hewer (1987) pinpointing hemianopia (loss of side of visual field) as a second factor for functional recovery in post-acute phase of stroke. More recent work by Paolucci et al. (1998) has stated that absence of neglect is

**FIGURE 2 | Schematic illustration of left hemisphere associations with performance in tool use and ADL based on the reviewed studies; middle frontal gyrus (MFG); inferior frontal gyrus (IFG); inferior parietal lobe (IPL); supramarginal gyrus (SMG); angular gyrus (ANG); parietal-occipital junction (POJ); middle temporal gyrus (MTG).**

the most important prerequisite for the promising prognosis for ADL independence. In addition, Pedersen et al. (1997) identified within the group of neglect patients that anosognosia (compromised self-awareness of own mental and physical state) is in fact a more powerful predictor of recovery in those patients. Therefore, many of the therapeutic approaches are targeted at broadening the visual field in patients suffering from hemianopia or hemineglect, through multisensory stimulations (Làdavas, 2008) or spatiomotor cueing (Kalra et al., 1997). The underlying assumption is that an effective rehabilitation plan needs to incorporate multicomponent factors and, in order to regain independence during ADL, a multifaceted approach is recommended – targeted at different neuropsychological symptoms (Katz et al., 1999). However, until now, there is no conclusive scientific evidence linking the severity of AADS with other neuropsychological symptoms, in particular, neglect. It is however clear that each of these symptoms has its own neural representation and a lesion will affect an aspect of ADL. These considerations reflect the difficulty to define a circumscribed neural network related to ADL. Rather, the network will be widespread with soft boundaries between areas directly and indirectly involved in action planning and tool use.

# **CONSEQUENCES OF APRAXIA AND AADS ON ADL, RECOVERY RATE, REHABILITATION**

Although the incidence of apraxia is relatively high, the common view was that apraxia recovers spontaneously (Basso et al., 1987). However, this outlook is contested by the previous work of Hanna-Pladdy et al. (2003) and Smania et al. (2006) reporting that CVA patients struggle with ADL, due to residual traits of apraxia. Therefore, rehabilitation of apraxia maintains a significant challenge for the clinicians and occupational therapy workers. The research in this matter is inconsistent and limited in comparison to the number of studies investigating behavioral and neural correlates of apraxia (Goldenberg, 2013). According to Buxbaum et al. (2008)the common treatment approach is focused on teaching compensatory techniques for ADL tasks, which allow fostering independence despite the presence of apraxia. This strategy training comprises of the errorless training and high number of repetitions for particular task or verbalisation techniques (Goldenberg, 2013). In errorless approach the therapist guides the patient through the correct sequence of ADL and prevents the occurrence of action errors. In a similar vein, Buxbaum et al. (2008) reported that committing errors during training is disruptive for the outcome of retraining, thus compensatory strategies should be based on errorless approach. Goldenberg et al. (2001) states that intensive training improves specific task performance but cannot be generalized to other activities. In other words, training has to be task specific and does not transfer to other non-trained tasks (Goldenberg and Hagmann, 1998). Interestingly, in this report the majority of patients showed a deterioration of independence during ADL when therapy was withheld (2–5 weeks training period, daily 20–40 min). Exploration training, pointing out critical features of objects, without guidance how to use them did not bring improvement in patients (Goldenberg et al., 2001). Donkervoort et al. (2001) argues that strategy training may bring a short term benefit for patients and improve the global ADL functioning, but is the most effective

in conjunction with standard occupation therapy. In their study intervention was based on verbalisation techniques comprised of providing narrative to guide through the task performance. Furthermore, another approach with evidenced efficacy is based on gesture training, which is more related to pantomime function (Buxbaum et al., 2008). This training is dedicated to practicing gestures associated with tool use. Smania et al. (2000) reported significant reduction in praxis errors and gesture comprehension after 35 training sessions (50 min each). In a subsequent study Smania et al. (2006) showed retention of gains 2 months post treatment after 30 training sessions of the same length as in previous report. In both studies, limited generalization to other tasks was found, but no impact on the overall ADL independence was noted. In addition, the home environment for training was pointed out to be important factor of recovery in 8 week intervention study (Geusgens et al., 2007). Tasks should be important for daily routine and meaningful for the patient. As summarized by Goldenberg (2013) AADS is not a homogenous disorder thus therapy approaches are usually adjusted to the core manifestations. Another aspect is that even if efficacy of training is maintained, it addresses primarily the ability to use compensatory strategies promoting independence during ADL, but does not affect the "concepts of use" (Goldenberg, 2013). Furthermore, the generalisability of training one task to global impact on ADL independence is often not assessed or not found, along with limited evidence for follow-up effectiveness (Maher et al., 1991; Pilgrim and Humphreys, 1994; Ochipa et al., 1995; Goldenberg et al., 2001). Consequently there is lack of clear guidelines what period of time is the optimal for treatment of AADS, which intensity of training is recommended and how to prolong the effects of therapy. Study by Goldenberg and Hagmann (1998) suggests that effects of the intervention can be only sustained if patient continues at home training of ADL independence. Training over the period of a few weeks is feasible if outpatient clinics or day clinics are in place. This, however, is increasingly challenged in the current economic climate, due to restricted funding for the post hospitalization phase. Therefore technology driven solutions might be soon developed to provide continuity for ADL training. In addition, if restoration of the function is impossible, rapidly developing technologies might soon provide a real time "crutch" for independence for stroke survivors. Use of the assistive devices in the home environment could provide additional contextual information for the patient in the ecologically valid setting. Contextual cueing was demonstrated by Maher et al. (1991) to improve performance of a chronic patient with ideokinetic apraxia (case study), within 2 weeks of therapy, based on the shaping (slow withdrawal of cues) paradigm. A similar idea was posited by Buxbaum et al. (2008) discussing the possibility of using robot-assisted devices.

Current projects, which aim to provide autonomous systems of guidance for patients struggling with ADL, are primarily tailored for subjects with dementia and use the concepts of domotics (intelligent home environment). One of the projects currently under development is the COACH system, which is designed to provide assistance in hand washing action to residents of nursing home for people with dementia (Mihailidis et al., 2008). Based on computer vision the system is capable of recognizing problems with task performance. The interface provides prompts based on verbal and visual information, with the prompts adjusted to the needs of patients (for example video or auditory cues). Another project, based on similar type of modeling and solutions is TEBRA, dedicated to aid tooth brushing performance in people with dementia in the home setting (Peters et al., 2013). Finally CogWatch (www.cogwatch.eu) is a system that is currently under development, which is tailored to the needs of AADS patients. The aim is to create fully automatised computer–human interface that provides cues or prompts errors during the performance of ADL (i.e., tea making and tooth brushing). Creating an autonomous system that could aid rehabilitation of ADL in AADS group is a technological and theoretical challenge, which surely will be pursued in the further research developments and projects.

#### **CONCLUSION**

The review summarized the most significant research conducted on the impact of AADS on the ADL in stroke survivors. Behavioral, neuroimaging and lesion studies were presented to give an overview of the current state-of-art. Taken together, CVA resulting in lesions in the left or/and right hemisphere has profound consequences on the daily independence of patients during everyday tasks such as food and drink preparation, grooming, personal hygiene, and use of everyday objects. A new approach was adopted to provide a comprehensive description of the unique features of apraxic and action disorganization disorder. The difficulties with execution of ADL were categorized arbitrarily into three components: problems with sequencing of the multi-step actions, conceptual knowledge about tool use and spatiotemporal aspects of the movement. This classification is novel in comparison to the original descriptions of AADS. However, the aim of this approach was to provide a comprehensive insight into the global picture of difficulties CVA patients might experience. Although these themes were presented separately, the evidence suggests those deficits are often intertwined on the behavioral level and also share the neural substrates. In the neural correlates section of this review, the critical role of the left angular gyrus was pinpointed in the sequencing of the multi-step actions. The neural underpinnings of conceptual knowledge were located in the inferior frontal gyrus, the inferior parietal lobe, and middle temporal gyrus. The spatiotemporal features of the execution of the ADL have been linked to the integrity of posterior part of the parietal lobe including the angular gyrus, the parieto-occipital junction and the inferior frontal gyrus. In addition, other areas that were also identified as linked to the ADL performance were discussed, with a conclusion that a wide neural network is involved in cognitive and motor aspects of action planning and execution. In the final section of this review, a strategy training approach was identified as the most efficient and common therapeutic strategy currently used in the rehabilitation of AADS.

#### **ACKNOWLEDGMENT**

This work was funded by the EU STREP Project CogWatch (FP7- ICT- 288912).

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 31 January 2014; accepted: 04 April 2014; published online: 23 April 2014. Citation: Bie´nkiewicz MMN, Brandi M-L, Goldenberg G, Hughes CML and Hermsdörfer J (2014) The tool in the brain: apraxia in ADL. Behavioral and neurological correlates of apraxia in daily living. Front. Psychol. 5:353. doi: 10.3389/fpsyg.2014.00353 This article was submitted to Cognition, a section of the journal Frontiers in Psychology. Copyright © 2014 Bie´nkiewicz, Brandi, Goldenberg, Hughes and Hermsdörfer. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# The neural basis of human tool use

# *Guy A. Orban1 and Fausto Caruana1,2 \**

*<sup>1</sup> Department of Neuroscience, University of Parma, Parma, Italy*

*<sup>2</sup> Brain Center for Social and Motor Cognition, Italian Institute of Technology, Parma, Italy*

#### *Edited by:*

*François Osiurak, Université de Lyon, France*

#### *Reviewed by:*

*Sarah H. Creem-Regehr, University of Utah, USA Ferdinand Binkofski, University of Luebeck, Germany*

#### *\*Correspondence:*

*Fausto Caruana, Brain Center for Social and Motor Cognition, Italian Institute of Technology, Via Volturno 39, 43100 Parma, Italy e-mail: fausto.caruana@unipr.it*

In this review, we propose that the neural basis for the spontaneous, diversified human tool use is an area devoted to the execution and observation of tool actions, located in the left anterior supramarginal gyrus (aSMG). The aSMG activation elicited by observing tool use is typical of human subjects, as macaques show no similar activation, even after an extensive training to use tools. The execution of tool actions, as well as their observation, requires the convergence upon aSMG of inputs from different parts of the dorsal and ventral visual streams. Non-semantic features of the target object may be provided by the posterior parietal cortex (PPC) for tool-object interaction, paralleling the well-known PPC input to anterior intraparietal (AIP) for hand-object interaction. Semantic information regarding tool identity, and knowledge of the typical manner of handling the tool, could be provided by inferior and middle regions of the temporal lobe. Somatosensory feedback and technical reasoning, as well as motor and intentional constraints also play roles during the planning of tool actions and consequently their signals likewise converge upon aSMG. We further propose that aSMG may have arisen though duplication of monkey AIP and invasion of the duplicate area by afferents from PPC providing distinct signals depending on the kinematics of the manipulative action.This duplication may have occurred when Homo Habilis or Homo Erectus emerged, generating the Oldowan or Acheulean Industrial complexes respectively. Hence tool use may have emerged during hominid evolution between bipedalism and language. We conclude that humans have two parietal systems involved in tool behavior: a biological circuit for grasping objects, including tools, and an artifactual system devoted specifically to tool use. Only the latter allows humans to understand the causal relationship between tool use and obtaining the goal, and is likely to be the basis of all technological developments.

**Keywords: tool use, affordances, mechanical problem solving, anterior supramarginal gyrus, anterior intraparietal sulcus**

# **INTRODUCTION**

The purpose of this short paper is to review the functional magnetic resonance imaging (fMRI) evidence related to the presence in the human brain of a region devoted to tool use lying in the anterior supramarginal gyrus (aSMG) of the left hemisphere, and to describe the properties of this area by integrating our findings with those of other imaging studies. Next, we derive the connections of this region active during the execution and observation of tool action and confront the cognitive operations implied by these results with views of the cognitive processes involved in tool use derived from neuropsychological studies of apraxia. Finally we discuss the emergence of tool use during evolution.

# **THE DIFFERENCE BETWEEN HUMANS AND MONKEYS IN TOOL USE**

Historically, tool use was considered a typically human behavior and the emergence of tool use was considered an important step in the evolution of primates, even serving to delineate the appearance of the genus Homo (Ambrose, 2001). It has, however, become increasingly clear that other animals, particularly chimpanzees and other old and new world monkeys do employ tools (for a review, van Schaik et al., 1999; Baber, 2003). Yet, even if actions using tools are simply compared between humans and apes, it becomes apparent that humans understand the causal relationship between the use of the tool and the results obtained, while this appears not to be the case for chimpanzees (Povinelli et al., 2000). It seems that in other species tool use can be understood by a combination of the affordances provided by the object, which can be manipulated in a tool-like fashion, and associative learning linking the presumptive tool and the result. As pointed out by Osiurak et al. (2010), the differences between animals and humans become even clearer if one makes the comparison both over the lifetime of an individual, in that humans use tools frequently and spontaneously, and at the species level, because all human societies develop technological devices which evolve and improve (Osiurak et al., 2010). Accordingly, it has been recently suggested that the striking differences between human and primates tool use reflect evolutionary discontinuities in hand-eye coordination, causal reasoning, function representation, executive control, social learning, teaching, social intelligence, and language (Vaesen, 2012), thus suggesting important differences with respect to brain structures and functions involved in tool use. The starting point of this review is that the study of the neural basis of tool use has made sufficient progress to begin to understand

why primates such as monkeys can use tools and why the human tool use is so radically different from that of non-human primates.

More than 15 years ago, Iriki et al. (1996) observed that the body scheme could be modified by training macaques to use a rake to retrieve food that was otherwise out of reach. Bimodal neurons in the anterior intraparietal sulcus (IPS; presumably medial bank) having somatosensory receptive field (sRF) and visual receptive field (vRF) representing the finger tips expanded their vRF to include the entire tool after extensive tool use. A similar extension of the properties of the biological effector to the tool was observed in monkey ventral premotor cortex (vPMC) F5. Umiltà et al. (2008) showed that, after extensive training, hand grasping premotor neurons also become active during grasping with ordinary pliers and, more interestingly, with reverse pliers requiring finger extension, rather than flexion, to grasp the object. Such modifications of the neural apparatus involved in planning and controlling object manipulation are likely to underlie the capacity of macaques, and probably apes, to use simple tools, such as the twigs employed in fishing for termites. Some evidence has been obtained for similar changes in humans Maravita and Iriki (2004), and Rushworth et al. (2003) have suggested that such changes may underlie the responses to static tool images reported in left aSMG. Yet, these adaptations are unlikely to explain the causal understanding that humans have of tool use, or the extent of human tool use. In contrast, the sporadic use of tools in animals could be simply explained by such changes in the biological grasping circuit brought about by mere associative learning processes. Typically this animal behavior is based on using objects such as stones or twigs, readily available in the environment. This use may become conditioned by repeated success, including the choice of most appropriate objects. Indeed, in the above mentioned experiment of Umiltà et al. (2008), the use of complex pliers required an extensive training (6–8 months) and was actually based on associative learning, achieved in three subsequent, rewarded, steps: grasping the pliers, opening/closing the pliers and, finally, operating the pliers to grasp food.

Peeters et al. (2009) have more recently provided evidence for a neural substrate involved in tool use that is typically human. These authors compared observation of biological actions such as dragging or grasping, with observations of tool action having similar goals, in both humans and monkeys. They discovered that when human subjects observed tool actions a region in left aSMG was differentially active with respect to static controls, while the same region was not differentially active when subjects simply observed the biological actions, i.e., the factors *tool* and *action* interacted. This finding was very robust as it was observed in nearly 50 subjects, and tested with three different tool actions: using a screwdriver, a rake and pliers, as well as with actions performed by a robotic arm. Most interesting, when performing the same testing in monkeys, Peeters et al. (2009) failed to observe any similar interaction in the monkey inferior parietal lobule (IPL), even after extensive training when the animals had become proficient in using the rake and the pliers. These experiments indicated that the activation of left aSMG by tool action observations was a typically human trait. Since the same aSMG

region has been reported to be activated in humans by pantomiming of tool use and the execution of tool actions (see below for references), Peeters et al. (2009) proposed that human, but not monkey cortex includes a region in the left supramarginal gyrus, devoted to executing and observing tool actions. It should be noted that these results do not exclude the possibility that in monkeys a few scattered neurons in the biological action observation circuit (Nelissen et al., 2011) respond to observation of tool actions. Even then, the results would still imply that in the human case the neurons responding to tool action observation are grouped together and therefore are computationally more powerful than the isolated neurons in the monkey. Indeed, the grouping of neurons with similar function is an important principle of cortical processing, as columnar and topographic organizations demonstrate. The clustering of face selective neurons in the face patches also illustrates the same principle (Tsao and Livingstone, 2008). Importantly, these findings of a left aSMG activation by tool action observation have been replicated in a new group of 12 subjects (Peeters et al., 2013), confirming the robustness of this finding.

In conclusion these studies allow us to understand how on one hand monkeys and apes can become efficient tool users, by modifying their biological manipulative action observation/execution circuit, and why humans use tools so extensively and so proficiently, by possessing a uniquely human cortical region devoted to the observation/execution of tool actions.

# **THE LEFT aSMG REGION DEVOTED TO TOOL USE**

In the original experiments of Peeters et al. (2009), the left aSMG region was defined as the conjunction of all the interactions for observing the different tool actions and the robot hand actions. This conjunction yielded 75 voxels (red outlines in **Figure 1**) located in the anterior tip of left SMG and centered at Talairach coordinates −60, −21, 31. These voxels likely underestimate the region devoted to tool action observation. Indeed, the subsequent study (Peeters et al., 2013) yielded a similar region of interaction for observation of all three tool actions (rake, pliers, screwdriver) combined, but located just to either side of the original 75 voxels (yellow outlines in **Figure 1**). Furthermore, testing the interaction in individual subjects yielded a moderate dispersion of the local maxima, less so in female than in male subjects. Therefore in **Figure 1** we consider the combination of the three components (red and yellow outlines) showing significant interaction in either experiment (Peeters et al., 2009, 2013) as a more realistic estimate of the left aSMG, but these 180+ voxels most likely still remain an underestimate.

The area is located in the anterior part of the crown of the SMG (**Figure 1**), below SI, posterior to gustatory cortex (BA 43), and dorsal to the parietal opercular areas (Eickhoff et al., 2006). Thus any further extension of aSMG is likely to occur in the dorso-caudal direction. Area aSMG largely overlaps with cytoarchitectonic area PFt (Caspers et al., 2008; Peeters et al., 2013), a remarkable match given all the uncertainties of defining each of these functional and anatomical entities in different groups of subjects by very different means, and ensuring that they are properly registered to one another. Using on a tractographybased parcellation of IPL, Mars et al. (2011) assigned the aSMG

**FIGURE 1 | Location of aSMG on the left inflated brain (A) and flatmap (B) of left hemisphere using Caret (Van Essen, 2005).** Yellow and red outlines: aSMG (Peeters et al., 2013); ? in red: area involved in technical reasoning; horizontal white lines : separation between dPMC and vPMC (Tomassini et al., 2007); BA: Brodmann area; dPMC and vPMC dorsal and

ventral premotor cortex; OP: opercular areas, DIPSA, DIPSM, POIPS, VIPS; phAIP and phPFG: putative human homologs of AIP and PFG; pMTG: posterior MTG and MTGt: sematic tool processing in MTG; LOC: lateral occipital complex; LOCo, LOCa, LOCt parts of LOC devoted to objects, action observation, and tools.

region to PFop (Caspers et al., 2008), which they consider distinct from more posterior SMG regions activated during grasping movements. The aSMG region exhibits a considerable anatomical asymmetry with a left hemisphere bias (Van Essen et al., 2012), and the action observation activation is also completely asymmetric, being restricted to the left hemisphere (Peeters et al., 2009). Activation of the aSMG region is also strongly asymmetric in the planning of pantomimes of tool use (Króliczak and Frey, 2009) and its functional connectivity is also asymmetric (Zhang and Li, 2013). This left lateralization only partially reflects the leading role of the right dominant hand in tool manipulation, as these activations have been shown invariant for handedness (Króliczak and Frey, 2009; but see Lewis et al., 2006; Martin et al., 2011). Finally it is worth remembering that deficits in tool handling are a hallmark of apraxia, which is generally associated with left IPL lesions (Goldenberg and Hagmann, 1998). In the opposite hemisphere the symmetrical region is occupied by the higher-order parietal motion area devoted to the extraction of attention-based motion, including long range apparent motion (Claeys et al., 2003).

The aSMG region was originally discovered using movies showing actions performed with tools and compared with similar actions performed with the hand, a paradigm which at that time had never been tested in fMRI. The aSMG overlaps partially with the left lateralized tAIPS region (Mruczek et al., 2013) defined by the subtraction viewing static tools vs. viewing static animals. A long list of imaging studies clearly indicate that the aSMG area is also activated by the sounds produced by tools when used (Lewis et al., 2005), as well as by executing tool actions, imaging or pantomiming tool use or making decisions about tool use (Binkofski et al., 1998; Chao and Martin, 2000; Moll et al., 2000; Okada et al., 2000; Inoue et al., 2001; Rumiati et al., 2004; Bunzeck et al., 2005; Creem-Regehr and Lee, 2005; Johnson-Frey et al., 2005; Valyear et al., 2007; Jacobs et al., 2009; Króliczak and Frey, 2009; Gallivan et al., 2013). The activation during both observation and execution of tool actions suggests that the aSMG may house mirror neurons (Rizzolatti and Craighero, 2004) for tool actions. Finally, the rostral part of SMG is activated when individuals make prehistoric tools or observe their production (Stout and Chaminade, 2007; Stout et al., 2008, 2011). Most of these imaging studies report the activation of several parietal activation sites, in addition to the aSMG. With respect to activation by tool action observation, Peeters et al. (2013) were able to show the uniqueness of aSMG (**Figure 2**). This region was the only parietal region of interest (ROI) out of 11 exhibiting a significant interaction between the factors *action* and *tool*. Indeed the activity profile indicates that, as in the original Peeters et al. (2009) study, the differential activation for observing tool actions significantly exceeded the differential activation for observing biological actions.

Peeters et al. (2013) showed that the left aSMG was activated as much by the observation of rake actions as by the observation of a human hand dragging an object like a rake. Although the activation of the aSMG by rake action observation was relatively weak, the same observation was made with two different groups of subjects. This finding suggests that what is critical is the observation of tool actions are the kinematics of the action, which are very different for tools and biological effectors. It further suggests an important distinction between the activation of the putative human homolog of anterior intraparietal (phAIP) and aSMG during observation of tool actions. Area phAIP is activated by observing the tool being grasped (Jacobs et al., 2009), just as for any other object, explaining why phAIP is also activated when observing biological manipulative actions. On the other hand, aSMG is activated by observing the tool being moved to achieve the goal (picking up or dragging the object toward the actor). One may therefore extrapolate and suggest that during execution phAIP and aSMG play similar roles, with phAIP planning the grasp of the tool and aSMG planning the movement of the tool to obtain the goal.

In conclusionfunctional imaging provides compelling evidence that the aSMG region, localized in the anterior tip of left IPL is specialized for processing of diverse aspects of tool use, using kinematics as the main visual feature of tool action.

# **NEURAL NETWORKS FOR TOOL ACTION EXECUTION AND OBSERVATION**

In this section we propose schemes for the afferent and efferent connections of phAIP (bilaterally, but with left dominance if right-handed subject) and left aSMG, operating in parallel during action execution and observation. These schemes will combine knowledge of connections of monkey cortical regions for which homology is known or plausible, as well as some human connectivity data, which by nature are indirect and must be considered with circumspection.

### **HAND AND TOOL ACTION EXECUTION**

The connections of phAIP and aSMG operating when agents plan for the use of a tool are shown in **Figure 3**. Left phAIP and left aSMG act in parallel, possibly exchanging lateral connections, and send converging signals to vPMC. The parallel nature of this operation needs qualification as only phAIP is active in planning hand actions, while both phAIP and aSMG operate during planning of tool actions. Convergence upon vPMC is supported by the evidence that, in the monkey, the same F5 neurons are active during the last phase of the grasping, i.e., the closure of the effector, regardless of whether the action is performed with the hand, with a tool, or even with a tool requiring an opposite biomechanical movement (finger opening rather than closing) to grasp the object (Umiltà et al., 2008). It is noteworthy that the parallel operation of phAIP and aSMG is a further elaboration of the ventro-dorsal stream (Rizzolatti and Matelli, 2003) of the dorsal visual pathway, or its human homolog. This does not imply that planning tool and hand actions are the only function of the ventro-dorsal stream, or that the present scheme is the only possible elaboration of this stream. Our view is related to those of Daprati and Sirigu (2006) and Binkofski and Buxbaum (2013), who both consider a role for the ventro-dorsal stream in tool use, but also stress the role of the dorso-dorsal stream in planning hand actions toward objects.

The monkey AIP is involved in planning grasping and other manipulative action using visual signals indicating the shape and size of objects; more specifically, this region has a visual component, located in the posterior sector and housing neurons selective

**FIGURE 2 | Specificity of left aSMG amongst parietal regions for tool action observation (from Peeters et al., 2013).** Activity profiles of the 11 ROIs with locations of these ROIs shown in the middle panel. 10 ROIs were defined in left parietal cortex: aSMG, phAIP and DIPSA, DIPSM (Peeters et al., 2009), three tool related ROIs from Johnson-Frey et al. (2005; JF1-3), two from Valyear et al. (2007; Va1-2) and one from Mahon et al. (2007; Ma). The

eleventh ROI, located in the right parietal cortex (Johnson-Frey et al., 2005) is drawn on the symmetrical position in the left hemisphere (JF4). The conditions shown in the activity profiles are tool action observation, static tool, hand action observation, and static hand. Vertical bars indicate SEM. The black asterisks indicate the only ROI in which the interaction between tool and action was significant (*p* < 0.05 corrected for 11 comparisons): aSMG.

for 3D shape from stereo (Srivastava et al., 2009), and a motor component, located in the anterior sector and housing visuomotor and motor neurons (Murata et al., 1996, 2000). Like monkey AIP, its human homolog comprises an anterior sector with motor and visuomotor properties, phAIP, and a posterior visual sector, the dorsal IPS anterior (DIPSA) region, probably equivalent to IPS5 (Mruczek et al., 2013), which, most likely, play similar roles in grasping various objects, including tools. In the monkey, AIP receives from caudal IPS (CIP), that itself receives from V3A (Nakamura et al., 2001). In humans, V7 and its twin area V7A (Georgieva et al., 2009; Kolster et al., 2011), likely equivalent to IPS0-1 (Silver et al., 2005), may correspond in the monkey to the CIP1-2 pair (Arcaro et al., 2011; Janssens et al., 2013). V7 overlaps

heavily with a motion-sensitive area ventral intraparietal sulcus (VIPS; Sunaert et al., 1999) which is incorporated in the present scheme. In humans, the region corresponding to the monkey's V3A may have expanded and may include the four areas described in the V3A complex by Georgieva et al. (2009). Hence we indicate in **Figure 3** input to DIPSA from the V3A complex via VIPS. In the monkey AIP also receives input from the lower bank of superior temporal sulcus (STS; Borra et al., 2008) which supposedly provides input concerning object properties. In deference to the homology between the monkey IT and the human lateral occipital complex (LOC; Denys et al., 2004), we show an input from a subregion of LOC (LOCo), on the ventral occipito-temporal surface, to phAIP. LOC receives from the V4 complex in the ventral pathway.

With regard to aSMG, it has been demonstrated that this region is connected with vPMC and posterior middle temporal gyrus (pMTG; Ramayya et al., 2010). This latter region is presumed to represent the tool use associated motion (MTGt; Chao et al., 1999; Devlin et al., 2002; Johnson-Frey et al., 2005; Mruczek et al., 2013), corresponding to the "law" of the tool, i.e., defining its nature (the target-movement mapping of the tool; Massen and Prinz, 2007). It probably provides the main input to aSMG for its planning function. MTGt itself processes dynamic input (Beauchamp et al., 2002), and is located close to the MT cluster (Kolster et al., 2010), from which it may receive input. MTGt also receives input from the fusiform gyrus representing semantic knowledge about tool properties (LOCt), a fusiform region also projecting directly to aSMG (Mahon et al., 2007). Which exact part of LOC devoted to tools requires further exploration as more lateral regions have also been implicated (Peelen et al., 2013; but see Mruczek et al., 2013). The connection between LOCt and aSMG, thus linking ventral and dorsal streams, is similar in nature to that from LOCo to phAIP. It is unclear to what extent aSMG needs visual input from the ventro-dorsal stream. So far we have assumed that aSMG receives no specific visual input. However, to utilize the tool, it has to be positioned appropriately with respect to the target object (consider a screwdriver and the slot in the screw); in addition once the tool is moving, the target is subject to its influence and will also be moving (in this instance the screw getting deeper). Both aspects, as they relate to the application of the *law* of the tool, may require visual input for proper assessment. Possible sources of such information may be the dorsal IPS medial (DIPSM) or VIPS (**Figure 3**). In this way the aSMG region subserves the tool-object relationship, while phAIP takes care of the tool-actor relationship (Osiurak et al., 2009). Finally, aSMG is likely to receive tactile input indicating whether and how the tool is being held by the hand. These inputs may originate in neighboring SII and reach aSMG directly or indirectly via phAIP. Indeed, the anterior part of monkey AIP receives input from SII (Borra et al., 2008).

# **HAND TOOL ACTION OBSERVATION**

During the observation of tool actions phAIP and aSMG again operate in parallel and again their outputs converge onto vPMC (**Figure 4**). Although the exact homology of vPMC is under investigation (Orban et al., 2012) it likely includes the homolog of monkey F5c, an area which contains the mirror neurons. This convergence is supported by the evidences that, in trained monkeys, the observation of tool use triggers activity in hand grasping mirror neurons (Rochat et al., 2010), even if the firing rate of these neurons was higher during hand grasping observation. Furthermore, the lack of interaction between *tool* and *action* in the studies of Peeters et al. (2009, 2013), confirmed that vPMC is as differentially active for observing biological actions as for tool actions. The most plausible reason of this convergence in vPMC is that, during both hand and tool action observation, the motor cortical areas respond in relation to the goal of the action, which is the same in both types of action, rather than to the movement executed to accomplish the goal (Järveläinen et al., 2004; Cattaneo et al., 2009).

In the monkey the visual signals concerned with grasping observation transit through two parietal stations,AIP and PFG (Nelissen et al., 2011). The homolog of PFG (tentatively labeled phPFG) is still unknown but some recent data (Ferri et al., 2013) suggest that it is located ventrally and slightly caudally from phAIP. The two parietal regions, AIP and PFG, receive input from monkey STS, more precisely with middle superior temporal pole (STPm) in the upper bank feeding into PFG and a region in the rostral lower bank of STS located near that providing object property information, projecting to AIP. We assume the same holds in humans and given the assumed homology (Jastorff and Orban, 2009; Jastorff et al., 2012) of STPm with pMTG and of the rostral lower bank of STS with lateral part of LOC (LOCa; **Figure 1**) we indicate such connections in **Figure 4**. Both regions likely receive input indirectly from the MT cluster, which is very similar in monkeys and humans (Kolster et al., 2009, 2010). In both species this cluster

includes four areas sharing central representations in the center of the cluster.

While it is relatively clear how visual information about observed biological actions reach phAIP, it is less clear how visual signals related to observed tool actions reach aSMG. The most likely origin is from posterior parietal cortex (PPC). One possibility is through DIPSM, another motion-sensitive region which is the homolog of anterior LIP (Durand et al., 2009), also a motionsensitive region in monkeys (Freedman and Assad, 2006; Orban et al., 2006). LIP in the monkey receives from the MT cluster, and DIPSM likely does the same. The alternative is through VIPS, which we believe receives input from human V3A, which is also motion-sensitive (Tootell et al., 1997), unlike it monkey counterpart (Vanduffel et al.,2001). These routes over the PPC may explain why more parietal regions are activated by motion stimuli in humans than in monkeys, whether by 3DSFM motion (Vanduffel et al., 2002) or translation (Orban et al., 2006).

# **RELATIONSHIP WITH NEUROPSYCHOLOGICAL AND COGNITIVE STUDIES**

### **THE RELATIONSHIP BETWEEN TOOL ACTIONS AND AFFORDANCES**

A difficulty in studying brain mechanisms involved in tool use arises in attempting to formalize for scientific purposes the folk category of tool. In fact, each definition of a tool, attempting to distinguish between tool use and other behaviors, has proven elusive and often led to paradoxical conclusions. Hence, many investigations into what would correspond to tool use have generally concluded that this behavioral category is arbitrarily drawn, and that any definition of tool use is one of convenience rather than psychological (Beck, 1980; Preston, 1998; Osiurak et al., 2010). Part of the problem arises from studies in animals, for which the distinction between tool use and other purposeful behavior using natural materials such as branches, e.g., building a nest, is less clear. These difficulties also reflect the fact that humans, who use tools proficiently, use many different types of artifacts with diverse goals, i.e., extending various natural actions, e.g., using a car or a rifle. The solution might be that a tool is any artifact extending the class of manipulative actions, with the understanding that tools will generally be man-made artifacts for humans, but not for animals. Can a spoon or a knife be considered a tool? The answer depends very much on the goal (Osiurak et al., 2010): if they are used to eat, rather than to cook or prepare food, they may not, strictly speaking, belong to the category of tools, if we define tools as artificial implements intended to extend human manipulative capability. The same applies to a toothbrush (Sunderland et al., 2013), which extends the interaction capabilities with the own body and not with external objects. Recent evidence (Ferri et al., 2013) indicates that viewing actions directed to the own body activates different parietal regions from viewing manipulation. While the boundaries of what counts as a tool are still fuzzy, a number of manipulable objects can clearly not be considered tools and we would emphasize the need for greater care in the selection of the appropriate stimuli for studying tool use. In contrast, some studies make the opposite error, considering typical tools such as hammers or saws to be objects (Kalénine et al., 2010), and erroneously labeling the aSMG as a region involved in action understanding in general.

A different sort of inappropriate stimuli concerns the use of static images instead of movies. The dynamical aspects, such as kinematics, are intrinsic characteristics of the tool actions detected by aSMG and for this reason static photographs of a gesture (Osiurak et al., 2009; Bach et al., 2010) are not optimal stimuli for studying this region. Indeed, we used static gestures as control conditions. Finally, tool use implies a goal to be accomplished and for this reason one has to take care how pantomime tests are performed: if they simply imitate tool manipulation without addressing any goal (Osiurak et al., 2009), they may not actually fit the definition of a tool action which, by definition, has a goal. In fact, while little is known about the selectivity of aSMG for the goal of the action, it is well known that other regions involved in tool use show dramatic changes in activity between observing an object-directed tool action and observing the same actions devoid of any goal, i.e., when the object on which the tool operates is lacking (Järveläinen et al., 2004; Cattaneo et al., 2009).

The distinction between planning object-directed actions using the natural effectors and planning actions directed to the same object but using tools is consistent with the view (Osiurak et al., 2009) that affordances apply to tools only insofar as they are objects that can be grasped. Considering only tool affordances to gain insight into the use of tools may be counterproductive, as these characteristics fail to consider the relationship between the tool and the object, or target, of the tool action, e.g., the nail when using a hammer, or the screw when using a screwdriver (Osiurak et al., 2009). Affordances are egocentric, while we need an allocentric framework to plan tool use, particularly that of novel tools (Osiurak, 2013). Hence, it is highly unlikely that only the affordance of tools, even very familiar ones, can explain the repetition suppression effects observed by Valyear et al. (2012) during tool observation. Indeed the suppression effects were induced by repeated visual presentation of a tool, and not by repeating the tool action. Hence these effects are unlikely to track the existence of specialized neuronal mechanisms for the use of these familiar tools, the more that repetition suppression is known to overestimate selectivity (Sawamura et al., 2006).

The relationship between tool use and affordances can be clarified using **Figure 3**. It indicates that the tool action planning network is more complex, as two parietal regions are involved in planning tool use: phAIP to grasp the tool and aSMG to move it according to its nature. Considering the final goal to be reached by the tool action, affordances apply only to the phAIP component of the planning, which is indeed the component common to both objects and tools (**Figure 5**). However, this does not exclude the possibility that aSMG send some biasing signal to phAIP to favor the selection of the affordance that best suits the proper use of the tool. The phAIP affordance component uses the visual analysis of size and shape to plan the appropriate grip aperture, a function commonly associated with the canonical neurons described in monkey AIP and F5, i.e., with the ventro-dorsal stream, unlike what is proposed by Osiurak (2013). That visual analysis generally yields several affordances, one of which has to be selected according to the goal of the action and the agent's intentions, a function in which prefrontal cortex (PFC) afferents to AIP may play a role. As far as tool grasping is concerned, aSMG is the primary candidate region for signaling the most appropriate affordance to phAIP.

This view is supported by the finding that patients with ideomotor apraxia perform more poorly in grasping tools correctly than in grasping abstract objects (Sunderland et al., 2013), thus suggesting that the selection of the *more appropriate* grip of a tool depends on both phAIP coding multiple affordances and aSMG providing knowledge of the *law* of the tool.

# **APRAXIA AND MECHANICAL PROBLEM SOLVING IN TOOL USE**

The left lateralization of the planning of tool use is consistent the observation made repeatedly, that tool use is deficient in apraxia due to left parietal lesions (Goldenberg and Hagmann, 1998). The difficulty has been in appreciating that the various symptoms included in apraxia, typically deficits in imitation of meaningless gestures and in tool use, need not to be necessarily related. Attempts have been made to link these two deficits and find a common underlying factor such as the analysis of spatial relationships (Goldenberg, 2009), but recent studies have challenged this view showing that apraxic patients perform more poorly in tool-related actions than in hand actions, even if the demands of these tasks on postural or spatial representation are identical (Sunderland et al., 2013). The common association of deficits in reproducing meaningless gestures and tool use may simply indicate that the neural mechanisms of these two activities while distinct, occupy neighboring IPL regions. This juxtaposition could explain their common involvement in lesions which are generally due to stroke and typically involve large areas of cortex. More fundamental and productive was the realization that the use of familiar tools which is generally impaired in

apraxia may in fact be dependent on two distinct mechanisms (Goldenberg and Spatt, 2009). One concerns semantic knowledge of the conventional use of familiar tools and the other the inference of function from structure (Goldenberg and Hagmann, 1998; Daprati and Sirigu, 2006), also referred to as mechanical problem solving (Goldenberg and Spatt, 2009) or technical reasoning (Osiurak et al., 2010).

The present review provides clear indications regarding the neural pathways that may underlie semantic knowledge of the conventional use of familiar tools, a circuit which relies on the LOCt and MTGt, two areas providing input to the aSMG. Thus aSMG indeed constitutes the entry point of semantic information into the dorsal pathway, extending what was already known about AIP/phAIP. In contrast, the present review has provided no additional information with regard to the other component of tools use, mechanical problem solving, which can be applied to new tools as well as to non-conventional uses of familiar tools. This type of reasoning refers to the ability to contemplate the abstract principles and mechanics involved in tool use, and is based on mental simulations (Hegarty, 2004) relying on high-level allocentric spatial representations (Osiurak, 2013) and analog processes involved in rule-based reasoning. Importantly, technical reasoning does not require semantic knowledge (Osiurak et al., 2010). Indeed, a decision that the tip of a screwdriver is appropriate for the groove in a particular screw is relatively independent of our semantic knowledge about screwdrivers, in that the relevant information for turning screws is the fit of the tip of one object, whatever its' nature (a screwdriver, a knife, or a coin), into the slot of the other object.

That aspect suggests some interesting parallelism with affordances, which also do not require explicit knowledge of object identity, the main difference being that during tool use the hand-object relationship, typical of affordances, is replaced by the relationship between the tool and its receiver object (Osiurak et al., 2010).

There is now mounting evidence from apraxia studies that mechanical problem solving/technical reasoning must be localized in the left IPL (Goldenberg, 2009; Osiurak et al., 2010). One possibility is that the region located caudal to aSMG, between aSMG and phAIP, extending toward the angular gyrus, is involved in this function (? in **Figure 1**). A possible indication is provided by the original Peeters et al. (2009). In this study, one of the tools, the screwdriver, was used in an unconventional way, to pick up an object. The activation for observing this action extended much further posterior that that evoked by observing the rake or the pliers being used conventionally (compare Figure 2B with Figures 2C,D in Peeters et al., 2009). The mechanical problemsolving function was probably also active in the subjects of Stout et al. (2011), who observed Acheulean tool making, compared to Oldowan tool making. Again, the activation common to all type of subjects (novices, trained, experts) extended further caudally (LM −50, −36, 42) compared to the aSMG as defined in Peeters et al. (2009, 2013). Thus the aSMG (**Figure 5**) would be the entrypoint for not only semantic information into the dorsal stream, but also for the output of technical reasoning, hence the locus of the dialectic as described by Osiurak et al. (2010).

# **THE ORIGIN OF aSMG DURING THE EVOLUTION**

As mentioned earlier the chimpanzee tool use differs fundamentally from that of humans (Osiurak et al., 2010; Vaesen, 2012), depending primarily on a modification of the biological grasping circuit centered on AIP. This is supported by recent evidence that chimpanzees differ as much from humans as macaques do with respect to action observation (Hecht et al., 2013). Hence the question arises as to when tool use based on left aSMG arose during evolution. The first ancestors equipped with this new neural apparatus were most likely Homo Habilis, associated with the Oldowan industrial complex or Homo Erectus, associated with the Acheulean industrial complex. The former dates back 2.5 million years, the second to 1.7 million years before present (Asfaw et al., 1992; Susman, 1994; Roche et al., 1999). Either choice would place the development of tool use, between the emergence of the two other main human traits, bipedalism and language. Bipedalism can be traced back to the Australopithecus, a species much older than Homo Ergaster or Erectus, that emerged about 4 million years ago (Wood and Baker, 2011). However, Homo erectus was probably the first fully fledged biped (Ruff, 2009). Language on the other hand is a much more recent acquisition and may be as recent as 600 thousand years, or less, following the emergence of the premodern homo (Coqueugniot et al., 2004), who was a maker of composite tools.

This timing suggests that these three developments may to some degree be interdependent. Indeed bipedalism frees the hands for manipulative purposes, and must have been an important step toward tool use, insofar as tools are obviously manipulated by the hand. One possibility is that a region such as AIP was duplicated by a prolongation of the cell cycle and this region gradually came to be controlled by afferents carrying tool use related signals. On the other hand, if tool use did indeed precede the emergence of language, it may help understand the typical left lateralization of language. Indeed tool use may in fact have triggered the development of technical reasoning in left IPL, which in turn may have favored a development of language in the left hemisphere. The link between the emergence of tool making and language during evolution has been postulated previously (Stout et al., 2008; Stout and Chaminade, 2012). This view receives support from the involvement of certain IPL regions neighboring those involved in technical reasoning in literacy (Carreiras et al., 2009), the understanding of words, and probably also the planning of speech (Wernicke area or Spt, Hickok and Poeppel, 2007). An evolutionary link is also supported by the modest asymmetry favoring the left hemisphere which has been observed in non-human primates (Joly et al., 2012). It has been argued that the left asymmetry of language is much clearer for execution than for understanding (Hickok and Poeppel, 2007). Execution includes the planning of speech and thus a parietal component, and we may consider Wernicke's area as a region of sensori-motor transformation (Cogan et al., 2014), just as most other PPC regions.

# **CONCLUSION**

A large body of imaging studies implicates the left aSMG region as an area involved in the execution and observation of tool actions. The present review has attempted to make the implications of these findings explicit, particularly with respect to pathways centered on the anterior parietal sulcus regions: phAIP and aSMG, the latter appearing to be a specialization of phAIP for manipulating complex tools. Switching from hand to tool action requires, besides the visual input regarding features of the target object provided by the IPS to phAIP and aSMG (for hand-object and tool-object interaction, respectively), additional information specific to tool use and presumably converging upon aSMG. These afferents include semantic information, which is particularly relevant when using familiar tools, technical reasoning, more crucial during the use of uncommon or new tools, and somatosensory feedback. It follows that the affordances of a tool cannot, by themselves, account for tool use. Furthermore, postural and intentional constraints also play a role during the planning of tool actions and consequently are probably provided by inputs to aSMG, which were not discussed. This connectivity model is clearly more elaborated than the two substreams of the dorsal visual pathway. At present it displays a partial convergence with recent neuropsychological views in so far as the semantic input required for familiar tool use has been identified with some degree of confidence, while the cortical localization of technical reasoning can only be conjectured. These views also provide new support for the link between tool-making and language emergence during evolution.

#### **ACKNOWLEDGMENTS**

This work was supported by ERC Parietalaction to Guy A. Orban.

#### **REFERENCES**

Ambrose, S. H. (2001). Paleolithic technology and human evolution. *Science* 291, 1748–1753. doi: 10.1126/science.1059487


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 17 January 2014; paper pending published: 20 February 2014; accepted: 25 March 2014; published online: 09 April 2014.*

*Citation: Orban GA and Caruana F (2014) The neural basis of human tool use. Front. Psychol. 5:310. doi: 10.3389/fpsyg.2014.00310*

*This article was submitted to Cognition, a section of the journal Frontiers in Psychology. Copyright* © *2014 Orban and Caruana. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

**REVIEW ARTICLE** published: 05 March 2014 doi: 10.3389/fpsyg.2014.00151

# Contribution of the posterior parietal cortex in reaching, grasping, and using objects and tools

# *Guy Vingerhoets\**

*Department of Experimental Psychology, Ghent University, Ghent, Belgium*

#### *Edited by:*

*François Osiurak, Université de Lyon, France*

#### *Reviewed by:*

*Derrick L. Hassert, Trinity Christian College, USA Ferdinand Binkofski, University of Luebeck, Germany*

#### *\*Correspondence:*

*Guy Vingerhoets, Department of Experimental Psychology, Ghent University, Henri Dunantlaan 2, 4th floor, B-9000 Ghent, Belgium e-mail: guy.vingerhoets@ugent.be* Neuropsychological and neuroimaging data suggest a differential contribution of posterior parietal regions during the different components of a transitive gesture. Reaching requires the integration of object location and body position coordinates and reaching tasks elicit bilateral activation in different foci along the intraparietal sulcus. Grasping requires a visuomotor match between the object's shape and the hand's posture. Lesion studies and neuroimaging confirm the importance of the anterior part of the intraparietal sulcus for human grasping. Reaching and grasping reveal bilateral activation that is generally more prominent on the side contralateral to the hand used or the hemifield stimulated. Purposeful behavior with objects and tools can be assessed in a variety of ways, including actual use, pantomimed use, and pure imagery of manipulation. All tasks have been shown to elicit robust activation over the left parietal cortex in neuroimaging, but lesion studies have not always confirmed these findings. Compared to pantomimed or imagined gestures, actual object and tool use typically produces activation over the left primary somatosensory region. Neuroimaging studies on pantomiming or imagery of tool use in healthy volunteers revealed neural responses in possibly separate foci in the left supramarginal gyrus. In sum, the parietal contribution of reaching and grasping of objects seems to depend on a bilateral network of intraparietal foci that appear organized along gradients of sensory and effector preferences. Dorsal and medial parietal cortex appears to contribute to the online monitoring/adjusting of the ongoing prehensile action, whereas the functional use of objects and tools seems to involve the inferior lateral parietal cortex. This functional input reveals a clear left lateralized activation pattern that may be tuned to the integration of acquired knowledge in the planning and guidance of the transitive movement.

**Keywords: parietal cortex, reaching, grasping, tool use, intraparietal sulcus, inferior parietal lobule, dorsal stream, superior parietal lobule**

### **INTRODUCTION**

Despite the fact that genes encode an important deal of the information required by our motor system concerning locomotion, ingestion, and fight-and-flight responses, every individual must learn and remember a great deal of motor information during her or his lifetime. An important part of the human action repertoire that needs to be acquired consists of our remarkable ability to use a wide variety of objects as a means to achieve a diverse amount of goals. This unique quality of object-related (transitive) interaction is particularly developed in humans and involves the exposure to and learning of specific routines to master the correct gestures for functional object use, an ability called praxis.1 The neural basis of tool use is dramatically illustrated by the sudden deficits in the production of learned movements in patients suffering from apraxia following stroke. Tool perception and tool use have received a fair share of attention in recent functional neuroimaging research with paradigms ranging from visual tool perception to actual tool use.

What all of these paradigms seem to have in common is that they elicit robust neural responses in areas of the posterior parietal, premotor, prefrontal, and posterior temporal cortices, and that this pattern of activation is clearly lateralized to the left hemisphere (Johnson-Frey, 2004; Lewis, 2006). The finding that this particular neural activation pattern is triggered by a diversity of tool-related tasks and stimuli underlines the importance of tools for our brain (and species) and also suggests that the neural network of operations underlying tool-related behavior is highly interconnected. The co-activation of distant neural regions during different types of tool-related tasks has obscured a detailed record of the functional role of each of these regions to tool use. In addition, the expanse of the neural response in the parietal, frontal, and temporal lobes has hampered the identification of a mosaic of specialized foci within each region as well as their specific contribution to transitive gestures.

Central to a functional transitive gesture are two other components of upper limb behavior that have been associated with a complex cortical organization, namely reaching and grasping. In contrast to the functional manipulation of objects, reaching, and grasping are readily observed in newborns and improve dramatically through practice within the first year of

<sup>1</sup>Although I realize that the words "object" and "tool" do not convey exactly the same meaning, I will use them interchangeably in the manuscript. In both cases, I consider them as external objects that serve a functional purpose in the transitive action, such as a scissors applied to cut a piece of paper or a pebble thrown to demonstrate my aiming skills.

life. Much of the research on the neural correlates of reaching and grasping has been performed on non-human primates, but the emergence of neuroimaging has allowed a more finegrained study in humans as well. Surprisingly, the scientific study on reaching and grasping and the research on object and tool manipulation have evolved as relatively independent fields with remarkably limited cross referencing in their literatures. Here, I will try to review the major observations on reaching, grasping, and the purposeful use of objects and tools. The focus is on the posterior parietal lobe and the action-related sub-regions within it, and how they contribute to goal-directed visuomotor action.

#### **ANATOMY OF THE POSTERIOR PARIETAL REGION**

Situated between the somatosensory cortex in the postcentral gyrus and the visual cortex in the occipital lobe, the posterior parietal cortex (PPC) is well positioned to bridge visual and somatosensory input and to contribute to the sensory control of action via output to the frontal (pre)motor areas. Anatomically, the lateral part of the PPC is divided in the superior and inferior parietal lobules separated by the intraparietal sulcus (IPS; **Figure 1A**). Anteriorly, the parietal lobules emerge out of the postcentral sulcus (PoCS) and posteriorly the small parieto-occipital sulcus (POS) forms the lateral boundary with the occipital lobe. The superior parietal lobule (SPL) consists of two cytoarchitectonically different regions, a smaller anterior Brodmann area (BA) 5, and a larger posterior BA area 7. These BA areas extend medially into the longitudinal fissure where they give rise to a similar division of the precuneus (PCu), the medial surface of the parietal lobe. The inferior parietal lobule (IPL) also consists of two different cytoarchitectonical regions that by and large correspond to two anatomical structures, namely the supramarginal gyrus (SMG) or BA 40, and posterior to it, the angular gyrus (AG) or BA 39. The IPS separating both lobules, is roughly about 4.5 cm long and ascends anteriorly from the postcentral sulcus (aIPS), runs a horizontal course over its middle segment (mIPS), and then descends caudally [segment called caudal IPS (cIPS)] where at its most posterior end (pIPS) merges with the POS. The IPS is quite deep, in some regions up to more than 3 cm, and a lateral (sometimes called horizontal) and medial bank are distinguished. In order to expose these intraparietal regions that remain concealed in a classical lateral view of the brain I have constructed a schematic image of the PPC (**Figure 1B**).

Although our knowledge on the sensory control of action has benefitted a lot from macaque neurophysiology, anatomical and functional homologies of the primate brain in humans are highly tentative. First, the monkey's parietal lobe is cytoarchitectonically quite different from ours. Macaques do not have BA's 39 and 40, rather their IPL is made up of (subdivided) BA 7, whereas their SPL is BA 5 (**Figure 1D**). In addition, their IPS consists of many specialized regions that appear to be organized differently in human IPS. Second, compared to non-human primates, the magnitude and complexity of human tool use reflects a profound discontinuity between us and our close relatives with regard to the cognitive capacities underlying tool use (Vaesen, 2012). As a consequence of these important differences between species, we will focus on findings from human neuropsychology and neuroimaging, although

we will refer to monkey research when it comes to more basic components of transitive gestures such as reaching and grasping.

# **REACHING**

# **DEFINITION**

Reaching can be described as the transportation of the hand to the object by the upper limb2. Obviously, this requires an integration of the hand and target positions into a single reference frame, thus combining proprioceptive and visual information. A detailed review of the vast literature on the sensorimotor integration of eyehand coordination, gaze modulation, and (near) space coding is beyond the scope of this contribution (Carey et al., 2002; Culham et al., 2008). Instead, we will focus on the parietal correlates of simple reaching tasks in humans.

#### **NEUROPSYCHOLOGICAL RESEARCH**

The classical deficit associated with difficulties in reaching is optic ataxia (OA). Although these patients typically do not exhibit problems when reaching for objects in central vision and show no signs of motor or sensory disturbances, neglect, or apraxia, they are severely impaired when reaching to targets in the peripheral visual field (Perenin andVighetto,1983,1988; Prado et al.,2005). Patients with OA present with reaching errors of their contralesional hand in both visual fields, and also on the presentation of the object in the contralesional hemifield. This hand and field effect acknowledges the problems of visuomotor integration in OA. Recent data of a patient with a selective lesion in left PPC demonstrates the function specificity of OA, showing impairment of reaching (but not grasping) and effector independence (disturbances of arm *and* leg reaches; Cavina-Pratesi et al., 2013). Common lesion sites of patients with OA include the IPS and SPL.Voxel-based lesion function mapping later contradicted the involvement of the SPL proper, and pointed to the parieto-occipital transition zone spanning the IPL, SPL, and PCu border with the superior occipital cortex instead (Karnath and Perenin, 2005).

### **NEUROIMAGING RESEARCH**

Because of the difficulty of studying arm and hand movements within the scanner bore, researchers have turned to different solutions for the viewing and motor limitations of the MR-setting. Instead of true reaching, some studies asked participants to orient the wrist and point with the index finger, thus eliminating the transport component of the reach movement. In addition, target presentation during fMRI is classically achieved by means of back projection through a mirror fixed to the head coil. This arrangement, however, induces a discrepancy between the spatial reference frame of the movement and that of the target, and mirror conditions have been shown to impact neural activations patterns and even behavior (Binkofski et al., 2003). Solutions have been proposed to offer direct viewing of the target that allow for more natural reaching responses (Prado et al., 2005; Beurze et al., 2007).

Blangero et al. (2009) performed a meta-analysis on neuroimaging studies of reaching to identify the relevant parietal foci and to compare these foci with lesion studies from OA patients.

<sup>2</sup>Although one can also reach to a location or to a person, I will focus on reaching toward an external object. Pointing may also encompass a transportation of the hand, but is not considered here.

Using an activation likelihood estimation (ALE) method on thirteen empirical studies they found four bilateral parietal foci along an antero-posterior axis involved in reaching (**Figure 2A**). The most posterior pair is located on the inferior side of the POS, a location also referred to as the parieto-occipital junction (POJ). The second pair is located on the opposite (superior) bank of the POS in posterior IPS (pIPS). The third pair is situated in mIPS, and finally, the last pair is located in aIPS, a region that is often associated with grasping. It is of importance to note that reaching generally elicits bilateral activation along the IPS, but that the hemisphere contralateral to the moving hand is substantially more involved than the ipsilateral hemisphere. In addition, Blangero et al. (2009) were able to show that the more posterior foci (pIPS and POJ) displayed greater lateralization for contralateral visual stimulation, whereas the more anterior foci (aIPS and mIPS) revealed higher lateralization for the use of the contralateral hand,

brain: AG, angular gyrus; aIPS, anterior intraparietal sulcus; cIPS, caudal

thereby demonstrating an antero-posterior gradient of these modules to somatic-to-visual integration (Vesia et al., 2010; Vesia and Crawford, 2012). Finally, lesion overlap of 11 OA patients superimposed on the parietal clusters derived from the meta-analysis showed overlap with the three most posterior foci (Blangero et al., 2009).

intraparietal area; V6, visual area 6; V6A, visual area 6A.

It has proven difficult to disentangle the functional contribution of each of these regions to reaching. A popular option is to compare the functional organization of monkey IPS to that of humans (Culham and Kanwisher, 2001; Grefkes and Fink, 2005; Culham et al., 2006; Vesia and Crawford, 2012). Electrophysiological and anatomical research in the macaque revealed a mosaic of interconnected IPS areas that encode combinations of sensory and effector information that define their functions. Roughly, anterior parts are involved with sensorimotor processing and posterior parts with visual processing. In addition, neurons in the

medial bank respond more to arm movements, and neurons in the lateral bank are more concerned with eye movements. Neurophysiological testing of IPS neurons according to their preferences resulted in the differentiation of several functional regions named after their anatomical location in the anterior (AIP), middle (MIP, LIP), fundus (VIP), and posterior (CIP, V6 complex) portions of the monkey IPS (**Figure 1D**). Two of these areas seem of particular importance for reaching, the V6 complex and MIP. The monkey V6 complex consists of a purely visual V6 area that receives input from early visual areas and sends output to the V6A visuomotor area and MIP. Together with MIP, the V6 complex forms the monkey parietal reach region (PRR) that makes a circuit with macaque area F2, a dorsal premotor region located anterior to primary motor cortex. Similar to the monkey, human pIPS and POJ are on each side of the POS, making this area a likely candidate for the putative human V6 complex (Pitzalis et al., 2013). A recent series of experiments confirmed the specific contribution of the human superior parieto-occipital cortex (SPOC) for visually guided reaching (Quinlan and Culham, 2007; Culham et al., 2008; Filimon et al., 2009; Gallivan et al., 2009; Cavina-Pratesi et al., 2010).

Neuronal discharge in the macaque MIP is dependent on the direction of hand movements toward a visual target and appears involved in the coordination of hand movements and visual targets (Johnson et al., 1996; Eskandar and Assad, 1999; Grefkes and Fink, 2005). In humans, true reaching tasks also show increased neural activity in mIPS and this region is deemed crucial for transforming visual coordinates into motor programs (Grefkes et al., 2004; Grefkes and Fink, 2005; Prado et al., 2005). The association between visual and motor coordinates is in line with the observation that the mIPS is also robustly activated in paradigms requiring visually guided saccadic eye movements and may be involved in the planning of eye movements in relation to the goal that is to be achieved (Beurze et al., 2009; Filimon et al., 2009; Vingerhoets et al., 2010).

Finally, the anterior bilateral IPS pair reported in the Blangero et al. (2009) overview can be associated with monkey AIP. Situated on the lateral bank, monkey AIP contains neurons that are highly selective to the size, shape, and orientation of objects, and are active during fixation and manipulation of objects (Grefkes and Fink, 2005; Culham et al., 2006). Although aIPS is also activated during reaching, it is more active during grasping, and we will discuss the AIP-aIPS comparison in more detail below (Culham et al., 2003).

Whereas most reaching related parietal activation appears to be located bilaterally along the IPS, recent studies have documented activation of superior regions within the PPC. Filimon et al. (2009) and Filimon (2010) reported a reach selective area in the anterior precuneus (aPCu). This medial parietal region was equally active in a reaching-to-target task with the hand visible as in a similar (darkened) task in which the participant could not see his/her hand, suggesting that aPCu is a sensorimotor region whose sensory input is primarily proprioceptive. This finding underlines the observation that there may be multiple reach-related areas within the PPC with greater visual dominance in posterior parts, mixed responses in between, and greater or even exclusive somatosensory dominance in anterior PPC regions (Filimon, 2010). Cavina-Pratesi et al. (2010) reported, besides the already mentioned SPOC, also activation in the rostral SPL during a task that manipulated the transport component by positioning the target in a far or near location. A recent neuroimaging study that directly compared reaching and grasping movements reported a functional gradient of specificity with grasp-specific regions being located in the left anterior IPS extending in the PoCS, regions along the IPS showing activation during reaching *and* grasping movements, and reach- specific activation in the left PCu and SPL (Konen et al., 2013). Together, these data suggest that in comparison to tasks involving object grasping or manipulation (see below), reaching without grasping activates more dorsal and medial parts

of the PPC (Filimon et al., 2007; Cavina-Pratesi et al., 2010; Konen et al., 2013).

#### **OBSERVATION AND IMAGERY OF REACHING**

Only few studies investigated the observation and/or mental simulation of reaching movements. Compared to passive viewing of objects (baseline), observed reaching and imagined reaching activated the IPS, SPL, and PCu (Filimon et al., 2007). Imagined reaching also included SMG activation. In both conditions left hemispheric activations were much stronger as these right handed volunteers observed/imaged the reaching tasks with the right hand. Significant overlap between activations during executed, observed and imagined reaching was found in the left medial IPS and the left SPL extending medially into the superior PCu. The contrast between observed and imagined reaching showed no difference in parietal activation suggesting an equal neural response in observation and imagery of reaching (Filimon et al., 2007).

# **GRASPING**

### **DEFINITION**

Whereas reaching requires correspondence between the spatial location of hand and target, grasping is focused on another visuomotor match, namely the correspondence between the object's form and the hand's posture. Grasping requires the extraction of visual features of an object, such as its size, shape, orientation, texture, and estimated weight in order to properly preshape the hand during the approach, adjust grasping speed during contact, and close the fingers around the object applying the correct grip force. Although reaching and grasping can be distinguished conceptually, in practice they form a continuum as revealed by the kinematics of a reach-and-grasp movement showing adaptation of the grip aperture during the reaching phase of the gesture (Jeannerod et al., 1995; Castiello, 2005). The close relation between the transport and grip components of prehension make it difficult to separate the neural correlates underlying each component. As a result, several areas are activated during reaching *and* grasping, although often a preference in responsiveness to reaching or grasping can be observed.

### **NEUROPSYCHOLOGICAL RESEARCH**

Binkofski et al. (1998) performed a lesion analysis in nine patients with parietal lesions who showed no or only minor visuomotor difficulties and underwent kinematic analysis during reach-andgrasp movements. Patients showing kinematic deficits (*n* = 5) all revealed lesions in the lateral bank of the aIPS, whereas in patients showing normal grasping this region was spared. Kinematics further revealed that the deficit was more pronounced for grasping than for reaching, and especially affected the contralesional hand. To the best of my knowledge, this is the only study that investigated grasping in a clinical population with parietal lesions.

### **NEUROIMAGING RESEARCH**

Similar to reaching, the technical limitations of the scanner have influenced the ecological validity of the grasping paradigms used. As grasping involves objects *and* movement – two well known sources of artifacts during MR-data acquisition – researchers have used pantomimed grasping (no object) or imagined grasping (no object, no movement) instead. Again, these are rather unnatural, or at least uncommon, tasks that question the validity of these paradigms' claims on the neural representation underlying real grasping. Fortunately, methodological and technical solutions have been presented over the last few years that allow a more natural setup within the scanner environment (Culham et al., 2006).

Data from neuroimaging studies have confirmed the importance of the junction between the PoCS and the aIPS for human grasping (**Figure 2B**; Grafton et al., 1996b; Culham et al., 2003; Shikata et al., 2003; Frey et al., 2005; Pierno et al., 2009). The cortex in this region is considered to be part of the IPL. As described above, the region also responds to reaching movements, but its response to grasping is generally stronger (Culham et al., 2003, 2006). Similar to non-human primates, the aIPS is activated by visually guided grasping, object manipulation without vision, and visual inspection without grasping (Culham et al., 2006). The latter effect is only achieved when 3D objects are presented or when 2D pictures of objects with particular hand associations are shown, such as tools (Chao and Martin, 2000; Culham et al., 2003; Creem-Regehr and Lee, 2005). Pure perceptual processing of object features unrelated to grasping does not activate aIPS (Culham et al., 2003). Grasping with either hand evokes bilateral aIPS activity, but the extent and magnitude of the activation is much larger in the aIPS contralateral to the hand used and appears influenced by handedness (Culham and Kanwisher, 2001; Culham et al., 2003; Begliomini et al., 2008). Finally, TMS applied to the left aIPS (but not mIPS or cIPS) disrupts on-line grasping execution (Rice et al., 2006), and selectively results in impaired judgments of tool-related grip configurations in right handers (Andres et al., 2013). World-wide replications of anterior IPS activation during grasping paradigms in humans and macaques result in a growing consensus that human aIPS is the most likely functional equivalent of monkey AIP. In humans, the role of the aIPS region has also been extended to higher-order motor functions as it appeared involved with action planning, recognition of goaldirected hand-object movements, and motor semantics (Shmuelof and Zohary, 2005; Tunik et al., 2005, 2007, 2008b; Hamilton and Grafton, 2006; Ortigue et al., 2009; Vingerhoets et al., 2009a; Cross et al., 2012).

Macaque AIP forms a circuit with macaque F5, the rostral part of the monkey ventral premotor cortex (vPM) which, in turn, projects to the hand region of the primary cortex F1 (Jeannerod et al., 1995). Inactivation of either the monkey AIP or F5 area gives rise to impaired hand shaping relative to the object's features (Gallese et al., 1994; Fogassi et al., 2001). It was suggested that AIP uses visual input to highlight grasp-relevant object features and that F5 uses this information to select the most appropriate grasp. Continuous feedback between both regions monitors the ensuing grasp movement (Fagg and Arbib, 1998). A similar fronto-parietal link has been proposed in humans, linking aIPS with the putative human homologue of monkey F5, the pars opercularis, the posterior part of the inferior frontal gyrus, also known as the vPM or Broca's region (**Figure 1C**). Recent neuroimaging studies have corroborated this idea (Tunik et al., 2005; Cavina-Pratesi et al., 2010; Gallivan et al., 2011; Makuuchi

et al., 2012;Vingerhoets et al., 2013b). As in the monkey F5 region, vPM in humans is modulated by grip type, in particular precision grips (Ehrsson et al., 2000, 2001). But also aIPS shows selective responses to different hand configurations. Multivariate pattern classification analysis of BOLD responses during a rock-paperscissors game was able to accurately classify the pattern of aIPS activity unique to each hand movement (Dinstein et al., 2008). Accurate classification was obtained within modality (either during observation or execution), but not between modalities, leading the authors to suggest that observed and executed movements may be represented by different subpopulations of neurons within aIPS (Dinstein et al., 2008). Although this study investigated hand movements (postures) rather than grasps, these results disclose the central role of aIPS in the perception and execution of hand configurations.

Grasping also elicits activation in other parietal regions besides aIPS. Activation during visually guided grasping was reported in the posterior section of IPS (Culham et al., 2003). In order to explain the pIPS activation during grasping, another comparison with the monkey brain appears relevant. Macaque CIP is situated in the lateral bank of caudal IPS and appears involved in the analysis of object features such as surface texture and orientation (**Figure 1D**). It is believed to analyze the 3D shape and orientation of objects by integrating binocular and monocular depth cues and feed this to the grasping area AIP (Sakata et al., 1998; Tsutsui et al., 2003; Grefkes and Fink, 2005). In humans, caudal activation in the medial bank of IPS was uncovered in a surface orientation discrimination task (Faillenot et al., 1999; Shikata et al., 2001, 2003).

A second parietal region linked with aIPS during grasping lies in SPL (Tunik et al., 2008b). Tunik et al. (2008b) had right handed participants grasp target objects that could or could not undergo rotation after the initiation of the reach and grasp movement. Electrophysiological recordings of evoked brain responses revealed a two-stage process. Response duration in a first stage activated left aIPS region and was longer when there was an object perturbation, whereas initiation of the corrective movement coincided with SPL activity. The authors suggested that aIPS is involved in the initial state activation and the emerging action plan. With increasing discrepancy between the desired and actual state, aIPS activation is prolonged to initiate corrections that are mediated in part by the SPL (Tunik et al., 2008b).

#### **OBSERVATION AND IMAGERY OF GRASPING**

Observed and imagined grasping actions have been studied frequently with neuroimaging as they require no actual movements in the scanner. Based on the temporal coupling between executed and imagined movements a similarity, in neural terms, was expected between the state where an action is simulated and the state of execution of that action (Jeannerod, 2001). In monkeys an extensive overlap of parietal networks activated during grasp execution and grasp observation have been established (Evangeliou et al., 2009). Most studies on imagined grasping in humans indeed reported similar activation of the IPS, SPL, and IPL areas compared to executed grasping (Decety et al., 1994; Grafton et al., 1996a; Binkofski et al., 1999; Grezes and Decety, 2002). Also the observation of grasping actions is believed to elicit the same mechanisms in the

observer's brain that would be activated were that action intended by the observer. This prediction was confirmed by several neuroimaging studies that compared observed versus executed object grasping (Grafton et al., 1996a; Buccino et al., 2001; Grezes et al., 2003a). A recent study required volunteers to judge videos of transitive reach and grasp gestures and decide whether the object was grasped with the intention to use or to displace. Discrimination of action intention during observed grasping revealed bilateral activation of aIPS, mIPS, and cIPS foci suggesting that regions very similar to those involved with executed grasping are recruited by the observer to determine the purpose of the grasp (Vingerhoets et al., 2010). Lateralization of the posterior parietal activation during observed grasping, in particular of the aIPS, appears influenced by the observer's perspective. In a first-person perspective, anatomical congruence is observed showing contralateral activation to the modeled hand. In third-person viewpoint, specular or spatial congruence is seen with parietal activation ipsilateral to the modeled hand (Shmuelof and Zohary, 2008; Vingerhoets et al., 2012b).

#### **INTEGRATION OF REACHING AND GRASPING**

Although the transport and grip components of a transitive gesture can be separated conceptually, in everyday life they present as a single fluid action. Much of the research thus far has strived toward the study of one single component as if reaching and grasping were completely independent. As shown above, supporting evidence from neuropsychology and neuroimaging indeed points to a aIPS – vPM circuit (also termed the dorsolateral circuit) relevant for grasping that can be distinguished from a POTZ (SPOC)/mIPS – dPM circuit (the dorsomedial circuit) underlying reaching (**Figure 1C**). Novel paradigms combining reaching and grasping uncovered brain regions (supplementary motor area, SMA and dPM) that seem to be active during both components and may be relevant for the coordination of reach and grasp (Cavina-Pratesi et al., 2010). In addition and adding to the observations reported above (Tunik et al., 2008b), recent evidence suggests that the aIPS centered dorsolateral circuit and the superior POS centered dorsomedial network appear to specify the same grasping parameters but are temporarily dependent on each other, and thus seem to be organized in a hierarchical manner (Verhagen et al., 2013).

# **USING DEFINITION**

Transitive movements are performed with a purpose. The purpose of the interaction dictates how we will grasp and manipulate an object. This is very obvious when we interact with tool objects. Manipulating a pair of scissors to cut a piece of paper for example, is quite different from the gestures required to move the scissors from the desktop to the drawer. But goal-directed differences are also observed when we interact with objects that have no particular function; the way I will pick up a stick to throw it away for my dog to fetch is different from the movements I use to move the stick out of the way (Ansuini et al., 2006, 2008). Using an object for a particular purpose requires the generation of an action plan. Usually, this plan is already present during the reach and grasp components of the transitive action. Top-down motor planning in grasping is

nicely demonstrated by the end-state comfort effect, the tendency of people to adaptively structure their initial grasp in order to end up with a comfortable posture for the intended action, even when this necessitates them to use an awkward grasp at the start of the movement (Rosenbaum et al., 1992, 1996). When grasping a cup that is upside down, we would use a different grip when we want to pour tea in it compared to placing it in the dishwasher. Behavioral research has provided support for a left hemisphere dominance in the motor planning of end-state comfort effects in right and left handers (Janssen et al., 2011). It remains a matter of debate whether the planning of reach and grasp actions for object use versus object transport are guided by different mechanisms (Osiurak et al., 2008). In addition to reach-and-grasp planning, we also must recall and apply the appropriate object-related movements to achieve the planned goal. Again, the different components of the transitive action, reaching, grasping, and using, are closely intertwined, and difficult to separate in natural action.

Purposeful behavior with objects can be assessed in a variety of ways, of which the *actual use* of tools appears to be the most ecological method. But other approaches have been fruitful too. Clinical work revealed that *pantomiming* tool use is a more sensitive method to elicit symptoms of apraxia, as patients are unable to rely on the physical properties of the tool that may afford toolrelated gestures (Randerath et al., 2011)*. Imaging* the use of tools has been applied to make abstraction of the actual movements and investigate the neural and behavioral correlates of motor imagery. Finally, researchers have also investigated the *receptive*, rather than *productive* aspects of tool use by having participants observe actual or pantomimed tool-related behavior performed by others. Motor imagery and action observation are sometimes referred to as action simulation states or S-states, because they appear to be based on the activation of the brain's motor system, yet in contrast to actual or pantomimed movements, they require no execution of the motor action (Jeannerod, 2001). We will offer an overview of the most relevant findings for each of these tasks below.

# **PRODUCTIVE PARIETAL RESPONSES OF TRANSITIVE GESTURES** *Actual use of objects and tools*

*Neuropsychological research.* Misuse of everyday tools and objects is one of the three categories of symptoms that qualify for the diagnosis of apraxia {the other two being dysfunctions in the imitation of gestures and the production of communicative gestures [symbolic gestures (also called emblems) or pantomimes] respectively; Goldenberg, 2008, 2009}. Apraxia occurs predominantly following left brain lesions and affects both sides of the body, not just the (often hemiplegic) contralesional side. Patients with apraxia may present with multiple or just one of the core symptoms indicating that apraxia is not a unitary disorder and that the different symptoms rely on a (partially) different neural representation. Clinical neuropsychology has traditionally associated limb apraxia with left parietal dysfunction. In particular the left IPL and IPS region are assumed to store knowledge about hand and finger postures/movements required for the use of tools (Sirigu et al., 1995; Haaland et al., 2000; Buxbaum et al., 2003, 2007), but also see (Schnider et al., 1997). Although Goldenberg (2009) claimed that no clear relation between defective actual tool use

and left parietal lesions has been established, other than a number of case studies, his voxel-wise lesion-function study revealed selective impairment on certain tool tasks following parietal damage (Goldenberg and Spatt, 2009). In this study Goldenberg and Spatt (2009) investigated 38 patients with left brain damage on semantic tool knowledge, mechanical problem solving, and use of familiar tools and objects. Parietal lesions, in particular of the IPL and SMG, interfered with the latter two tasks, but not with semantic tool knowledge. The authors concluded that the parietal lobe's role concerns general principles of tool use and comprehension of mechanical interactions, rather than prototypical tool use gestures or the selection of grip formations (Goldenberg and Spatt, 2009). A related observation of a dissociation between functional object knowledge (action semantics) and mechanical problem solving skills has been voiced earlier (Hodges et al., 1999). Patients with semantic dementia and temporal atrophy showed impaired object identification and functional semantics and displayed markedly impaired use of familiar objects, yet retained mechanical problem solving ability as demonstrated in a novel tool task and the correct use of familiar objects with obvious structure-function relationships (Hodges et al., 2000). In contrast, a patient with corticobasal degeneration and biparietal atrophy demonstrated impaired mechanical problem solving and common tool use despite near normal semantic knowledge about the tool's function (Hodges et al., 1999). It appeared to the authors that object-specific conceptual knowledge is crucial for object use, and may be supplemented to some degree by sensory input of object affordances into a parietal "how" system that may trigger mechanical reasoning and the correct use of (some) objects (Hodges et al., 1999, 2000). Later research challenged this view by presenting two patients with degraded semantic knowledge (including functional object knowledge), who showed preserved object use over a twoyear follow-up (Negri et al., 2007). The existence of a separate representation of semantic *and* kinematic/motor knowledge of functional object use in the brain thus remains to be elucidated.

*Neuroimaging research.* Given the limitations for tool interaction in the scanner environment, only a handful of fMRI studies examined actual tool use in humans. Their paradigms compared real tool manipulation against pantomimed or imagined use, or both. In general, the tasks produced widespread activation in parietal, posterior temporal, frontal, and subcortical regions. We will again focus on specific task differences within the parietal region. One study investigated the actual, pantomimed, and imagined right hand use of chop sticks (Imazu et al., 2007). Compared to the pantomimed performance, actual chop stick use showed increased parietal activation in the left PoCG and right IPL (BA 40). Another study compared the actual use of 16 common tools or their imagined use with a control condition without mental task (Higuchi et al., 2007). In the latter two conditions participants were allowed to hold the tools. Actual use revealed unique activity in the left postcentral gyrus and shared activity with the imagery task in left pIPS compared to the control task. A third study compared pantomimed and actual use of 32 familiar objects during a presentation phase, a preparation phase, and an execution phase during which they were either handed the tool for actual use, or were required to pantomime its use (Hermsdörfer et al., 2007). During

the execution phase, actual tool use revealed increased activation in left PoCG, and bilateral SPL and IPL (BA 40).

As expected, all studies report increased activation during actual tool use over the left primary somatosensory region (PoCG, **Figure 3**). Modulation of several bilateral posterior parietal regions is also reported, but there is little consensus regarding a specific location which is probably due to substantial methodological differences between studies. The additional somatosensory modulation during real tool manipulation suggests that the physical demands of the object may serve as cues during actual performance and might explain why apraxic patients perform typically better during actual tool use than during pantomimed tool use (Laimgruber et al., 2005; Hermsdörfer et al., 2006; Randerath et al., 2011).

Another interesting approach compared the neural activation in healthy right handers during the manipulation of a small object with a pair of tongs or with the fingers (Inoue et al., 2001). The PET study revealed that when volunteers used their dominant hand, the left aIPS was activated similarly in the tool and fingers condition, but in the tool condition an additional region in the ipsilateral (right) posterior IPL/IPS became active. The authors interpreted this region to be involved in the integration of visuosomatosensory information during the use of a tool as required during the incorporation of an external object into the body schema (Inoue et al., 2001; Maravita and Iriki, 2004).

# *Pantomimed use of objects and tools*

*Neuropsychological research.* The crucial difference between pantomimed versus actual use of a tool is that the former has to be mentally elaborated and stored in the absence of an external image of the object and the hand acting on it. In other words, it is a creative process that cannot rely on the physical cues provided by the action scene. Movement kinematics of actual versus pantomimed prehensile actions demonstrated qualitative differences between both tasks in apraxic patients *and* healthy controls (Laimgruber et al., 2005; Hermsdörfer et al., 2006). Apparently, patients with apraxia experience difficulties with the absence of the mechanical affordances and constraints of real tools and objects as their performance during naturalistic execution is often superior to their pantomiming of similar actions (Goldenberg and Hagmann, 1998; Buxbaum et al., 2000; Westwood et al., 2001; Goldenberg et al., 2004). Similar to actual tool use, the involvement of parietal lesions with deficits in pantomiming meaningful gestures on verbal command is mainly supported by single case observations, but not corroborated by lesion studies (Goldenberg et al., 2003, 2007; Goldenberg, 2009). The lack of a clear relationship between left parietal lesions and tool use pantomiming, and a somewhat more convincing (though not absolute) relation of parietal damage with deficits in actual tool use and imitation of meaningless gestures, had led Goldenberg to the question of what the latter tasks have in common. Goldenberg proposes that tool use and imitation of meaningless gestures rely on the categorical apprehension of spatial relationships between body parts, tools, and objects. Rather than a repository for the representation of motor acts, the parietal lobe acts to spatially configure multiple (parts of) objects, that may be body parts, external objects, or both (Goldenberg, 2009).

*Neuroimaging research.* In contrast to the limited evidence of a relation between parietal lesions and pantomime dysfunction, most (if not all) of the neuroimaging studies using pantomime paradigms reported robust posterior parietal activation in their healthy subjects. One of the first studies compared tool use pantomimes versus a non-symbolic gesture sequence and found predominant left IPS activation (Moll et al., 2000). Choi et al. (2001) compared left and right hand tool pantomiming against a motor control task and demonstrated dominant left parietal activation with either effector. Tool pantomiming resulted in activation of the SPL (BA 7) and SMG (BA 40), with stronger activation in the former (Choi et al., 2001). Rumiati et al. (2004) used PET in a paradigm that controlled for perceptual, semantic, and sensorimotor aspects to reveal that skilled pantomimes elicited parietal activation in two left IPL foci. The more dorsal and posterior one is particularly responsive to pantomiming triggered by object stimuli, whereas their more ventral IPL focus was also active during imitation of pantomimes and was taken by the authors to be associated with tool grasping (Rumiati et al., 2004). Ordinary tool pantomimes and body-part-as-object gestures showed left SPL (BA 7) and SMG (BA 40) activity irrespective of the hand used in the study of Ohgami et al. (2004). Body-partas-object gestures additionally activated the right SMG (Ohgami et al., 2004). Johnson-Frey et al. (2005) compared the planning and execution of tool use gestures with either hand against a movement control task. For either limb planning tool use pantomimes activated two left parietal foci in the IPL, one more anterior and inferior in SMG, another more superior and posterior in SMG extending to AG (Johnson-Frey et al., 2005). The authors noted that the anterior focus did not match with the putative human AIP coordinates derived from human grasping studies, and suggested that representations of tool manipulation are stored in a separate region, that is near to, but not identical with the area involved in computing sensorimotor transformations during grasping. Johnson-Frey et al. (2005) further hypothesize that their posterior parietal SMG/AG site is involved in the representation of motor programs for acquired tool use skills. Imazu et al. (2007)found increased activity in the left IPL (BA 40) during pantomimed compared to actual chop stick use. ROI-analysis over the left parietal region also revealed an IPL focus of the pantomime versus actual use contrast in the Hermsdörfer et al. (2007) and suggested that this region may be necessary for pantomiming, but unnecessary when sensorimotor feedback is available (Hermsdörfer et al., 2007). Kroliczak and Frey (2009) investigated tool pantomime versus a linguistic control task. Two left IPL foci appeared to be independent of the hand used to prepare the pantomime. A first was positioned dorsal and posterior in the IPS, the second more ventral and anterior in the SMG. Three recent studies byVingerhoets et al. (2011, 2012a, 2013a) investigated pantomiming of familiar tools to explore action semantics and the effects of handedness and atypical language lateralization respectively. In a first study, a distinction was made between execution and planning of tool pantomimes controlled for non tool transitive movements. Execution of familiar tool pantomimes bilaterally modulated SPL and two foci within IPL. The more dorsal focus lies in the superior part of SMG, whereas the ventral focus is located in the part of the SMG that descends into the lateral fissure (Vingerhoets et al., 2011). A hand-independent paradigm comparing tool pantomiming versus control gestures was used in two other studies that evaluated neural lateralization effects due to hand preference and language dominance respectively. Tool pantomimes elicited robust left parietal activation regardless of handedness and hand-effector, although lateralization in the parietal region was stronger in right handers compared to left handers (Vingerhoets et al., 2012a). Typical (left) language dominant volunteers exhibited activation in the left PoCG, PCu, and two SMG foci that appeared switched to homologous regions in the right hemispheres in participants with atypical (right) language dominance (Vingerhoets et al., 2013a).

What most of the studies investigating tool pantomiming in normal volunteers seem to have in common is the activation of one or both of two foci in the left SMG that appear to differ along a superior/inferior and anterior/posterior dimension (**Figure 3**). A summary of the peak coordinates grouped along these dimensions is provided in **Table 1**. We only listed peak voxel coordinates if provided by the studies, and if in the IPL (the SMG coordinate reported by Choi et al., 2001 for example is not even near SMG). MNI coordinates were Lancaster-transformed to Talairach coordinates if necessary and vice versa (Lancaster et al., 2007). Coordinates were ordered along the *Z*-axis with more superior peak voxels on top. If you now look at the *Y*-axis Talairach coordinates, you see that in the upper part of the table most *Y*-coordinates are close to 45 (indicated in blue), whereas in the lower part they are more close to 35 (indicated in green). Two outliers, indicated in red, were not used in calculating the mean coordinates of both foci. The more superior and posterior focus of

**FIGURE 3 | Human parietal foci responsive during different manifestations of tool use.** AG, angular gyrus; iaSMG, inferior anterior supramarginal gyrus; IPS, intraparietal sulcus; PCu, precuneus; PoCG, postcentral gyrus; pSPL, posterior superior parietal lobule; spSMG, superior posterior supramarginal gyrus.

the two (−42, −46, 48, Talairach coordinate) is positioned on the convex portion of the posterior part of the SMG. Its activation has been attributed to the triggering of object-related action schemata in humans (Rumiati et al., 2004), the representation of motor programs for acquired tool use skills (Johnson-Frey et al., 2005), and the production of object manipulation without sensory feedback (Imazu et al., 2007). The more inferior and anterior focus (−53, −33, 31, Talairach coordinate) is located on the ventral part of the SMG were the gyrus descends in the lateral fissure. This focus has been related to tool use and grasping in particular (Rumiati et al., 2004), and appears active during both planned and executed gestures (Johnson-Frey et al., 2005; Hermsdörfer et al., 2007). Despite its anterior position, this anterior SMG focus is not similar to the aIPS, the prototypical region underlying grasp formation (Tunik et al., 2007). Tunik et al. (2007) listed 22 coordinates of human aIPS in grasping studies and calculated the mean coordinates for the left and right hemisphere (−40,−39, 43 on the left, indicated in brown in **Table 1**, and 41, −40, 45 on the right). Clearly, the anterior SMG focus found in tool pantomime studies lies inferior and lateral to this region. In addition, tool pantomime neuroimaging studies show robust left lateralized activation, irrespective of the hand used, which is fundamentally different from the more robust contralateral aIPS activation that depends on the hand performing the grasping movement. Taken together, these results suggest that tool use pantomiming elicits activation in left hemispheric anterior and posterior SMG foci that can be distinguished from the prototypical grasp formation aIPS locus. Given the diverse interpretations, it is difficult to speculate on the role of these different foci. The left anterior SMGfocus may seem to be particularly tuned to the grasping of familiar objects (compared to unknown objects or shapes), but this was not confirmed in a study that explicitly tested this assumption and in fact revealed more posterior SMG activation for such a contrast (Vingerhoets et al., 2009a). The only contrast that demonstrated modulation of the anterior SMG in the latter study was when participants had to imagine displacing familiar versus unfamiliar tools (−58, −23, 33, Talairach coordinate), suggesting that this region may indeed be object-specific, but not particularly tuned to functional prehension (Vingerhoets et al., 2009a). An alternative explanation may be found in a transcranial magnetic stimulation (TMS) study during planning of grasp actions (Tunik et al., 2008a). These researchers administered TMS over left SMG during goal specific grasping movements toward a familiar object (a cup placed upside down) with the intent to use or to move the object. Although no specific site coordinates were provided, the SMG focus illustrated in their figure lies inferior and anterior to the aIPS focus (another TMS target in their study) and thus seems to coincide with a more anterior SMG location. Stimulation to SMG (but not aIPS) during the planning phase of the action significantly delayed the goal-oriented actions, although the execution of arbitrary stimulus-response mappings was not affected. Based on these and previous data, Tunik et al. (2008a) argued that SMG may be involved in goal-oriented formation of plans and selection of actions (planning of actions), whereas aIPS may be responsible for monitoring the fit between hand-object interactions and its intended outcome (guidance of actions). More recently, and partially based on diffusion tensor imaging to identify connections between tool-relevant brain regions, it was


*location. Coordinates*

 *with red markings are not used in averaging. Brown coordinates*

 *refer to an averaged aIPS coordinate. Bold coordinates*

 *reflect mean coordinates*

 *of multiple foci.*

proposed that the anterior SMG is responsible for the integration of non-spatial and semantic information to generate a gesture plan as it appears to show a strong and almost completely left lateralized connection with the posterior middle temporal gyrus, a region that is considered to be a repository of semantic information (Ramayya et al., 2010). Taken together, the anterior SMG region may be involved in goalspecific movement planning toward tool-like objects, but more specific research is needed to corroborate this idea. There is more consensus on the role of the posterior SMG subserving functional motor schemata for familiar objects. The imagined use of familiar tools repeatedly demonstrated increased activation in the left posterior SMG region when compared to imagined displacement of tools (familiar and unfamiliar) or shapes (Vingerhoets et al., 2009a).

Why is this robust left IPL activation during tool use pantomime in healthy participants not reflected in a clear cut relation between parietal lesions and pantomime dysfunction? Goldenberg (2009) suggests that the unusual position and awkward visual and spatial context of the participant during imaging studies may give rise to additional spatial demands that induce this parietal activation. It can be argued that this possible confound would not hold for planned or imagined pantomimes, and that the latter tasks nevertheless reveal significant left IPL activity. An alternative explanation may be that a left parietal lesion often does not suffice to elicit deficient pantomiming. Although the left PPC may be involved in the initiation and control of motor schemata for pantomiming the use of familiar objects, the existence of multiple and potentially redundant left parietal foci, and the modulation of right parietal regions during the pantomiming of familiar and unfamiliar tools hints at the availability of compensatory mechanisms (Vingerhoets et al., 2011). Additional frontal or white matter damage may be required to disrupt the execution of the (sub-optimal) pantomime plan.

### *Imagined use of objects and tools*

Imagined tool use is a strategy used in some neuroimaging studies to avoid possible noise of overt movements in the magnet or to explore the neural correlates of mental imagery. The drawback of imagined gestures is of course a lack of performance and compliance data, although the temporal coupling between imagined and executed movements can be used for a timed estimation of the imagery performance (Vingerhoets et al., 2009a). In addition, it is unclear how "imagined tool use" differs from "planned pantomime," an approach that introduces a delay period between stimulus presentation and the execution command (Johnson-Frey et al., 2005; Kroliczak and Frey, 2009). During the delay period participants are required to prepare the instructed gesture, but it is unclear whether this required keeping the task active in working memory or to imagine its execution. Here, I will focus on studies that explicitly requested imagined tool use.

Imazu et al. (2007) found increased activity in the left IPL (BA 40) during imagined and executed pantomimes compared to actual chop stick use. They suggested that this left IPL focus was involved in the explicit retrieval and production of grasping and manipulation of objects without sensory feedback. Moll et al. (2000) reported IPS patterns of activation during imagined tool-use performance (versus an imagined control motor task) that were identical to those during a similar pantomimed contrast. Vingerhoets et al. (2009a) compared the parietal activation during imagined use versus imagined displacement of the same tools and uncovered activation in left SPL extending to mIPS and aIPS. Unfortunately this study only compared different manipulations of target objects and tasks, it did not include a condition with executed pantomimes. Interestingly, a very strict conjunction analysis aimed to reveal voxels that are activated while using a familiar tool (in imagination) while correcting for differences in object qualities or non-functional aspects of reach-and-grasp movements, detected significant activity on the convex border of the left SPL/SMG in a region that is very close to the posterior SMG focus described in the pantomime section (Vingerhoets et al., 2009a). Again, this finding confirms the involvement of the posterior SMG region for the representation of motor schemata for the functional use of tools. The few studies on imagined (rather than executed) pantomimes seem to indicate that imaging tool use produces activation in the same regions that are active during the real pantomiming of tool use, confirming the close neural match between motor imagery and executed movement (Jeannerod, 2001; Kosslyn et al., 2001).

# **RECEPTIVE PARIETAL RESPONSES OF TRANSITIVE GESTURES** *Observed actual use of objects and tools*

Viewing tool objects facilitates motor responses that are compatible with its manipulation (Humphreys, 2001). The object is believed to possess affordances, properties that are relevant for its use and potentiate associated motor actions (Gibson, 1979). Effects of action priming or motor affordances have been described in particular for physical object properties such as its size or spatial orientation (Tucker and Ellis, 1998, 2001; Phillips and Ward, 2002; Grezes et al., 2003b; Vingerhoets et al., 2009b). As tools, in contrast to most other classes of objects, are able to activate cortical areas associated with motor functions, action priming is believed to result from the neural activation elicited by the tool object that partially overlaps with regions involved with actual tool use (Decety et al., 1997; Chao and Martin, 2000; Grezes and Decety, 2002; Grezes et al., 2003b; Creem-Regehr and Lee, 2005; Lewis, 2006; Vingerhoets, 2008). Here, we will focus on the neural correlates of observed tool use, rather than of static images of tools. A meta-analysis of the neural patterns of execution, simulation of execution, and observed execution of actions revealed clear overlap in SMG and SPL, among other extraparietal motorrelated regions (Grezes and Decety, 2001). Other studies reported strong activation of the aIPS and IPL during the observation of transitive actions (Buccino et al., 2001; Manthey et al., 2003), or overlap in the left IPL in a conjunction analysis of observed and executed transitive actions (Hetu et al., 2011). Peeters et al. (2009) scanned human volunteers, untrained monkeys, and two monkeys trained to use tools, while they observed hand actions and actions performed using simple tools. During tool use observations, human participants exhibited specific activation of a rostral region in left IPL (aSMG) that was not observed in the untrained and trained monkeys. The authors claim that this uniquely human region underlies a specific way of understanding tool actions in terms of causal relationships between the intended use of the tool

and the results obtained by its use, and represents a fundamental evolutionary leap enlarging the motor repertoire of humans (Peeters et al., 2009). Interestingly, the aSMG coordinate reported in this comparative fMRI study (−52, −26, 34, MNI coordinate) is very similar to the mean iaSMG coordinate reported in the tool use pantomime section and **Table 1** (−56, −31, 34, MNI coordinate).

# *Observed pantomimed use of objects and tools*

Halsband et al. (2001) showed patients with left or right parietal or premotor lesions video clips of familiar pantomimed gestures. They were asked to recognize the gestures and subsequently imitate them from memory. The patients showed little problems with gesture comprehension, but the left parietal volunteers were most severely disturbed on imitation performance, especially with gestures on their own body (combing one's hair) rather than with an external object (hammering a nail). In a related study, healthy volunteers observed similar sets of pantomimes while undergoing fMRI (Lotze et al., 2006). aIPS was activated in body-referred and isolated hand pantomimes, whereas left inferior SMG and AG showed a significantly increased response to body-referred pantomimes compared to an isolated hand pantomiming an external object.

# **CONCLUDING REMARKS**

# **PARIETAL REGIONS IMPLICATED IN REACHING, GRASPING, AND TOOL USE**

*Reaching* involves the transportation of the limb effector toward the target, a task that is usually performed under visual guidance and thus requires the integration of visual and proprioceptive coordinates in a network that is able to respond flexibly to changing target positions and effector facilities. Human parietal areas associated with this ability include pIPS and superior PPC areas. pIPS regions appear to deal with the correspondence of visual and motor coordinates, whereas more rostral superior SPL/PCu areas seem to provide input regarding target related proprioceptive information. Both regions are believed to interact during reaches and share specifics of grasps.

*Grasping* is regarded as the act that completes the transitive movement, the merging of hand and object. As in reaching, it is likely to be guided visually and also requires close interplay with proprioceptive information. As a result, there is a substantial overlap in parietal regions subserving reaching and grasping tasks, especially along the IPS. Grasp specific areas include the aIPS and probably also cIPS. The former is a well-established grasping region involved in the perception and execution of prehensile hand configurations and very similar in function to monkey AIP, although in humans its function also appears to encompass goal-directed action planning. cIPS is believed to play a role in prehension-related texture analysis. As in reaching, more superior and medial (SPL) areas are shown to respond when on-line adjustments of the grasping movement seem necessary.

In contrast to reaching and grasping that seem present at birth, the *functional use* of objects and tools requires the recall of learned object interactions. Tool manipulation knowledge can be demonstrated in a variety of ways. In general, neuropsychological and neuroimaging studies agree on a strong left hemispheric lateralization for praxis, although they don't always agree on the key role

of the PPC. Compared to reaching and grasping, the left hemispheric dominance of praxis, regardless of which hand performs the task, underscores the more conceptual level of the mental operations involved. When contrasted to simple motor control tasks, tool use paradigms demonstrate widespread activation along the IPS and adjacent areas. But given its many possible task comparisons tool use paradigms usually explore more fine-grained task differences concerning stimulus type, movement goal, effector choice, assessment method, etc. Subtraction of similar tool use activation patterns reflecting subtle task differences offer detail on the functional role of particular cortical areas, but also filter away most of the basic prehensile PPC activation, and may make us unobservant of its key role in every goal-directed transitive action. This being said, comparison of different tool use tasks reveals that actual tool use is accompanied by activation of the sensorimotor cortex and this might help apraxic patients in the recall of the appropriate tool use gestures. When this proprioceptive feedback is absent, as during tool use pantomiming or tool use imagery, individuals are more reliant on memorized tool interactions, and this appears to elicit neural responses in the IPL. Possibly, multiple IPL foci exist, mediating different types of information of learned transitive movements and interactions. Similar to the reaching and grasping of objects, tool use also seems associated with SPL activation, although it may not be the same SPL regions that contribute to each of the action components. The core regions underlying reach and grasp gestures, however, are organized along the IPS, whereas the core regions subserving functional object use activate the phylogenetic new inferior parietal cortex.

# **THE PUTATIVE ROLE OF THE POSTERIOR PARIETAL REGION**

Traditionally, the parietal cortex is considered as a major component of a dorsal visual pathway (occipital-parietal route) that encodes spatial location ("where" an object is) and can be differentiated from a ventral visual pathway (occipital-temporal route) responsible for object identification ("what" an object is; Ungerleider and Mishkin, 1982). Later, Goodale and Milner (1992) re-interpreted the functional role of the dorsal visual stream from "where" to "how," taking into account its prominent role in the control of skilled motor action.

Both the reaching and grasping literature, and the research on tool use – which seem to have evolved as two relatively separate lines of research – have proposed further subdivisions of the dorsal stream. Based on animal research and clinical data a differentiation of the dorsal stream was proposed into a dorso-dorsal part important for the online control of the transitive action and a ventro-dorsal stream involved with action organization (Tanne-Gariepy et al., 2002; Rizzolatti and Matelli, 2003). In the monkey brain, a dorso-dorsal stream originates from an extrastriate visual node V6, and connects with areas V6A and the medial intraparietal area (MIP) in the medial bank of the IPS (SPL), which is closely linked to the somatosensory system. Its major functional role is described as important for the "on-line" control of action. A ventro-dorsal stream stems from another extrastriate node MT (middle temporal), and connects to the IPL and medial superior temporal area (MST). In addition to action organization, the ventro-dorsal stream is hypothesized to play a role in object awareness, control of hand posture, and action understanding (Rizzolatti and Matelli, 2003). Based on neuropsychological evidence and functional imaging data, it is suggested that a comparable segregation might exist in humans, one for acting on and another for acting with objects, and that hand-object interactions follow different streams dependent on the goal to be achieved (Johnson and Grafton, 2003; Johnson-Frey, 2004; Buxbaum et al., 2006, 2007; Daprati and Sirigu, 2006; Vingerhoets et al., 2009a). If an object is to be moved (acting on), visual information regarding the object's intrinsic (shape and size) and extrinsic (orientation and location) qualities will guide the movement's reach trajectory and grasp aperture. Conceptual knowledge about the target is not required and the movement is guided by an IPS/SPL network. If we wish to use the object (acting with), stored knowledge about its functional properties is required and is integrated with the perceptual affordances of the "on-line" pathway. The conceptual input is believed to rely on IPL structures and guides a functional grasp and purposeful movement with the object.

Similarly, reach and grasp research proposed a distinction between dorsomedial (transport) and dorsolateral (grip) substreams within the parieto-frontal cortex (Grol et al.,2007;Cavina-Pratesi et al., 2010; Vesia and Crawford, 2012). Dorsomedial parietal regions include SPOC, whereas dorsolateral regions include aIPS, although a significant crosstalk between both substreams is expected (Grol et al., 2007).

Both the tool use and the reach and grasp lines of research suggest a division of the dorsal stream in two functionally different substreams based on empirical findings within each research tradition. Although there are clear similarities between the proposals, there are also differences, with the transport/grip distinction focusing on SPL/IPS regions and their frontal projections and the more transitive "use" literature concentrating on the separate contribution of perceptual versus semantic information in action control and the importance of IPL for the latter. A possible integration of both views would be to consider a reach-and-grasp movement as the backbone of every transitive gesture. Such an act requires the integration of visual and somatosensory information that in primates appears to be organized mainly along the IPS in a mosaic of areas that have graded input to the different components (reach or grasp), modalities (visual, somatosensory), and effectors (hand, arm, eye) that contribute to the reach and grasp movement. Its complexity reflects the primate's ability of performing complex transitive actions associated with independent bilateral control over hands with opposable thumb*s* and its cortical network occupies a bilateral IPS territory that bridges incoming visual and somatosensory input. At the same time, the core IPS-centered reach-and-grasp process is supported (and if necessary corrected or adapted) by two different sources of information. The first is predominantly sensory and perceptual in nature and subserves corrections due to more demanding visuo-spatial/tactile-proprioceptive matches. It contributes to the on-line control of the transitive action and is mainly performed in dorsomedial PPC, in particular SPL. The second source of information that contributes to the reach-and-grasp process is more semantic in nature as it relies on functional knowledge about the object, previous experience with the associated actions, and probably also on acquired insight in mechanical relations. The IPL may

not necessarily be the repository of all of this knowledge, but it somehow controls the way in which action semantics influences the transitive gesture. This source of information and its influence on the transport/grip action is clearly lateralized, usually to the left. It is especially well-developed in humans and may constitute a major difference with non-human primate transitive movements. For a proper understanding of the role of each of these processes, it is important to note that a reach and grasp action is not completed once the target object is held. Subsequent object manipulation requires continued adaptations of visuospatial coordinates (transport) and hand posture (grip) in order to carry out the desired transitive movement. Depending on the type and phase of the transitive action, differential input from both information sources is continuously necessary to steer transport and grip components adaptively in a given situation and environment.

# **EFFECTOR-SPECIFIC, SIDE-SPECIFIC, AND ACTION-SPECIFIC PARIETAL MAPPING**

In the paragraphs on reaching and grasping, we already pointed to the lateralized effects of hemi-field presentation (reaching to targets in right or left hemi-space produces more robust contralateral pIPS activation) and effector performance (reaching with the right or left effector evokes more robust contralateral activation in anterior IPS; Blangero et al., 2009). Many findings also have led to the view that PPC is organized in an effector-specific manner, with different subregions mediating movements for hand, arm, and eye respectively. What remains uncertain is the degree of effector and computational specificity of these regions in particular in the human PPC (Vesia and Crawford, 2012). In a review on reach function these authors investigated effector specificity of reach versus saccades and reach versus grasp to come to the conclusion that there is empirical evidence for the existence of effector specificity *and* for a substantial overlap. Increases in the spatial resolution of current neuroimaging techniques appears required to shed more light on the effector specificity of human reach and grasp movements.

In tool use research lateralized effector-specificity seems to be of lesser importance. The side of the hand performing the tool manipulation has little effect on the strong leftward parietal activation, and even left handers show a clear left dominant praxis network (Vingerhoets et al., 2012a). The type of effector used however does seem to elicit some effector-specific mapping in PPC. Buccino et al. (2001) had volunteers observe transitive and intransitive gestures performed by hand, leg, and mouth and their results demonstrated effector specificity in the PPC of both hemispheres. Later research also pointed to the existence of PPC regions that showed overlapping activation during similar actions observed by different effectors, suggesting a form of action mapping that is independent of the effector performing the task (Jastorff et al., 2010; Heed et al., 2011; Lorey et al., 2013). As this research mainly focused on S-states (observation and imagery) and not on actual effector performance, and effector and action specificity appear to depend on the type of S-state applied (Lorey et al., 2013), further research appears necessary. The discovery of action-specific mapping is an intriguing finding that begs the question along which dimensions transitive action-specificity is organized. Given its association with the functional meaning of transitive actions, action mapping regions may be expected to be found in an IPL location and this indeed seems to be the case (Jastorff et al., 2010; Lorey et al., 2013).

# **ACKNOWLEDGMENT**

This study was supported by grant number G.0555.11 attributed to the author by the Fund for Scientific Research – Flanders (www.fwo.be).

# **REFERENCES**


*Hum. Brain Mapp.* 12, 1–19. doi: 10.1002/1097-0193(200101)12:1<1::AID-HBM10>3.0.CO;2-V


resonance during natural action observation. *Neuropsychol. Rehabil.* 22, 752–767. doi: 10.1080/09602011.2012.686885


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 06 December 2013; paper pending published: 07 February 2014; accepted: 07 February 2014; published online: 05 March 2014.*

*Citation:Vingerhoets G (2014) Contribution of the posterior parietal cortex in reaching, grasping, and using objects and tools. Front. Psychol. 5:151. doi: 10.3389/fpsyg.2014. 00151*

*This article was submitted to Cognition, a section of the journal Frontiers in Psychology. Copyright © 2014 Vingerhoets. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

#### *Josselin Baumard1 \*, François Osiurak2 \*, Mathieu Lesourd2 and Didier Le Gall 1,3*

*<sup>1</sup> Laboratoire de Psychologie des Pays de la Loire, Université d'Angers, Angers, France*


#### *Edited by:*

*Cristina Massen, Leibniz Research Centre for Working Environment and Human Factors, Germany*

#### *Reviewed by:*

*Guido Gainotti, Policlinico Gemelli, Italy Tim Vanbellingen, University Hospital Bern, Switzerland*

#### *\*Correspondence:*

*Josselin Baumard, Laboratoire de Psychologie des Pays de la Loire, Maison des Sciences Humaines, Université d'Angers, 5 bis, Boulevard Lavoisier, 49045 Angers, France e-mail: josselin.baumard@ univ-angers.fr; François Osiurak, Laboratoire d'Etude des Mécanismes Cognitifs, Institut de Psychologie, Université Lyon-2, 5 Avenue Pierre Mendès-France, 69676 Bron, France e-mail: francois.osiurak@ univ-lyon2.fr*

# **INTRODUCTION**

Over the past century, a body of evidence has indicated that lesions in the left hemisphere can impair the ability to use tools, hereafter referred to as "apraxia of tool use." Nevertheless, there is neither consensus on the underlying cognitive processes (semantic knowledge about tool function, sensorimotor knowledge about tool manipulation, mechanical knowledge), nor on the way they are assessed (pantomime of tool use, single tool use, real tool use, mechanical problem solving). So, it may be difficult for students and researchers to obtain a comprehensive overview of tool use impairments after left brain damage. The major aim of this paper is to fill this gap by providing a synthesis of experimental results over the last 30 years. This will lead us to address two crucial issues: (1) The role of mechanical knowledge in tool use, which has received growing attention in recent years; (2) The cognitive processes supporting the most widely employed task, namely, pantomime of tool use.

Before discussing these issues directly, let us specify which studies are eligible for inclusion in the present review. Apraxia covers a wide range of disorders (e.g., constructive apraxia, gait apraxia, apraxia of speech, dressing apraxia) as well as several types of gestures (tool use, symbolic and meaningless gestures). However, we will only emphasize tool use impairment. Besides, the historical ideomotor/ideational apraxia dichotomy has been argued to be confusing, reflecting either a task-based or a process-based distinction (e.g., Hermsdörfer et al., 2006;

In this paper we review studies that investigated tool use disorders in left-brain damaged (LBD) patients over the last 30 years. Four tasks are classically used in the field of apraxia: Pantomime of tool use, single tool use, real tool use and mechanical problem solving. Our aim was to address two issues, namely, (1) the role of mechanical knowledge in real tool use and (2) the cognitive mechanisms underlying pantomime of tool use, a task widely employed by clinicians and researchers. To do so, we extracted data from 36 papers and computed the difference between healthy subjects and LBD patients. On the whole, pantomime of tool use is the most difficult task and real tool use is the easiest one. Moreover, associations seem to appear between pantomime of tool use, real tool use and mechanical problem solving. These results suggest that the loss of mechanical knowledge is critical in LBD patients, even if all of those tasks (and particularly pantomime of tool use) might put differential demands on semantic memory and working memory.

**Keywords: apraxia, tool use, pantomime, mechanical problem solving, stroke**

Osiurak et al., 2011; Lesourd et al., 2013a). Therefore, for the sake of clarity, we decided not to use this dissociation to select studies.

### **TOOL USE ASSESSMENT**

Apraxia of tool use can be assessed in at least four ways depending on the amount of information given to patients.

#### **PANTOMIME OF TOOL USE**

Critical to this task is that patients are asked to demonstrate the use of tools *without holding them* in hand. The input modality may vary (visual presentation of the tool, verbal command, imitation) and the examiner may provide more or less information as to the name of the tool, its function, its usual corresponding object1 or the necessity of imagining holding the tool in hand. Imitation tasks can be performed without referring to tool knowledge, as in imitation of meaningless postures (Della Sala et al., 2006; see also Goldenberg, 1995, 1999; Goldenberg and Hagmann, 1997). Therefore, we did not consider results about imitation and we only included studies on pantomime of tool use on visual presentation and/or to verbal command.

<sup>1</sup>We shall use the terms tool and object to refer to the implement performing the action (e.g., screwdriver) and the recipient of the action (e.g., screw), respectively.

# **SINGLE TOOL USE**

Single tool use consists in demonstrating the use of a tool *while holding it* in hand but *without the usual, corresponding object*. Contrary to pantomimes, the tactile input is present, suggesting that patients do not need to form a mental representation of the tool. Additional information may be provided (name of the tool, the action or the goal of the action) but, for our purpose, we did not take these criteria into account.

# **REAL TOOL USE**

In this task, patients are asked to *actually* use tools *with the usual, corresponding object*. We distinguished between two conditions (no-choice versus choice). In the no-choice condition, patients are presented with only the tool and its corresponding object. In the choice condition, several tools and objects are given. Two criteria can be found in the literature, namely, the presence/absence of tools/objects not useful for the action to be done (i.e., distractors) or the presence/absence of a sequence of at least two actions involving more than two tools/objects (i.e., multiple object task). This latter condition can be viewed as a choice condition since each time an action is performed with two tools/objects (e.g., striking the match on the matchbox), the remaining tools/objects (e.g., the candle) become distractors for this specific action.

# **MECHANICAL PROBLEM SOLVING**

These tasks require using novel tools in order to solve an unfamiliar tool use situation (e.g., extracting a target from a box or lifting a cylinder). The solution can be found out from the mere observation of the device, perhaps without adopting trial-anderror strategy. This covers situations wherein familiar tools have to be used in a non-conventional way (e.g., screwing a screw with a knife). As for real tool use, two conditions exist: choice (i.e., selection of the correct tools among an array of novel tools) and no-choice (i.e., only the correct, novel tool is present).

# **THEORETICAL BACKGROUND**

It is commonly assumed that tool use is supported by two systems: The conceptual and the production system. The role of the conceptual system is to form a mental, tool action representation. Three kinds of knowledge have been proposed in the literature. The first one corresponds to semantic knowledge about tool function, which contains information about the usual relationship between a familiar tool and its corresponding object or the context wherein it can be used (e.g., a hammer is commonly used with a nail and can be found in a workshop; Roy and Square, 1985; Rothi et al., 1991; Buxbaum, 2001). In other words, it refers to allocentric relationships (i.e., tool-object), and is associated with left anterior, temporal lobe lesions (Hodges et al., 2000; Goldenberg and Spatt, 2009; Goldenberg, 2013a).

Second, sensorimotor knowledge about tool manipulation comprises information about the movements associated with the usual manipulation of a specific tool (e.g., the use of a hammer requires ample elbow oscillations; Rothi et al., 1991; Buxbaum, 2001). So, contrary to semantic knowledge, sensorimotor knowledge is supposed to encode egocentric relationships (i.e., tooluser). Damage to the left inferior parietal lobe might impair this

Third, mechanical knowledge provides information about relationships between the physical properties of tools and objects (e.g., hammering requires that the hammer is heavier than the nail; Goldenberg and Hagmann, 1998b; Goldenberg and Spatt, 2009; Osiurak et al., 2010, 2011, 2013; Osiurak, 2014). This kind of knowledge refers to allocentric relationships (i.e., tool-object) and might be also supported by the left inferior parietal lobe (Goldenberg and Spatt, 2009).

The role of the production system is to generate a specific movement pattern by taking into account both the environmental constraints and the tool action representation built by the conceptual system (for discussion, see Osiurak, 2013a,b). The dorsal stream would be the neural basis of this production system (Heilman et al., 1986; Buxbaum, 2001; Binkofski and Buxbaum, 2013).

The aforementioned kinds of knowledge have been suggested to be differentially involved depending on the given task (pantomime of tool use, single tool use, real tool use, mechanical problem solving). Special attention has to be paid to pantomime of tool use given that it might be grounded on processes that are not tool-specific. Indeed, the most widespread interpretation of impaired performance in this task stresses damage to sensorimotor knowledge (i.e., the sensorimotor knowledge hypothesis; Heilman et al., 1982; Buxbaum et al., 2005). However, it has also been hypothesized that it is a non-routine, creative task requiring working memory in order to temporarily maintain information about how the tool has to be held in hand and should be used with the corresponding, absent object (i.e., the working memory hypothesis; Roy and Hall, 1992; Bartolo et al., 2003). At last, pantomime of tool use has been assumed to be nothing else but a kind of symbolic gesture (i.e., the symbolic hypothesis; Goldenberg et al., 2003). In this view, the demonstration by pantomime would aim to communicate the idea of the action rather than to attempt to reproduce the gesture strictly speaking. We shall return to these three hypotheses in more detail below.

# **METHODS**

The purpose of the present paper was to review the experimental data published on pantomime of tool use, single tool use, real tool use and mechanical problem solving since 1985 (i.e., the year Roy and Square published the conception-production model). To this end, several databases (i.e., PubMed, ScienceDirect, Eric, Francis, PBSC, Psycarticles, Web of Knowledge) were searched in 2013–2014 for the following keywords: "tool use," "object use," "apraxia," "limb apraxia," "ideational apraxia," "apraxia of tool use," and "stroke," "left brain damage," "left hemisphere."

### **SELECTION OF PAPERS**

Only English language experimental studies were included. They had to meet the following criteria:

(1) *Presence of right-handed patients with lesions confined to the left hemisphere*. Studies were not included if they involved healthy subjects only or if they investigated disconnection syndromes.


Our keywords led us to create a corpus of 176 studies. Only 36 out of 176 studies fitted our criteria (see **Figure 1**). In this pool we counted 59 different tasks, considering that several studies included more than one relevant task. Regarding our criteria, tool use is frequently assessed through pantomime of tool use (25/36, 69%) whereas single tool use (12/36, 33%), real tool use (14/36, 39%) and especially mechanical problem solving (8/36, 22%) were only occasionally investigated over the last 30 years. This can be explained by a lack of consensus in this field (see Dovern et al., 2012).

# **DATA EXTRACTION**

In many papers only *apraxic* left-brain damaged (LBD) patients are included, most often on the basis of imitation or pantomime tasks. However, although some manifestations of apraxia are more prevalent following left rather than right hemisphere lesions (Goldenberg, 2009), this is not the case for real tool use and naturalistic actions (Schwartz et al., 1999; Hartmann et al., 2005; Rumiati, 2005). We did not select these studies because our purpose was to analyze the consequences of left-brain-damage, rather than apraxia, on tool use. Indeed, if we did so, this would have led us to follow a pointless, circular reasoning, namely, apraxic patients are apraxic. Nevertheless, we reviewed these studies if they secondarily included non-apraxic LBD patients. In this case, we calculated the mean performance of apraxic and non-apraxic LBD patients (by taking into consideration, of course, the number of patients in each category). We acknowledge that this may be a bias since it does not display the performances of consecutive patients. However we believe it reflects the state of literature, and it prevented us from eliminating too many relevant studies.

# **DATA ANALYSIS**

In order to make data from these 36 studies comparable, we converted mean performances and standard deviations into percentages. Then, we calculated the mean performance level for each task, weighted by sample sizes. Furthermore, for each study, we computed the difference between controls' scores minus LBD patients' scores (for a similar method, see Lesourd et al., 2013a,b). This procedure appears suitable for several reasons. First, given the low number of studies available and methodological heterogeneity, it was not relevant to conduct a meta-analysis. Second, the performances of control subjects can vary between studies, therefore focusing on differences rather than raw scores avoids a bias when comparing papers. At last, this procedure expresses the severity of the impairment in each task, which is a good way to determine whether different tasks call upon similar or different cognitive mechanisms (e.g., a difference of 50% in one task and 10% in another may lead us to infer divergent cognitive

demands). However, results from this procedure have to be taken with caution in light of the high frequency of ceiling effects in control groups, which can artificially reduce the difference to patients.

# **RESULTS**

#### **COMPARISONS BASED ON WEIGHTED MEANS**

As can be seen in **Table 1**, performance of LBD patients is lower for pantomime of tool use than for single and real tool use (mean scores 66, 78, 84%, respectively; range 34–88, 68–100, 72–100), with the two latter producing similar results at first sight. Actually, in terms of level of performance, pantomime of tool use is closer to mechanical problem solving (mean score 68%, range 30–94), than to other conditions.

### **COMPARISONS BASED ON GROUP DIFFERENCES**

Control-patient differences are presented in **Figure 2** (raw scores are displayed in **Table 1**). Each circle corresponds to one task in one study. The Y-axis displays the distance in percentage between control subjects and LBD patients: The greater the difference, the

#### **Table 1 | Performances of control subjects and LBD patients (mean scores and standard deviations).**


*\*We included this paper although some data have already been published in a larger sample (Hartmann et al., 2005). Bolded values are non-significant differences.*

higher the impairment. Colors were assigned to some circles in order to stress studies in which two, three or four tasks were administered. Gray circles represent studies that investigated only one of the four tasks. Circles in bold are non-significant differences. Circle surfaces express sample sizes and curves were drawn for the only two studies that investigated all of the four tasks (Heilman et al., 1997; Jarry et al., 2013).

At first sight, there is more discrepancy between studies concerning pantomime of tool use, and there is a gradient from pantomime of tool use (mean difference 25%, range 6–52) to real tool use (mean 13%, range 0–28) with single tool use being intermediary (mean 17%, range 0–28). Actually, the first is systematically more difficult than the two latter if we focus on studies in which at least two tasks were administered (i.e., colored circles in **Figure 2**).

Second, this gradient is less obvious between single and real tool use. Although previous studies found no significant association between these tasks (Butler, 2002; Bickerton et al., 2012; see also Riddoch et al., 1989), only two studies investigated both of these tasks in respect of our criteria: According to Jarry et al. (2013), both of them are equally difficult whereas Randerath et al. (2011) found real tool use to be much easier. This gap is most likely due to methodological variations since the latter authors employed a very structured task (i.e., patients were assessed with only two items, they did not have to select tools in real tool use and they were provided with verbal information about the action to be done).

Finally mechanical problem solving appears to be the most difficult task after pantomime of tool use (mean difference 18%, range 6–34). More specifically, the control-patient difference is almost always greater than in single and/or real tool use. Only one out of eight studies reported the opposite finding in LBD patients with posterior lesions only (Goldenberg et al., 2007). Nevertheless, real tool use and mechanical problem solving are assessed with a wide array of tasks. In light of these results, it appeared necessary to control for this methodological discrepancy.

### **EFFECT OF CHOICE AND DISTRACTORS IN REAL TOOL USE AND MECHANICAL PROBLEM SOLVING**

We divided data from real tool use and mechanical problem solving into two categories: In the no-choice condition, patients are presented with only one tool and its corresponding object (e.g., a match and a matchbox) whereas in the choice condition, they are presented with three or more tools/objects (e.g., a match, a matchbox and a candle; also referred as to multiple object tasks). Within this latter condition, studies were also distinguished by the presence/absence of distractors, defined as tools/objects not useful for the task to be done (e.g., a match, a matchbox and a *hammer*). Situations in which at least two tasks are presented simultaneously (e.g., making coffee, fixing a tape recorder) were judged to include distractors since tools that are useful for one task are useless for the other.

As can be seen in **Table 2** and **Figure 3**, only two studies investigated both choice and no choice in both real tool use and mechanical problem solving (Heilman et al., 1997; Jarry et al., 2013). Overall, although mechanical problem solving is more difficult than real tool use, these tasks produce similar results in that reducing the number of tools/objects enhances performances in both conditions. The only study that investigated real tool use (choice) without distractors (Neiman et al., 2000) found similar results.

Finally, as shown in **Table 3**, pantomime of tool use is more difficult than single tool use, which is more difficult than real tool use (no choice), with choice conditions being intermediary between pantomime of tool use and single tool use. Nevertheless, it is noteworthy that even in no choice condition, LBD patients' performance is significantly impaired as compared to controls.

#### **ASSOCIATIONS BETWEEN TASKS**

We intended to determine whether associations can be found between the tasks of interest. However, given that too few studies explored more than one condition, we only described association tendencies. To this end, we displayed control-patient differences from each study in which at least two tasks where investigated, among pantomime of tool use, single tool use, real tool use and mechanical problem solving (see **Figure 4**).

As can be seen, stronger positive associations were found between pantomime of tool use, real tool use and mechanical problem solving than between single tool use, real tool use and mechanical problem solving. A negative association was observed between single tool use and mechanical problem solving, but this observation has to be taken with caution given that it concerned only two studies. Interestingly, a slight impairment in mechanical problem solving coincides with more substantial impairment in pantomime of tool use than in real tool use. Furthermore, there is a positive association between mechanical problem solving and pantomime of tool use.

# **DISCUSSION**

The aim of the present paper was to provide an overview of tool use impairments after left brain damage. More precisely, we shall discuss the role of mechanical knowledge in tool use as well as the cognitive mechanisms supporting pantomime of tool use.

#### **THE ROLE OF MECHANICAL KNOWLEDGE IN TOOL USE**

Three kinds of conceptual knowledge have been proposed to support real tool use: Semantic knowledge about tool function (Roy and Square, 1985; Rothi et al., 1991; Buxbaum, 2001), sensorimotor knowledge about tool manipulation (Rothi et al., 1991; Buxbaum, 2001) and mechanical knowledge about the physical properties of tools and objects (Goldenberg and Hagmann, 1998b; Goldenberg and Spatt, 2009; Osiurak et al., 2010, 2011). We shall address these hypotheses in turn.

Semantic knowledge provides individuals with information about the usual relationship between familiar tools and objects (e.g., a hammer is usually used with a nail). Therefore, it might be required in at least four situations: When matching pictures of tools with the corresponding, usual object (e.g., hammer/nail) or the context in which they can be used (e.g., hammer/workshop); when it is necessary to select tools/objects to be used together; when pantomiming the use of tools; and when performing single tool use. Indeed, given that objects are not present in the two latter situations, access to semantic knowledge is necessary to produce the right conventional action (e.g., hammering is relevant with a nail but not with a shoe). Interestingly, patients with semantic dementia, who have lost semantic knowledge about tools, have been demonstrated to perform better in no-choice situations and mechanical problem solving, suggesting that these tasks put less demands on functional knowledge (Hodges et al., 2000; Bozeat et al., 2002; Silveri and Ciccarelli, 2009).

In our data, LBD patients perform better in real tool use (no choice; mean control-patient difference 8%, range 0–16) than in pantomime of tool use (25%, 6–52) and single tool use (17%, 0– 28). In other words, the more contextual information patients receive, the better they perform. Presumably, this contextual advantage may be a semantic advantage in that the presence of objects in real tool use provides sufficient information and makes retrieval from semantic memory unnecessary. Furthermore, the choice condition of real tool use (18%, 9–28) is more difficult than the no-choice condition of real tool use. These results are consistent with the semantic hypothesis. Nevertheless, patients still perform worse than controls in real tool use (no choice) and mechanical problem solving (18%, 6–34). As a consequence, disruption of semantic knowledge accounts for some, but not all, of tool use impairments. In other words, this kind of knowledge is not sufficient to support tool use (see also Buxbaum et al., 1997).

Sensorimotor knowledge links specific movements to specific tools (e.g., using a hammer requires ample elbow oscillations). Three predictions can be derived from this hypothesis. First, this kind of knowledge should be necessary in any task involving the production of tool-related movements, among which are pantomime of tool use, single tool use and real tool use. Second, choice situations should not be more difficult than no-choice situations because the same movement is required in both cases (e.g., hammering does not vary depending on the number of tools on the desk). Third, the loss of sensorimotor knowledge should not interfere in the use of novel tools, such as in mechanical problem solving.

Our results do not confirm these predictions. Indeed, LBD patients are not impaired to a similar extent in pantomime of tool use (25%), single tool use (17%), and real tool use (no-choice, 8%). Moreover, the choice condition of real tool use is more difficult than the no-choice condition of real tool use even though this dissociation has been assessed in only two studies (Heilman et al., 1997; Jarry et al., 2013) and remains to be confirmed. At last, the sensorimotor hypothesis does not account for impaired performance of LBD patients in mechanical problem solving. On these accounts, experimental data did not prove that apraxia of tool use in LBD patients is due to the loss of sensorimotor knowledge.

Finally, mechanical knowledge about the physical properties of tools and objects (e.g., hammering requires that the hammer is heavier than the nail) may be necessary to use both familiar and novel tools, and might be supported by the left inferior parietal


 **solving.**


*Gray scores indicate the presence of distractors (i.e.,* 

*Bolded values are* 

*non-significant*

 *differences.*

*tools/objects*

 *that are not useful for the task to be done).*

#### **Table 3 | Mean control-patient differences.**


lobe (Goldenberg and Spatt, 2009). So, LBD patients are supposed to be concurrently impaired in both of these tasks.

Overall, our results confirmed this prediction (real tool use, mean control-patient difference 13%; mechanical problem solving: 18%). Moreover, LBD patients are constantly impaired in mechanical problem solving and, in studies that investigated both conditions, failure to solve mechanical problems was systematically associated with failure to use familiar tools (Heilman et al., 1997; Goldenberg and Hagmann, 1998b; Hartmann et al., 2005; Bartolo et al., 2007; Goldenberg et al., 2007; Osiurak et al., 2009; Jarry et al., 2013). Additionally, as shown in **Figure 4**, there is a clear positive association between the two tasks. These results lead us to suggest that mechanical knowledge is necessary to use familiar tools and objects.

To conclude, experimental data obtained over the last 30 years indicate that real tool use might be supported by at least two kinds of knowledge, both of them referring to allocentric relationships: Semantic knowledge about tool function and mechanical knowledge. These two types of knowledge might partially compensate for each other, in that studies on semantic dementia already described impaired use of familiar tools in the context of preserved mechanical problem solving (Hodges et al., 2000; Bozeat et al., 2002; Silveri and Ciccarelli, 2009). However, to our knowledge, this pattern has never been found in LBD patients. On the other hand, the reverse pattern (i.e., impaired, but better performance in real tool use than in mechanical problem solving; see **Table 1**) was frequently observed, suggesting that loss of mechanical knowledge can be partially compensated by intact semantic knowledge although it is critical to account for tool use disorders in LBD patients. We shall now discuss the cognitive processes underlying pantomime of tool use.

#### **THE COGNITIVE PROCESSES SUPPORTING PANTOMIME OF TOOL USE**

Three hypotheses have been proposed to explain the cognitive basis of pantomime of tool use: The sensorimotor knowledge hypothesis (Heilman et al., 1982; Buxbaum et al., 2005), the symbolic hypothesis (Goldenberg et al., 2003) and the working memory hypothesis (Roy and Hall, 1992; Bartolo et al., 2003).

According to the sensorimotor knowledge hypothesis, pantomime of tool use requires individuals to implicitly recover gesture representations that contain invariant, egocentric relationships, and that are specific to particular tools. Therefore, as suggested above, there should be no difference between

**FIGURE 4 | Associations between pantomime of tool use and single tool use, real tool use, and mechanical problem solving (left panel) and between mechanical problem solving and pantomime of tool**

**use, single tool use and real tool use (right panel).** Each point corresponds to control-patient differences. Slopes illustrate the degree of association.

pantomime of tool use, single tool use and real tool use. Indeed, because these representations are egocentric and invariant, the presence/absence of tools and objects should not modify controlpatient differences. However, the present review confirmed that pantomime of tool use is much more difficult than real tool use (see also Riddoch et al., 1989; Roy and Hall, 1992; Butler, 2002; Bartolo et al., 2003; Bickerton et al., 2012). Moreover, pantomime of tool use seems to be poorly associated with single tool use compared with real tool use and even mechanical problem solving (see **Figure 4**). These results thus do not favor the sensorimotor knowledge hypothesis.

The symbolic hypothesis assumes that defective pantomime of tool use is due to asymbolia, that is, a "general inability to express concepts by means of learned signs" (Goldenberg et al., 2003). As an example, drawing from memory implies to select typical features of the object to be drawn (e.g., the shape of both the handle and head of a hammer). Presumably, asymbolia should impair any activity that requires access to semantic memory, such as language, drawing from memory and pantomime of tool use (see Goldenberg, 2013b). Indeed, this hypothesis also presumes that pantomimes are part of communicative gestures in that they require patients to select distinctive features of the sensory appearance of absent tools/objects (e.g., the shape of the handle of a hammer) and to abstract properties that do not contribute to recognizability (e.g., the color or the material of the handle) in order to produce a canonical, recognizable gesture.

So, pantomime of tool use should be more difficult than single tool use since in the latter, patients do not need to communicate the idea of the tool because they already handle it. Our data are consistent with this hypothesis: Pantomime of tool use (mean control-patient difference 25%) and single tool use (17%) appear to be weakly associated and the first is consistently more difficult than the latter over studies. Nevertheless, pantomime of tool use is closer to mechanical problem solving than to single tool use (see **Table 3**, **Figures 3**, **4**) and in previous studies, asymbolia alone could not account for pantomime disturbances in LBD patients (Goldenberg et al., 2003). Further research is thus required on this point.

In line with the working memory hypothesis, pantomiming the use of tools leads individuals to form a mental representation of the tool in hand, the object on the desk and the action to be performed. Once this layout has been imagined, it has to be maintained in working memory until the gesture is finished. This implies that holding a tool in hand and/or seeing the object provides cues, hence reducing the degrees of freedom and so the number of possible errors (Roy and Hall, 1992; Bartolo et al., 2003). As a consequence, the presence of actual tools reduces the load on working memory and enhances performance. The gradient we already described is consistent with this hypothesis.

To sum up, the present review found the working memory hypothesis and, to a lesser extent, the symbolic hypothesis, to be most relevant as regards pantomime of tool use. On the other hand, the sensorimotor knowledge hypothesis remains to be demonstrated. Another key finding is the similar difficulty level and the relationship between mechanical problem solving and pantomime of tool use (see **Table 3** and **Figure 4**). Previous studies reported significant correlations between these tasks (Heilman et al., 1997; Goldenberg and Hagmann, 1998b; Jarry et al., 2013). This finding is not compatible with cognitive models of apraxia (Roy and Square, 1985; Rothi et al., 1991; Buxbaum, 2001) but rather suggests that pantomime of tool use is a composite task that may call for mechanical knowledge, in addition with semantic knowledge and working memory. In fact, this task can be viewed as a kind of problem solving for it may require forming a mental representation through identification and combination of distinctive features of tools and actions (see Goldenberg et al., 2003; Goldenberg, 2009) or, put differently, technical means and technical ends (Osiurak et al., 2010, 2011).

#### **Table 4 | Cognitive demands depending on the task.**


++*High demands on the cognitive process.*

+*Moderate demands.*

−*Low or absent demands.*

Before concluding, let us discuss results indicating differences between choice and no choice conditions. On the whole, the presence of numerous tools seems to be a major obstacle to LBD patients but not to control subjects. Note that although this finding is intuitive, the cognitive models of apraxia (Roy and Square, 1985; Rothi et al., 1991; Buxbaum, 2001) do not address the issue of how humans choose tools and objects. Interestingly, the choice effect is true for familiar as well as novel tools and, as a result, questions the relationship between mechanical knowledge and tool substitutions. Unfortunately, only one of the selected studies investigated real tool use (choice) without distractors (Neiman et al., 2000). Consequently, it remains unknown whether LBD patients fail multiple object tasks because of a planning impairment, interference from distractors or inability to select and combine useful/useless tools. Nevertheless, these results remain to be confirmed because ceiling effects prevented us from computing the real difference between choice and no-choice conditions in control group. Therefore, such a difference among patients could be accounted for by the intrinsic difficulty of choice conditions. Future research is needed to disentangle the origin of the choice effect in LBD patients.

### **CONCLUSION**

To conclude, pantomime of tool use, single tool use, real tool use and mechanical problem solving seem to have at least one cognitive mechanism in common, which may be the ability to retrieve mechanical knowledge on the basis of identification and combination of distinctive features of tools and objects. Nevertheless, each task calls for differential demands depending on presence/absence, familiarity/novelty and number of tools/objects (see **Table 4**). This theoretical distribution challenges the idea that tool use in general, and pantomime of tool use in particular call for sensorimotor knowledge. Note also that data reported here focus on left brain damage but do not exclude a role of the right hemisphere in tool use (Schwartz et al., 1999; Hartmann et al., 2005; Rumiati, 2005). In sum, although apraxia of tool use is classically viewed as a disorder of movement representations/motor control, the present review emphasizes that apraxia of tool use in LBD patients may be first and foremost a cognitive disorder involving the understanding of how tools and objects have to be used together (Osiurak et al., 2010, 2011; Goldenberg, 2013a).

# **ACKNOWLEDGMENTS**

This work was supported by grants from ANR (Agence Nationale pour la Recherche; Project Démences et Utilization d'Outils/Dementia and Tool Use, N◦ANR 2011 MALZ 006 03), and was performed within the framework of the LABEX CORTEX (ANR-11-LABX-0042) of Université de Lyon, within the program "Investissements d'Avenir" (ANR-11- IDEX-0007) operated by the French National Research Agency (ANR).

# **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at: http://www*.*frontiersin*.*org/journal/10*.*3389/fpsyg*.*2014*.* 00473/abstract

# **REFERENCES**


Implications for action representation and apraxia. *Cortex* 49, 184–199. doi: 10.1016/j.cortex.2011.10.010


right hemisphere stroke. *Neuropsychologia* 37, 51–66. doi: 10.1016/S0028- 3932(98)00066-9


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 05 March 2014; accepted: 01 May 2014; published online: 21 May 2014. Citation: Baumard J, Osiurak F, Lesourd M and Le Gall D (2014) Tool use disorders after left brain damage. Front. Psychol. 5:473. doi: 10.3389/fpsyg.2014.00473 This article was submitted to Cognition, a section of the journal Frontiers in Psychology.*

*Copyright © 2014 Baumard, Osiurak, Lesourd and Le Gall. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

<sup>∗</sup>Data displayed in tables and figures were extracted from references preceded by a ∗.

# Tool use ability depends on understanding of functional dynamics and not specific joint contribution profiles

# *Ross Parry1,2\*, Gilles Dietrich3 and Blandine Bril <sup>4</sup>*

*<sup>1</sup> Sorbonne Universités, UPMC Univ Paris 06, UMR\_S 975, CRICM, Paris, France*

*<sup>3</sup> UFR STAPS, Université Paris Descartes, Paris, France*

*<sup>4</sup> Groupe de Recherche Apprentissage et Contexte, Ecole des Hautes Etudes en Sciences Sociales, Paris, France*

#### *Edited by:*

*François Osiurak, Université de Lyon, France*

#### *Reviewed by:*

*François Osiurak, Université de Lyon, France Joachim Hermsdörfer, Technische Universität München, Germany*

#### *\*Correspondence:*

*Ross Parry, Institut du Cerveau et de la Moelle épinière, GH Pitié-Salpêtrière, 47 Bld. de l'Hôpital, Paris 75013, France e-mail: r.e.parry@hotmail.com*

Researchers in cognitive neuroscience have become increasingly interested in how different aspects of tool use are integrated and represented by the brain. Comparatively less attention has been directed toward tool use actions themselves and how effective tool use behaviors are coordinated. In response, we take this opportunity to consider the mechanical principles of tool use actions and their relationship to motor learning. Using kinematic analysis, we examine both functional dynamics and joint contribution profiles of subjects with different levels of experience in a primordial percussive task. Our results show that the ability to successfully produce stone flakes using the Oldowan method did not correspond with any particular joint contribution profile. Rather, expertise in this tool use action was principally associated with the subject's ability to regulate the functional parameters that define the task itself.

**Keywords: tool use, motor learning, motor equivalence, synergy, expertise, mechanical constraints, stone knapping, mechanical reasoning**

# **INTRODUCTION**

The study of human tool use necessitates the observation of interactions with the surrounding environment. Indeed, the very notion of tool use itself implies the appropriation of an object (external to the organism) from the environment. More importantly though, the purpose of tool use generally is to extend one's ability to effect change upon the environment (Leroi-Gourhan, 1964; Baber, 2006). Any instance of tool use behavior should therefore be regarded primarily as a goal directed action and thus can only be effectively evaluated in relation to the demands of the situation or task at hand (Bril et al., 2010; Nonaka et al., 2010).

Determining the efficacy of a tool use action however, is not as simple as it may appear. Like most other motor tasks, an effective tool use action may be generated using a multitude of different postural combinations (refer to **Figure 1**). The question of how the brain goes about choosing one particular movement is a central theme in the study of motor control and is commonly referred to as the motor equivalence problem (Bernstein, 1967). In effect, successful achievement of the desired outcome in any motor problem requires only that the actor satisfy the constraints of the task at hand. It is thus the mechanics of the task that impose the characteristics of the action (Bril et al., 2009, 2010). Adaptive behavior then emerges as the nervous system learns to exploit the mechanical properties that exist in the different bodyenvironment configurations (Bernstein, 1967; Chiel and Beer, 1997).

When compared with other motor tasks though, the distinguishing feature of tool use behavior is the incorporation (by the actor) of an external device in order to mediate the physical interaction constitutive of the goal (Preston, 1998; Bongers et al., 2004; Baber, 2006). It is a permutation that effectively entails several consequences. On one hand, the introduction of the tool adds greater complexity to the existing body-environment system. And beyond the evident addition of the physical characteristics of the tool itself, each and any variation of the tool's relationship to the body and to the environment may have significant repercussions upon task performance (van Leeuwen et al., 1994; Bongers et al., 2004). In purely mechanical terms, introducing a greater number of degrees of freedom to the system necessitates more sophisticated methods of control. On the other hand, modifying the dynamics of the system can afford potential benefits, not the least of which may be greater precision or mechanical advantage. In this respect, tool use may be considered a game of functional dynamics, of learning and mastering a complex system of mechanical conditions in a body-task-environment interaction to the desired effect (Roux et al., 1995; Smitsman, 1997).

The ability to perceive and manage varying complexity of physical interactions is thus fundamental. It defines adaptive tool use behavior. Remarkably though, very few studies of tool use place emphasis upon these phenomena as part of their experimental design, focusing rather on action plans and neural representations. Frequently, studies on tool use behavior have either sought to eliminate the need for subjects to negotiate the physical interactions for which that tool is conceived, (e.g., pantomime, naming, or recognition of tools, observing tool action, imagining tool use) or otherwise significantly reduced the degrees of freedom existing between actor and tool (see among others Choi et al., 2001; Kellenbach et al., 2003; Johnson-Frey et al., 2005; Lewis, 2006; Goldenberg et al., 2007; Stout et al., 2008; Peeters et al., 2009, 2013; Ramayya et al., 2010; Massen and Sattler, 2012).

Importantly, several recent papers have confronted the cognitive processes involved in understanding the physical interactions

*<sup>2</sup> Inserm, U 1127, CRICM, Paris, France*

In these examples, the weight of the box (mass × gravity), indicated by the white arrow remains constant. The force applied by the actor (indicated here by the red arrow) may however vary as conditions change. **(A)** In the example of lifting a box, an individual may use any number of postural combinations. Successfully lifting the box requires only that the person generates enough force to overcome the mass of the box with the combined effect of gravity. **(B)** Upon introduction of a simple tool such as a lever, the dynamics of the task change considerably. **(C)** Adjusting relationships between tool-person-environment configurations may significantly change functional dynamics of the situation. Here this may be due to adjusting the location of the load and pivot, or alternatively by increasing the size of the lever.

involved in tool use. In examples of clinical studies, Goldenberg and Hagmann (1998) and Hodges et al. (1999) proposed that a form of "mechanical reasoning" may support functional tool use, enabling an individual to determine appropriate actions by means of comparison between structural properties of the objects (both tool and target material) with respect to task demands. More recently, Osiurak et al. (2009) expanded upon this work using the notion of "technical reasoning"—a capacity that presents as being distinct from those involved in object representation. In all cases though, the basis of these types of mechanical reasoning processes have been examined in the context of tool selection or by classification of the action demonstrated (e.g., correct/incorrect; object error/action error). As yet these studies have not yet been extended to include the use of quantitative evaluation (by means of kinetic analysis, for example) of subject ability to control the physical interactions critical to the task.

Undoubtedly, these technical and mechanical reasoning frameworks above have proven themselves to be rather informative, most notably in the study of apraxia. Still, the interest of these models has been found primarily in their utility for determining the roles played by the various cognitive processes involved in tool use (Goldenberg and Spatt, 2009; Osiurak et al., 2009). Indeed for the most part in tool use research, the human ability to engage in complex tool use has been perceived predominantly as a function of cognitive capacity. The overwhelming prevalence of research methods focused primarily upon cognitive and cerebral activity does seem to be somewhat at odds with the problematic itself. After all, unlike certain other skills that frequently occur as exclusively internal cognitive processes (e.g., planning, recall, or arithmetic), tool use does not "happen in the brain." The tool use action itself may be seen to embody the actor's capacity to perceive relevant stimulus and coordinate an efficient response with respect to the situation at hand (Preston, 1998; Baber, 2006; Bril et al., 2009). As such, functional approaches to the analysis of tool use behavior may provide particularly rich information regarding the cognitive abilities of actor (Bril et al., 2009).

Accordingly, it is imperative to recognize that that locus of control does not rest exclusively in the brain. Effective tool use necessitates organization across an exceptionally intricate system spanning both the central and peripheral nervous systems. More than just a question of internal representation, adaptive tool use is equally a question of dexterity (Bernstein, 1996). To focus exclusively upon mechanisms for transforming sensory representations of the body and environment into motor programs is thus insufficient for explaining the complexity of tool use behavior. Moreover, the division made between cognitive and motor aspects of performance implied by such methods appears to be more an academic convenience than a physiological reality (Newell, 1991; Summers and Anson, 2009). Indeed, the very notion of the motor program, though an ever-present paradigm in both research and clinical perspectives, is obscure at best and no real consensus exists on whether it should be regarded as a literal or metaphorical concept (Newell, 1991; Morris et al., 1994; Ostry and Feldman, 2003; Latash, 2008a; Summers and Anson, 2009).

The study of motor control thus provides a rather privileged manner for evaluating cognitive and neural bases of tool use. Through the study of motor control, one may effectively see what is controlled in terms of mechanical principles (e.g., velocity, force, energy). Further to this, motor control allows the observer to see how the action is controlled, most commonly in the form of kinematic organization. Looking at a series of tool use actions in this manner provides valuable insight into how the nervous system as a whole prioritizes or controls different aspects of the action. This is, in essence, the same logic used by Bernstein in some of the earliest studies of motor control in tool use (reviewed by Latash, 2000; Biryukova and Bril, 2002). Conducted during the 1920s and at the height of Taylorism, Bernstein's studies had been organized under the direction of the Soviet Ministry for Scientific Labor Organization. Their purpose had been to facilitate the standardization of labor techniques and thereby increase worker efficiency. When analyzing the hammering techniques of expert blacksmith's however, Bernstein made a rather remarkable observation. Although it had been expected that variability in joint contributions would be indicative of poor hammer control, this was not the case. Rather, despite considerable variability of joint angle contributions through the striking arm, expert blacksmiths exhibited minimal variability of the hammer's working point trajectory.

The results of these experiments highlight in a simple yet elegant manner some interesting points regarding expert movement and neural organization. Evidently, in the case of these expert subjects, the nervous system did not seek to exploit any unique movement pattern, a variety of functionally equivalent movements were used to comparable effect. For Bernstein, it seemed unlikely that the brain would specifically prescribe different kinematic and kinetic profiles upon each trial, with individually programmed joint trajectory and muscle activation patterns. He concluded that during these expert movements, the ensemble of joints comprising the multi-segmental effector system was compensating for variability arising from each individual articulation.

Today the terms "synergy" and "coordinative structure" are commonly used to describe this functionally specific organization of neural, muscular, and skeletal elements evoked in Bernstein's observations (Latash, 2008b; Kelso, 2009). It is maintained that the arrangement of motor apparatus in such a way permits the highly flexible and responsive movement characteristic of dexterous tool use. Assembled across the nervous system as the situation or context evolves, the synergy facilitates sensory and mechanical feedback—effectively modulating network activity so that the task specific objectives may be stabilized. This theory that motor control is organized by these coordinative structures is also consistent with physiological literature. For example, it has been demonstrated that descending tract activity is in fact unable to directly prescribe muscle activity in terms of torque or trajectory. The central commands instead appear to regulate postural and movement responses by changing threshold values of muscle length (Matthews, 1959; Ostry and Feldman, 2003; Houk and Rymer, 2011). It has been argued that the existence of synergies is evidenced by the exceptionally rapid adaptation of movement in response to perturbation during goal directed activity (Kelso, 2009). The Uncontrolled Manifold Hypothesis (UCM; Scholz and Schöner, 1999) provides a method of measuring the coordinative structure by separating the movement variability that does not affect the performance outcome (compensated variability) from the movement variability that does compromise one's ability to satisfy the task requirements (non-compensated variability).

Whilst the blacksmiths of Bernstein's early work on percussive tool use indicated that experts tend to exploit the abundant degrees of freedom at their disposal in actual tool use activity, the relationship between functional dynamics and movement variability is less clear during other phases of motor learning. In a recent theoretical article, Latash (2010) described a novel view on stages of motor learning using this principle of synergies. It was proposed that initial stages of learning, obliged the actor to explore functional dynamics of the task at hand to allow for the discovery of effective movement parameters. With increased experience, task performance would then become stabilized as synergies became more robust—thus allowing greater flexibility of movement as the neuromotor system became more adept at regulating the mechanical conditions necessary for successful task performance. Finally, once movement synergies reached a stage where uncompensated variability could not be further reduced, their composition may be altered in order to optimize other relevant factors secondary to task performance, favoring for example, energy conservation or the aesthetic features of movement.

In this paper we use an experimental protocol to explore how movement synergies develop in tool use. This will be done through the observation of coordinative structure at different levels of expertise. We propose here that tool use capacity is first and foremost a learned ability to manipulate the functional dynamics of the task at hand. Given this, our analysis will focus upon subject ability to satisfy task constraints and its relationship to kinematic movement patterns.

The data presented here adds to the existing body of work based around the technique of stone knapping. It is a technique that involves the removal of stone flakes from a flint core and was widely employed by prehistoric man in the production of edged cutting tools. Certain knapping techniques continue to be used today, in the production of architectural flint and artisanal crafts for example. Given that stone knapping provides the earliest known evidence of human tool use and tool production, it is often considered to reflect both cognitive and manual skills that distinguish human tool use abilities from those of other species. Previous studies have linked stone knapping to the evolution of anatomical and biomechanical properties of the upper limb (Marzke and Marzke, 2000; Rolian et al., 2011; Williams et al., 2012); the expansion of cortico-cerebellar circuitry supporting motor control (Bril et al., 2012) and the acquisition of the cognitive capacities supporting language and communication (Toth et al., 1993; Stout et al., 2008; Stout and Chaminade, 2012; Uomini and Meyer, 2013).

In certain respects, the removal of a stone flake is similar to other percussive tasks such as driving a nail into wood, hitting a golf ball, or breaking the hard shell of a nut. All require the use of forceful, striking movements to achieve the goal at hand. What distinguishes one of these tasks from any other though are the objectives of the activity, the materials involved and the functional dynamics of each situation. Here we use the framework developed by Bril and colleagues (Bril et al., 2009, 2010, 2012; Nonaka et al., 2010; Rein et al., 2013) in their previous studies on the mastery of percussive techniques as a practical framework for defining the characteristics of the task and the performance of the actor (see **Figure 2**).

In this model, the task constraints are the conditions necessary to effectuate the desired goal. To satisfy task constraints, the actor must generate specific values of functional parameters (kinetic energy, angle of blow, and point of percussion). The actor may do this by using any one of a variety of mutually dependent combinations of control parameters (hammer mass, velocity at impact). In turn, a multitude of potentially valid strategies (potential energy, trajectory, muscular effort) are at the disposal of the actor as he attempts to regulate the relationship between these combinations of control parameters. Finally, given that the number of degrees of freedom defining the task constraints is fewer than the number found in the multi-segmental effector system, there exist an infinite number of combinations of movement parameters (kinetics, kinematics, muscle control) that could serve as valid motor

solutions. This framework facilitates the study of percussive tasks by highlighting interplay between the complexity of a percussive task, the strategy chosen by the actor, and the coordination of the movement during the performance of the action.

As with other fine grain materials such as glass, cornelian and quartz, the intentional shaping, or reduction or a flint core is made possible through conchoidal fracture (Roux et al., 1995). Successfully removing a stone flake by this action is dependent upon relationships between several variables; the external platform angle, the point of percussion, the angle of the blow and the kinetic energy delivered to the point of impact (see **Figure 3**). The conchoidal fracture is contrasted with "split breaking," which can occur independently of these other variables upon the application of a sufficiently large force. The removal of flakes by split breaking offers very limited control over the form and number of flakes produced (see Pelegrin, 2005; Bril et al., 2012 for further discussion).

In practice, producing a stone flake of a pre-determined form is by no means a trivial accomplishment; it requires highly attuned perceptual-motor capacities. For example, in the research presented by Nonaka et al. (2010), participants of varying levels of experience in Oldowan stone knapping techniques were required to trace an outline indicating the dimensions of the flake they intended to produce prior to each attempt. Only those subjects having extensive knapping experience demonstrated the capacity to reliably predict and control the flake removal process, effectively revealing their expert appreciation for the higher order relationships existing between the multiple functional parameters at play.

As part of the experiments presented by Bril et al. (2010), participants in a series of stone knapping activities were required to use hammerstones with varying mass. Several particularly interesting observations were made through the course of the analysis. Firstly, only expert knappers were able to adapt their movements in such a way that left the resulting kinetic energy unaltered between hammer conditions, proving their remarkable sensitivity to this key functional parameter. Secondly, whereas novice and intermediate subjects tended to compensate for lower hammer masses by increasing the muscular force they applied, expert subjects maintained resultant kinetic energy by increasing trajectory length and thereby the potential energy upon initiation of the movement. In other words, when adapting to different control parameters, experts sought motor solutions that harnessed external forces, namely that of gravity.

The movement parameters involved in stone knapping tasks have also drawn the attention of several recent studies. In a paper by Williams et al. (2010), kinematic analysis revealed a proportionately high level of movement at the wrist in four beginner/novice stone knappers. Together with a proximal-distal sequencing pattern, the high level of wrist activity was judged to be advantageous in developing greater accuracy and velocity. Conversely, in a study with four experts and eight novices, Rein et al. (2013) observed that the elbow joint provided a greater contribution to the knapping task than both the shoulder and wrist joints. Reflecting the findings of Bril et al. (2010), expert subjects in this experiment were also observed to exhibit smaller hammer velocities than subjects of the novice group.

As part of the data analysis process, Rein et al. (2013) also attempted to characterize movement variability using the UCM and determined that stone knappers coordinate their movement to minimize the variability of the hammerstone's working point trajectory (one should be mindful however, that the UCM analysis of this study was limited to movement characteristics pertaining to the striking arm and did not test hypotheses regarding control as it related to the functional parameters themselves). And while novice subjects did exhibit greater variability of working

the strike. point trajectory than experts, this fact may be more a symptom of their poor understanding of functional parameters than an inability to coordinate the movement itself. Overestimating

from the highest position of the striking hand through to the completion of

the importance of velocity at the time of impact will inevitably have negative consequences upon precision (Fitts, 1954; see also Domkin et al., 2002, for another example of this effect in a UCM analysis).

In addition to studies on Oldowan stone flaking, stone knapping by counterblow, a technique used by artisanal craftsmen in India, has also been the subject of several studies on complex tool use behavior. In relation to the movement capacities of these artisans, Biryukova and Bril (2008) found the kinematic patterns of a group of expert subjects to be strikingly individual. More intriguing still was the fact that the most expert and versatile subject amongst the craftsmen demonstrated far greater joint angle contribution variability than other subjects. They concluded that the number of joints involved and similarly the potential number of effective joint angle contributions available to a subject increased as a function of skill.

In contrast to the prior studies, the following experiment incorporates actors of varying levels of expertise. Included are subjects having no prior experience at all on the set task, nor an academic appreciation of stone tool techniques (referred to hereon as uninitiated subjects). Other subjects of this study present with a varying range of skill and experience in stone knapping. This design hence permits study of the stone flaking action at novice, intermediate, and expert levels. Instead of exploring movement performance through tools for measuring central tendency, this study will place the emphasis on the features of a series of individual movements. In doing so, we intend to observe if certain movement patterns or strategies are typical at a given stage of expertise in tool use.

Here, tool use is considered primarily as a goal directed activity, fundamentally defined by task specific mechanical principles. We suggest that rather than seeking to learn a specific movement, the nervous system seeks to learn the action (Bernstein, 1996; Reed and Bril, 1996) through exploration of the functional dynamics of the body-tool-environment system. As such, we hypothesized that numerous kinematic patterns would prove effective in the stone flaking task. In other words, we anticipated that successful tool use actions would not be characterized by any particular kinematic profile.

Our second hypothesis was that kinematic movement variability would fluctuate according to a subject's sensitivity to functional dynamics. It was expected that subjects having no prior experience on this novel task (the uninitiated group) would employ highly variable movement patterns as they would be obliged to explore the dynamics of this complex tool use activity. Having discovered a limited set of body-tool-environment configurations in satisfying task constraints, novice, and intermediate subjects were expected to have more regular kinematic movement profiles. Lastly, expert tool use performance was expected to demonstrate a high level of sensitivity to the functional parameters during the stone flaking task (Bril et al., 2010; Nonaka et al., 2010). We anticipated that these subjects would have more variable kinematic movement profiles as robust synergies ensured the stability of functional parameters through the flexible covariation of upper limb segments.

# **MATERIALS AND METHODS**

# **PARTICIPANTS**

A total of 19 human subjects (8 males, 11 females) participated in this study. The mean age of the sample was 35.3 years (median 28 years; standard deviation 14.5 years; range 23–71 years). The absence of pathology impacting upon upper limb function was a condition for participation in this experiment. Only one subject, a flint worker by profession, was remunerated for his participation. All other subjects were unpaid volunteers. The majority of the subjects having a background understanding of lithic tool production were recruited through academic institutions in the Paris region. The flint-working professional (P18-JL) and one experienced hobbyist (P19-BM) were recruited separately through existing professional relationships. Subjects having no knowledge or experience in stone knapping techniques (the uninitiated group) were sourced from visitors and staff at the Paris-Descartes University campus.

### **APPARATUS**

Basalt hammerstones were used for this experiment. Each was specifically selected as having the properties required for hardhammer percussion in Oldowan lithic tool production (corresponding to the lower Paleolithic period between 2.6 Myr to 1.7 Myr ago; see Roche, 2005 for further detail). The task analyzed for the purposes of this study involved the use of hammerstones that were roughly ovoid in shape. Flint stone cores served as the raw material for stone flake production. Having been acquired from one unique source, there was limited variability in the quality of the flint itself. All cores had been pre-formed into the shape of a frustrum (a truncated pyramid) by a commercial flint working professional prior to the experiment. This measure served to facilitate immediate flake production by all participants whilst also ensuring that all subjects started under relatively similar conditions.

Movement parameters were recorded using a spatial tracking system (Polhemus Liberty, Polhemus Corporation; Colchester, VT—referred to heron as STS), a device which determines position and orientation of its associated sensors relative to a stationary system by means of an electromagnetic field. This STS permits the recording of movements in six degrees of freedom (x,y,z and rotation along the axes, x,y,z). All data was sampled at a frequency of 240 Hz and recorded online using MotionTracker v1.43 (BIOMETRICS France; Gometz-le-Châtel, Île de France).

# **PROTOCOL**

The experimental protocol used here respected the ethical guidelines of the American Psychological Association (APA). Following the provision of clear information on the conditions of participation, each subject gave their written consent. Prior to commencing the experimental procedure, personal data (e.g., height, weight, age) was collected and anthropometrical features of the striking arm were recorded in order to permit geometric modeling of the upper limb at a later stage.

STS sensors were applied to the striking arm with adhesive tape at the dorsal surface of the hand, the dorsal surface of the lower arm, the lateral aspect of the upper arm and the dorsal aspect of the coracoid process of the scapula, reflecting the protocol used by Biryukova and colleagues (Biryukova et al., 2000; Biryukova and Bril, 2008) (see **Figure 5**).

The final stage of preparation involved using the STS stylus to record the location of various anatomical landmarks of the upper limb and thorax in relation to the STS sensors, following the calibrated anatomical system technique (CAST; Cappozzo et al., 1995) and in accordance with the International Society of Biomechanics (ISB) recommendations on joint coordinate systems (Wu et al., 2005). In addition to the stated anatomical landmarks, working surfaces were also defined. This was done by using the STS stylus to record the striking surface of the frustrum in relation to an STS sensor fixed at its base. Similarly, the point of impact used on each hammerstone was recorded in relation to the STS sensor fixed to each subject's striking hand as it was held during the habitual striking grasp.

All subjects received the same instructions and model stone flakes (small and large) were provided to subjects in order to demonstrate the general dimensions of the desired end product. No restriction was placed upon the subjects' seated posture and no time was imposed for completion of the task, allowing subjects to freely explore the materials at hand. Each subject was then required to carry out a series of flaking tasks in a total of six conditions (small and large flake production with hammers of three various masses) in order to determine level of expertise (Bril et al., 2010). Three strikes only were permitted in an attempt to produce one stone flake and subjects were requested to produce three flakes in each condition. All stone flakes removed during this process were collected, weighed and labeled. The image of an expert subject carry out a stone flaking task is provided in **Figure 3C**.

# **ALLOCATION OF SKILL LEVEL**

Each individual's level of expertise was next attributed based upon their ability to produce small and large flakes with a series of different tools (per Protocol). No qualitative distinction was made to classify the form of a flake as being characteristic of conchoidal fracture or split breaking. Measures of mean flake mass and standard deviation (SD) were used as the basis to determine one's ability to intentionally and consistently control stone flake dimensions, as per **Figure 4**.

Allocation to both intermediate and expert groups was dependent upon the ability to consistently produce both small and large flakes upon command. Expert status was then attributed to those individuals who produced flakes with a high level of regularity—as indicated by mean flake masses relative to SD.

Novices were defined as subjects with irregular stone flake production. This included firstly those subjects who produced small and large flakes on an inconsistent basis and; secondly, subjects who removed flakes of regular mass and dimension, incapable of producing flakes of varying size during the experiment. Subjects of the uninitiated group were unable to produce flakes of the prescribed size, having highly variable flake masses over the different conditions. The final composition of each group is presented in **Table 1**.

# **BIOMECHANICAL MODELING**

The first stages of biomechanical modeling involved the creation of the anatomical frame of reference by calculating the offsets of the anatomical landmarks (recorded with the STS stylus) from the adjacent sensors (refer to **Figure 5**). The geometric model of the arm, drawn from the manually measured anthropometrical features, was then integrated using the method described by Hanavan (1964). With upper limb and thorax positions in place, the offset of the shoulder joint center from the acromion process was calculated using the sphere fitting process (Leardini et al., 1999; Stokdijk et al., 2000). Following this, elbow and wrist joint centers were calculated based upon the assumption that all joint centers could be found at the center of the axes constructed, respectively, by the two epicondyles at the elbow and the two styloid processes at the wrist (see **Figure 5**). This method provided one unique joint center at each articulation around which joint axes could then be calculated (Grood and Suntay, 1983; Zatsiorsky, 1998).

An optimization process was then used to eliminate artifacts in the movement data, evident in the form of temporal variations in the distance between adjacent joint centers—a consequence of STS sensor displacement relative to the underlying anatomical landmark and usually the consequence of the deformation of skin and underlying muscular and adipose tissues during the rapid, forceful movement characteristic of percussive tasks. This process involved the recalculation of segment lengths for each

**Table 1 | Allocation of subjects to groups according to level of skill.**


subject from sequential frames during a sedentary period of recorded movement data (Lu and O'Connor, 1999; Roux et al., 2002). These recalculated segment lengths were then imposed upon the axes already in place. The final model presented here presents movement relative to three degrees of freedom at each joint for a total of nine degrees of freedom. All movement analysis was performed using customized scripts which were coded using MotionInspector v1.43 (BIOMETRICS France; Gometz-le-Châtel, Île de France).

### **ANALYSIS OF TASK PERFORMANCE**

Only one specific flaking task was analyzed for the purposes of the present study, that of small flake production with a 600 g hammerstone. This measure ensured that the striking action analyzed corresponded to reasonably equivalent task constraints. This particular task was chosen firstly because most knappers typically prefer hammerstones of this approximate weight and secondly because small flakes tend to be easier to produce than larger flakes (Bril et al., 2010). Measures of flake mass variability were used to determine each subject's ability to intentionally and consistently control stone flake production. These measures included range, SD, and coefficient of variation (CV = SD/mass × 100). The statistical significance of CV between each group was calculated using one way *t*-tests with a Bonferroni correction (*p* = 0*.*05*/*6) to determine if CV reduced as a function of expertise.

# **REGULATION OF FUNCTIONAL PARAMETERS**

Relationships between key functional parameters were calculated using data extracted from the biomechanical model. Maximal kinetic energy was determined with reference to the working point of the hammer according to the formula *Ek* <sup>=</sup> <sup>1</sup>*/*<sup>2</sup> *mv*2. Potential energy was calculated with respect to vertical distances between the frustrum and the working point of the tool according to the formula *Ep* <sup>=</sup> *mgh* (*<sup>g</sup>* <sup>=</sup> <sup>9</sup>*.*81 ms−2). Ratios of kinetic energy to potential energy (*Ek/Ep*) were also calculated in order to highlight movement strategies in terms of muscular effort (refer to **Figure 2**).

Two way *t*-tests were used to determine if statistically significant differences existed between the four groups in terms of maximal kinetic energy, potential energy, and the ratio of kinetic energy to potential energy. The Bonferroni correction described above was used in all cases. Whilst data relating to the angle of blow and point of percussion may be extracted from the task specific biomechanical model presented here, this is unfortunately outside the scope of the present paper.

#### **POSTURAL ORGANIZATION AND MOVEMENT ANALYSIS**

Video recordings of flaking tasks were synchronized with their corresponding biomechanical reconstruction and permitted an initial qualitative evaluation of joint coordination profiles. The striking movement itself was determined according the displacement of the STS sensor fixed to the striking hand. The beginning of the movement was defined as the moment where the sensor reached its highest point on the vertical axis prior to the strike. The movement was deemed to have ended when the STS sensor reached either its lowest point or when it slowed to a speed inferior to 4*.*<sup>17</sup> <sup>×</sup> <sup>10</sup>−<sup>3</sup> m/s (the time between two frames at 240 Hz).

#### **ANALYSIS OF COORDINATION BY SEGMENTAL CONTRIBUTION**

Principal component analysis (PCA), one of the more classic statistically driven techniques for recognizing patterns in movement data was applied to kinematic data of striking movements. This method was chosen for two reasons. Firstly the PCA facilitated the comparison of movements through compression of the multidimensional datasets. Secondly, use of the covariation matrix served to represent the data in a way that reflects the underlying movement synergies (Ting and Chvatal, 2011). The PCA was applied to each striking movement, following the equation ϕ*<sup>i</sup> (t)* − ϕ*Mi (t)* = - *<sup>k</sup> wki*ξ*<sup>k</sup> (t)*, where the vector of temporal variation of joint angles around their mean values is defined as being equal to the sum of the principal components. Using this mathematical technique, each principal component is presented according to its magnitude so that the first principal component (PC1) corresponds with the axis along which the dataset is most spread; the second principal component (PC2) describes an orthogonal axis which describes the next most important data variance and so on. Use of this method generally permits the description and analysis of a significant percentage of movement data through one or two matrices (Rein, 2012).

Finally, the regularity of joint angle contributions to PC1 was then calculated for each subject's three successful attempts. This was done using by comparing absolute values of Pearson's correlation coefficient from the PC1 loadings across each of the nine degrees of freedom represented in the biomechanical model strikes (successful strike 1 vs. successful strike 2; successful strike 1 vs. successful strike 3; successful strike 2 vs. successful strike 3). The correlation coefficient was considered to be significant at a value greater than 0.7.

# **RESULTS**

#### **TASK PERFORMANCE**

The data pertaining to the specific task of small flake production with the 600 g hammerstone yielded a total of 89 strikes, of which 53 produced a flake. From the three attempts granted to each subject per trial, uninitiated subjects employed an average of 2.2 strikes in order to remove a flake while novices used an average of 1.6 strikes. Intermediate and expert group subjects used an average 1.3 and 1.4 strikes per trial respectively in removal of the stone flakes.

Given that successfully controlling the size of stone flake production was the goal of the set task, flake mass variability was used as the primary indicator of task performance. Summary statistics of flake mass production by group are provided in **Table 2**. Overall, uninitiated subjects produced flakes with highly variable results (*SD* = 65*.*1 g; range = 1–232 g), as did members of the novice group (*SD* = 30*.*5 g; range = 2–94 g). The intermediate (*SD* = 23*.*7 g; range = 1–75 g) and expert groups (*SD* = 19*.*7 g; range = 2–56 g) were more consistent in their stone flake production. Importantly, SD and range of the mass of flakes produced by each group can be seen to decrease according to the level of expertise (as can be seen in **Table 2**). It is also interesting to note that the median flake mass of the expert group is the same as the model flake provided (12 g).

No statistically significant difference of CVs between the respective groups was found following *t*-tests. It should be recognized however that the power of any statistical test would be limited given the small sample sizes.

#### **REGULATION OF FUNCTIONAL PARAMETERS**

Novice, intermediate and expert subjects all produced relatively similar levels of kinetic energy across the task in question (mean *Ek* = 4*.*20 J, *SD* = 1*.*27 J; mean *Ek* = 5*.*21 J, *SD* = 2*.*36 J; mean *Ek* = 4*.*44 J, *SD* = 2*.*56 J, respectively). Uninitiated subjects demonstrated exceptionally high levels of kinetic energy, at an average of 12.30 J (*SD* = 4*.*88 J), a factor which proved to be statistically significant to all other groups.


**Table 2 | Summary statistics of flake mass and regulation of kinetic energy by group.**

Similarly, potential energy upon initiation of the striking movement was also particularly high amongst subjects of the uninitiated group (*Ep* = 3*.*39 J, *SD* = 1*.*06 J) and was again statistically significant using two tailed *t*-tests. Mean values for potential energy did however show a tendency to decrease as a function of expertise with mean values of 2.24 J (*SD* = 0*.*47 J) for the novice group, 2.13 J (*SD* = 1*.*34 J) for the intermediate group and 1.37 J (*SD* = 0*.*45 J) for the expert group. This difference proved to be statistically significant between the novice and expert groups.

The average ratio of kinetic energy to potential energy was also very high amongst uninitiated subjects (mean *Ek/Ep* = 3*.*64, *SD* = 0*.*82). Conversely, novice subjects demonstrated particularly low average ratios of kinetic energy to potential energy (*Ek/Ep* = 1*.*90, *SD* = 0*.*50), with intermediate and expert subjects demonstrating respectively higher ratios of mean kinetic energy to potential energy (*Ek/Ep* = 2*.*82, *SD* = 0*.*94 and *Ek/Ep* = 3*.*04, *SD* = 1*.*39, respectively). Multiple *t*-tests with Bonferroni corrections proved all groups to be significantly different to each other. Data on the regulation of functional parameters is presented in both **Table 2** and **Figure 6**.

#### **POSTURAL ORGANIZATION AND MOVEMENT ANALYSIS**

Individual differences in terms of postural preference and movement profiles were evident upon analysis of video data synchronized with the reconstructed kinematic model. Some subjects positioned the core upon or against their leg whilst other subjects held the core in front of their body (see also Bril et al., 2010). Whilst the majority of subjects carried out the task whilst seated on a stool, certain subjects chose to be seated on the ground for the duration of the flaking tasks.

Spatiotemporal aspects of striking movements were also seen to be vary both on intraindividual and interindividual bases. Some subjects demonstrated striking movements characterized by high levels of wrist contribution, other subjects demonstrated movements characterized by high levels of elbow contribution. No apparent relationship between kinematic movement organization and expertise was evident. An example of two expert strikes (P18-JL strike 2 and P19-BM strike 2) is provided in **Figure 7**. It is interesting to note that while angular variations and the duration of the striking movements are quite different in these two strikes, the working point trajectories appear remarkably similar.

#### **JOINT ANGLE CONTRIBUTION**

Overall, PC1 accounted for 71% of joint angle variation (median = 72%; *SD* = 8%; range 55–81%), while PC2 accounted for 17% (median = 18%; *SD* = 5%; range = 11–25%) of joint variation in the flaking task. The percentage of joint angle variation accounted for by PC1 in the uninitiated group members was observed to be considerably lower than that of other groups at 64%. Limited differences were evident in the percentages of joint angle variation accounted for by PC1 in the other three groups, with PC1 accounting for 74, 75, and 76%, respectively, for the novice, intermediate and expert groups. PC1 and PC2 combined accounted for greater than 90% of joint angle variation in all groups except for the uninitiated group for whom the sum of PC1 and PC2 accounted for 84%. Having this amount of variance expressed by the first two principal components is indicative of a level of compression sufficient for valid data analysis (Rein, 2012).

The loading factors of PC1 and PC2 were used to analyze relative joint contributions to each movement and the regularity of coordinative structure in each subject's attempts at the task in question. **Figure 8** provides joint angle loadings on PC1 for three subjects representative of each group. This Figure highlights the exceptional variability of movement strategies employed by the subjects of this experiment, on both interindividual and intraindividual bases.

The variability of coordinative structure in strikes that successfully removed a flake was also examined on an individual basis using Pearson's correlation coefficients (these strikes are indicated by an "F" in **Figure 8**). No subject had joint angle loadings with significant correlation coefficients in all three comparisons. These results indicate that successful flake production was not dependent upon consistent patterns of joint angle coordination.

# **DISCUSSION**

This study examined tool use actions in a healthy adult population by means of kinematic movement analysis. From the outset, it was proposed that tool use capacity was based upon a learned ability to manipulate the functional dynamics of a given situation, as opposed to being a skill determined by movement characteristics *per se*. As such, we expected individuals to demonstrate a variety of motor patterns during a functionally equivalent (having the same task constraints) tool use activity. Using stone flake production by hardhammer percussion (based upon the Oldowan tradition) as the tool use activity in the experimental procedure, this study included participants of varying levels of experience at the task; from those with no prior exposure to the activity, through to individuals with many years of regular practice.

Relationships between the regulation of functional parameters and movement parameters (see **Figure 2**) were observed

**FIGURE 6 | Regulation of functional parameters according to level of expertise. (A)** Mean values of maximal kinetic energy of the hammer. **(B)** Potential energy relative to the striking surface of the core. **(C)** Ratios of kinetic energy to potential energy. Whisker values indicate the confidence interval at 0.95.

of the working point along the vertical axis is depicted in the bottom-most panel. No data presented here has been filtered. Max and min indicated the beginning and the end of the striking movements themselves.

hammerstone. Three subjects from each skill level group (uninitiated, novice, intermediate, and expert) are provided as examples. Black traces represent

flake are indicated with a "0."

that produce flakes are indicated with an "F," strikes that failed to produce a

in these four different groups. We hypothesized that given the task constraints, countless combinations of kinematic patterns may constitute a viable action, capable of producing the desired result. That is to say, we proposed that no particular kinematic movement pattern would be necessary or characteristic of successful task performance. In addition to this, it was hypothesized that the variability of a subject's joint angle contributions in the striking action would vary as a function of expertise. Specifically, we expected that uninitiated group subjects would use varied combinations of movements as they explored functional dynamics while novice and intermediate subjects would have comparably less movement variability, indicative of the limited number of task-specific movement patterns in their motor repertoire. Finally, movement variability was expected to be relatively high in experts, as robust synergies would exploit the multiple degrees of freedom at play in the body-tool-environment system in order to stabilize the functional parameters of the task.

The results of this experiment proved the first of these hypotheses to be correct; successful task performance was not correlated with any specific movement pattern. The second of these hypotheses proved incorrect. Indeed, both subjects with highly variable joint contribution profiles and subjects with comparably less variable joint contribution profiles were present in each skill level group (see **Figures 8**, **9**). These particular results hold interest firstly in the context of the existing body of work on stone knapping and skill acquisition in early man; and secondly, in the broader context of understanding the cognitive and neurological bases of human tool use.

The analysis of the kinematic movement data presented here revealed that successfully removing stone flakes from a flint core by the Oldowan method was not contingent upon specific

contributions of the wrist, elbow or shoulder. Effective stone flaking actions characterized by relatively high contributions of each joint were observed through the course of the experiment (see **Figure 8**, P7-AG; P4-CKB; and P18-JL for respective examples). This result may be seen to validate previously contrasting findings of experiments that found stone flaking actions to be characterized by high levels of wrist contribution (Williams et al., 2010) and others finding the elbow to have the most important contribution to the action (Rein et al., 2013).

Although based upon a different technique, the highly individual nature of upper limb kinematics demonstrated by the subjects in this study reflects the findings of Biryukova and colleagues (Biryukova and Bril, 2008; Biryukova et al., in press). Further to this, the results of the present study also indicate that patterns of kinematic movement variability are not a reliable measure of skill (cf. Biryukova et al., in press). Rather, relative tool use ability—expertise as it were, was manifested principally by the stability and intentional control of task performance. Further differences between those with no task specific experience (uninitiated) through to expert practitioners of the stone knapping technique were also apparent through the regulation of and sensitivity to key functional parameters (Bril et al., 2010). Relationships between these different parameters is highlighted here in **Figure 2** and facilitates the analysis of tool use ability in terms of action.

With respect to actual stone flake production, members of the uninitiated group demonstrated particularly erratic performance as evidenced by high measures of statistical dispersion. This variability of stone flake production (indicated by flake mass) was generally observed to reduce with each respective level of expertise, as shown by the corresponding reduction in range and SD of flake mass. Expecting identical flake dimensions upon successive strikes is of course unrealistic as even despite pre-shaping, each core varied slightly in terms of form, reflective of a real life situation.

The values of kinetic energy produced by members of the respective groups was coherent with the characteristics of the flakes produced. Subjects of the uninitiated group exhibited exceptionally high levels of kinetic energy, signifying their lack of understanding in fracture mechanics. Whilst no qualitative classification of stone flakes was conducted to discern between split breaking and conchoidal fracture in this experiment, it may safely be assumed that such amounts of kinetic energy would typically have been in excess of the threshold at which split breaking occurs, thereby producing flakes of highly variable dimensions. In contrast, the values of kinetic energy produced by novice group members is approximately half that of the uninitiated group. One may infer that these subjects, having already been inducted into basic stone knapping techniques, possessed a sound awareness of managing this key functional parameter when attempting to control stone flake dimensions. Although having somewhat greater ranges, the intermediate and expert groups produced mean values of kinetic energy similar to that of the novice group. It is interesting to note however, that expert subjects tended to produce larger flakes on average than their intermediate counterparts (as indicated by mean and median flake mass, see **Table 2**). The utilization of more energy efficient motor solutions exhibited by expert subjects here reflects those findings of Nonaka et al. (2010). It suggests that beyond an appreciation of kinetic energy, higherlevel stone knappers have a greater appreciation of the nested relationships between existing between the angle of blow, point of percussion, and external angle (see **Figures 2**, **3**).

A general trend in the use of potential energy is also evident between the four skill level groups. As can be observed in **Figure 6B**, subjects tend to reduce the amplitude of their movements (and in such a way the amount to which they harness gravity in generating the necessary energy for stone flake removal) as expertise increases. In the case of the uninitiated group, generating large amounts of potential energy is clearly representative of their limited understanding of task constraints (per above). The ratios of kinetic energy to potential energy shown in **Figure 6C** give further insights into these movement strategies and indicate an increase in the use of muscular energy from novices to experts. The reason for this trend however, is not clearly evident. It may be position that this effect is simply a reflection of confidence, whereby subjects who are sure of their actions employ less ample movement but with notably greater velocity. Or it could be that this strategy is of functional importance, and that the greater levels of acceleration (deduced here from the relationship between resultant velocity and length of hammer trajectory) may improve propagation of the shock involved in conchoidal fracture, reducing the likelihood of step fractures or the production of other undesirable features upon the core.

As opposed to task performance and the regulation of functional parameters, kinematic aspects of upper limb movement in terms of joint angle contribution were not capable of distinguishing subjects of different levels of tool use ability. Subjects with highly variable movement patterns and subjects with comparably regular movement patterns were present in each of the four skill level groups. Furthermore, it is also interesting to note that on several occasions, two movements with quite similar PC1 loadings produced different results. For example, P18-JL is shown to employ movements with almost identical PC1 loadings on both his first and second strikes (see **Figure 8**), but whilst the first is unsuccessful, the second strike successfully produces a flake. Of course, what cannot be discerned from the present analysis, is if and how this subject may have adjusted other factors, such as the orientation of the core at the time of the strike. Again, one cannot truly determine the efficacy of the movement independently of the tool-environment system with which it must be synchronized.

This finding that the regularity of kinematic movement patterns does not correspond with levels of expertise in this percussive tool use activity may initially appear to be contrary to intuition. Indeed, movement variability is often (incorrectly) perceived as being related to error, and is typically thought to decrease with improved task performance (Stergiou and Decker, 2011). From a strictly mechanical perspective however, countless combinations of joint angle contributions may produce viable motor solutions to a given problem. The considerable variability of movement patterns demonstrated in and across each skill level group in this experiment support our assertion that during tool use, the nervous system learns to manage the action rather than the movement. In any case, perfect reproduction of a certain movement in no way affords the possibility of adaptable behavior. Instead, it seems apparent that humans learn to improve the effects of their actions by increasing their understanding of functional dynamics. And rather than a hindrance, this variability witnessed in patterns of joint angle coordination could be seen to enhance this process of motor learning (Wu et al., 2014).

At the outset of this experiment, the hypothesis regarding the evolution of movement variability and expertise was founded upon this idea of exploring, then exploiting the mechanical properties comprising the body-tool-environment system. In addition, we contended that synergies provided a viable means for the neural control of functional dynamics during dexterous tool use activities. The experimental results presented here did not however, reflect a varying composition of coordinative structures across the skill level continuum (see **Figures 8**, **9**). Despite this outcome, we do not interpret this result as meaning that individuals of all skill levels possess equally flexible families of task specific motor solutions. The simple fact is that predicting whether movement variability is a good or a bad thing is not a straightforward matter. In the study of Rein et al. (2013) for example, novice subjects actually exhibited greater magnitudes of compensated variability of working point position than their expert counterparts—the effect of this was just negated by a level of uncompensated variability that was also higher than that of the experts.

In the present study, certain differences in joint contribution profiles appear simply to reflect changes in the positioning of the core, adjustments of tool grip or other such postural preference. The causes of these postural variations may in effect have limited relation to an individual's level of expertise; to maintain exactly the same posture over any duration of time is, under normal circumstances, not only uncommon but rather challenging in itself. In order to observe the highly adaptable movement synergies that would distinguish novice subjects from expert subjects, it is likely that more challenging conditions would need to be introduced to the experimental protocol (e.g., using a different tool, performing under stress/fatigue, responding to external perturbations).

In conclusion, this study has found that tool use ability depends primarily upon an understanding of the functional dynamics that exist across the body-tool-environment system, and that kinematic movement profiles alone are not sufficient to indicate relative skill levels. In other words, when learning a tool use activity, what the individual learns is the functional dynamics of the task rather than any particular movement *per se*. We argue that functional approaches such as the one employed here are imperative to the understanding of goal directed activity in cognitive neuroscience. Logically, it must first be understood what a person is controlling in terms of task relevant parameters in order to understand that which is being represented by the brain.

# **ACKNOWLEDGMENTS**

The work presented here uses experimental data collected as part of the Masters thesis of Mr. Ross Parry (Master RIM, UPMC, Paris), under the direction of Blandine Bril and Gilles Dietrich. This project was supported by the EHESS and funded in part by the PHC Programme Sakura, 2013–2014 Bilateral Joint Research Projects between the French Ministères des Affaires Étrangères et Européennes (MAEE) and the Japan Society for the Promotion of Science (JSPS). We wish to thank all the participants for generously donating their time to our study. A special thank you must be extended to Tetsushi Nonaka who participated in the recording of a handful of participants and to Cezary Sieluzycki for his valuable input during revision of the manuscript.

# **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 30 November 2013; paper pending published: 21 January 2014; accepted: 25 March 2014; published online: 23 April 2014.*

*Citation: Parry R, Dietrich G and Bril B (2014) Tool use ability depends on understanding of functional dynamics and not specific joint contribution profiles. Front. Psychol. 5:306. doi: 10.3389/fpsyg.2014.00306*

*This article was submitted to Cognition, a section of the journal Frontiers in Psychology.*

*Copyright © 2014 Parry, Dietrich and Bril. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Using tools with real and imagined tool movements

# *Jochen Müsseler 1\*, PeterWühr <sup>2</sup> and Michael Ziessler <sup>3</sup>*

*<sup>1</sup> Work and Cognitive Psychology, RWTH Aachen University, Aachen, Germany*

*<sup>2</sup> Technical University of Dortmund, Dortmund, Germany*

*<sup>3</sup> Liverpool Hope University, Liverpool, UK*

#### *Edited by:*

*Cristina Massen, Leibniz Research Centre for Working Environment and Human Factors, Germany*

#### *Reviewed by:*

*Thomas Kleinsorge, Leibniz Research Centre for Working Environment and Human Factors, Germany Miriam Gade, University of Zurich, Switzerland*

#### *\*Correspondence:*

*Jochen Müsseler, Work and Cognitive Psychology, RWTH Aachen University, Jägerstrße 17-19, 52056 Aachen, Germany e-mail: muesseler@psych. rwth-aachen.de*

When using lever tools, subjects have to deal with two, not necessarily concordant effects of their motor behavior: the body-related proximal effects, like tactile sensations from the moving hand, and/or more external distal effects, like the moving effect points of the lever. As a consequence, spatial compatibility relationships between stimulus (S; at which the effect points of the lever aim at), responding hand (R) and effect point of the lever (E) play a critical role in response generation. In the present study we examine whether the occurrence of compatibility effects needs real tool movements or whether a similar response pattern can be already evoked by pure mental imaginations of the tool effects. In general, response times and errors observed with real and imagined tool movements showed a similar pattern of results, but there were also differences. With incompatible relationships and thus more difficult tasks, response times were reduced with imagined tool movements than compared with real tool movements. On the contrary, with compatible relationships and thus high overlap between proximal and distal action effects, response times were increased with imagined tool movements. Results are only in parts consistent with the ideomotor theory of motor control.

**Keywords: tool use, sensorimotor transformation, imagery, imagination, stimulus-response compatibility, action effect, ideomotor theory**

# **INTRODUCTION**

Responding to a stimulus is faster and more accurate when stimulus location and response location spatially corresponds than when they do not. This effect is well known as spatial stimulusresponse compatibility (SR compatibility). Explanations of SR compatibility often assume that the presentation of a stimulus activates automatically the ipsilateral response. This activation is advantageous in spatially corresponding conditions, but results in a response conflict in spatially non-corresponding conditions. The solution of the response conflict increases the time needed to select the response and the probability of selecting the wrong response (for an overview see Proctor and Vu, 2006).

Furthermore, in the last two decades studies demonstrated that response times and errors in spatial compatibility tasks are not only determined by the spatial relationship between stimulus and response, but also by the location of the intended action effects (e.g., Hommel, 1993; Kunde, 2001). The theoretical background of these studies was that actors select, initiate, and execute a movement by anticipating the movement's sensory effects (ideomotor principle, see, e.g., Hommel et al., 2001a,b; Shin et al., 2010). These may be representations of body-related effects, like tactile sensations from the moving finger, and/or representations of more external effects, like the illuminating bulb when the switch is turned on.

However, when considering the chain "Stimulus → Response → Effect," different compatibility relationships come into play. Beside the mentioned SR compatibility, performance might be also influenced by the spatial relationships between stimulus and action effect (SE compatibility) and/or by the spatial relationships between response and action effect (RE compatibility). Investigating the use of lever tools where the moving effect points of a lever represent the (anticipated) action effect, proofed to be an easy way to decouple – at least in parts – the different compatibility relationships (e.g., Kunde et al., 2007; Massen and Prinz, 2007; Müsseler et al., 2008; Beisert et al., 2010; Massen and Sattler, 2010). In these studies the SR relationship is the correspondence (or non-correspondence) between stimulus location and handresponse direction. The SE relationship is the correspondence (or non-correspondence) between stimulus location and the direction of the lever's effect point. Thus, a compatible SE relationship represents the situation in which the lever's effect points have to reach at the stimulus and an incompatible SE relationship represents the situation in which the effect points have to be shifted away from it. Finally, the RE relationship reflects the correspondence (or non-correspondence) between hand-response direction and the direction of the spatial effect point of the lever. With a compatible RE relationship, the hand and the lever's effect points move in the same direction, while with an incompatible RE relationship the hand and the lever's effect points move in the opposite direction. Thus, an incompatible RE relationship requires an inverse tool transformation.

To our knowledge, SR, SE, and RE compatibilities were varied simultaneously only in a study by Müsseler and Skottke (2011). In their experiment the authors used an U-lever and an inverted U-lever with a pivot: the tool consisted of a vertical rod with a grip at the bottom part and a centrally placed crossbar in the upper part (**Figure 1**). The pivot point was in the middle of the horizontal rod and the tool's effect points were at the ends of additional upward

or downward oriented rods attached to the crossbar. Using these tools made it possible to manipulate SR, SE, and RE compatibilities independently of each other in a full 2 × 2 × 2 design allowing to examine the contribution of each compatibility relationship and their interactions to response times and errors1.

The main outcome of this study was that response times and errors were drastically increased with an inverse tool transformation, that is when hand movement and the lever's effect point move in opposite direction (incompatible RE relationship see also Kunde et al., 2007; Müsseler et al., 2008; Massen and Sattler, 2010). For instance, in **Figure 1A**, a hand movement to the left results in an effect point movement to the right. This situation is disadvantaged compared with the situation when hand movement and the lever's effect points move in the same direction (compatible RE relationship; **Figure 1B**). Additionally, it turned out to be easier to reach with the levers' effect points at the stimulus (compatible SE response) than to shift the effect points to the contrary side (incompatible SE response). However, at least this finding has to be interpreted with the significant SE–RE interaction and the significant three-way SR-SE-RE interaction. In short, the interactions came about by substantial differences within the compatible RE conditions, while only minor differences were observed within incompatible RE conditions.

The aim of the present study was twofold. The first aim was to replicate the findings of Müsseler and Skottke (2011) with a simpler tool. One objection against the U-shaped and inverted Ushaped lever is that hand movements and tool-effect movements are only indirectly coupled through the pivot point. Therefore, in the present experiment participants operated with the index finger and middle finger a rocker switch. With a key-press on the rocker switch, a rocker presented on a display moved in direct

correspondence (or non-correspondence) to the hand movement (see **Figure 2**). The rocker was the participant's tool for pointing to the imperative stimuli.

**Figure 3** illustrates the various SR, RE, and SE relationships with the rocker. A compatible SR relationship was present when the side of the key-press corresponded with the side of disk presentation, otherwise it was SR incompatible. An imperative light blue disk indicated to move the nearest rocker's effect point toward the stimulus, exposing the compatible SE relationships. A dark blue disk indicated to move the rocker's effect point away from the stimulus and thus represent an incompatible SE relationships. Further, the rocker on the display moved in correspondence with the hand movement, which resembles a compatible RE relationship. A non-correspondence between rocker movement and hand movement notify an incompatible RE relationships, which agreed with the inverse tool transformation2.

The second aim of our study was to contrast real vs. imagined rocker movements, that is, participants were asked to operate the

<sup>1</sup>In the present study SR, SE, and RE compatibilities were also varied independently of each other with another, more simpler tool. In **Figure 3**, the concrete implementations of the compatibility relationships were explicated for the new tool, which have been also applied to the U-lever and inverted U-lever (see Müsseler and Skottke, 2011, Figure 4).

<sup>2</sup>Analyzing the SR settings without the rocker revealed further possible relationships between stimuli and responses. Beside the spatial SR relationship, left–right responses could be differently influenced by the upper–bottom arrangement and/or by the color of the stimuli. Several findings indicate that responding to an upper stimulus is somewhat faster and more accurate with a right response than when with a left response and vice versa (orthogonal compatibility effects, for an overview see Proctor and Vu, 2006, Chap. 8). However, in the present design orthogonal compatibility effects cancel each other out. For example, an upper left light blue disk required a right response (which should be facilitated by the orthogonal relationship), but an upper right light blue disk required a left response (which should be hampered by the orthogonal relationship). Only the average of both conditions entered into the analyses. Additionally, we see no reason to assume different effects of light and dark blue stimuli on left–right responses.

**FIGURE 3 | SR, RE, and SE relationships with the rocker.** In the figure the imperative light blue disk indicated to move the nearest rocker's effect point toward the stimulus (compatible SE relationships) and the dark blue disk indicated to move the rocker's effect point away it (incompatible SE relationships). Further, the rocker on the display moved in correspondence with the hand movement (compatible RE relationships) or in non-correspondence (incompatible RE relationships, which indicated an inverse tool transformation). Finally, the key-press on the rocker switch corresponded spatially with the side of disk presentation (compatible SR relationships) or did not correspond spatially (incompatible SR relationships). In the figure, only left stimulus presentations are depicted, right stimulus presentations were varied correspondingly. Circles represent pivot points, vertical arrows the side of the key-press.

rocker switch with corresponding movements of the rocker and without such movements displayed on the screen. In the latter condition, they should only imagine that the rocker (the tool for the pointing movements) moved.

Imagined tool movements are often studied in the context of mental training demonstrating generally an improvement in performance through previous imaginations [e.g., playing tennis with a tennis racket (Noel, 1980) or playing golf using a golf club (Taylor and Shaw, 2002)]. By comparing real with imagined movement times, other studies showed, for instance, that mental models of the tool mechanics are used during imagery (Schwartz and Holton, 2000), that the thickness of a painting tool and thus characteristics of the tools' effect influence imagery time (Rieger and Massen, 2014), or that the speedaccuracy relationships of Fitts' law are present in real and imagined tool use (Macuga et al., 2012). Consequently, movements with real and imagined tool use are assumed to recruit similar processing mechanisms (see also Jeannerod, 1994, 2001; Davidson and Wolpert, 2005; Higuchi et al., 2007; Munzert et al., 2009).

However, the movements investigated in previous studies of imagined tool movements are relatively long lasting and complex. They also often require closed-loop control when performed under real conditions. Consequently, the observed effect of imagery can depend on many factors and it remains open to what extent they include the imaginations of tool transformations as part of the

planning of the complex action. In contrast, only short ballistic, open-loop movements are necessary to press the rocker switch in the present experiment. Moreover, as we used a computeranimated version of a lever3, the lever "moves" in direct response to the key-press within one vertical retrace of the monitor to its end position. In other words, the responses are fully preplanned and executed before the tool-effect movements actually take place. Consequently, compatibility effects should be fully visible in response time differences, independently of whether a rocker movement occurred in reality or in imagination.

Another motivation for comparing conditions with real and imagined tool movements arises from ideomotor theory. With regard to the ideomotor principle, tool movements are initiated by both the anticipation of the body-related kinesthetic effects and the anticipation of the tool effects in the environment. Whenever a tool is used, actors' intentions are usually directed to the tool's effect points. We have already pointed out that especially when feature overlap between hand movement and tool movement is high, actors are less aware of their own hand movements and the distal tool effects become predominant (Sutter et al., 2013; for empirical evidence see, e.g., Rieger et al., 2005; Müsseler and Sutter, 2009; Sülzenbrück and Heuer, 2009). In other words, with compatible RE relationships, the information processor seems to work in an automated manner with the tool effects. If the anticipated distal tool effects are used to initiate the key-press response, as is assumed by the ideomotor account, key-press times should not vary between conditions of real and imagined tool movements, especially under compatible conditions.

However, with incompatible relationships in the chain "Stimulus → Response → Effect," the situation might be different. If feature overlap between hand movement and tool movement is low and if a transformation between them is obvious, proximal action effects might interfere with the distal tool actions. The incompatible relationship requires a tool transformation and consequently the actors might perceive the task as more difficult (cf. Sutter et al., 2013). In this case, it might be easier for the actor to apply SR-translation rules to the task and thus to ignore the distal tool movements. If so, in conditions with real tool movements, the anticipation of the discordant tool effects might hamper response execution, while in conditions with imagined tool movements, discordant tool effects are easier to "ignore." Therefore, we expect increased response times with real tool movements especially under incompatible conditions.

# **MATERIALS AND METHODS**

### **PARTICIPANTS**

Twelve students (10 female, between 19 and 31 years of age, mean age 22.3 years) from RWTH Aachen University participated in the experiment for pay or course credit.

<sup>3</sup>Note, that comparing response times with a real and a computer-animated lever have shown amazingly consistent results [cf. the results of Kunde et al. (2007, Experiment 1) with a real lever and Müsseler et al. (2008, Experiment 2) with a comparable computer-animated lever; see also Müsseler and Skottke, 2011; for a direct comparison of both types of levers see Müsseler et al., 2008, Experiments 1a and 1b].

# **APPARATUS AND STIMULI**

The experiment was carried out in a dimly lit chamber and was controlled by an Apple Macintosh computer with Matlab software (Mathworks) using the Psychtoolbox-3 extension (Kleiner et al., 2007). Participants responded with their index or middle finger of their preferred hand by pressing down the left or right side of a rocker switch, which could release a corresponding or noncorresponding movement of the rocker displayed on a 22" color CRT monitor (Iiyama Vision Master Pro 513, 100-Hz refresh rate, 1024 × 768 pixel). The rocker was presented at screen center and consisted of a black straight line (200 pixel) with arrows at both ends and a pivot (8 pixel) in the middle (cf. **Figure 2**).

A dark blue or light blue disk was displayed in one of four gray disks (each with a diameter of 40 pixel), which formed a virtual square (240 × 240 pixel) surrounding the rocker. The participant's head was placed on a chin rest 500 mm in front of the monitor. The blue disks served as imperative stimuli, the gray disks were placeholders for possible stimulus positions.

# **DESIGN**

The experiment had a 2 (real vs. imagined tool movement) ×2 (SR compatible vs. incompatible) × 2 (RE compatible vs. incompatible) × 2 (SE compatible vs. incompatible) repeated measurement design. Thefour combinations of thefactors"real vs. imagined tool movement" and "RE compatibility" were presented block-wise at two different days with the sequence of blocks balanced between participants. The only restriction was that the conditions of real tool movements and imagined tool movements were presented consecutively at 1 day. For example, at the first day a participant performed the RE-compatible trials with the real tool movements and then with the imagined tool movements (or vice versa). At least 1 week later, the participant performed the RE-incompatible trials with the real tool movements and then with the imagined tool movements (or vice versa). Within these blocks, all combinations of SR and SE compatibility were presented in a randomized order.

Altogether, participants worked through a total of 960 trials. The first blocks were considered as practice trials and were not analyzed. Thus, each cell of the design was filled with 50 repeatedmeasurement trials. Dependent measures were median response times and the error percentages of each participant.

### **PROCEDURE**

Participants were instructed in written form. They were informed that a left or right key-press on the rocker switch produced a corresponding (RE compatible) or a reversed turn (RE incompatible) of the rocker on the screen. In conditions of imagined tool use, participants were asked to imagine the corresponding tool movement only.

The experiment started with the presentation of the four gray disks and the rocker, which remained visible until the end of the experiment. At the beginning of each trial, the rocker was in the horizontal home position. When one of the four gray disks changed its color to light or dark blue, participants were required to press the right or left side of the rocker switch as fast and accurately as possible. The light blue disk indicated to move the nearer effect point of the rocker (the left or right

arrow) toward the stimulus and the dark blue disk indicated to move the nearer effect point away from the stimulus. The keypress immediately caused a corresponding or non-corresponding shift of the rocker to the end position with the next vertical retrace of the monitor. Through the phi phenomenon observers perceived a movement of the rocker between the home position and the end position. The rocker turned back to the home position after the release of the key. The next trial started after 1.5 s.

An error feedback was given, if participants had made the wrong response (a tone of 440 Hz with a duration of 50 ms) or if response times were lower than 100 ms or exceeded 2,000 ms (a tone of 880 Hz with a duration of 50 ms). At each day, the experiment lasted about 30 min including short breaks every 40 trials.

# **RESULTS**

Median response times and percentage of errors of each participant were entered into 2 (real vs. imagined tool movement) × 2 (SR compatible vs. incompatible) × 2 (RE compatible vs. incompatible) × 2 (SE compatible vs. incompatible) analysis of variance (ANOVA) with repeated measurements. Results are shown in **Figure 4**. We first focus on the results of SR, RE, and SE compatibility and their interactions averaged over real and imagined tool movements. After that we look at this factor and its possible interactions with the compatibility conditions.

A significant RE effect was observed in the reaction-time analysis, *<sup>F</sup>*(1,11) <sup>=</sup> 23.09, *<sup>p</sup>* <sup>=</sup> 0.001, <sup>η</sup><sup>2</sup> <sup>p</sup> = 0.677, and in the error analysis, *<sup>F</sup>*(1,11) <sup>=</sup> 12.99, *<sup>p</sup>* <sup>=</sup> 0.004, <sup>η</sup><sup>2</sup> <sup>p</sup> = 0.541. Responses under compatible RE relationships were performed 138 ms faster and with 4.6% less errors than under incompatible RE relationships (673 vs. 811 ms and 3.7 vs. 8.3%). In other words, when the rocker movement on the display was in correspondence with the hand movement on the rocker switch (left panel of **Figure 4**), response times and errors decreased compared with a noncorrespondence of hand and rocker movements (right panel of **Figure 4**).

Other significant main effects were that responses were performed faster (708 ms) under compatible SE relations than incompatible SE relations (775 ms), *F*(1,11) = 29.14, *p* < 0.001, η2 <sup>p</sup> = 0.726, and that errors in SR compatible trials were significantly increased as compared to errors in SR incompatible trials (6.6 vs. 5.3%), *<sup>F</sup>*(1,11) <sup>=</sup> 10.63, *<sup>p</sup>* <sup>=</sup> 0.008, <sup>η</sup><sup>2</sup> <sup>p</sup> = 0.491. However, these findings have to be qualified by significant interactions. The two-way interaction between SE and SR compatibility was significant in the response time analysis, *F*(1,11) = 7.78, *p* = 0.018, η2 <sup>p</sup> = 0.414, and in the error analysis, *F*(1,11) = 8.04, *p* = 0.016, η2 <sup>p</sup> = 0.422. Furthermore, the three-way interaction between SE, SR, and RE compatibility was significant in the response time analysis, *<sup>F</sup>*(1,11) <sup>=</sup> 25.81, *<sup>p</sup>* <sup>&</sup>lt; 0.001, <sup>η</sup><sup>2</sup> <sup>p</sup> = 0.701, and in the error analysis, *<sup>F</sup>*(1,11) <sup>=</sup> 29.04, *<sup>p</sup>* <sup>&</sup>lt; 0.001, <sup>η</sup><sup>2</sup> <sup>p</sup> = 0.725. The three-way interaction reflects the finding that under compatible RE relations (**Figure 4**, left panel) responses in SR compatible trials were advantaged if the SE relationship was compatible, but disadvantaged if the SE relationship was incompatible. Under incompatible RE relations (**Figure 4**, right panel) this pattern of

results was reversed. However, there is also a possibly more simple description of those results, if the left–right SR compatibility is replaced by the compatibility between the stimulus position and the direction of the required response. A closer inspection of the three-way interaction revealed that responses were faster and less error prone when the stimuli appeared in the lower position as compared to their appearance in the upper position (697 vs. 786 ms, 4.0 vs. 8.0%; cf. **Figures 3** and **4**). Corresponding *post hoc t*-tests were significant for response times with *t*(11) = 5.08, *p* < 0.001 and for errors with *t*(11) = 5.39, *p* < 0.001, two-tailed.

When comparing response times and errors with regard to real and imagined tool movements, the findings appear amazingly consistent. However, two differences showed up as interactions in the reaction-time analysis. First, the SE compatibility effect was larger with real tool movements (difference= 83 ms) than with imagined tool movements (difference = 52 ms; cf. **Figure 5**, left panel), producing a significant interaction between "SE compatibility" and "real vs. imagined tool movement," *F*(1,11) = 8.74, *p* = 0.013, η2 <sup>p</sup> = 0.443.

Second, there was a tendency toward an interaction of "RE compatibility" and "real vs. imagined tool movement," *F*(1,11) = 4.18, *<sup>p</sup>* <sup>=</sup> 0.066, <sup>η</sup><sup>2</sup> <sup>p</sup> = 0.275 (cf. **Figure 5**, right panel). This result indicated that also the RE compatibility effect was larger with real tool movements (difference = 199 ms) than with imagined tool movements (difference = 77 ms; cf. **Figure 5**, right panel). Other effects were not significant, also not in the error analysis.

### **GENERAL DISCUSSION**

In a reaction-time experiment the spatial compatibility relations between stimulus and response location, between stimulus and SE and between response and RE were varied independently of each other. In addition, in half of the trials the response-effect was actually visible whereas in the other half of trials the participants only imagined the effect. As we did in the results section, we will first discuss the results of SR, RE, and SE compatibility and their interactions.

The analyses of the response times and errors indicate that all types of compatibility relationships were involved in the planning and execution of the responses. However, the strength of the compatibility effects varied between the different relationships. The most prominent compatibility effects were observed for RE compatibility. Under all conditions and independent on the other two compatibility relationships participants needed more time if the movements of the fingers on the rocker switch were incompatible to the movements of the rocker on the screen (see also Kunde et al., 2007; Müsseler et al., 2008; Massen and Sattler, 2010). Under compatible conditions, participants can plan the responses using the same features than those used for the anticipation of the intended effect. However, if the RE relations are incompatible, an additional transformation process is necessary that translates the features of the intended effect into the features of the response to achieve this effect. In line with the ideomotor principle, this underlines the importance of the response-effects for the control of the responses. However, the result also shows that the assumption of the ideomotor principle that the responses are directly activated by the anticipation of their effects might be too simple. To activate a response, the effect features need to be translated into response features and this translation process is facilitated under compatible conditions, i.e., if there is an overlap between features of the effects, the stimuli and the responses.

Compared to the RE compatibility, the effects of SE compatibility were on average about 50% smaller and depended on the other compatibility relationships. Only if the RE and SR relations were both compatible or both incompatible, an SE compatibility effect was observed. Similarly, SR compatibility effects were only observed, if RE and SE relations were

both compatible or both incompatible. It should be noted that SR compatibility effects were smaller than the SE compatibility effects and for two conditions the SR compatibility effects were negative. If the RE relations were compatible and the SE relations incompatible, compatible SR relations were associated with longer response times than incompatible SR relations. The same inverted compatibility effect was found if the RE relations were incompatible and the SE relations compatible (see also Müsseler et al., 2008, for a similar observation). In other words, the control of the responses is dominated by the compatibility between features of the responses and the effects and between features of the stimuli and the effects. Within the complex pattern of compatibility relations, SR compatibility seems to play only a minor role. It might be that the participants rarely use the left–right location of the stimuli for the coding of the responses because the location of the stimulus does not allow a decision on the response. Instead they could use configurative features. For example, under the RE compatible condition, if a dark blue stimulus appears on the main diagonal of the screen (upper left or lower right stimulus position) a right response would be required. Possible SR compatibility effects are overwritten by other compatibility relations. As we have shown in the results section, if the stimuli appeared in the lower position responses were faster and more accurate. Because all four responses consisted in downward responses, for stimuli in the lower position the stimulus position and the movement direction were compatible whereas this relationship was incompatible for stimuli in the upper position. However, it is also worth to note that the effects of the lower and upper positions on response times and errors were probably evoked by the tool, that is by the rocker switch. Müsseler et al. (2008, Experiment 2, Figure 4) observed in a comparable setup without a tool only minor effects of a few milliseconds at upper and lower positions, but strong effects as in the present experiment when the tool was presented.

In sum, when considering the results with regard to the SR, SE, and RE compatibility, the present findings replicated successfully the study of Müsseler and Skottke (2011) with a simpler lever tool. The observed main effects of SE and RE compatibility as well as the interactions were found in both studies and demonstrate the robustness of the results with lever tools (see also Kunde et al., 2007; Müsseler et al., 2008; Massen and Sattler, 2010). The only obvious difference seems to be that the differences within the incompatible RE conditions were more pronounced in the present study than in the previous study of Müsseler and Skottke (2011).

If the participants only imagined the movements of the rocker on the screen a similar pattern of compatibility effects emerged. This is further evidence that real and imagined movements might recruit similar processing mechanisms (cf. Jeannerod, 1994, 2001; Davidson and Wolpert, 2005; Higuchi et al., 2007; Munzert et al., 2009; Macuga et al., 2012; Rieger and Massen, 2014).

However, we also observed a significant interaction between factors SE compatibility and the tool movement (real vs. imagined) and a trend toward an interaction between RE compatibility and tool movement. If the tool movement was present on the screen as response-effect, both compatibility effects involving the tool movement (i.e., the distal effect) were more pronounced as compared to imagined tool movements. The physical absence of the tool movements reduced both the facilitating effect of compatible relationships and the inhibiting effect of incompatible relationships (**Figure 5**). On the one hand, there is evidence that the participants still use the tool effects for the control of their responses as assumed by the ideomotor principle, even if the effects are only imagined. On the other hand, the reduction of the SE and RE compatibility effects suggests that the participants rely less on the effects in controlling their responses if the tool movements are not real. The latter is difficult to explain in the theoretical framework of the ideomotor principle. If the effects were necessary for the selection of the responses, it should not matter whether the effects were physically present or only imagined. In both cases the effects have to be anticipated and the anticipated effects should then activate the response. Thus, the pattern of the compatibility effects indicates that the imagined tool movements were involved in the control of the responses, but the reduced size of the compatibility effects is also evidence that the selection of the responses does not fully depend on the anticipation of the effects.

This leads to the interesting question of what is the function of effect anticipation in the control of motor responses. As it seems from the present results, the central assumption of the ideomotor principle that effects are used for response selection is too narrow. In motor control, effects are also involved in the monitoring of the responses and the evaluation of the executed responses (e.g., Schmidt, 1975). Both functions include that the effects of a selected response are anticipated as part of response planning (Nikolaev et al.,2008; Ziessler and Nattkemper,2011; Ziessler et al., 2012). If effects are anticipated depending on a selected response, it becomes possible to perform an internal test of the selected response: the response can be executed if the anticipated effect is in accordance with the intended effect.

Taking this idea into account, there is an alternative interpretation for the observed compatibility effects. The participants might select their responses via simple stimulus-response transformation rules (e.g., under RE compatible conditions: dark blue stimulus on the main diagonal [upper left or bottom right] → press left key, light blue stimulus on antidiagonal [upper right or bottom left] → press left key etc.). The anticipation of the response-effects, i.e., the anticipation of the rocker movement, depends on SE and RE compatibility. The faster participants get the confirmation from the internal test that the planned response will generate the intended effect the earlier the response will be executed. Under compatible conditions this procedure will facilitate the responses, under incompatible conditions the procedure might hold up the responses.

The described interpretation applies in particular to the condition in which the movement of the rocker was physically present. Under the condition of imagined rocker movements the participants learned that there was no distal tool effect of their responses. It has been shown in other experiments that participants stop to anticipate learned response-effects if the effects were removed from the experimental setting (Ziessler et al., 2012). Consequently, under the imaging condition there was no reason for the participants to anticipate non-existing effects during the planning of the responses. This could have led to the leveling of the response times. Effect related compatibility effects are still there as long as the participants follow the instruction to imagine the effects. But the compatibility effects become weaker because the anticipation of rocker movements is in conflict with the actual effects.

In conclusion, the present experiment underlines again that the anticipation of effects is an important component of response planning. This includes distal effects that are generated by tools. The function of effect anticipation does not seem to be limited to response selection. Anticipation of effects for selected responses also constitutes an internal test if the selected response will generate the intended effect. For the planning of a response including the anticipation of its effects the cognitive system uses very flexibly all existing relationships between the stimuli, the responses and the effects. The impact of distal response-effects on response planning

diminishes if the effects are removed from the setting. Under theoretical aspects, that means the ideomotor principle needs at least to be amended to provide an explanation for the multiple ways of response selection and response preparation.

#### **ACKNOWLEDGMENTS**

We wish to thank Laura Makowski for carrying out the experiment. This work was supported by a grant from the Deutsche Forschungsgemeinschaft to the first author (Mu 1298/10). Correspondence concerning the manuscript should be addressed to Jochen Müsseler, Work and Cognitive Psychology, RWTH Aachen University, Jägerstrße 17-19, 52056 Aachen, Germany. e-mail: muesseler@psych.rwth-aachen.de

# **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 18 February 2014; accepted: 11 May 2014; published online: 17 June 2014. Citation: Müsseler J,Wühr P and Ziessler M (2014) Using tools with real and imagined tool movements. Front. Psychol. 5:515. doi: 10.3389/fpsyg.2014.00515*

*This article was submitted to Cognition, a section of the journal Frontiers in Psychology. Copyright © 2014Müsseler,Wühr and Ziessler. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Effects of angular shift transformations between movements and their visual feedback on coordination in unimanual circling

# *Martina Rieger1,2 \*, Sandra Dietrich1,3 and Wolfgang Prinz1\**

*<sup>1</sup> Department of Psychology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany*

*<sup>2</sup> Institute for Psychology, Department for Medical Sciences and Health Systems Management, University for Health Sciences, Medical Informatics and Technology, Hall in Tirol, Austria*

*<sup>3</sup> Department of Education, Leipzig University, Leipzig, Germany*

#### *Edited by:*

*Cristina Massen, Leibniz Research Centre for Working Environment and Human Factors, Germany*

#### *Reviewed by:*

*Mathias Hegele, Justus Liebig University Giessen, Germany Franz Mechsner, Northumbria University, UK (formerly affiliated)*

#### *\*Correspondence:*

*Martina Rieger, Institute for Psychology, Department for Medical Sciences and Health Systems Management, University for Health Sciences, Medical Informatics and Technology, EduardWallnöfer Zentrum 1, A-6060 Hall in Tirol, Austria e-mail: martina.rieger@umit.at; Wolfgang Prinz, Department of Psychology, Max Planck Institute for Human Cognitive and Brain Sciences, Stephanstraße 1a, D-04103 Leipzig, Germany*

*e-mail: prinz@cbs.mpg.de*

Tool actions are characterized by a transformation between movements and their resulting consequences in the environment. This transformation has to be taken into account when tool actions are planned and executed. We investigated how angular shift transformations between circling movements and their visual feedback affect the coordination of this feedback with visual events in the environment. We used a task that required participants to coordinate the visual feedback of a circular hand movement (presented on the right side of a screen) with a circling stimulus (presented on the left side of a screen). Four stimulusvisual feedback relations were instructed: same or different rotations of stimulus and visual feedback, either in same or different y-directions. Visual speed was varied in three levels (0.8, 1, and 1.2 Hz). The movement-visual feedback relation was manipulated using eight angular shifts: (−180, −135, −90, −45, 0, 45, 90, and 135◦). Participants were not able to perform the different rotation/different y-direction pattern, but instead fell into the different rotation/same y-direction pattern. The different rotation/same y-direction pattern and the same rotation/same y-direction pattern were performed equally well, performance was worse in the same rotation/different y-direction pattern. Best performance was observed with angular shifts 0 and −45◦ and performance declined with larger angular shifts. Further, performance was better with negative angular shifts than with positive angular shifts. Participants did not fully take the angular shift transformation into account: when the angular shifts were negative the visual feedback was more in advance, and when angular shifts were positive the visual feedback was less in advance of the stimulus than in 0◦ angular shift. In conclusion, the presence and the magnitude of angular shift transformations affect performance. Internal models do not fully take the shift transformation into account.

**Keywords: unimanual coordination, visuo-motor transformation, angular shift, sensorimotor integration, tool transformation, circling, synchronization**

# **INTRODUCTION**

Tool actions are characterized by a transformation between movements and their resulting consequences in the environment. For instance, when pushing a lawn-mower movements result in consequences further ahead in the environment, or when pulling a sledge by a cord the consequences are behind the position of the actual movement. Transformations between movements and the consequences have to be taken into account when tool actions are planned and executed, and they are an important part of the cognitive representation of tool actions (Massen and Prinz, 2007). Some tool actions require the coordination of a tool's consequences in the environment with other events in the environment. For instance, in baseball the tip of the baseball bat has to be coordinated with the position of the ball or in hockey the tip of the hockey stick needs to be coordinated with the position of the puck in order to achieve the intended goal. This is the topic of the present study: coordination performance when a transformation between a movement and its consequences exists.

Coordination principles have been studied using unimanual as well as bimanual tasks. In research on unimanual coordination (i.e., the coordination of one hand with an event or a stimulus) mostly tasks with discrete structuring events have been used. This is the case in tapping tasks (e.g., Aschersleben and Prinz, 1995), coincidence anticipation tasks (e.g., Fleury et al., 1992), and sometimes tracking tasks which include movement reversals (e.g., Alaerts et al., 2007). Research on bimanual coordination (i.e., the coordination of the two hands) has also included tasks without discrete structuring events, like circling (e.g., Swinnen et al., 1997; Mechsner et al., 2001; Tomatsu and Ohtsuki, 2005). Unimanual coordination of continuous movements in tasks without structuring events has rarely been investigated, and little is known how tool transformations affect coordination in such tasks (but see Dietrich et al., 2012; Rieger et al., 2014). Specifically, to the best of our knowledge it has not been investigated how the magnitude of an angular shift transformation between a movement and its visual feedback in the environment affects coordination performance in

circling movements. Therefore, this was investigated in the present study.

Research on bimanual coordination has shown that coordination stability depends on (a) the relation between the hands in reference to movements toward and away from the body midline (we refer to this as the x-axis in the following) and (b) the relation between the hands in reference to movements toward or away from the body (we refer to this as the y-axis in the following). Coordination is more stable when the two hands move in opposite directions on the x-axis (one hand moves to the left and one hand moves to the right) than with any other type of movement pattern between the limbs (Swinnen et al., 1997; Swinnen and Wenderoth, 2004; Tomatsu and Ohtsuki, 2005). The second most stable mode is moving the two limbs into the same x-direction, (both hands move to the left and to the right at the same time, e.g., Swinnen et al., 1997). With reference to the y-axis, performance is most stable when the hands move in same y-directions (toward and away from the body at the same time), the second most stable mode is when the hands move in opposite y-directions (one hand is moving away and one hand is moving toward the body). However, with high frequencies movements in different y-directions often become instable, resulting in a transition to more stable same ydirection patterns (e.g., Swinnen and Wenderoth, 2004). All other coordination patterns are less stable (Haken et al., 1985; Tomatsu and Ohtsuki, 2005). Thus, the most stable coordination performance is obtained when movements of the hands have opposite x-directions, and same y-directions, i.e., mirror symmetric movements (Swinnen et al., 1997). These effects are ascribed to motor constraints (the way the central nervous system issues motor commands, Swinnen et al., 1997; Cardoso de Oliveira, 2002; Heuer et al., 2004; Salter et al., 2004), motor related feedback (kinesthesis and proprioception, Mechsner, 2004), visual feedback (Mechsner et al., 2001; Bogaerts et al., 2003; Mechsner, 2004; Tomatsu and Ohtsuki, 2005; Kovacs et al., 2010a,b), and cognitive constraints (Weigelt et al., 2007).

Coupling phenomena found in bimanual coordination are often similarly observed in unimanual coordination (e.g., Wimmers et al., 1992; Buekers et al., 2000). In unimanual coordination there is no second limb with which movements are coordinated, but rather a coordinative stimulus/event. As there can be no motor constraints related to the second hand moving, unimanual coordination depends on the perceptual characteristics of the movement feedback of the moving hand, which can be either visual and/or proprioceptive/kinesthetic. Studies indicate that coordination is predominantly governed by visual feedback in many situations (Buekers et al., 2000; Roerdink et al., 2005; Dietrich et al., 2012), even though proprioception/kinesthesis must also be taken into account (Wilson et al., 2005a,b; Dietrich et al., 2012). It also depends on the type of task whether visual or kinesthetic/proprioceptive information is more beneficial for unimanual coordination (Alaerts et al., 2007).

Transformed visual feedback has been experimentally deployed to study how motor related feedback (kinesthesis and proprioception) and visual feedback interact and contribute to coordination performance (e.g., Mechsner et al., 2001; Alaerts et al., 2007). In a task similar to the one we used in the present study participants were asked to coordinate the visual feedback of a circular hand movement with a circling stimulus in order to produce different visual patterns on the screen (Dietrich et al., 2012). Those visual patterns consisted of visual feedback and stimulus rotating in same or different directions and moving in same or different y-directions. To dissociate movements and the associated proprioceptive/kinesthetic feedback from visual feedback, participants performed the task under regular and transformed visual feedback (180◦ angular shift between movement and visual feedback on the screen). A 180◦ angular shift of the visual feedback implies that when stimulus and visual feedback have same y-directions, the y-direction of the hand movements is opposite to the ydirections of the stimulus and the visual feedback. However, when stimulus and visual feedback have different y- directions, the y-direction of the hand movement is opposite to the ydirection of the visual feedback, but corresponds to the y-direction of the stimulus. In this task, coordination occurred mainly in visual space, (similar data patterns with regular and transformed feedback, vision-to-stimulus coordination), but subtle effects of coordination in movement space were also observed (smaller differences between same and different y-directions in visual space with transformed feedback, movement-to-stimulus coordination). The presence of a transformation affected performance negatively.

In the present study we used a similar task. However, in contrast to Dietrich et al. (2012) we used a wider range of angular shift transformations between movements and the visual feedback on the screen, in order to disentangle effects of the presence of a transformation and the magnitude of a transformation on performance. Participants were asked to coordinate a visual feedback dot (produced by the participants'movement and presented on the right side of a screen) with a continuously circling stimulus dot (presented on the left side of the screen). They were asked to produce four different patterns of the dots on the screen. Two aspects of the stimulus-visual feedback relation were varied. First, we varied the rotation direction which was either the same or different. The stimulus dot always moved clockwise. In one condition participants were asked to move counterclockwise (correspondingly the visual feedback dot also moved counterclockwise), therefore stimulus and visual feedback have different rotation directions (i.e., different directions on the x-axis). In another condition participants were asked to move clockwise, resulting in same rotations of stimulus and visual feedback (i.e., same directions on the x-axis). Second, the y-direction of the stimulus-visual feedback relation was varied. We asked participants to produce same y-directions and different y-directions of stimulus and visual feedback. Based on the study by Dietrich et al. (2012), in which a similar task was used, we expected that performance would be better when the coordinative pattern required same ydirections in visual space. We were further interested in whether we could replicate the previous finding that participants have difficulties performing the different rotation/different y-direction pattern.

The movement-visual feedback relation was transformed by using angular shift transformations. We used 0 and ±180◦ angular shifts as in the previous study, and in addition three positive angular shifts (45, 90, and 135◦, visual feedback is ahead of the movement), and three negative angular shifts (−45, −90, and −135◦, visual feedback lags behind the movement). This was done in order to investigate the influence of magnitude and direction (in advance or behind the hand movement) of the angular shift transformations on coordination performance. If only the pattern in visual space is important for unimanual coordination, the different angular shifts transforming the movement-visual feedback relation should have no effect on performance, i.e., the *accuracy* of performance should be equal for different angular shifts, and should depend only on instructed patterns in visual space. However, if it matters that a transformation modifies the movement-visual feedback relation, best performance should be observed with 0◦ angular shift and performance should be worse with all other angular shifts. The latter was expected based on previous results (Tomatsu and Ohtsuki, 2005; Wilson et al., 2005a; Dietrich et al., 2012; Rieger et al., 2014). Most importantly, we were interested in whether the magnitude of the transformation matters for performance. On the one hand, one could expect that all angular shifts which are not equal to 0◦ are performed equally well (or bad), because they all imply that movement and visual feedback do not match in angular position. On the other hand, this mismatch is more drastic in larger angular shifts than in smaller angular shifts. One may therefore expect that performance varies gradually, depending on the magnitude of the shift. The latter prediction would be in accordance with previous results on gain transformations (Rieger et al., 2014). However, even though the 180◦ angular shift is the most drastic one (visual feedback and movement are a maximal distance apart) it might be easier than smaller angular shifts. A similar effect is found in bimanual coordination, concerning the relation between hands. Opposite y-directions of the hands are (apart from same y-directions) more stable than other relations between the hands (Haken et al., 1985; Zanone and Kelso, 1992; Tomatsu and Ohtsuki, 2005). The difficulty of the movement-visual feedback relation (and/or movement stimulus relation) might follow similar principles as the difficulty of hand–hand relations. A particular benefit of the 180◦ angular shift condition might be observed in the different y-direction conditions: here stimulus and movement have the same y-direction, i.e., participants can rely on movementto-stimulus coordination, which may benefit performance (see Dietrich et al., 2012).

In addition, we varied the speed of the stimulus dot in three levels, because previous studies have shown that coordination performance deteriorates with increasing speed (Kelso, 1984; Haken et al., 1985; Heuer, 1993; Byblow et al., 1995; Carson et al., 1997; Roerdink et al., 2005), especially under transformation conditions (e.g., Salter et al., 2004; Alaerts et al., 2007). Spontaneous switches from difficult to easy coordination patterns more likely occur with higher speed (e.g., Semjen et al., 1995). Therefore, we expected that performance would deteriorate with increasing speed.

In addition to accuracy of performance, we were interested in *how* participants perform the task. Specifically, we were interested in whether participants' movement feedback is on the ideal position as instructed, or whether it systematically lags behind or is advance of (leads) that position. We assumed that the visual feedback dot would be in advance of the stimulus dot when no

transformation is present. Such a lead was previously shown in a similar experimental setup with gain transformations (Rieger et al., 2014). This effect most likely occurred because movements were performed with the right (dominant) hand and the visual feedback was presented on the right side of the screen (as in the present task). In bimanual coordination the dominant hand usually shows a slight lead over the non-dominant hands (Treffner and Turvey, 1995), which seems to be due to attentional factors, because the lead of the dominant hand disappears when attention is directed to the non-dominant hand (Amazeen et al., 1997). However, this lead might be affected by the shift transformation, because the shift causes the visual feedback to lag behind or to be in advance of the movement, which needs to be compensated.

# **MATERIALS AND METHODS**

#### **PARTICIPANTS**

Sixteen adults (nine female and seven male, aged 20–39 years, *M* = 25.6 years, SD = 3.6 years) took part in the experiment. All participants were right-handed according to the Edinburgh Handedness Inventory (Oldfield, 1971) and reported normal or corrected-to-normal vision. They were paid 7 euros/h to participate in a single session. Participants gave informed consent. The study was conducted in accordance with the Declaration of Helsinki and was approved by an ethics committee.

#### **APPARATUS AND STIMULI**

The experiment was programmed using the C-language in a Microsoft DOS environment. Movements were recorded using a Wacom UD A3 writing pad (resolution: 500 pixels per cm, sampling rate 100 Hz), which was connected to the computer via a serial port and positioned on a desk in front of participants. Stimuli were presented on a 17 inch cathode ray tube monitor (screen refresh rate: 75 Hz, resolution: 800 × 600 pixels). The center of the screen was aligned with the midsagittal axis of the participant's body and located 15 cm higher than and behind the writing pad. The background of the screen was black.

The stimulus was a white dot (diameter = 0.43 cm, stimulus dot), moving clockwise on a circular trajectory (radius = 4.32 cm). A second white dot (visual feedback dot, radius 0.43 cm) was controlled by a stylus for the writing pad, which participants held. The stylus was fixed inside a crank (radius 5 cm) and could only be moved in circles. The crank was fixed below a wooden board (15 cm above the writing pad), which also served to shield the hand from view. The center of the circular trajectory of the hand was positioned 10 cm to the right of the body midline. The distance between the centers of the stimulus trajectory and visual feedback trajectory on the screen was 17.27 cm. Participants sat on a height-adjustable chair; eye-screen distance was approximately 60 cm.

### **PROCEDURE AND DESIGN**

Participants were instructed on two characteristics of the visual patterns they were asked to produce. The first instruction concerned the rotation direction of the stimulus-visual feedback relation. Rotation direction could be the same, i.e., both dots moved clockwise, or different, i.e., the stimulus moved clockwise while the visual feedback dot moved counterclockwise. Second, participants were instructed on the y-directions of the stimulusvisual feedback relation. If y-direction was same, stimulus and visual feedback dots both moved upward and downward on the screen at the same time. If y-direction was different, the stimulus dot moved upward while the visual feedback dot moved downward and *vice versa*. An illustration of the patterns in visual space resulting from those instructions can be seen in **Figure 1**, upper part.

The movement-visual feedback relation was manipulated by introducing angular shift transformations. A certain angular value was added to (or deducted from) the hand position before being displayed on the screen. There were eight different angular shifts: 0 and ±180◦ angular shifts, three positive angular shifts (45, 90, and 135◦, visual feedback is ahead of the movement), and three negative angular shifts (−45, −90, and −135◦, visual feedback is behind the movement). For an illustration see **Figure 1**, lower part. Note that in the same rotation direction condition a positive angular shift meant that the visual feedback was shifted clockwise. In the different rotation direction condition a positive shift meant that visual feedback was shifted counter-clockwise. The reverse was the case for negative angular shifts.

The experiment started with a short trial in which participants were asked to turn the crank in order to check whether the writing pad worked properly and to allow participants to familiarize themselves with the apparatus. After that participants read the instructions which stated that the task was to coordinate the visual feedback of circular hand movements with a circling stimulus in four different patterns on the screen. They were explained that those patterns differed with respect to whether the visual feedback trajectory should be rotating in the same or in the opposite direction of the stimulus trajectory, and whether the stimulus dot and visual feedback dot should be on same or on opposite positions of the respective circles. Opposite meaning for example that when the dot of the stimulus trajectory was in the highest position of the stimulus circle, the dot of the visual feedback trajectory should be in the lowest position of the visual feedback circle. To illustrate those patterns, they then saw demonstrations of the four patterns they were asked to produce in visual space. The demonstration consisted of two dots in the positions of the stimulus dot and visual feedback dot, moving in the respective patterns. Participants had the opportunity to ask questions in the instruction phase as well as later prior to each trial, as the experimenter was present during the whole experiment.

After that, the procedure was the same for every trial. At first a two-word instruction for the next trial appeared on the screen, defining the stimulus-visual feedback relation (in terms of rotation direction and y-direction). Participants started trials themselves by pressing the space bar on a keyboard with their left hand. As soon as the space bar was pressed the stimulus dot appeared at the rightmost position of the stimulus trajectory and started moving. The stimulus dot increased its speed every 10 circles by 0.2 Hz (from 0.8 to 1.2 Hz, one trial thus consisted of all three speeds). Each trial lasted 30.83 s. The four visual patterns were blocked. The order of visual pattern blocks was randomized for each participant. Within each visual pattern block each of the eight

angular shifts was presented in one block for six trials, the order of angular shift blocks was randomized. Thus, altogether 192 trials were performed (4 patterns × 8 angular shifts × 6 trials). It took participants between 2 h and 2 h 30 min to complete an experimental session. The duration of the experimental sessions varied, as participants had the opportunity to take breaks between trials.

#### **DATA ANALYSIS**

Because we were interested in performance after participants had adjusted to a certain transformation, we excluded the first trial of each condition from analysis, as this was regarded a training trial. Further, we excluded the first three circles of every speed level, to allow time for adaptation to the new speed requirements. For the remaining data we calculated the angular difference by subtracting the ideal position of the visual feedback from the actual position of the visual feedback (see **Figure 2**). Because the shortest distance between the two points was used, the angular difference cannot be smaller −180◦ or larger than 180◦.

Based on the angular differences, we calculated the percentage of time participants spent in the instructed mode (IM; angular differences between −45 and 45◦) and the opposite mode (OM; angular difference smaller than −135◦ or larger than 135◦; see Dietrich et al., 2012; Rieger et al., 2014 for a similar procedure calculating IM and OM). The expected value for these variables is 25% (random performance). We also calculated the spatial constant error (CE), a signed value indicating the average angular difference between the ideal and the actual angle, which indicates whether participants are in lead of or lag behind the stimulus. We also calculated the temporal CE. The data patterns of the spatial and temporal CE were very similar (as they are related in our task). We therefore decided to report the spatial CE only.

Because participants were not able to perform the instructed pattern in the different rotation/different y-direction condition (see **Figure 2** and analysis below), but rather fell into a different rotation/same y-direction pattern, we did not include this condition in the analysis in which we investigated the effects of the magnitude of the angular shift transformation on performance. Rather, we calculated ANOVAs with the factors Coordination Pattern (different rotation/same y-direction, same rotation/same y-direction, and same rotation/different y-direction), Angular Shift (−180, −135, −90, −45, 0, 45, 90, and 135◦), and Speed (0.8, 1.0, and 1.2 Hz). Because the factor speed did not result in switches to other patterns (performance only declined with faster speeds), we do not report any effects in which this factor is involved.

For the investigation of the roles of movement-to-stimuluscoordination and vision-to-stimulus-coordination only the angular shift 180◦ in comparison to the angular shift 0◦ is of interest. With 180◦ angular shift, performance in the same y-direction condition may suffer, not only because visual feedback and stimulus have different y-directions, but also because movement and stimulus have different y-directions. However, performance in the different y-direction condition may profit, because movement and stimulus have the same y-direction. Thus, one may expect (a) that differences between the same and the different y-direction conditions are smaller with 180◦ angular shift than with 0◦ angular

**and visual speed, separately for each angular shift.**

shift and (b) better performance in the different y-direction condition with 180◦ angular shift than 0◦ angular shift. To investigate this we performed an ANOVA with the factors Rotation Direction (same, different), Y-direction (same, different), Shift (0 and 180◦), and Speed (0.8, 1.0, and 1.2 Hz) on IM. In this analysis we were only interested in interactions involving the factors y-direction and angular shift.

If Mauchly's test indicated that the assumption of sphericity was violated we report Greenhouse–Geisser corrected *F*-values and *p*-values, and Greenhouse–Geisser'ε. *Post hoc* comparisons were conducted using *t*-tests. The significance level for *post hoc* tests was corrected using the Holm–Šídák procedure. Where appropriate exact, minimum (*p*min) and/or maximum (*p*max) *p*-values are reported.

# **RESULTS**

# **ACCURACY OF PERFORMANCE: INSTRUCTED MODE**

In a first step IM and OM were compared to chance in each condition. This analysis indicated that participants tended to be most frequently in the IM in all but the different rotation/different ydirection condition. In the different rotation/different y-direction condition IM was not above chance even with no transformation (0◦ angular shift) but rather below chance (*p*max = 0.005). OM was above chance in this condition (all *p* < 0.001). A similar pattern of results was observed in Dietrich et al. (2012). Thus participants produced predominantly a different rotation/same y-direction pattern when they were instructed to produce a different rotation/different y-direction pattern (see **Figure 2**).

Results for IM are depicted in **Figure 3A**. A significant main effect of Pattern, *<sup>F</sup>*(2,30) <sup>=</sup> 52.66, *<sup>p</sup>* <sup>&</sup>lt; 0.001, <sup>η</sup><sup>2</sup> <sup>p</sup> = 0.78, indicated that IM was significantly lower in the same rotation/different ydirection condition (*M* = 39.4%) than in the other two patterns (different rotation/same y-direction: *M* = 60.7%, same rotation/same y-direction: *M* = 57.5%, both *p* < 0.001). IM in did not significantly differ between the latter two patterns (*p* = 0.11).

**Constant Error (B), and Instructed Mode calculated using Constant Error (C) depending on instructed pattern and angular shift.** For the different rotation/different y-direction condition only the values of 0◦ angular shift and −180◦ angular shift are depicted.

A significant main effect of Angular Shift, *F*(7,105) = 42.38, *p* < 0.001, η<sup>2</sup> <sup>p</sup> = 0.74, was also observed. IMs were highest with 0◦ (*M* = 62.2%) and −45◦ (*M* = 63.4%) angular shifts, which were not significantly different from each other (*p* = 0.55). IM with 0◦ angular shift also did not significantly differ from IM with −90◦ angular shift (*M* = 59.1%, *p* = 0.15), however, IM with −45◦ angular shift was significantly higher than IM with −90◦ angular shift (*p* = 0.017). IM with 0◦ angular shift was significantly higher than IM with all other shifts (*p*max = 0.001). In all other conditions IM successively decreased the greater the angular shift diverged from 0◦ (−180◦: *M* = 45.4%, −135◦: *M* = 50.9%, 45◦: *M* =53.1%, 90◦: *M* =45.0%, and 135◦: *M* =41.3%, *p*min<0.001, *p*max = 0.022).

The decline in IM around 0◦ angular shift was asymmetric: performance was lower with angular shifts 135◦ than −135◦, 90◦ than −90◦, and 45◦ than −45◦ (all *p* < 0.001). Performance was however symmetric around −45◦ angular shift, i.e., was not significantly different between angular shifts −90 and 0◦, −135 and 45◦, and −180 and 90◦ (*p*min = 0.13), and lowest with 135◦ angular shift (*p*min < 0.001, *p*max = 0.022). The significant interaction between Angular Shift and Pattern, *F*(14,210) = 3.26, *p* = 0.006, η2 <sup>p</sup> =0.18, ε =0.43, slightly modified this pattern. In the same rotation/different y-direction condition successive angular shifts did not significantly differ from each other (*p*min = 0.026 in 0 vs. 45◦ angular shifts). Still, there was some indication of a decline in IM the larger the angular shifts were: most (but not all) shifts that were further apart than one step significantly differed from each other (*p*min < 0.001, *p*max = 0.4). The observation of less pronounced decline in IM with higher shifts in the same rotation/different ydirection condition may be due to a floor effect, as performance in this condition was worse than in the other two conditions, leading to less pronounced differences between different angular shifts.

The ANOVA to investigate whether we find subtle effects of movement to stimulus coordination showed significant interactions between Y-direction and Angular Shift, *F*(1,15) = 23.35, *p* < 0.001, η<sup>2</sup> <sup>p</sup> = 0.61, and Y-direction, Angular Shift, and Rotation, *<sup>F</sup>*(1,15)=14.62, *<sup>p</sup>*=0.002, <sup>η</sup><sup>2</sup> <sup>p</sup> =0.49. The difference between same and different y-directions in the different rotation condition was higher with 0◦ angular shift (*M* = 59.7%) than with −180◦ angular shift (*M* = 35.7%, *p* < 0.001). However, in the differenty-direction condition performance was not better with −180◦ angular shift than with 0◦ angular shift (*p* = 0.29). In the same rotation condition the difference between same and different ydirection conditions was not significantly different between 0◦ angular shift (*M* = 19.4%) and −180◦ angular shift (*M* = 14.1%, *p* = 0.24).

# **LEAD/LAG: CONSTANT ERROR**

The results for CE are depicted in **Figure 3B**. With 0◦ angular shift CE was 17.8◦, which was significantly higher than 0◦ (*p* < 0.001), indicating that it may be the default mode of participants to be in advance of the stimulus. The main effect of Pattern was not significant, *<sup>F</sup>*(2,30) <sup>=</sup> 0.39, *<sup>p</sup>* <sup>=</sup> 0.59, <sup>η</sup><sup>2</sup> <sup>p</sup> = 0.03, ε = 0.63. However, a significant main effect of Angular Shift, *<sup>F</sup>*(7,105) <sup>=</sup> 13.61, *<sup>p</sup>* <sup>&</sup>lt; 0.001, <sup>η</sup><sup>2</sup> <sup>p</sup> = 0.48, was observed. With −135◦ angular shift (*M* = 14.2◦) and −180◦ angular shift (*M* = 19.5◦)

CE did not significantly differ from 0◦ angular shift (*p* = 0.13 and *p* = 0.67, respectively). With other negative angular shifts participants were significantly less in advance of the stimulus (−90◦: *M* = 8.3◦, −45◦: *M* = 10.0, *p*max = 0.005) with positive shifts participants were significantly more in advance of the stimulus (45◦: *M* = 26.3◦, 90◦: *M* = 29.2◦, 135◦: 25.8◦, *p*max = 0.019) than with 0◦ angular shift. The interaction between Pattern and Angular Shift, *<sup>F</sup>*(14,210) <sup>=</sup> 2.06, *<sup>p</sup>* <sup>=</sup> 0.08, <sup>η</sup><sup>2</sup> <sup>p</sup> = 0.12, ε = 0.35, did not reach significance.

#### **CONTROL ANALYSES: IM CALCULATED USING MEAN CE**

One may argue that variations in IM are due to systematic variations in CE. Because IM was calculated by using CE values within ±45◦ around the ideal position, it may be that when the mean CE is not 0, parts of the distribution around it are systematically not used in the calculation of IM. To rule out this possibility, we recalculated IM, using a window around participants' mean CE ±45◦ for each condition. The results for IM calculated using mean CE are depicted in the **Figure 3C**. Overall, IM calculated using mean CE (*M* = 74.2%) was significantly higher than IM calculated using the ideal position (*M* = 52.5%, *p* < 0.001).

The ANOVA on IM calculated using mean CE only revealed significant main effects of Pattern, *F*(2,30) = 40.7, *p* < 0.001, η2 <sup>p</sup> = 0.73, ε = 0.68, and Angular Shift, *F*(7,105) = 25.9, *p* < 0.001, η<sup>2</sup> <sup>p</sup> = 0.63, ε = 0.57, but no significant interaction between Pattern and Angular Shift, *F*(14,210) = 1.5, *p* = 0.21, η2 <sup>p</sup> = 0.09, ε = 0.41. Again, IM did not significantly differ between the different rotation/same y-direction condition (*M* = 79.2%) and the same rotation/same y-direction condition (*M* = 79.3%, *p* = 0.95). In those two conditions IM was higher than in the same rotation/different y-direction condition (*M* = 64.0%, both *p* < 0.001).

Instructed mode was significantly higher with 0◦ angular shift (*M* = 82.5%) than with all other angular shifts apart from −45◦ angular shift (*M* = 79.7%, *p* = 0.08, others *p*max = 0.003). IM again successively decreased the greater the angular shift diverged from 0◦ (−180◦: *M* = 68.0%; −135◦: *M* = 71.1%, −90◦: *M* = 76.8%, 45◦: *M* = 77.8%, 90◦: *M* = 70.7%, and 135◦: *M* = 66.9%, *p*max = 0.025, apart from 45 vs. 90◦, *p* = 0.06 and 135 vs. −180◦, *p* = 0.49). The decline in IM around the 0◦ angular shift was again asymmetric: performance was lower with angular shifts 135◦ than −135◦ (*p* = 0.001), and 90◦ than −90◦ (*p* = 0.013), but not 45◦ than −45◦ (*p* = 0.36). Performance was however symmetric around the middle of the angular shifts of −45 and 0◦, i.e., was not significantly different between angular shifts −90 and 45◦ (*p* = 0.53), −135 and 90◦ (*p* = 0.78), and −180 and 135◦ (*p* = 0.49).

### **DISCUSSION**

To investigate how the perceptual-motor system deals with shift transformations in unimanual circling we asked participants to coordinate the visual feedback of their hand movement with a continuously circling stimulus in order to produce four different patterns in visual space. The patterns they were asked to produce consisted of same and different rotations of stimulus and visual feedback, either in same or different y-directions. The movement-visual feedback relation was manipulated using eight angular shifts: (−180, −135, −90, −45, 0, 45, 90, and 135◦). Participants were not able to perform the different rotation/different y-direction pattern. Instead they fell into the different rotation/same y-direction pattern (defined in terms of visual space). The different rotation/same y-direction pattern and the same rotation/same y-direction pattern were performed equally well, performance was worse in the same rotation/different ydirection pattern. Best performance was observed with 0◦ angular shift and with −45◦ angular shift. Performance declined with increasing shift, the 180◦ angular shift condition was no exception. The decline was symmetric around −45◦/between −45 and 0◦ angular shift. Participants did not fully take the angular shifts into account: when angular shifts were negative, the visual feedback was less in advance of the stimulus than with 0◦ angular shift, and when angular shifts were positive, the visual feedback more in advance of the stimulus than with 0◦ angular shift. However, this diminished with higher angular shifts, the CEs of −135◦ angular shift and −180◦ angular shift did not significantly differ from the CE of 0◦ angular shift. No clear indication of movement-to-stimulus coordination in the different y-direction conditions with −180◦ angular shifts was observed.

Similar to Dietrich et al. (2012) the relative difficulty of the coordinative patterns resembles results from bimanual coordination studies (e.g., Swinnen et al., 1997). Participants were not able to produce the different rotation/different y-direction pattern in visual space, but tended to produce the different rotation/same y-direction pattern. This is also the most difficult of the four patterns in bimanual coordination (Swinnen et al., 1997), and participants tend to fall into the easier coordination pattern (Semjen et al., 1995). Further, same y-directions of stimulus and visual feedback were advantageous for performance in comparison to different y-directions between stimulus and visual feedback, an observation which has also been made concerning the y-directions of the two hands in bimanual coordination (e.g., Swinnen et al., 1997). This suggests that the principles by which bimanual and unimanual coordination are governed are similar (see also Buekers et al., 2000; Dietrich et al., 2012). Similar results are also obtained when two people perform coordination patterns together (Schmidt et al., 1990). Therefore, the stimulus circle in the present task may have been represented in a way similar to the way another person performing a movement is represented.

However, in contrast to studies on bimanual coordination, we observed no difference in performance between the different rotation/same y-direction condition (which results in a mirror symmetric pattern on the screen) and the same rotation/same y-direction condition (which results in a parallel pattern on the screen). A similar observation has been made by Dietrich et al. (2012). In bimanual coordination mirror symmetric movements are generally associated with better and more stable performance than parallel movements (e.g., Swinnen et al., 1997; Mechsner et al., 2001; Tseng et al., 2006). Swinnen et al. (1997; see also Kelso, 1984) argue that the performance advantage of mirror symmetric movements is due coactivation of the same (homologous) muscles of the two limbs. Alternatively, or in addition, it has been argued that the specification of equal

movement parameters for both limbs plays a role for this effect (Heuer, 1993; Cardoso de Oliveira, 2002). Because we used a unimanual task, coactivation of homologous muscles cannot occur, and movement parameters are specified only for one hand. The similar performance in the different rotation and the same rotation conditions (with same y-directions between stimulus and visual feedback) can thus be attributed to the absence of such motor constraints. In terms of perceptual constraints, different rotations and same rotations may be equally difficult. Indeed, participants sometimes even prefer parallel motions over symmetric motions when they have to rely on visual feedback (Alaerts et al., 2007).

Performance patterns in visual space were similar under all angular shift conditions, indicating that vision-to-event coordination dominated performance. This is in accordance with unimanual and bimanual coordination research showing dominance of visual information over proprioceptive or kinesthetic information (e.g., Mechsner et al., 2001; Bogaerts et al., 2003; Roerdink et al., 2005) and also research on tool transformations using other tasks (Sutter, 2007; Sutter et al., 2011). The dominance of vision might be due to the quality of visual feedback: visual information is less noisy than proprioceptive information, and visual feedback is usually readily available (Wilson et al., 2005a,b). Another reason for the dominance of vision rather than proprioception may be that vision is more distal than proprioception. It has been suggested that distal rather than proximal movement consequences provide the main reference frame for movement planning and execution (Prinz, 1997; Hommel et al., 2001). Therefore movement representation in the external world may be on the highest level of a hierarchical structure of movement planning and execution (Rieger et al., 2005). However, given that the task was defined in terms of the stimulusvisual feedback relation, the dominance of vision-to-event coordination over movement-to-event coordination may not be surprising.

Nevertheless, performance with 0◦ angular shift (regular visual feedback) and −45◦ angular shift was more accurate than performance with other shifts. Thus, producing visual patterns is not sufficient for coordination, as the patterns were the same in all shift conditions. If only the visual pattern had mattered for performance, the transformations of the movement-visual feedback relation should have had no effect on performance. Thus, in accordance with other studies (Roerdink et al., 2005; Kunde et al., 2007; Massen and Prinz, 2007; Lepper et al., 2008; Sülzenbrück and Heuer, 2010; Dietrich et al., 2012; Rieger et al., 2014) there are costs when a transformation is present. Importantly, performance declined with increasing shift. Thus, the magnitude of the transformation mattered. Larger shifts may have been experienced as more incongruent and therefore more difficult. This is in accordance with findings showing that the likelihood of consciously detecting a transformation depends on its magnitude (Fourneret et al., 2002; Knoblich and Kircher, 2004; Rieger et al., 2014). Performance with small negative angular shifts (i.e., −45◦) was comparable to performance with no angular shift, and the performance decline was symmetrical around an angular shift of less than 0◦. Even though the asymmetry around 0◦ diminished slightly when performance accuracy was corrected for the CE, it was still

present. Thus, it was more advantageous when the hand was in advance of the visual feedback than when it was behind the visual feedback.

The 180◦ angular shift condition was no exception to the decline in performance with larger angular shifts. We had thought that the difficulty of the movement-visual feedback relation might follow similar principles as the difficulty of hand–hand relations in bimanual coordination. In bimanual coordination moving in opposite y-directions is easier than moving at other phase relationships between the hands (apart from moving in the same y-direction; Tomatsu and Ohtsuki, 2005). However, such an effect was not observed. Even in the different y-direction condition the 180◦ angular shift condition was not beneficial. Here, movements and stimulus move in the same y-directions which may have been used to benefit performance. The observation that no movement-to-stimulus coordination occurred in the same rotation condition is in accordance with a previous study in which a similar task was used (Dietrich et al., 2012). However, previously it was observed that movement-to-stimulus coordination occurred in the different rotation condition (180◦ angular shift resulted in better performance than 0◦ angular shift), which indicated that proprioceptive information from the hand was used to aid performance. This was not the case in the present study. Even though the difference between same and different y-directions was larger with 0◦ angular shift than 180◦ angular shift, performance in the 180◦ angular shift condition was not better than in the 0◦ angular shift condition, which speaks against movementto-stimulus coordination. It was particularly surprising that this effect was not found, because vision-to-stimulus coordination was difficult in the different rotation/different y-direction condition. Performance was below chance. Movement-to-stimulus coordination may have been used to improve performance. How can the differences between the two studies (previously we found evidence for movement-to-event coordination, here we do not) be explained? In the present study we used several shifts, but only one shift was used in the previous study. The use of several shifts may have made it harder for participants to detect that they can make use of movement-to-stimulus coordination, as it could not be used in most shifts of the experiment. Thus, the experimental context may have prevented participants from applying this strategy. The failure to detect that such a strategy is possible most likely occurred, because proprioceptive information may not have been perceived with a high spatial accuracy. In similar tasks participants are not very good in knowing their actual hand positions (Fourneret and Jeannerod, 1998; Rieger et al., 2014).

Overall visual feedback was more likely to be in lead of the stimulus, which was also the case when no transformation was present (0◦ angular shift). This may be due to participants' use of the dominant hand in the task. The dominant hand shows a slight lead over the non-dominant hand when coordinating symmetrical movements in bimanual coordination (Treffner and Turvey, 1995). This seems to be due to attentional factors, because the lead of the dominant hand disappears when attention is directed to the non-dominant hand (Amazeen et al., 1997). Participants probably paid more attention to the visual feedback than the stimulus.

The CE was systematically influenced by the magnitude of the transformation. Participants did not fully take the transformation into account: when the angular shifts were negative the visual feedback was less in advance of the stimulus than in 0◦ angular shift, and when angular shifts were positive the visual feedback was more in advance of the stimulus than in 0◦ angular shift. These results are also in accordance with results on shift transformations in bimanual coordination (Tomatsu and Ohtsuki, 2005). Tomatsu and Ohtsuki (2005) asked participants to perform circling movements with the two hands (one clockwise and one counterclockwise) in four different relative phases between the hands: 0, 90, 180 and 270◦. In a transformed feedback condition visual feedback of the right hand was shifted such that performing those patterns in movement space resulted in mirror symmetry in visual space. Thus, as in our task, the shifts were present in movement space but not in visual space. Similar to our results, the right hand was in advance of the ideal angle in the 90◦ shift condition and lagged behind the ideal angle in the 270◦ shift condition (comparable to −90◦ angular shift in our study). This indicated that movements tend to shift toward 0◦ phase relations. Thus, like in our study, the transformation was not sufficiently taken into account. The results are also in accordance with a previous study in which gain transformations were applied in a unimanual coordination task (Rieger et al., 2014): with high gains the visual feedback was in advance of the stimulus suggesting that the magnitude of the gain might be underestimated. With low gains the visual feedback lagged behind the stimulus, suggesting that the magnitude of the gain might be overestimated. Altogether, the results suggest that the magnitude of a transformation is insufficiently taken into account.

However, the CE with −135◦ angular shift and −180◦ angular did not significantly differ from the CE of 0◦ angular shift, indicating that the transformation was accounted for in more extreme shifts. This is in contrast to the results on performance accuracy, which indicate that the 180◦ angular shift condition was not performed better than other shifts. Thus, in terms of *how* the 180◦ angular shift condition and a shift close to it are performed, i.e., the applied strategy, performance resembles the 0◦ angular shift condition. Even though the 180◦ angular shift is the most drastic one (visual feedback and movement are a maximal distance apart), applying the same strategy with 0◦ angular shift might be easier than at smaller angular shifts because the hand is exactly opposite of the visual feedback.

It is assumed that the nervous system controls movements using internal models (Wolpert and Flanagan, 2001) Inverse models choose appropriate motor commands for desired action goals and forward models predict the sensory consequences of motor commands. These predictions can refer to bodily consequences (e.g., kinesthesis and proprioception of the hand movement) and to consequences in external space (e.g., visual feedback). When a movement is transformed as in tool use external consequences do not coincide with the bodily consequences (Wolpert and Flanagan, 2001). In tool use people develop internal models of/adapt internal models to the tool transformation in order to choose motor commands and to make predictions about resulting sensory consequences which take the tool transformation into account (Imamizu et al., 2003, 2007; Rieger et al., 2008; Sülzenbrück and Heuer, 2012). Our data suggest that in the present task internal models do not fully take the shift transformation into account. If that were the case, the case, the CE should not differ between the different angular shifts. However, in accordance with previous studies (Tomatsu and Ohtsuki, 2005; Rieger et al., 2014) the transformation is represented as smaller than it actually is, resulting in imprecision. This is also in accordance with findings that the nervous system does not necessarily completely adapt to observed errors (Wei and Kording, 2009).

The present results have implications for the use of tools with shift transformations. First, such movements are more difficult to perform than untransformed movements. Thus, there are limits to the dominance of visual feedback in controlling actions involving tool transformations (see also Sutter et al., 2013). Second, the representation of the transformation in internal models can be flawed. It is important to note that the performance decrements and flaws in the representation of the transformation were observed even though initial adaptation to gains and speeds was excluded from data analysis. However, with extended practice further adaptation processes may take place. Also, telling participants about the exact nature of the shift transformation may be beneficial for performance, as it has been shown that cueing the transformation is in some cases more beneficial than cueing the action goal in tool actions (Massen and Prinz, 2007; Massen and Sattler, 2012). Knowledge of the nature of the transformation may result in participants consciously choosing strategies to aid performance.

We conclude that the mere presence of a transformation has a negative impact on performance. The representation of the transformation may be flawed. When designing machines or tools that involve transformations between movements and their external consequences, this should be taken into account.

# **ACKNOWLEDGMENTS**

We thank Gudrun Henze and Silke Meissner for their contribution in helping with the experimental setup, data collection, and data preparation. Further thanks go to Andreas Romeyke und Henrik Grunert for helping with the programming of the experiments and building the experimental apparatus. We also thank Victoria Bart for her help with figures and the reference list.

### **REFERENCES**


The CE was systematically influenced by the magnitude of the transformation. Participants did not fully take the transformation into account: when the angular shifts were negative the visual feedback was less in advance of the stimulus than in 0◦ angular shift, and when angular shifts were positive the visual feedback was more in advance of the stimulus than in 0◦ angular shift. These results are also in accordance with results on shift transformations in bimanual coordination (Tomatsu and Ohtsuki, 2005). Tomatsu and Ohtsuki (2005) asked participants to perform circling movements with the two hands (one clockwise and one counterclockwise) in four different relative phases between the hands: 0, 90, 180 and 270◦. In a transformed feedback condition visual feedback of the right hand was shifted such that performing those patterns in movement space resulted in mirror symmetry in visual space. Thus, as in our task, the shifts were present in movement space but not in visual space. Similar to our results, the right hand was in advance of the ideal angle in the 90◦ shift condition and lagged behind the ideal angle in the 270◦ shift condition (comparable to −90◦ angular shift in our study). This indicated that movements tend to shift toward 0◦ phase relations. Thus, like in our study, the transformation was not sufficiently taken into account. The results are also in accordance with a previous study in which gain transformations were applied in a unimanual coordination task (Rieger et al., 2014): with high gains the visual feedback was in advance of the stimulus suggesting that the magnitude of the gain might be underestimated. With low gains the visual feedback lagged behind the stimulus, suggesting that the magnitude of the gain might be overestimated. Altogether, the results suggest that the magnitude of a transformation is insufficiently taken into account.

However, the CE with −135◦ angular shift and −180◦ angular did not significantly differ from the CE of 0◦ angular shift, indicating that the transformation was accounted for in more extreme shifts. This is in contrast to the results on performance accuracy, which indicate that the 180◦ angular shift condition was not performed better than other shifts. Thus, in terms of *how* the 180◦ angular shift condition and a shift close to it are performed, i.e., the applied strategy, performance resembles the 0◦ angular shift condition. Even though the 180◦ angular shift is the most drastic one (visual feedback and movement are a maximal distance apart), applying the same strategy with 0◦ angular shift might be easier than at smaller angular shifts because the hand is exactly opposite of the visual feedback.

It is assumed that the nervous system controls movements using internal models (Wolpert and Flanagan, 2001) Inverse models choose appropriate motor commands for desired action goals and forward models predict the sensory consequences of motor commands. These predictions can refer to bodily consequences (e.g., kinesthesis and proprioception of the hand movement) and to consequences in external space (e.g., visual feedback). When a movement is transformed as in tool use external consequences do not coincide with the bodily consequences (Wolpert and Flanagan, 2001). In tool use people develop internal models of/adapt internal models to the tool transformation in order to choose motor commands and to make predictions about resulting sensory consequences which take the tool transformation into account (Imamizu et al., 2003, 2007; Rieger et al., 2008; Sülzenbrück and Heuer, 2012). Our data suggest that in the present task internal models do not fully take the shift transformation into account. If that were the case, the case, the CE should not differ between the different angular shifts. However, in accordance with previous studies (Tomatsu and Ohtsuki, 2005; Rieger et al., 2014) the transformation is represented as smaller than it actually is, resulting in imprecision. This is also in accordance with findings that the nervous system does not necessarily completely adapt to observed errors (Wei and Kording, 2009).

The present results have implications for the use of tools with shift transformations. First, such movements are more difficult to perform than untransformed movements. Thus, there are limits to the dominance of visual feedback in controlling actions involving tool transformations (see also Sutter et al., 2013). Second, the representation of the transformation in internal models can be flawed. It is important to note that the performance decrements and flaws in the representation of the transformation were observed even though initial adaptation to gains and speeds was excluded from data analysis. However, with extended practice further adaptation processes may take place. Also, telling participants about the exact nature of the shift transformation may be beneficial for performance, as it has been shown that cueing the transformation is in some cases more beneficial than cueing the action goal in tool actions (Massen and Prinz, 2007; Massen and Sattler, 2012). Knowledge of the nature of the transformation may result in participants consciously choosing strategies to aid performance.

We conclude that the mere presence of a transformation has a negative impact on performance. The representation of the transformation may be flawed. When designing machines or tools that involve transformations between movements and their external consequences, this should be taken into account.

# **ACKNOWLEDGMENTS**

We thank Gudrun Henze and Silke Meissner for their contribution in helping with the experimental setup, data collection, and data preparation. Further thanks go to Andreas Romeyke und Henrik Grunert for helping with the programming of the experiments and building the experimental apparatus. We also thank Victoria Bart for her help with figures and the reference list.

#### **REFERENCES**


# The emergence of use of a rake-like tool: a longitudinal study in human infants

# *Jacqueline Fagard\*, Lauriane Rat-Fischer and J. Kevin O'Regan*

*Laboratory Psychology of Perception, CNRS UMR 8242 – Université Paris Descartes, Paris, France*

#### *Edited by:*

*François Osiurak, Université de Lyon, France*

#### *Reviewed by:*

*Luc Augier, Centre National de la Recherche Scientifique, France Sarah Ruth Beck, University of Birmingham, UK*

#### *\*Correspondence:*

*Jacqueline Fagard, Laboratory Psychology of Perception, CNRS UMR 8242 – Université Paris Descartes, 45 rue des Saints-Pères, 75006 Paris, France e-mail: jacqueline.fagard@ parisdescartes.fr*

We describe the results of a longitudinal study on five infants from age 12 to 20 months, presented with an out of reach toy and a rake-like tool within reach. Five conditions of spatial relationship between toy and rake were tested. Outcomes and types of behavior were analyzed. There were successes observed around 12 months in the condition of spatial contiguity between rake and toy, but these could not be interpreted as corresponding to full understanding of the use of the rake. At this age and for the following months, in the conditions involving spatial separation between rake and toy, infants' strategies fluctuated between paying attention to the toy only, exploring the rake for its own sake, and connecting rake and toy but with no apparent attempt to bring the toy closer. Only between 16 and 20 months did infants fairly suddenly start to intentionally try to bring the toy closer with the tool: at this stage the infants also became able to learn from their failures and to correct their actions, as well as to benefit from demonstration from an adult. We examine the individual differences in the pattern of change in behaviors leading to tool use in the five infants, and find no increase in any one type of behavior that systematically precedes success. We conclude that sudden success at 18 months probably corresponds to the coming together of a variety of capacities.

**Keywords: tool use, infants, spatial contiguity, longitudinal, observational learning**

# **INTRODUCTION**

Tool use is the ability to use one object (the "tool") to manipulate other objects, and hence move beyond the limits imposed by the length of one's limbs or the type of one's end-effector (Nabeshima et al., 2006). Tool use has often been recognized as an important step during evolution (van Schaik et al., 1999), and as a marker of the evolution of human intelligence (Wynn, 1985). Its importance as a milestone in human development has also long been recognized (Piaget, 1952) and is still emphasized ("a royal road to the study of problem solving," Keen, 2011, p. 2). And more recently, understanding the basis of tool use has come to be seen as fundamental for robotics (Nabeshima et al., 2006).

Curiously, the development of tool use in human infants has received relatively little interest until recently (see Keen, 2011, and Greif and Needham, 2011, for reviews). In addition, most of the existing studies have been concerned more with describing stages of skill development or factors that induce success, than with suggesting precise learning mechanisms. Furthermore, very few of these studies have been longitudinal.

One possible exception, both for being longitudinal and for looking for mechanisms, is Piaget. Piaget first described "la conduite du bâton" (stick behavior) in 1952. He noticed that his children started to use a stick to move far away objects by the end of the first year. Piaget had noted that his son Laurent discovered the use of the stick "almost without trial and error" (Piaget, 1952, p. 290). The question asked by Piaget in 1952 was "how to explain the transition from trial and error to invention, from motor scheme to representative scheme." Another longitudinal study is that of Connolly and Dalgleish (1989). In this study, Connolly and Dalgleish observed two groups of infants, aged 11 and 17 months, at monthly intervals over a 6-month period, as they tried to use a spoon. However, Connolly and Dalgleish were more interested in changes in the shape of the movement leading to expertise (hand use, grip pattern, spoon trajectory) than in the underlying mechanisms leading to an understanding of the use of the spoon to retrieve the food.

Another exception, not for being longitudinal but for being interested in underlying mechanisms, is a study by Bruner, who observed how children progress from one level of organization to the next when using a primitive form of tool, a lever with fixed fulcrum (Koslowski and Bruner, 1972). From their crosssectional study of 12-to-14, 14-to-16, and 16-to-23-month-old children learning to use the lever to obtain a toy attached to the end of the lever, Koslowski and Bruner extracted some principles to explain how the child progresses from one level of organization to the next. For them, the transition seems to involve the child concentrating on the two individual components of the task (how the rotation of the lever affects the position of the goal, and how the child can effect a rotation of the lever). Once each of the components has been modularized and is less attention-demanding, the child becomes able to attend simultaneously to the movement of both the lever's goal end and its hand end. This allows them to finally envision the solution to the problem.

However, Koslowski and Bruner's lever task is not a real tool use task *stricto sensu*, because the tool is not completely independent of the object to be retrieved. In Koslowski and Bruner's task, the child had to select among several means the appropriate one for achievement of the end state. This kind of task, involving the planful execution of a sequence of steps to achieve a goal, is referred to as a means-end task, and has been amply studied since Piaget (1952), mostly with cross-sectional studies. Examples of means-end tasks are for instance pulling a support to retrieve an object resting on it (Willatts, 1999), pulling a string to retrieve an object attached to it (Uzgiris and Hunt, 1975; Brown, 1990), pulling one of two strings to retrieve an object attached to it one of the strings (e.g., Chen et al., 1997). Tool use, where the tool is completely independent of the toy to be retrieved, as in the study presented here, may be considered as belonging to an extended category of means-end tasks. However it differs from the simplest means-end tasks by the complete spatial discontinuity between the tool and the object to be retrieved.

There have been some cross-sectional studies on tool use, with the tool being independent of the object to be acted upon. For instance, use of a spoon, already the object of Connoly and Dalgeish's longitudinal study, was also considered in a crosssectional study focusing on progress in action planning (McCarty et al., 1999). Nine-, 14-, and 19-month-old children were given a spoon presented in such a way that the bowl part was on the side of the preferred hand. Only the 19-month-olds anticipated the problem and directly grasped the handle with the ipsilateral non-preferred hand, whereas younger infants used their ipsilateral (preferred) hand to grasp the bowl part of the spoon or the handle part with an awkward movement. Another spoon study, also cross sectional, focused on the role of prior experience on tool use (Barrett et al., 2007). In this study, the task was to turn on a light inside a box. The infants showed much less flexibility in grasping the spoon in an unusual way (different from the habitual grasp of the spoon) demonstrated by the experimenter than in grasping a new tool. For the authors, this meant that "rather than learning about tool function. . . infants learn about which part of the tool is meant to be held. . . " (op. cit., p. 352).

Infants generally have ample opportunity to familiarize themselves with use of a spoon, so it is not the best tool on which to study the emergence of tool use from scratch. A few crosssectional studies have investigated what factors contribute to the difficulty to use a new tool to get an out-of-reach object. They all stress that difficulty in tool use increases with an increasing spatial gap between the tool and the object to be acted upon (Bates et al., 1980; van Leeuwen et al., 1994), or more generally with an increasing number of steps needed to achieve the required result (Smitsman and Cox, 2008). In their 1980 study, Bates et al. compared 40 10-month-old infants retrieving an out-of-reach toy placed either on a cloth, at the end of a string, or at different positions near three kinds of tool likely to help the children retrieve it (hoop, crook or stick). The conditions where toy and tool are physically linked ("unbreakable contact") were succeeded most, followed by the conditions in which there was breakable contact (toy placed against/inside the hoop or the curved part of the crook). The conditions with no contact (toy beside the crook or the stick) were succeeded least. The authors concluded that at 10 months solving the problem is easier when the link between the tool and the toy is suggested by the spatial array. Van Leeuwen et al. proposed that the role of spatial contact between tool and toy in helping infants solve the problem was partly linked to the number of mental transformations that the infants must perform to imagine the solution ("number of elements to be integrated," 1994, p. 189).

In summary from this brief review of the literature, we can conclude that spatial proximity, number of transformations, and also familiarity with the tool are important factors, and that planning of action improves with practice. However, we still lack an understanding of the cognitive mechanisms that underlie the acquisition of tool use ability in the course of the second year. In particular, we cannot as yet answer Piaget's question as to whether tool use appears through sudden insight or emerges gradually through progressive familiarization with tool affordances. According to Lockman (2000), tool use emerges from a long period of object manipulation that familiarizes infants with the use of an object to interact with other objects. On this view, the progressive discovery of the various affordances of an object allows infants to later ascertain which affordance will solve their problem. This ecological view contrasts with the more radical view that tool use results from sudden insight (Köhler, 1927).

To more precisely explore the mechanisms underlying the acquisition of tool use, and in particular to ascertain whether this acquisition occurs gradually or through sudden insight, a longitudinal study is called for. We decided to take a small number of infants and study their evolution from ages 12 to 20 months on a regular basis, carefully analyzing their behavior longitudinally as they learned to use a rake-like tool to obtain an out-of-reach toy. The questions we posed were: why is spatial proximity an important factor? What will allow infants to understand the affordance of the rake in conditions of no spatial contact: Observation of their success in easier conditions of spatial contact? Exploration of the rake? Trial and error? Observation of a demonstration? Sudden insight?

Regularly following a small number of infants during the months preceding the acquisition of a skill has in the past proven to be a good way to gather useful information on mechanisms underlying skill acquisition (Piaget, 1952; Thelen et al., 1993). It is also one way to look at individual trajectories as well as common patterns.

# **METHODS**

We constructed a T-shaped rake-like tool made out of white cardboard with a 20-cm-long handle. We used a selection of small toys that we had previously determined to be interesting to children in a day-care nursery. Infants were comfortably seated at a testing table during the whole session. They were either on the parent's lap or, at older ages, in a high chair. By using a white rake, we ensured that the rake was not itself strongly attractive, as compared to the visually highly salient toys that we used so as to attract the attention and trigger the desire of the infants.

Five infants (two girls and three boys) were observed regularly in five conditions: toy on top of rake, attached to it (C1), toy inside/against the rake (C2), toy inside the rake but not against it (C3), toy to the side of the rake (C4) (see **Figure 1**), and rake handed to the infant (C5). The toy was always just out of arm's reach.

All infants were brought in at 11 months for familiarization with the experimental room and the experimenters. Testing started when the infants were 12 months old. They were tested

every month until they could use the rake with success (16 months for one infant, 18 months for three of the infants, 20 months for the fifth). Mean age was 12 months at session 1 (Ses1), 13 months 1 day at Ses2, 14 months 4 days at Ses3, 15 months 6 days at Ses4, 17 months 1 day at Ses5, and 18 months 8 days at Ses6. Four infants took part in 6 sessions. Infant 2 (I2) was seen at 11 months but missed sessions 1 and 2 for family reasons; we kept him in the study for two reasons: first because of the small number of infants; second, because we thought it interesting to compare his performance on his first session with that of the other infants of the same age but who had had two practice sessions. Infant 5 succeeded at session 4 (16 months) and was seen again at 20 months to check the stability of performance. Condition C1 was only tested once at the beginning of each session since it does not represent a challenge for infants at the ages tested here and it is not strictly speaking tool use. Results from this condition are briefly mentioned at the beginning of the Results section but not included in later analyses. The other conditions were tested several times per session. Since this study was exploratory, we decided to test the infants for as long as they were willing to participate, rather than to have exactly the same number of trials per infant. We checked that the difference in number of trials between infants was not related to a difference in age of success. At the beginning of each session, the order of presentation was from C1 to C5, but when an infant was willing to continue, conditions C2, C4, and C5 were retested in unsystematic order. After the first failure on C4 and C5, a demonstration was provided by one of the adults present in the room, either a parent or an experimenter. A demonstration consisted in two or three repetitions of showing the infant, while he or she was looking at the toy/rake, how to bring the object toward himself or herself. The demonstrator always showed how to use the rake from the infant's perspective and accompanied the demonstration by encouraging small talk ("*Look how you can do it, look what I do to get the toy. . .* "). The rake was then either put back in front of the infant (C4) or handed directly to the infant (C5). There were 389 trials in all, and between 1 and 3 demonstrations per session. A trial was terminated if the infant did not try to obtain the toy within one minute, or after failing to retrieve the toy. After getting the toy, the infants were allowed to play with it for about one minute.

The research was approved by a local ethical committee.

#### **CODING OF BEHAVIORS**

We first coded elementary behaviors in each condition of each trial for each of the five infants for the 362 trials of conditions C2–C5. These elementary behaviors involved looking (infant looks at toy, at rake, at adult, or elsewhere); pointing toward toy (with bare hand or with rake); grasping the rake (after touching it by chance, spontaneously, encouraged by the experimenter, or put in the baby's hand by the experimenter), moving the rake (rakes it or lifts it from the table with inside or outside lateral movements, or with a straight movement toward himself; makes a detour around the object or not); refusing the rake (refuses it when handed by experimenter, places it on table, throws it away); manipulating the rake *per se* (puts rake into mouth, bimanually explores rake) or on the table (swipes table, rubs table, hits table); manipulating the rake in connection with the toy with no clear intention to bring it back (hits toy or pushes toy with rake); interacting with the adult, clearly asking for help (gives rake to adult, takes the adult's hand and places it on rake); and manipulating the rake with clear intention to bring the object back (brings object to hand with rake; with wrong movements, peculiar but effective movements, or direct movements; prepares the second hand while raking the toy with the first hand). These elements of behavior occurred together in several ways during trials, leading to a count of 26 whole-trial behaviors among all 362 trials (see **Table 1**). The whole-trial behaviors are grouped into categories as a function of the level of performance they reflect and these categories give a raw score. The notation NT (No Try) means that the child did not try either to retrieve the toy or to explore the rake. T (Toy) indicates that the child was interested in the toy; R (Rake) means that the child was interested in the rake but in neither case was s/he interested in their interaction. T+R indicates that the child was interested in the interaction between rake and toy without showing a clear intention to retrieve the toy. S1 (Success level 1) indicates that the child appeared to show clear understanding of the rake as a possible tool to retrieve the toy but did not yet know how to use the rake. S2 (Success, top level) means that the child clearly knew how to retrieve the toy with the rake.

Notice that in our classification, the last category of behaviors, which we call "A" for "Ambiguous" (behaviors 22–26), has a special status. Behaviors 22–25 occurred in conditions C2 and C3 where it was possible for the child to succeed without any understanding of the functionality of the rake, as we shall see below. This is because, due to the physical position of the toy inside the rake, simply pulling the rake automatically brought the toy into reach. Success in this condition could thus be due to the contingency between rake and toy, and would not necessarily indicate understanding of the function of the rake: hence our coding of "Ambiguous." Finally, there was also another behavior (26) that we could not interpret and that we have included in the "ambiguous" category: sometimes the infant simply grasped the rake and gave it to the adult. She may have done so because she wanted the adult's help and had understood that the rake was the key element, or because she wanted to get rid of the rake. This behavior was observed only eight times in all, in three of the infants.

#### **DATA ANALYSES**

Each infant received a score for each trial, depending on the category it fit in: 0 (No Try), 1 (interested only either in the toy, or in the rake, T or R), 2 (using the rake in connection with the toy, not for retrieval, T+R), 3 (using the rake for retrieval but with

#### **Table 1 | Different strategies observed during a whole trial (in a few trials two strategies, or more rarely three, occurred in succession).**

# **Whole-trial behaviors**

#### **NO TRY (NT)**


3. Refusal

#### **T: BEGGING FOR TOY AND NOT USING RAKE AFTER ITS GRASPING LEADS TO FAILURE**

4. Points to toy and refuses or ignores the rake


7. Grasps rake, gets rid of it (throws it away, places it on the table), and points to the toy

8. Looks at toy, pulls rake while looking at toy, stops action with rake when sees that toy does not come, points to toy

#### **R: EXPLORING RAKE BUT NOT USING IT IN CONNECTION WITH THE TOY**

9. Points to toy, then grasps rake and plays with it (puts into mouth or rubs, swipes, hits, etc. on table)

10. Grasps rake, interested in rake only (puts into mouth or rubs, swipes, hits, etc. on table)

11. Grasps rake, swipes table with it and sweeps toy away by accident

12. Grasps rake, plays with it and then rejects it, may be interested in toy again

# **T+R: USING RAKE IN CONNECTION WITH TOY BUT NOT FOR RETRIEVAL**

13. Points to toy, then grasps rake (spontaneously or encouraged by the experimenter) and touches or pushes toy with it

14. Grasps rake, touches or pushes object with rake

15. Grasps rake (after pointing first to toy or not), points to toy with rake

#### **S1: USING RAKE FOR RETRIEVAL: TRIAL AND ERROR, DIFFICULT OR PARTIAL SUCCESS, OR ONLY AFTER DEMONSTRATION**

16. Grasps rake, moves rake, tries to bring back toy, partial success

17. Grasps rake (after pointing first to toy or not), retrieves or tries to retrieve toy after demonstration

18. Grasps rake after being encouraged (after pointing first to toy or not), moves rake and retrieves toy with it

19. Grasps rake (after pointing first to toy or not), awkward movements to bring toy to hand, success

20. Grasps rake (after pointing first to toy or not), retrieves toy after several attempts

#### **S2: USING RAKE FOR RETRIEVAL: INTENTIONAL MATURE SUCCESS**

21. Grasps rake, moves rake to retrieve toy, success

#### **AMBIGUOUS CASES (NOT INTERPRETABLE, THUS NO SCORE)**

22. Points to toy, hand on rake more or less by chance, grasps rake, rakes with it, toy comes by contingency (at C2 or C3)

23. Points to toy then grasps rake encouraged by experimenter and brings the toy to hand possibly by contingency (at C2 or C3)

24. Points to toy, grasps rake spontaneously, retrieves toy possibly by contingency (at C2 or C3)

25. Grasps rake spontaneously, retrieves toy possibly by contingency (at C2 or C3)

26. Grasps rake (spontaneously or encouraged by the experimenter) and gives rake to adult or grabs adult's hand

*S1 and S2 were coded for C4 and C5 only, when the rake had first to be displaced laterally to be used.*

difficult or partial success or only after demonstration, S1), or 4 (intentional spontaneous mature success, S2).

For some statistical tests we pooled C2 and C3, the two conditions without spatial gap, and C4 and C5, the two conditions with spatial gap.

For each significant effect of ANOVA, the effect size was calculated as partial eta2, using the formula: <sup>η</sup><sup>2</sup> <sup>=</sup> *SS*effect*/SS*total, where *SS*effect = the sums of squares for sessions or conditions, and *SS*total = the total sums of squares for sessions or conditions and errors.

#### **RESULTS**

#### **RETRIEVAL OF THE TOY AS A FUNCTION OF CONDITION AND SESSION**

Before considering the detailed behaviors as classified in our detailed coding scheme, we present in this first section an analysis of overall success, including the ambiguous successes, at retrieving the toy. The results for overall success bear on 389 trials: 27 for C1, 89 for C2, 60 for C3, 118 for C4, and 95 for C5, in all. Most of the time the infants were interested in the task. They sometimes expressed frustration at not being able to get the toy, but they rarely refused a trial. NT (No Try) was coded for 31 trials (7.9%). NT never occurred in C1. For the four other conditions the percentage of NT did not change with condition (*p* = 0*.*52).

#### *Toy attached to the rake (C1)*

When the toy was attached to the rake (C1), the infants grasped the rake without hesitation and then detached the toy from the rake (see **Figure 2A**). They almost never first reached or pointed toward the toy in this condition (see below results on pointing as first behavior and **Figure 2B**). All infants looked clearly at the toy from the start of their pulling movement. This shows that visual information sufficed for them to understand that the toy was connected to the rake. Success was always 100%, starting on the first trial.

#### *Toy inside the rake (C2 and C3)*

The rate of toy retrieval was high as of the first session, particularly for C2, as can be seen in **Figure 3** which represents the mean percentage of success in which the toy was retrieved successfully. Rates of success for C2 and C3 did not differ significantly. In C2, in the first session infants most often immediately grasped the rake to make a raking movement leading to successful retrieval. When the toy was not against the rake (C3), these successful retrievals represented only 39% of trials.

The rate of toy retrieval in C2 showed a U-shaped form. After the first session and the rather stereotyped behavior seen in it (the majority of observed strategies were A25), infants demonstrated various behaviors in C2, as we shall see below. An ANOVA on the frequency of object retrieval in C2 as a function of session showed a significant and moderate effect [*F*(5*,* 15) = 4*.*3, *p <* 0*.*02; partial <sup>η</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>*.*59]. An LSD *post-hoc* test indicated that the percentage of retrieval was almost significantly higher at the first compared to the third session (*p* = 0*.*06). The percentage of retrieval was significantly lower at sessions 2, 3, and 4 than at sessions 5 and 6. Percentage of success in C3 showed an increase across sessions but no statistics were calculated on C3 alone because of missing data.

#### *Toy to the side of the rake (C4 and C5)*

All infants younger than about 16 months failed to retrieve the toy when it was not inside the rake, except two infants who succeeded once in the third session but did not repeat it. Successes in C4 and C5 showed a rather sudden increase between sessions 5 and 6 (see **Figure 3**). In session 6, all five children succeeded

**FIGURE 2 | (A)** Grasping the rake directly (C1), **(B)** Reaching/pointing toward the toy (C4).

in C4 and C5, although they still did not succeed on all trials, as can be seen in **Figure 3**. An ANOVA on the frequency of object retrieval in C4 and C5 combined (percentage of "S1" + "S2") as a function of session showed a significant and large effect [*F*(5*,* 15) <sup>=</sup> <sup>15</sup>*.*9, *<sup>p</sup> <sup>&</sup>lt;* <sup>0</sup>*.*001; partial <sup>η</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>*.*73]. A LSD *post-hoc* test indicated that the percentage of retrieval was significantly different in session 6 as compared with all the other sessions, which did not differ significantly from each other. Interestingly, infant 2 who missed sessions 1 and 2, and is compared with the others for age (that is, he is included in session 3 at age 14 months as if it was his third session even though it was his first session) is well within the mean of all infants (see **Figure 7** for individual results).

#### *Comparison between conditions*

In term of success, an ANOVA on the frequency of retrieval as a function of condition and session was calculated. For this calculation we used the mean frequency of success at C2 and C3, the mean frequency of success at C4 and C5 and compared both of them to success at C1 (See **Table 2**). Results show a significant main significant and large effect of condition [*F*(2*,* 30) = 94*.*2, *<sup>p</sup> <sup>&</sup>lt;* <sup>0</sup>*.*0001; partial <sup>η</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>*.*84], a significant small main effect of session [*F*(5*,* 30) <sup>=</sup> <sup>11</sup>*.*2, *<sup>p</sup> <sup>&</sup>lt;* <sup>0</sup>*.*001; partial <sup>η</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>*.*80], and a significant large effect of condition x session interaction [*F*(10*,* 30) = <sup>6</sup>*.*5, *<sup>p</sup> <sup>&</sup>lt;* <sup>0</sup>*.*0001; partial <sup>η</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>*.*84]. A LSD *post-hoc* test shows that the condition effect is due to a difference between all conditions, C1 being better than C2–C3, itself better than C4–C5. For the session effect, it is due to a difference between sessions 1, 2, 3, 4 on one side and 5 and 6 on the other side. The first four sessions do not differ significantly from each other. The difference between sessions 5 and 6 almost reach significance (*p* = 0*.*05). A LSD *post-hoc* analysis on the condition x session interaction indicates that condition 1 is better than conditions 2–3 at the first 4 sessions only, and better than conditions 4–5 at all sessions, and that conditions 2–3 are significantly better than conditions 4–5 at all sessions.

Another analysis that we did on all conditions before moving to the more qualitative analyses of strategies concerns reaching toward / pointing to the toy as a first behavior. Infants frequently pointed to the toy before grasping the rake. As already mentioned, they almost never did it in condition C1 when the toy was attached on the rake. Pointing as a first behavior increased with task difficulty. An ANOVA on the percentage of reaching/pointing as a function of condition (C5 excluded, since in this condition the rake was handed directly to the infant) indicated a significant and large effect of condition on reaching/pointing [*F*(3*,* 12) <sup>=</sup> <sup>12</sup>*.*8, *<sup>p</sup> <sup>&</sup>lt;* <sup>0</sup>*.*001; partial <sup>η</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>*.*73]. An LSD *posthoc* test indicated that the percentage of reaching/pointing was

#### **Table 2 | Mean success (%) as a function of session and condition (pooling C2–C3 and C4–C5).**


significantly lower in C1 than in all other conditions, and lower in C3 than in C4 (see **Figure 4**).

In conclusion, as of 12 months of age, retrieval of the toy was always successful when the toy was attached to the rake (C1), often successful when the toy was inside and against the rake when infants were 12 and 13 months old (C2) but less so on the next two sessions, and not successful at all when the toy was to the side of the rake until 16–20 months of age depending on the infants (C4–C5). Thus, early successes in C2 did not appear to help much in allowing the infants to understand how to use the tool, since these early successes were followed by many failures in C2 and by almost total failure in C4 and C5. In order to get cues to understand the U-curve shape observed in C2 and the relatively sudden onset of success observed in C4 and C5, and to answer our other questions (Why is spatial proximity an important factor? What helps infants understand the affordance of the rake in conditions of no spatial contact: Exploration of the rake? Trial and error? Observation of a demonstration? Sudden insight?), we shall undertake a finer analysis of behaviors as a function of condition and session. This is the purpose of the following section.

# **FINER ANALYSIS OF BEHAVIORS AS A FUNCTION OF CONDITION AND SESSION: WHAT DO THESE BEHAVIORS TELL US ABOUT INFANTS' UNDERSTANDING OF THE RAKE'S FUNCTIONALITY?** *Toy inside the rake (C2)*

In the following paragraphs we shall analyse more finely the behaviors observed in condition C2 in order to try to understand the origin of the U-shaped curve in retrieval rate observed over

the successive sessions. As mentioned above, in Session 1 the most frequent behavior was elementary behavior A25 (60% as a mean for all infants), in which the child almost immediately grasps the rake and pulls it. Because the toy is spatially inside the tool, the toy generally comes along with the rake, and the child is able to retrieve it. This stereotyped direct pulling of the rake observed in session 1 for C2 decreases in frequency in sessions 2 (46.7%) and 3 (23.3%), being replaced by more varied behaviors in the next three sessions. By then, infants often pointed to the toy before pulling the rake (behaviors A23 and A24) or they started to rake the toy but the object was not brought near enough to be retrieved, they did not use a further raking movement to retrieve the toy and instead pointed to the toy with the empty hand (behaviors 5–7). Another frequent behavior was grasping the rake and playing with it (behaviors 13–15). More generally at this stage infants frequently took an interest only in the toy (reaching/pointing to the toy while ignoring the rake or after discarding it), or the rake (exploring the rake by itself, putting it into the mouth, rubbing, sweeping or hitting the table with it) (see **Figure 5**). Connecting rake and toy not for retrieval (touching or hitting it), was not often observed, except for one infant who used it from the first session.

We interpret all these behaviors typical of the few sessions following the first one as showing that the high rate of toy retrieval observed in the first session did not reflect a real understanding of the rake's functionality. There were three main reasons why we consider the first successes at C2 as ambiguous/uninformative and not reflecting a clear understanding of the affordance of the rake: the first is that the rate of toy retrieval decreased marginally significantly from session 1–3. The second reason is that when the infants started to rake the toy but failed to bring it close enough to grasp it, they never tried a second time to pull the toy with the rake: instead, they discarded the rake and pointed toward the toy with the empty hand. The third reason is that in several cases during the second to fourth session the infants did not pull the rake on the table but grasped it and lifted it over or around the toy before pointing toward the toy with the empty hand (see **Video S1**). In order to understand the origin of this pattern of behaviors, we may suppose the following. Infants may have grasped the rake as their first action either because it was the closest object or because they believed the toy to be attached to it as in C1 (and it may have taken them some time to realize that this was not the case). In any case, because the toy was touching the rake or almost touching the rake, the simple

**of condition.**

pulling of the rake was enough to bring the toy closer most of the time. When, in the following sessions, rather than pulling the rake directly, infants moved the rake around the toy and then pointed to the toy, begging for it with their bare hands after getting rid of the rake, they may have been showing their interest in the toy but also that they, at this stage, knew that the rake and the toy were not connected. Such behavior may also show that they did not know that the rake could be used to bring an unconnected object closer. Ultimately however, by sessions 5 and 6, the rate of successful retrieval in C2 went up again, probably corresponding to a true understanding of the rake's affordance.

It is worth noting that there were interindividual differences in the way new behaviors replaced the systematic pulling of the rake in the first sessions. Infant 1 was more interested in the toy than in the rake and behavior "T" replaced "A" at the following sessions. Infant 2 frequently explored the rake by itself on the third session (his first session). Infant 3 was very interested in exploring the rake from the beginning, either in connection with the toy or alone. For him, behavior "T+R" was frequent especially at sessions 2–4. Infant 4 was the infant whose ambiguous successes in C2 decreased the least after the first sessions. For her, behaviors "T" and "R" were frequent in sessions 3 and 4. Infant 5 showed the lowest rate of ambiguous success at the first session and either pointed to the toy (T) or was interested in exploring the rake from the beginning (R).

In conclusion, observation of behaviors in condition C2 across sessions indicates that after the early successes of the first sessions, infants' behavior changed in sessions 2–4. Instead of immediately pulling the rake, they either pointed to the toy, sometimes after discarding the rake, or they grasped the rake and explored it. Thus, in those sessions, infants tended to pay attention either to the toy or to the rake but they seldom connected the two objects and when they did, it was not to retrieve the toy. These switches between different strategies across sessions are comparable to the overlapping wave patterns described by Chen and Siegler (2000) in their microgenetic study of tool use at 18–35 months of age.

If, as we suggest, the early successes in toy retrieval in condition C2 were only due to the physical proximity between rake and toy (so that any movement of the rake would tend to bring the toy closer), rather than to a true understanding of the rake's functionality, this may explain why there was no rapid transfer from "successes" in C2 to successes in C4 and C5.

We will next analyze behaviors in conditions C4–C5 in order to elucidate how children understood how to use the rake when the toy was clearly separated from the tool.

#### *Toy not near the rake (C4 and C5)*

We have seen that despite all their experience of success (expected or not) when the toy was inside the rake, when it was clearly separate from the rake (in C4 and C5) it took the infants several sessions and many trials to understand how to use the rake to retrieve the toy. In particular, it took the infants 23–35 trials in all in C4 and C5 (median: 28 trials) across 4–6 sessions (mean: 5) to succeed, and they were aged 16–20 months (mean 17.8 months) when they reached this stage. If the infants did not learn much from their own "unexpected" success in C2, then how did they learn to use the rake in conditions C4 and C5? By exploring the rake? By trial and error? By watching a demonstration by an adult? In the following section we explore these alternatives by checking which behaviors preceded success in C4 and C5.

*Exploring the rake alone and in connection with the toy.* In this section we ask whether exploring the affordances of the rake over successive sessions allowed the child to accumulate enough knowledge to finally make the link between rake and tool, and thereby accomplish the task.

First of all, exploring the rake itself ("R") was very frequent over the successive sessions (see **Figure 6**). It was the second most frequent behavior (20.5%, all sessions considered) after behavior "T" (36.4%). Connecting the rake with the toy (T+R) was less frequent (16.4%). Note that the "T+R" behaviors of the first sessions seemed not to be directed toward retrieving the toy (see **Video S2**), and were very different from behaviors 16 or 17 of S1 (see **Table 1**) observed in the last sessions where infants clearly connected the rake with the toy to try to retrieve it even though they failed. Hitting the toy with the rake seemed to be a game *per se* in the first sessions, and infants who used this strategy did not even grasp the toy systematically when it happened to come within reach after they hit it.

A second point is the following: individual patterns showed that all five infants fluctuated between the different strategies across sessions (see **Figure 7**). Sometimes they mostly pointed toward the toy, sometimes they mostly explored the rake, and at other times they mostly connected rake with toy. Doing statistics on the evolution of the different strategies across sessions would be misleading as it is clear that the five infants switched in different ways between pointing to the toy, exploring the rake, and connecting the rake with the toy (T+R) during sessions preceding success. What is common across infants is the large amount of fluctuation and the lack of a clear, single tendency: we might have expected, for instance, to observe an increase in the connection

between rake and toy in the session preceding success, but this was not the case.

Thus, we found little evidence that gradual accumulation of knowledge about rake affordances leads to the ability to make the connection between rake and toy. There was frequent rake behavior but it did not gradually increase; nor was rake + toy behavior systematically preceded by frequent rake behavior.

*Learning from trial and error.* While the infants appeared not to have learned from their unexpected successes in C2 during the first session, we wondered if they learned from their errors in C4–C5. In other words, did they correct their movements after trying unsuccessfully to grasp the toy with the rake? There is some indication of this, since behavior S1, which reflects awkward or partly successful attempts to use the rake to obtain the toy (trial and errors), was more frequent in the first half of the first successful session (22.2%) than in the second half of the

same session (17.5%), whereas S2 increased from 1.6 to 11.1% (see **Video S3**).

*Learning from demonstration by an adult.* Another mechanism to learn how to use a rake might be to observe others doing it. This would be a more economical method than trial and error. As mentioned above, in all sessions, after the first failure in C4 and C5, infants received a demonstration from either the parent or one of the experimenters (usually two demonstrations in a row). Infants clearly did not learn much from the adult's demonstration until late in the study. With only one exception (infant 1, session 3), none of the infants succeeded in retrieving the toy with the rake in C4 or C5 right after a demonstration before the sixth session. In addition, infant 1 did not repeat her success before the sixth session, either before or after demonstration. To check whether the behavior had been influenced by the demonstration despite not sufficing to lead to retrieval of the toy, we compared the level of performance, indexed by the obtained score, on the trials preceding and following demonstration for C4 and C5 considered together (see **Figure 8**). It can be seen that the score on trials just following demonstration did not differ greatly from the score of the trials preceding a demonstration until the last session. An ANOVA was performed on the score as a function of condition (×2, before and after the demonstration), and of session (×4, we choose to start at session 3 to be able to include infant 2) with repeated measures. It showed no main effect of condition, a significant and large main effect of session [*F*(3*,* 12) = 31*.*3, *<sup>p</sup> <sup>&</sup>lt;* <sup>0</sup>*.*001; partial <sup>η</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>*.*89], and a significant and moderate condition x session interaction [*F*(3*,*12) = 7*.*5, *p <* 0*.*01; partial <sup>η</sup><sup>2</sup> <sup>=</sup> <sup>0</sup>*.*65]. A *post-hoc* LSD test indicated that on the last session the score after demonstration differed significantly from the score before demonstration (*p <* 0*.*0001). Thus, infants started to benefit from demonstration relatively late, and not before 18 months.

In sum, when the toy was not inside the rake, infants started to use the rake to retrieve the toy between 16 and 20 months of age. Before that, they either explored the rake *per se* or focused on the toy, or to a lesser extent made some connection between rake and toy but apparently without the intention to retrieve the toy with

the rake. When successes first appeared, they often were awkward and partial, or they happened after demonstration by an adult. But neither of these behaviors was observed during the first sessions. It could be that the capacity to correct inadequate motor planning (trial and error strategy), and the capacity to benefit from a demonstration (observational learning) require that the infant already have some intuition of the solution.

### **DISCUSSION**

As noted in the introduction, few controlled experimental studies have considered in detail what mechanisms might underlie the acquisition of the use of a rake-like tool in infancy, and among these, only one is longitudinal. The essential conclusion from the existing studies is that spatial proximity, number of planning steps, and familiarity with the tool are factors that play a role. However these studies present no hypotheses about what drives progress in tool acquisition. In particular, they do not elucidate the question of whether the child learns to use the rake through progressive familiarization with use of the rake in interacting with other objects, or whether more or less sudden insight is involved.

Even though our conclusions are incomplete, the present longitudinal study makes some preliminary qualitative steps toward answering this question, and toward sketching out possible mechanisms underlying tool acquisition. The approach was to study a small number of infants, and analyse their detailed behavior at regular intervals. In this respect, the study differs from most contemporary approaches, and comes closer to the old, more qualitative observational methods of Piaget. As such the conclusions do not have the same statistical value as is usual in today's studies, but they provide valuable ideas for further work.

A first interesting point that our results brings to light concerns the interpretation of early successes observed in the literature in cases when there is no spatial gap between rake and toy.

#### **EARLY SUCCESSES WHEN THERE IS NO SPATIAL GAP**

Our study confirms previous work showing that situations where there is no spatial gap between the rake and the toy, infants as young as 9–12 months can have little difficulty retrieving the toy (Bates et al., 1980).

But our results allow us to provide more insight into these early successes than has previously been coming forth.

First, with respect to the situation where the toy is attached to the tool: here our results clearly show that at 12 months (and most probably before that) children already know that they can move one part of a rigid object by moving another part: all infants succeeded as of the first trial and looked clearly at the toy from the start of their pulling movement. The result is compatible with extensive work using purely perceptual measures at even earlier stages of development (e.g., Spelke and Van de Walle, 1993).

But second, an important contrast exists with respect to the situation where the toy is contiguous but not attached: it can be touching or with a small spatial gap but within the trajectory of the rake. Here our results show quite distinctly that successes in such cases do not correspond to real understanding of the function of the rake as a tool, but to the fact that because of its spatial proximity, playing with the rake will likely cause the toy to move. Evidence that the infant had no notion that the rake would bring the toy closer is first: when the child moves the toy with the rake but not far enough to grasp it, the child will often not continue using the rake but stretch out with its hand to try to get the toy; and second, after a successful trial, in a subsequent trial an infant will often grasp the tool and move it around the toy before pointing to the toy (see Voulomanos, 2011, for other examples of u-shape developmental curves).

#### **MEANS-END BEHAVIOUR**

The conclusion from these considerations is thus that real understanding of the use of the rake as a tool only emerges in our data after about 18 months, and that early successes without a spatial gap do not correspond to proper understanding. This is consistent with our previous cross-sectional study (Rat-Fischer et al., 2013), but it raises the question of the relation to the literature on means-end behavior.

In the literature it is sometimes claimed (Willatts, 1999) that means-end behavior of various types is observed as early as 8 months, examples being given of the case of a cloth support, string, or of an extended tool like a stick, to obtain an out of reach object.

Our results lead us to ask whether success in such studies could be reinterpreted in a way similar to what we proposed for the rake task in the no-spatial-gap condition: Could it be that in many classical means-end tasks, successes before the second year of life were accidental, and due to the fact that any small motion of the cloth/string/tool will have tended to bring the object into motion, thereby drawing attention toward the object, causing the child to look at it, and then allowing the child to attain it by manual grasping. For instance, Willatts' (1999) experiment showing apparent clear presence of intentional cloth-pulling to retrieve a toy at 8 months can be re-interpreted as infants' having (1) an automatic cloth-pulling action which they put into play whenever confronted with the cloth/toy situation; and (2) having a larger attentional span in peripheral vision, and as a consequence being more likely to notice the toy moving and thus to look at it; and (3) having overall larger arm motions, thus making the cloth move further on every pull. These three mechanisms would result in coding as "intentional" (measured among others by probability of attaining the toy, probability of looking at it). A further suggestion that younger infants might not actually fully, practically, understand the function of the tool, can be got from experiments in which the infant is given the choice of several strings, or between different tools, in order to solve the task. In such situations it is known that children do not succeed immediately until well into the second year (Brown, 1990). Similarly, in one study on 14 infants aged 16 months, we also observed that infants rarely chose the correct string among a set of four when three were non-connected (Rat-Fischer et al., under revision). It can always be claimed that difficulty in such situations derives from confusion, attentional load, or goal/sub-goal competition induced by the visually more complicated set-up, but a more parsimonious account of the results when taken together with our present findings, might be that infants in fact do not have proper practical understanding of the notion of tool as a means to attain an object until about 18 months. Sommerville and Woodward's (2005) observation that 10-month old infants, as a group, do not "planfully" succeed in a cloth support task is consistent with these ideas. Note, however, that in the same study, the authors observed that when a rectangular box was substituted for the cloth, 10-month-old infants "planfully" succeeded in pulling the box to retrieve the toy, thereby demonstrating a sensivity to the causal structure.

The above re-interpretation of 8-month-old's successes at many classical means-end tasks, does not, we suggest, apply to our 12-month-old's success at C1 (toy attached to the rake). There we claim the children's behavior is truly intentional, reflecting knowledge that moving one part of the rake will move another part. This seems likely because the children were older, but also the data clearly demonstrate that all infants succeeded as of the first trial and looked clearly at the toy from the start of their pulling movement.

# **PATH TO SUCCESSFUL USE OF THE TOOL WHEN THERE IS A SPATIAL GAP**

A major purpose of the present study was to try to cast light on the process that leads to infants finally understanding the notion of tool. We were hoping that among the different behaviors, behavior T+R, that is, the behavior of bringing the rake into contact with the toy, would allow the infant to test the affordances of the rake and bring the child closer to understanding its functionality. We thus expected that T+R would generally increase before the child demonstrates success. Such findings would have been in line with Kahrs et al.'s (2012) observation of progress in the kinematics of banging movements between 7 and 14-months, that the authors considered as a pre-adaptation for later instrumental hammering, as well as with the observation of "non-random errors" and exploration of objects preceding tool-use in young animals (Meulman et al., 2014).

Curiously however this was not what we found. Taking together the data for all infants, we found that the T+R category of behaviors was not obviously correlated with subsequent successes at C4–C5, and occurred about equally often in all sessions preceding Sessions 4 and 5 where successes started occurring. Looking at the data individually for each infant also did not reveal any tendency for T+R or any other behavior to increase clearly just before success for any infant.

#### **FACTORS LIMITING SUCCESS**

One first limiting factor could have been that infants actually did not have the goal of retrieving the toy. However this does not seem to be the case. Even though some infants showed an interest in playing with the rake, the desire to get the toy was evident at some point in all sessions for all infants.

As already mentioned, in C4–C5 infants frequently pointed toward the toy as their first action. This pointing might be an example of pre-potent action patterns that young learners must inhibit in order to solve the problem (by using the tool). The difficulty of overcoming prepotent actions has been demonstrated with young animals (cf. review by Meulman et al., 2014). Inhibition of prepotent action patterns may be facilitated by maturation of the prefrontal cortex (Diamond and Gilbert, 1989).

Attentional limitations might be another limiting factor. In C4-C5 it is generally the case that after first trying to attain the toy by pointing toward it, infants lose interest and then switch their attention to the rake. However this attention shift does not imply that infants know that the rake can be used to get the toy. On the contrary, our evidence suggests that infants' goal is now purely to explore the rake for its own sake. After some tool exploration, infants then often revert to pointing toward the toy. It could be the case that dividing attention between the task at hand (how to retrieve the toy?) and the affordance of a novel object (what kind of actions can be done with a rake?) involves excessive cognitive load for the infant.

Another limiting factor to be considered is manual dexterity. It could be argued that physical inability to move the tool with an effective movement limits the chance of success. There are two reasons why we do not think this to be the case: first, once the infants started to try to use the rake to retrieve the toy, these partial successes were very rapidly followed by efficient successes, within the same session. In other words, after trying to use the rake to retrieve the toy, infants might be awkward for the first trial but corrected their error almost immediately. Second, in another study, pure visual exposure to a parent using the rake several times at weekly intervals, without the infant itself being allowed to manually manipulate the rake, was enough to significantly advance the age of success by between two weeks and two months (Somogyi et al., under revision).

#### **LACK OF LEARNING FROM OBSERVATION IN OUR STUDY**

Another piece in the puzzle that must be integrated into a theory explaining the emergence of rake use is our striking result on the effect of demonstration from an adult: infants were only able to profit from a demonstration precisely around the age when they would in any case be able to do the task spontaneously1. This is consistent with our previous cross-sectional study bearing on 60 infants, aged 14, 16, 18, 20, and 22 (Rat-Fischer et al., 2013) and with other work showing that proper understanding of the causal structure of means-end tasks in observational learning only matures in the second half of the second year (Meltzoff, 1995; Bellagamba and Tomasello, 1999; Huang et al., 2002).

However, it is in contrast to some other recent research which has investigated the ability of infants to solve means-end problems by observing an adult perform the task. In these experiments it was found that in certain means-end tasks, infants are able to profit from observation of a demonstration as early as 12 months (Provasi et al., 2001; Esseily et al., 2010; see Elsner, 2007, for a review). Such findings seem incompatible with our current finding that even after demonstration, infants were unable to succeed in using the rake until about 18 months.

As a way to explain the incompatibility, a possibility might be to claim that the particular materials employed by Esseily et al. and Provasi et al. had the property that even a fairly approximate imitation of the adult's demonstration would tend to lead to success. The means-end tasks used by these authors thus more closely resembled the *no spatial gap* conditions of our experiment, where infants were frequently successful because, however they moved the rake, the toy was likely to come closer. This is in contrast to

<sup>1</sup>Although we have very recently demonstrated that emphasizing the demonstrator's intentions may advance the age of success, cf. Esseily et al. (2013).

the spatial gap conditions of our experiment, where a particular, precise form of raking motion is necessary for success.

To explain the ability to learn from demonstration from an adult in Esseily et al. and Provasi et al., we could then appeal to the fact that infants as early as 6 months have the capacity to *imitate* actions that they are shown (or have seen) (see Poulson et al., 1989; Elsner, 2007; Elsner et al., 2007, for reviews). Because of the relative simplicity of the tasks involved in Esseily et al. and Provasi et al., such imitation might then have led to higher success rates in the demonstration conditions. But under this hypothesis, these successes would not have corresponded to real understanding of the functionality of the means that led to success. In our experiment, where the task is somewhat more complex involving two stages (first grasping the rake, then adequately manipulating it), such imitation without understanding will not have led to success.

In conclusion, this longitudinal study of five infants learning how to use a rake reveals the interest (and difficulty!) of studying individual behaviors in a particular task over several months. No single type of behavior in our study seemed to lead systematically to success, leading us to suggest that many processes are involved. It may be that a variety of experiences involving familiarization with objects, exploration of object affordances, attentional factors, social cues, action planning, some of them associated with personal experience, others associated with brain maturation, each contribute small amounts of expertise that all come together fairly suddenly around 18 months to allow the child to understand that the rake can extend the body's range of action. There may be different routes to success and the mechanisms leading to success may differ from one infant to the next. An interesting question for future work will be to manipulate factors or conditions that allow infants to acquire tool use earlier than the second year—an example being our finding that simple visual exposure to the parent using the rake several times at weekly intervals alone can accelerate progress by many weeks (Somogyi et al., under revision). Future work should also include using different materials to check to what extent the children transfer their knowledge to different tools and different situations.

#### **ACKNOWLEDGMENTS**

J. Kevin O'Regan was supported by ERC Advanced Grant 323674 FEEL.

#### **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at: http://www*.*frontiersin*.*org/journal/10*.*3389/fpsyg*.* 2014*.*00491/abstract

**Video S1 | Infant 1, 15 months, C2: discarding the tool.**

**Video S2 | Infant 3, 12 months, C4: hitting the toy with the rake.**

**Video S3 | Infant 1, 18 months, C5: trial and error and observational learning before first full success.**

#### **REFERENCES**

Barrett, T. M., Davis, E. F., and Needham, A. (2007). Learning about tools in infancy. *Dev. Psychol.* 43, 352–368. doi: 10.1037/0012-1649.43. 2.352


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 11 February 2014; accepted: 06 May 2014; published online: 23 May 2014. Citation: Fagard J, Rat-Fischer L and O'Regan JK (2014) The emergence of use of a rake-like tool: a longitudinal study in human infants. Front. Psychol. 5:491. doi: 10.3389/fpsyg.2014.00491*

*This article was submitted to Cognition, a section of the journal Frontiers in Psychology.*

*Copyright © 2014 Fagard, Rat-Fischer and O'Regan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Tool use imagery triggers tool incorporation in the body schema

# *Matteo Baccarini 1,2†, Marie Martel 2,3†, Lucilla Cardinali 1,2†, Olivier Sillan1,2, Alessandro Farnè1,2† and Alice C. Roy 2,3 † \**

*<sup>1</sup> Lyon Neuroscience Research Center, INSERM U1028, CNRS UMR5292, ImpAct Team, University Lyon1, Lyon, France*

*<sup>2</sup> Hospices Civils de Lyon, Mouvement et Handicap, Neuro-immersion, Lyon, France*

*<sup>3</sup> Laboratory on Language, Brain and Cognition (L2C2), CNRS UMR 5304, Cognitive Sciences Institute, University Lyon 1, Lyon, France*

#### *Edited by:*

*Cristina Massen, Leibniz Research Centre for Working Environment and Human Factors, Germany*

#### *Reviewed by:*

*Martina Rieger, UMIT - University for Health Sciences, Medical Informatics and Technology, Austria Irina M. Harris, University of Sydney, Australia*

#### *\*Correspondence:*

*Alice C. Roy, Laboratory on language, brain and cognition (L2C2), CNRS UMR 5304, Cognitive Sciences Institute, University Lyon 1, 167 bd Pinel, Bron 69500, France e-mail: aroy@isc.cnrs.fr*

*†These authors have contributed equally to this work.*

Tool-use has been shown to modify the way the brain represents the metrical characteristics of the effector controlling the tool. For example, the use of tools that elongate the physical length of the arm induces kinematic changes affecting selectively the transport component of subsequent free-hand movements. Although mental simulation of an action is known to involve -to a large extent- the same processes as those at play in overt motor execution, whether tool-use imagery can yield similar effects on the body representation remains unknown. Mentally simulated actions indeed elicit autonomic physiological responses and follow motor execution rules that are comparable to those associated with the correspondent overt performance. Therefore, here we investigated the effects of the mental simulation of actions performed with a tool on the body representation by studying subsequent free-hand movements. Subjects executed reach to grasp movements with their hand before and after an imagery task performed with either a tool elongating their arm length or, as a control, with their hand alone. Two main results were found: First, in agreement with previous studies, durations of imagined movements performed with the tool and the hand were similarly affected by task difficulty. Second, kinematics of free-hand movements was affected after tool-use imagery, but not hand-use imagery, in a way similar to that previously documented after actual tool-use. These findings constitute the first evidence that tool-use imagery is sufficient to affect the representation of the user's arm.

#### **Keywords: tool-use, mental imagery, body representation, action, kinematics**

# **INTRODUCTION**

Tool-use modifies our perception of the world around us. Several studies on tool-use in both healthy and brain-damaged populations have consistently reported that tool use alters our perception of space. Two main interpretations have been put forward to account for perceptual changes observed following tool-use: either space perception *per se* would be altered in such a way that far stimuli become processed as if they were nearer (Berti and Frassinetti, 2000; Maravita et al., 2001; Farnè et al., 2005; Witt and Proffitt, 2008; Holmes, 2012; Osiurak et al., 2012; Bourgeois et al., 2014), or alternatively tool-use would displace the attentional focus to the tip of the tool (Holmes et al., 2007). These effects have been related to plastic features of the multisensory processing of the peripersonal space, as identified electrophysiologically in non-human primates: in monkeys trained to retrieve distant objects with a rake, Iriki et al. (1996) revealed that visuotactile hand centered receptive fields appeared to extend along the tool axis (see for review, Maravita and Iriki, 2004; Cardinali et al., 2009a; Brozzoli et al., 2012). In addition, tool-use modifies the spatial metric of our own body. When asked to point to touched landmarks on their arm (middle fingertip, wrist, elbow) after using a mechanical grabber to reach and grasp objects, neurotypical participants localized these landmarks as if their touched body-parts were more distant from each other than before tooluse (Cardinali et al., 2009b, 2011a; Sposito et al., 2012; Miller et al., 2013). Most interesting for the present study, besides modifying space and body perceptual metrics, tool-use shapes our actions. The body representation we use for action (i.e., the body schema) is modified when using tools in a way such that the tool is incorporated and becomes part of our body (Baccarini and Maravita, 2013). In humans, we demonstrated that using a mechanical grabber that extends the arm's functional length by 40 cm, extends the subject's arm length representation (Cardinali et al., 2009b, 2012). Our sensorimotor system seems to be able to immediately transfer the control from the arm to the new arm + tool configuration (Van der Steen and Bongers, 2011). The motor control of free-hand reaching movements performed right after use of this tool exhibits an altered kinematics: the representation of an elongated arm in the body schema translates in the later occurrence and reduced amplitude of some kinematics events (acceleration, velocity, deceleration peaks). Such changes in motor control of the arm have been considered as the key kinematics signatures for the incorporation of the tool into the body schema (Cardinali et al., 2011b) and revealed the latter is a highly plastic representation that quickly builds-up on previous experience.

Strikingly, mere mentally simulated motor experiences are sufficient in some cases to trigger subsequent actions modifications and athletes commonly use motor imagery to improve their performance (Driskell et al., 1994; Roure et al., 1999). Motor imagery might be sufficient to acquire functional object knowledge, however the built representations have been shown to be less detailed than when experiencing actual movement with the object (Macuga et al., 2012; Paulus et al., 2012). Moreover, brain areas recruited to perform actual or imagined movement execution are not strictly overlapping (Imazu et al., 2007; see for review Dietrich, 2008). Nevertheless a contagion from movement imagery to movement execution is possible as both evolve on a similar time-scale and follow very similar biomechanical rules. Motor imagery follows so faithfully the constraints imposed to the motor system that the postural adjustments normally accompanying a voluntary reaching movement while standing up are also present in an imagined reaching situation (Boulton and Mitra, 2013). Execution time of mentally simulated movements has been shown to be comparable in duration to actually executed movements (Papaxanthis et al., 2002; for review, see Jeannerod and Frak, 1999; Guillot and Collet, 2005). An important constraint of the motor system is the speed accuracy trade-off known as Fitts law (1954). According to this law, increasing the velocity of execution of an action leads to decrease in accuracy, and conversely, increasing the accuracy demands increases the time needed to perform the task. Several studies have demonstrated that Fitt's law holds in motor imagery, imagined movement times linearly increasing with task difficulty (Decety and Jeannerod, 1995; Maruff et al., 1999). In a prehension task paradigm, Frak et al. (2001) had subjects to physically or mentally grasp a cylinder between the index and thumb while varying the orientation of the axis formed by the opposed fingertips on the object, the so-called opposition axis. When free to adopt a natural finger positioning on the object, subjects typically tend to keep the opposition axis invariant from trial to trial, as changing it determines an additional cost on the musculo-articulatory system (Paulignan et al., 1997). Frak et al. (2001) elegantly demonstrated that prehension movements requiring different pre-determined orientations of the opposition axis induce similar modulations of movement time for both physically executed and imagined movements. Recently, Jacobs et al. (2010) used a similar paradigm to investigate free-hand grasping and grasping with a handheld tool. Subjects' performance during mental imagery respected the bio-mechanical constraints imposed by the tool during real movement execution (see also Rieger and Massen, 2014). Tooluse imagery has been less explored but is known to modify space perception as tool execution does (Witt and Proffitt, 2008; Davoli et al., 2012; Gabbard and Caçola, 2013) and to follow Fitt's law (Macuga et al., 2012). Most recently it has been reported that expert tool-users are sensitive to the held tool during imagery whereas naive tool-users are not (Bisio et al., 2014).

On the one hand, thus, evidence from real tool-use indicates that it modifies the kinematics of subsequent free-hand movements as if they were performed with a longer arm; on the other hand, mental imagery of tool-use seems to reproduce tool-use execution quite accurately. Taken together, these findings raise the question of whether mere tool-use imagery is sufficient to modify the representation of the arm's length. To answer this question we designed an experiment in which the rationale was the following: if imagining using the same mechanical grabber that extends the arm's length by 40 cm (Cardinali et al., 2009a,b, 2012) is sufficient for this tool to be incorporated into the body schema and thus increases the subject's represented arm length, then the real execution of free hand prehension movements subsequent to tool-use imagery should display those kinematics signatures that we observed after actual tool-use. Since motor imagery is known to be modulated by task difficulty, varying task difficulty is an efficient way to control that motor imagery was properly performed (Lotze and Halsband, 2006). We therefore applied the paradigm introduced by Frak et al. (2001), and manipulated the orientation of the opposition axis to be used to grasp a cylinder in order to vary movement's difficulty. In different sessions separated by one day, participants were required to perform prehension movements toward objects with different oppositions axes before and after having mentally simulated these movements with their free hand (as a control), or using the mechanical grabber.

# **MATERIALS AND METHODS**

#### **PARTICIPANTS**

Sixteen neurologically healthy subjects (8 male; mean age 22.4 years; *SD*: 3.7; range from 18 to 32) participated in the study. All were right-handed and had normal or corrected-to-normal vision. All participants gave written informed consent to participate in the study, which was approved by the local ethics committee and conformed to the Helsinki Declaration.

#### **APPARATUS AND PROCEDURES**

Participants were comfortably seated in front of a table with the right hand closed in a pinch-shaped grip on a switch. The left hand, palm down, was pressing a response button. The target object was a plastic cylinder (5 cm in diameter and 17 cm height) placed on the table at a distance of 35 cm along the sagittal axis, in line with subjects' right shoulder. Two colored dots on the upper edge of the cylinder marked the grasp landing positions required for the tips of the thumb (red) and index fingers (yellow). The virtual line connecting these two points of contact determined the Opposition Axis (OA) of the grip. At the beginning of each trial, the cylinder was presented with one of three possible OA, namely −22◦, 0◦ and +22◦ with respect to the subject trunk. Each OA was presented an equal number of times in a pseudorandomized order. A horizontal arrow was taped at 13 cm of height from the table on a wooden block, located about 10 cm to the left of the cylinder and served to indicate the height at which the participants had to lift the object (**Figure 1**).

The experiment consisted of three tasks, each presented over two consecutive days: Pre-imagery free-hand grasping task (18 trials), Motor Imagery task (54 trials), and Post-imagery free-hand grasping task (18 trials). During the Pre- and Postimagery free-hand grasping task, participants were required to reach, grasp and lift the target object up to the arrow with their right hand (see **Figure 2**). They were instructed to grasp the object using a precision grip by placing their thumb and index fingertips on the respective colored dots. Once the trial was performed, participants got back to the starting position, closed their eyes and

**panel).** Subjects placed their right hand on a starting switch (purple) and the left hand on a response button (green). The target object was a cylinder, located 35 cm from the starting point. On its upper side were

waited for an acoustical "go" signal to open their eyes and perform the next trial.

During the Motor imagery task of day 1, subjects were required to imagine using their right hand to reach for, grasp and lift the cylinder up to the height indicated by the arrow. They were instructed to wait for an acoustical go signal to open their eyes and start imagining performing the task. Participants had to raise their left hand to release the switch as they started their imagery the opposition axis, which could be of three orientations: −22◦, 0◦ and +22◦. On the left an arrow indicated the height to which the object should be lifted.

trial, and to put their hand back down to press the switch once it was accomplished (i.e., the object was lifted at the height indicated by the arrow). Participants' right hand was kept still on the right switch during the whole duration of the imagery task. Two pauses were planned after 18 trials and 36 trials respectively. Other pauses were delivered if required. During pauses participants were allowed to open and close the fingers of their right hand and to move their arms. The Motor imagery task of day 2 was identical to that of day 1, except that subjects had to imagine performing the prehension movement with a grabber they were holding still in their right hand. The grabber was constituted of an ergonomic handle (9 cm) fitted with a lever, a 33-cm-long rigid shaft, and a "hand" with two articulated fingers (10 cm). Squeezing the lever (vertically) made the "fingers" of the tool close (horizontally). The grabber used here was identical to that used in previous work documenting effects of actual too-use on subsequent free-hand kinematics (Cardinali et al., 2009b). During the whole duration of the imagery task the "tool fingers" were kept in a pinch grip posture on the start switch. During pauses, subjects were allowed to move the arm, but could not drop the tool. In order to be able to imagine using the grabber, at the end of day 1 subjects were familiarized with the tool by performing 18 grasping trials (6 for each opposition axis). Tool-use imagery never took place on day 1 to avoid potentials tool integration effects to carry over on day 2.

#### **KINEMATIC RECORDING**

Three infrared light emitting diodes (IREDs) were placed on the subjects' right hand: on the medial lower corner of the thumb nail, on the lateral lower corner of the index finger nail and on the skin proximal to the styloid process of the radius at the wrist. The reaching component of the movement was characterized by the wrist marker displacement, while the grip component was characterized by the thumb and index displacement. Spatial localization of the markers was recorded with an Optotrak 3020 (Northern Digital Inc; sampling rate: 200 Hz; 3D resolution: 0.01 mm at 2.25 m distance). Analyzed parameters included latencies and amplitudes of acceleration, velocity and deceleration peaks for the transport component, and latency and amplitude of the maximum grip aperture for the grip component. The total movement duration of imagined movements (from release to press of the left response button, corresponding to the same events of actual movements) was also extracted.

#### **STATISTICAL ANALYSIS**

To assess the effect of the OA on imagined movements, subjects' average imagined movement durations (MD) were submitted to a repeated measure ANOVA with Effector (hand/tool) and Opposition Axis (−22◦/0◦/+22◦) as within-subject factors. In order to establish the effect of motor imagery with the tool on subsequent free-hand movements, we performed a repeated measure ANOVA on movement kinematic parameters with type of Imagery (hand/tool), Session (pre / post imagery) and Opposition Axis (−22◦/0◦/+22◦) as within-subject factors. When necessary, Newman-Keuls *post-hoc* test were used.

#### **RESULTS**

#### **MOVEMENT DURATIONS DURING MOTOR IMAGERY**

As shown in **Figure 3**, the analysis revealed no significant difference between hand and tool imagined movement durations [*F*(1*,* 15) = 0*.*60, *p* = 0*.*45; *MD* = 2538 vs. 2633 ms]. A main effect of Opposition Axis [*F*(2*,* 30) <sup>=</sup> <sup>16</sup>*.*0, *<sup>p</sup> <sup>&</sup>lt;* <sup>0</sup>*.*001; <sup>η</sup><sup>2</sup> *p* = 0*.*52] highlighted that the most difficult OA(−22◦) required longer performance time (*MD* = 2702 ms) compared to the other orientations (0◦ *MD* = 2489 ms; +22◦ *MD* = 2565 ms, all *p*-values *<* 0.002), which tended to differ between them (*p* = 0*.*055). The interaction between Effector and Opposition Axis almost reached significance [*F*(2*,* 30) = 3*.*0, *p* = 0*.*065], potentially suggesting that OA may have a slightly different impact on tool and free-hand imagery.

Taken together these results highlight the difficulty raised by the most unnatural opposition axis (−22◦) irrespective of the used effector, indicating that participants performed free-hand and tool imagery tasks reliably.

#### **EFFECT OF FREE-HAND vs. TOOL MOTOR IMAGERY ON SUBSEQUENT FREE-HAND MOVEMENTS**

To investigate the effects of tool-use imagery on the subsequent free-hand movement execution, the following section focuses on the critical interaction between the factors type of Imagery (hand vs. tool) and Session (pre vs. post imagery) across the kinematic parameters of the transport and grasping components (see **Tables 1, 2** for an exhaustive report of the statistical results and means respectively). Two of such interactions were found to be significant, for the wrist velocity peak [*F*(1*,* 15) = 11, *p <* 0*.*01; η<sup>2</sup> *<sup>p</sup>* = 0*.*42] and the deceleration peak [*F*(1*,* 15) = 9*.*76, *p <* 0*.*01; η<sup>2</sup> *<sup>p</sup>* = 0*.*39; see **Figure 4**]. Free-hand imagery did

not induce any significant modifications on the subsequent movements' kinematics (velocity peak: pre: 773 mm/s vs. post: 793 mm/s, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*30; deceleration peak: pre: <sup>−</sup>2511 mm/s<sup>2</sup> vs. post: <sup>−</sup>2632 mm/s2, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*28). The pre imagery session of day 1 differed from that of day 2, in that subjects reached higher velocity and deceleration peaks -before motor imagery- in day 2 as compared to day 1 (all *p <* 0*.*01), compatible with some practice effects. Critically, participants' free-hand movements performed after tool-use imagery exhibited significantly decreased wrist velocity peak (pre 827 mm/s vs. post: 785 mm/s, *p <* 0*.*02) and deceleration peak (pre: <sup>−</sup>2829 mm/s2 vs. post: <sup>−</sup>2569 mm/s2, *p <* 0*.*04) with respect to those performed before tool imagery. As expected, no significant interaction was found on the kinematic parameters of the grasping component.

#### **DISCUSSION**

Here we investigated the effects of tool-use imagery vs. free hand imagery on subsequent free-hand grasping movements. As movement imagery is sensitive to task difficulty we asked participants to conform their final grip to three opposition axes of varying difficulty. In line with our expectations, the opposition axes differently taxed imagined movement durations, thus confirming that participants successfully engaged in the imagery tasks. Our analysis then focused on the differences of free-hand imagery vs. tool-use imagery on subsequent movements. While free hand grasping imagery did not affect actual free-hand movements, the latter movements performed after tool-use imagery were characterized by significant decrease of both wrist velocity and deceleration peaks. Together with previous findings from our group, these results indicate that imagery of tool-use may be sufficient to update the representation of the arm length used to execute free-hand movements. We have indeed reported previously that using a tool to grasp an object modifies the kinematics of subsequent free-hand movements as if the participant performed object prehension with a longer arm (Cardinali et al.,




*MGA, maximum grip aperture. Significant p values (<0.05) are reported in bold.*

# **Table 2 | Main values ± 1 s.e.m. of each kinematic parameter according to the full factorial design.**


*MGA, maximum grip aperture.*

2009b), and we proposed that these kinematic modifications are the fingerprint of the tool incorporation in the body schema (Cardinali et al., 2011a,b, 2012). Here we investigated whether tool-use imagery could be sufficient to induce such modifications of the body schema. While imagery has been largely explored in psychology and cognitive sciences, tool-use imagery has become a field of investigation only recently. Rieger and Massen (2014) have examined how different tools translate in different tool imagery performances by requiring participants to color a rectangle using pens with different thicknesses. As it was the case for physically executed actions, imagined actions were influenced by the pen's thickness, the thinnest one giving rise to longer movement times

to fill-up the rectangle. In the same vein, Macuga et al. (2012) reported that despite some inaccuracy, the Fitt's law holds for movements imaginarily performed with tools.

To make a step forward, here we tested whether tool-use imagery effects, besides influencing ongoing performance *during* the tool-use imagery task, can last sufficiently to modify actual movements performed *afterwards*, without the tool. We first aimed to ensure that imagery was accurately performed. To this aim, we varied the difficulty of an object prehension task by requiring participants to grasp a cylinder putting their index and thumb or the tool's "fingers" in predetermined positions on the cylinder, thus creating different orientations of the opposition axis(OA) between the fingertips. Our findings confirm and extend those of Frak et al. (2001) as we show that the −22◦ orientation of the OA is the most difficult and hence time consuming one, irrespective of whether movements were imagined with the hand or the tool.

When considering the effects of tool imagery on subsequent movements, our results make a considerable step further by demonstrating that tool-use imagery is sufficient to warrant tool incorporation in the body schema (i.e., the representation the brain uses to plan and execute actions). When comparing free hand movements performed before and after tool-use imagery, movement kinematics presented wrist velocity and deceleration peaks of decreased amplitude. Previously, after physical tool-use, we reported such reductions in amplitude for the very same kinematic parameters, accompanied by protracted latencies and discussed these kinematics modifications as the hallmark of tool incorporation in the body schema (Cardinali et al., 2009b, 2012). Similar to previous work, here the direction of the changes triggered by tool-use imagery on the subsequent movement kinematics (i.e., the reduction of maximum velocity and deceleration peaks) is compatible with a change of the represented length of the arm in the direction of its elongation. Compared to short-arm people, long(er)-armed participants naturally tend to perform the same grasping action with reduced velocity and deceleration peaks. For such movements, they also tend to display longer latencies of these parameters (see supplemental data in Cardinali et al., 2009b). In the present study, a relatively brief tool-use imagery task appeared sufficient to reduce the maximal amplitude of transport component parameters, thus suggesting profound consequences for real movements, from imagined movement execution. In contrast to our previous work, the latencies of the same parameters were not significantly modified by tool-use imagining suggesting that although very similar, tool-use imagery is not in all respects identical to actual tool-use execution. Nevertheless the modifications in motor control did replicate those found after actual tool-use both in the direction (i.e., reduction) and specificity, affecting selectively the transport component parameters and leaving the grasping ones unaltered (Cardinali et al., 2009b, 2012).

Noteworthy, the modifications on real hand movements induced by tool-use imagery unambiguously points to a tool incorporation in the body schema (e.g., reduced wrist velocity) and as such differ from the learning effects typically reported after mental practice (increased performance due to increased velocity). This observation finds additional support in the results of free-hand imagery performed in day 1. Indeed, normal subjects are by essence experts in performing manual prehension and hence mental training with the very same effector was ineffective in triggering any significant kinematic modification of subsequent executed movements (Allami et al., 2007). Moreover, the pre imagery session of day 2 as compared to that of day 1 displayed increased velocity and deceleration peaks, an effect that is exactly opposite to the one observed after tool-use imagery.

Finally, potential limitations of our study need to be addressed. First, the lack of execution session with the tool, before motor imagery, prevented us from directly comparing execution and imagery movement duration with the tool. Our main aim was not to compare tool execution and tool imagery (see Rieger and Massen, 2014 and Macuga et al., 2012 for this comparison), rather our study focused on tool-use vs. free-hand imagery effects on subsequent free-hand movements. A second potential limitation arises from the fact that to avoid potential carry-over effects freehand imagery and tool-use imagery were not counterbalanced, tool-use imagery occurring always on day 2 after hand imagery. The post-test performed on day 2 is thus the fourth time subjects executed the free-hand grasping task. One could have expected a facilitation effect similar to that observed between the pre session of day 1 and 2; by contrast, velocity and deceleration peaks decreased after tool-use imagery, an effect that is thus compatible with our previous results obtained after physical tool-use. Third, it might have been of interest to directly compare the consequences of both tool-use execution and tool-use imagery. Since we used the very same paradigm and grabber as the one we used for evaluating the effects of tool use execution (Cardinali et al., 2009b, 2012), the results obtained here nevertheless point to some differential effect of imagined vs. real tool-use, as the velocity and deceleration peaks, but not the latencies of these parameters, were affected by tool-use imagery.

To conclude, tool-use imagery not only adheres to most of the physical rules of actual movement execution, but has protracted consequences on the real execution of movements performed afterwards without the tool that are readily understandable as the product of previous tool incorporation.

#### **ACKNOWLEDGMENTS**

This work was supported by FRC (Fondation pour le Recherche sur le Cerveau), the Labex/Idex (ANR-11-LABX-0042), the James S. McDonnell Foundation and the ANR Samenta ASD-BARN 01502. Matteo Baccarini was supported by the Fondazione Carlo Fornasini-PoggioRenatico.

#### **REFERENCES**


Holmes, N. P. (2012). Does tool use extend peripersonal space? A review and reanalysis. *Exp Brain Res*. 218, 273–282. doi: 10.1007/s00221-012-3042-7


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 16 January 2014; accepted: 06 May 2014; published online: 30 May 2014. Citation: Baccarini M, Martel M, Cardinali L, Sillan O, Farnè A and Roy AC (2014) Tool use imagery triggers tool incorporation in the body schema. Front. Psychol. 5:492. doi: 10.3389/fpsyg.2014.00492*

*This article was submitted to Cognition, a section of the journal Frontiers in Psychology.*

*Copyright © 2014 Baccarini, Martel, Cardinali, Sillan, Farnè and Roy. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Differential mechanisms of action understanding in left and right-handed subjects: the role of perspective and handedness

# *Rachel L. Kelly and Lewis A.Wheaton\**

*Cognitive Motor Control Laboratory, School of Applied Physiology, Georgia Institute of Technology, Atlanta, GA, USA*

#### *Edited by:*

*Cristina Massen, Leibniz Research Centre for Working Environment and Human Factors, Germany*

#### *Reviewed by:*

*Thierry Chaminade, Centre national de la recherche scientifique, France Massimiliano Conson, Second University of Naples, Italy Aine Ni Choisdealbha, Lancaster University, UK*

#### *\*Correspondence:*

*Lewis A. Wheaton, Cognitive Motor Control Laboratory, School of Applied Physiology, Georgia Institute of Technology, 555 14th Street, Atlanta, GA 30332-0356, USA e-mail: lewis.wheaton@ap.gatech.edu* The ability to comprehend outcomes of skilled action is important for understanding the world around us. Prior studies have evaluated the perspective an action is performed in, but few have evaluated how handedness of the actor and the observer interact with action perspective. Understanding handedness affords the opportunity to identify the role of mirroring and matched limb action encoding, which may display unique strategies of action understanding. Right and left-handed subjects were presented with images of tools from egocentric or allocentric perspectives performing movements by either a left or right hand. Subjects had to judge the outcome of the task, and accuracy and latency were evaluated. Our hypothesis was that both left and right-handed subjects would predict action best from an egocentric perspective. In allocentric perspectives, identification of action outcomes would occur best in the mirror-matched dominant limb for all subjects. Results showed there was a significant effect on accuracy and latency with respect to perspective for both right and left-handed subjects. The highest accuracies and fastest latencies were found in the egocentric perspective. Handedness of subject also showed an effect on accuracy, where right-handed subjects were significantly more accurate in the task than left-handed subjects. An interaction effect revealed that left-handed subjects were less accurate at judging images from an allocentric viewpoint compared to all other conditions. These findings suggest that action outcomes are best facilitated in an internal perspective, regardless of the hand being used. The decreased accuracy for left-handed subjects on allocentric images could be due to asymmetrical lateralization of encoding action and motoric dominance, which may interfere with translating allocentric limb action outcomes. Further neurophysiological studies will help us understand the specific processes of how left and right-handed subjects may encode actions.

**Keywords: perspective, handedness, action understanding, limb-matched, mirror-matched, tool use**

"fpsyg-04-00957" — 2013/12/17 — 17:29 — page 1 — #1

# **INTRODUCTION**

Understanding skilled action is a basic aspect of our daily living. Skilled action in humans frequently involves the use of tools in order to complete action goals. In order to understand skilled tool-use actions, we must understand at least two elements: how to identify the tool needed for a specific task (contextual knowledge) and understand how the tool is used to complete the action goal (physical knowledge; Mizelle andWheaton, 2010). Our knowledge of a tool comes from the fact that we learn tool and action associations in our cognitive-motor system and from this knowledge we are able to use it to understand not only how to accomplish skilled actions ourselves, but also how to predict the ultimate goal of actions executed by others.

Previous research suggests how action understanding occurs through observation (Fadiga et al., 1995; Iacoboni et al., 1999; Bekkering et al., 2000; Rizzolatti et al., 2004). Action understanding likely requires an imitative capability that allows a persons' motor system to precisely organize body motion in order to achieve an observed movement. The ideomotor theory describes that action and the perception of action are related by common neural systems (Massen and Prinz, 2009). Thus perceiving another's actions or action outcomes elicits the same action in the observer's motor system. It has been proposed that when viewing a tool or object, not only are the physical elements of the scene being processed, but also an additional higher level of processing occurs which can prompt "functional affordance" representations (Mizelle et al., 2013). In this work, functional affordances are the possible object-based tool actions that best "afford" a desired action goal. When subjects looked at static correct tool and object images, sensorimotor activation was observed which indicated that action was being understood and the motor system was being driven. Type of tool or object also affects the ability to understand the ultimate action goal. New tools might not be able to simulate a motor plan as would a known tool; however, our previous work (Mizelle et al., 2011) has shown that after directly training with a novel tool one time, it activates the same neural tool network that known tools activate.

Seeing an action and being able to recognize the possible outcomes are vital for not only the potential of motor simulation of action, but also for understanding the tool-action outcomes themselves. What remains unclear is what particular variables impact the perception of action and the understanding of action goals.

One variable that has been studied is the perspective of observed actions. It has been suggested that perspective encompasses not only visual objects in a scene, but also how concentration is focused in order to determine specific judgments about the environment (Lindgren, 2012). In this work, first and third person perspectives in a virtual world simulation were analyzed. Results indicated that there was a significant advantage in subject's memory for tasks and task related elements when watching first person perspective simulations. Subjects also achieved higher accuracy during recall. How this applies to interpreting action based on other people's movements that are typically in the third person perspective is still unclear. Mentally simulated actions from an egocentric perspective are considered visually and motorically familiar (Ni Choisdealbha et al., 2011; Conson et al., 2012) as this affords optimization of motor imagery and action encoding. Alternatively, the allocentric perspective may not be motorically familiar to oneself, and in order to process allocentric action, motor imagery may necessitate visual transformations. In Ni Choisdealbha et al. (2011), they showed that right and left-handed subjects were faster at judging hand stimuli in an egocentric orientation that corresponded to their own dominant hand. It was proposed that this effect was due to better utilization of visual and sensorimotor information to facilitate judgments in the dominant limb. In allocentric orientations, behavioral strategies shifted to "visual only" so that subjects could reorient the stimuli to align with "self" as a method for interpretation. This in turn suggests that subjects use a self-centered motor strategy to interpret action.

However, it is unclear how a subject's handedness and the hand involved in seen actions may affect these results. In previous work, it has been shown that the left cerebral hemisphere is specialized for tool-use action (Raymer et al., 1999; Frey et al., 2005). Neuroimaging studies have shown left lateralization in right-handed participants for both left and right hand tool pantomime movements (Moll et al., 2000; Choi et al., 2001; Johnson-Frey et al., 2005; Bohlhalter et al., 2009; Cabinio et al., 2010). Further, left parietofrontal lateralization for performance of tool-use action was observed in left and right-handed subjects using their dominant hand (Vingerhoets et al., 2012). This evidence leads to the indication that damage to the left cerebral hemisphere resulting in ideomotor apraxia (which causes the inability to correctly perform tool-use and communicative gesture on command) should be a bilateral deficit (Wheaton and Hallett, 2007). Apraxia can be seen in both hands after left hemispheric damage, which suggests that the left hemisphere network controls skillful tool-use knowledge for both left and right hand movements (Heath et al., 2003).

It is worth considering that in left-handed subjects, there is a unique hemispheric dissociation which exists for motor planning of tool use (left parietofrontal) and primary motor cortices (right motor cortex). Whether this dissociation is disadvantageous to understanding action outcomes is a key goal in this work. It has been argued that encoding seen action utilizes a principle of motor resonance, where seen actions may be encoded in

"fpsyg-04-00957" — 2013/12/17 — 17:29 — page 2 — #2

the observer's motor system, perhaps using motor representations from the contralateral hemisphere of the seen arm (Gallivan et al., 2013). For actions seen in an egocentric (first person) perspective, limb-specific resonance is achievable. Under these circumstances, right-handed subjects watching a right-handed action would have no dissociation of motor planning and primary motor cortex. However, due to the diminished left lateralization of motor activation of left-handed action in right-handed subjects (Cabinio et al., 2010), there is the potential for some dissociation for right-handed subjects watching left-handed action. This assumes that action is encoded in the subject's limb that matches the seen action. It is unclear what would happen in left-handed subjects, where seeing a right-handed action may bring tool-use activation and motor activation into the same hemisphere. Further, we frequently have to understand actions in daily living, and we commonly view them from an allocentric (third person) perspective. There are two possible ways an action can be encoded in the allocentric perspective in order to understand that action: limb-matched and mirroredmatched (**Figure 1**). Limb-matched is a biological-limb match to the subject. For example, for a dominant right-handed person it would be a right-handed allocentric action. Mirror-matched would occur when watching a matched dominant limb perform an action as if you were looking in a mirror (for a dominant right hand person it would be a left-handed allocentric action). According to prior studies, mirror-matched movements are less challenging to imitate because they are spatially compatible and do not require a shift of reference (Chiavarino et al., 2007). Other studies show that both right and left-handed subjects were faster in egocentric perspectives when looking at their dominant hands and faster in allocentric perspectives when looking at other's nondominant hands (Conson et al., 2010). Thus, mirror-matched may be advantageous in this paradigm.

The motivation of this study is to evaluate how perspective and handedness interact to understand and identify tool-action

outcomes. Our hypothesis was that both left and right-handed subjects would identify action outcomes best from an egocentric perspective. When looking at stimuli from an allocentric perspective, identification of action outcomes would best occur in mirror-matched dominant limb for right and left-handed subjects. This study will help us better understand how we translate handedness and motor representations from different perspectives.

# **MATERIALS AND METHODS**

#### **SUBJECTS**

Twenty right-handed subjects (7 males; average age, 22.8, SD, 3.0) and 19 left-handed subjects (11 males; average age: 21.6, SD, 2.2) participated in the study. All subjects were neurologically normal and had normal or corrected-to-normal vision. Handedness was evaluated by the Edinburg Handedness Inventory (Oldfield, 1971) with right-handed subjects having an average score of 82.54 (SD: 15.87) and left-handed subjects averaging −57.65 (SD: 26.81). If the handedness score was > + 40 then the subject was right-handed and if the score was <−40 then the subject was considered left-handed. If the subject was between +40 and −40 inclusive, the subject was considered ambidextrous and was excluded from the study. The maximum score is +/−100. The experimental procedure was approved by the Georgia Institute of Technology Institutional Review Board and consent was obtained from all participants prior to experiment.

#### **TRAINING**

Subjects were first trained on inserting and extracting tools on an upright stationary wooden board with screws protruding facing the subject. The subject had to use three different tools to perform the task, two were unfamiliar and one was familiar. Familiarity of the tools was confirmed verbally by subjects when prompted if they knew what each tool was. If they were familiar with an "unfamiliar" tool or unfamiliar with a "familiar" tool they were excluded from the study. The familiar tool was a twist screwdriver, while the unfamiliar tools were a push style "Yankee" screwdriver and a rotating (plumber's) screwdriver being used by an actor (**Figure 2**). The use of multiple screwdrivers allowed us to maintain task and instruction consistency. These screwdrivers were particularly chosen because to use them, very different actions are required, but the action outcome is the same (insert or extract). The twist screwdriver uses a simple clockwise/counter-clockwise forearm rotation to insert or extract the screw. The push style screwdriver operates by pushing the driver handle that rotates the bit clockwise or counterclockwise based on the position of a toggle switch. The plumber's screwdriver is similar to the twist, except that it demands circular rotation at the elbow to insert or extract the screw. The twist is the most familiar with push and rotational being the least familiar. Of these three, the push only has one action to insert or extract the screw (the other two require clockwise or counterclockwise rotation) and it is treated as a control image. A training board was placed in front of the subject's visual field and was reachable at arm's length. Participants used each of the three screwdrivers to insert five screws all the way into the

board and then screw the same screws all the way back out to their initial starting position to obtain the motoric actions required to use each tool. Subjects were instructed to choose any five screws that were at a comfortable height for them to manipulate.

### **STIMULI AND TASK**

"fpsyg-04-00957" — 2013/12/17 — 17:29 — page 3 — #3

After all training was completed, subjects performed an action understanding task based on the trained tools. Subjects were seated comfortably in a chair and shown randomized action images of the three different tools on a 106.7 cm (42 inch) visual monitor (visual angle = 18.7◦). Images were high-resolution grayscale images of either a right or left-handed instructor holding one of the previously mentioned tools in either an allocentric or egocentric perspective.

While seated with a response pad comfortably in their hands, subjects were presented first with a circle (4–6 s), then a fixation cross which alerted subjects that the trial was about to start (500 ms), followed by the instructor-tool image (4 s). While the image (**Figure 3**) was on the screen, the subject was told the following: "The images on the screen will show you any of the tools you have just trained with, being used by either a left or right hand instructor, and can be shown either in an egocentric (as if you yourself are using the tool) or allocentric (as if you were watching me use the tool) perspective. On the image there will be a red arrow located on the wrist of the actor. Based on the direction of the arrow, you will need to simulate in your mind which way the hand is rotating, and answer if the hand is driving the screw into the board, or is it pulling the screw out of the board." If they thought the actor was inserting the screw into the board, they were instructed to indicate by pushing the left button with their left hand on the response pad. If they thought the actor was extracting the screw, they were instructed to indicate by pushing the right button with their right hand on the response pad. Based on the stimuli presented, this afforded an equal number of responses with the left and right hands without bias to the response hand matching the stimulus hand (i.e., a correct response would equally occur for the same number of left or right hand image actions). The subject was instructed to answer as quickly and accurately as possible from the onset of the image. If the subject did not respond before the 4 s time period, the circle reappeared and no response was counted. There were 12 different image types. Each type was displayed twice in each of the two blocks that lasted approximately 13 min each (**Figure 4**). All images were presented in a pseudorandom order and correctness and latency of responses were recorded.

#### **ANALYSIS**

Behavioral responses were recorded over two blocks of trials. All responses were recorded with Stim2 version 4.0 (Neuroscan 2003, El Paso, TX). Data sets were imported into Excel spreadsheets and organized by type into blocks. For each block, the response and latency average were calculated for each subject and every image type excluding any trials that the subject missed. Overall, there was no significant difference in missed trials for any image type (*p* = 0.685). All block averages were compiled into a grand average for each image type. Averages were then entered into IBM SPSS Statistics 19. A 4-way multivariate ANOVA (MANOVA) was computed with factors perspective (egocentric and allocentric) x hand of actor (left and right hand) x tool (traditional and rotational screwdrivers) x hand of subject (left and right-handed). Where appropriate, *t*-tests were used to identify interaction effects between the different image types. For *t*-tests, significance was assessed at *p* < 0.05 with Bonferroni correction for all comparisons.

# **RESULTS**

#### **LATENCY**

For latency of response time, there was a main effect of perspective (*F*(1,304) = 33.66, *p* < 0.05) and of tool (*F*(1,304) = 9.23, *p* < 0.05). In **Figure 5A** it is shown that when subjects look at egocentric images, they respond significantly faster than if they were looking at an allocentric image. Looking at novel tool images, subjects respond slower when compared to familiar tools.

There were no other main or interaction effects regarding latency.

### **ACCURACY**

Accuracy (percent correct) was also evaluated for each image type. There was a significant main effect in percent correct due to perspective (*F*(1,304) = 37.44, *p* < 0.05), with the egocentric perspective having lower error rates (**Figure 5B**). There was a second main effect with respect to percent correct for hand of subject (*F*(1,304) = 8.31, *p* < 0.05), with right-handed subjects having lower error rates than left-handed subjects.

An interaction effect was seen for perspective x hand of subject (*F*(1,304) = 4.06, *p* < 0.05). Right-handed subjects looking at images in the egocentric perspective had significantly lower error

**FIGURE 3 |The wooden board with screws that were mounted for subject training; (A) an exemplar image for right-handed egocentric rotating screwdriver driving a screw "in" and (B) an exemplar image for a left-handed allocentric rotating screwdriver pulling a screw "out".**

"fpsyg-04-00957" — 2013/12/17 — 17:29 — page 4 — #4

"fpsyg-04-00957" — 2013/12/17 — 17:29 — page 5 — #5

rates compared to allocentric images (*p* = 0.019). Left-handed subjects looking at images in an allocentric perspective had the highest error rates overall compared to all the other conditions (**Figure 6**). An additional interaction effect was seenfor tool x hand (*F*(1,304) = 4.88, *p* < 0.05), however when explored, there were no significant individual effects.

# **DISCUSSION**

Right and left-handed subjects were recruited in order to judge tool-use action outcomes while hand of instructor, perspective, and tool type used in the images were manipulated. Specifically, we sought to evaluate how perspective and handedness interact on a learned tool in order to accurately determine an action goal using a discrete motor task. In conformation of our first hypothesis, we found that egocentric perspective images had higher accuracy and faster latencies when compared to allocentric images. Our second hypothesis was refuted, as there was no effect of handedness of subject and limb performing the action. We will further discuss our findings based on the hypotheses presented.

# **ALLOCENTRIC VERSUS EGOCENTRIC PERSPECTIVES**

Our first hypothesis was that both left and right-handed subjects would be able to judge action best from an egocentric perspective. Results revealed there was a significant effect of accuracy and latency with respect to perspective for both right and left-handed subjects. The highest accuracy and fastest latency was found in the egocentric perspective for both sets of subjects, which supports our first hypothesis. These findings are in line with previous studies which suggest that action outcomes are best facilitated in an internal (egocentric) perspective, regardless of the hand being used (Conson et al., 2010; Lindgren, 2012; Oosterhof et al., 2012). Looking at previous neural studies, the left parietal lobe has been shown to be active in coding representations of the body, and the right parietal lobe is active for visuospatial orienting (Iacoboni et al., 1999; Watanabe et al., 2011). Specifically, Watanabe et al. (2011) studied right-handed subjects who viewed and then imitated limb-matched ("anatomical") and mirrormatched ("specular") images performing a finger touch task. The findings in this work suggested that the more dissimilar the actors hand was from the position of the participants, the more difficulty they had in interpreting the imitation task, and there was a corresponding notable increase in right posterior parietal cortex (PPC) activation. They suggested that the increase in activation was due to the demands of aligning visuospatial representations with kinesthetic signals from self and therefore it was more challenging to imitate the images. These findings could explain why our behavioral results showed effects of latency and accuracy, particularly disadvantageously in the allocentric perspective. Together, these authors suggest that when an action is observed in the allocentric perspective, it is possible that action resonates to either of the observer's limbs as a technique to interpret action more readily. Although visual areas associated with mental rotation were not assessed, this could be a future direction to further explore the neural mechanisms driving the behavioral effect.

### **EFFECTS OF HANDEDNESS IN ALLOCENTRIC PERSPECTIVE**

Our second hypothesis was that in allocentric perspectives, optimal action prediction would align best in mirror-matched dominant limb for right-handed and left-handed subjects. Handedness of subject showed an effect on accuracy, where right-handed subjects were significantly more accurate in the task than left-handed subjects overall. However, neither right nor left-handed subjects showed behavioral effects to the allocentric actions performed with a mirrored or matched hand, which does not support the second portion of our hypothesis. We studied

action prediction by testing if the ability for resonance to occur may be impacted in a limb specific way. In action perception, according to the ideomotor theory, a subject's motor system and the associated action representations are activated when perceiving action from another person (Massen and Prinz, 2009). Perceiving body movements and corresponding remote goals influences how those actions are understood. Functional affordances include all possible tool-based goal directed actions that best "afford" a desired action goal (Mizelle et al., 2013). In this work, we proposed that functional affordances are proposed to be critical for the ability to simulate action and understand all possible action outcomes. Importantly both body movements and action goals have a bidirectional association in order for the perception of action to trigger action in the observer (Massen and Prinz, 2009; Paulus, 2012). If the perception of action in an observer comes from bidirectional understanding of movements and goals, then mapping all seen action to the dominant or non-dominant limb in an allocentric perspective could facilitate action understanding. Although allocentric actions showed no bias to either limb for our behavioral study, Conson et al. (2010) did in fact see a limb bias in the allocentric perspective. This could be due to different experimental demands between the paradigms where our study was focused on action outcome and Conson et al. (2010) was focused on hand laterality and mental rotation. Future neurophysiological studies will further evaluate specific neural mechanisms that may relate to activation of left or right sensorimotor areas in a similar task.

"fpsyg-04-00957" — 2013/12/17 — 17:29 — page 6 — #6

**and hand of subject (\****p <* **0.05).** Right-handed subjects looking at images in the egocentric perspective were more accurate at the task when compared to allocentric images (\**p* = 0.019). Overall, left-handed subjects looking at images in an allocentric perspective were significantly worse compared to all other conditions (\**p* < 0.05).

When compared to right-handed subjects, left-handed subjects were significantly less accurate when judging the outcomes of allocentric images. The decreased accuracy for left-handed subjects on the allocentric images could be due to an asymmetrical lateralization of encoding action and motoric dominance in the brain, which may interfere with translating allocentric limb action outcomes within their own motor system. In prior work (Frey et al., 2005), left and right-handed callosotomy patients were studied in order to understand hemispheric specialization for tooluse. The left-handed patient performed worse at demonstrating tool-use actions with the dominant left hand compared to their right hand, but the right-handed patient performed best with the dominant right hand and worse with the left. These results indicate that the left hemisphere is specialized for tool-use information. This idea has been well validated in human neuroimaging experiments (Vingerhoets et al., 2012). For left-handed people (because the right hemisphere controls their dominant hand) a challenge is presented when trying to access tool representations from the opposite (left) hemisphere. However, performance of tool-use actions was not a disadvantage in their right-handed callosotomy patient. If tool-use information is stored in the left hemisphere for both right and left-handed people, then it is possible that because right-handed people have a dominant left motor hemisphere (creating a hemisphere match), they would have an advantage when interpreting action outcomes in our study. Extending these concepts, these results could suggest the reason left-handed subjects perform significantly worse in allocentric action outcome interpretation is because when they view the images they utilize an additional mechanism that is needed to facilitate coordination of information across the hemispheres. Specifically, we propose that when action is seen in the allocentric perspective, left-handed subjects have an additional demand of utilizing left hemisphere action encoding along with right hemispheric motor and visuospatial rotations to comprehend action

outcomes (Watanabe et al., 2011). Importantly, right hemispheric visuospatial rotation may relate to right-handed subjects performing worse on allocentric versus egocentric actions (**Figure 6**). Why this affects accuracy, but not latency is worth consideration in behavioral and neurophysiological studies to understand aspects of decision delay versus decision accuracy in similar tasks.

#### **EFFECTS OF LATENCY VERSUS ACCURACY**

The finding that latency was significantly increased for allocentric images contributes to previous research that states allocentric images are harder to interpret compared to egocentric images (Ni Choisdealbha et al., 2011; Zhou et al., 2012). However, latency effects did not persist through any other variable in this study. Given the difficulty of the task, there could possibly be no other latency differences because all images are moderately difficult, which would extend reaction time and ultimately interfere with accuracy due to the time constraints on response time. We removed the missed trials for each condition, which was 27.5% of trials in each condition (there was no significant difference in missed trials for any image type (*p* = 0.685), which suggests the task was equally difficult for all stimuli. Previous studies in our lab involving affordance have shown effects of action encoding in the latency domain but not in the accuracy domain (Borghi et al., 2012). Whether increasing the time constraint on response interval or reducing the difficulty of action images would alter latency effects is an issue to be investigated in future research.

#### **ALTERNATIVE EXPLANATIONS**

There is other existing evidence that would suggest it is possible that right and left-handed subjects have different strategies when it comes to interpreting action. Ni Choisdealbha et al. (2011) suggested that right-handed subjects rely primarily on sensorimotor mental rotation. On the other hand, left-handed subjects could depend initially on visual analysis and/or pictorial strategies followed by a mental rotation strategy.

Work has also been done to evaluate patients with frontal lesions on similar tasks (Chiavarino et al., 2007). The patients were asked to imitate mirror-matched or limb-matched stimulus. They discovered that patients had a selective deficit for imitating limbmatched responses which suggests that executive function of the frontal lobes drives the system to visually rotate the frame of reference in order for them to imitate the stimulus. They suggest that the imitation capacity was damaged for these particular patients. If this theory is true, then in our healthy population, left and righthanded subjects would have had a similar deficit when judging allocentric images. Although this is a valid explanation, we believe it is unlikely due to higher order executive function, but rather differences in the motor system. A limitation of their study was that they did not separate the patients into left and right sided brain lesion groups and they also had diverse locations where the lesions were located within the frontal lobe. Apraxia in left-handed patients with left or right hemisphere damage has been evaluated in a recent study by Goldenberg (2013). He found that in lefthanded patients, apraxia can occur as a result of damage to either the left or right hemisphere. Apraxia after left hemispheric damage (dissociating from manual dominance) may be explained as result of damage to the praxis relevant networks which remain in the left hemisphere. However, apraxia after right hemispheric damage could be explained as result of damage to a unique co-localization of praxis skills and spatial processing within the right hemisphere. Such findings could argue for a stronger bilateral organization of praxis control in left handed compared to right handed subjects.

#### **LIMITATIONS**

A limitation of the current study is that it is difficult to recruit left-handed subjects that are extremely left hand dominant. Most tools are designed for right-handed people, thus left-handers acclimate and become slightly more ambidextrous for some skilled unimanual tasks. This effect could confound the interpretation of potential hemispheric dissociations, as strength of left-handedness has been shown to augment the strength of right hemispheric laterality (Cabinio et al., 2010). Ambidextrous subjects were excluded from the present study, but left-handed subjects had a lower overall hand dominance score when compared to the right-handed subjects on the Edinburg Handedness Inventory scale. Each individual subjects score was, according to the Edinburg Handedness Scale, beyond the ambidextrous range.

Another limitation is although we were not seeking to understand the learning of new tools, a new tool was incorporated into the study in order to obtain selection of tools that had the same action of "screwing." Our study utilized direct training for all tools presented and there was no effect of accuracy for novel versus familiar tool observed. There was an effect on latency, with novel tools overall having an increased latency compared to that of familiar tools. We did not expect to see a difference behaviorally between tool types due to previous work indicating neural networks were the same; however, the addition of a neural study would be able to confirm this.

#### **CONCLUSION**

The current findings provide insight into how action-goals are encoded and interpreted by left and right-handed subjects. We have demonstrated that encoding of action of left and right-handed actors is not necessarily differentially encoded in left or right-handed subjects in a way that would demonstrate behavioral differences. We have shown there is a benefit in representation of actions encoded in the egocentric perspective. While the ideomotor theory can explain much of why this occurs, it is still unclear as to why left-handed subjects viewing allocentric action showed the pronounced deficit from other combinations of handedness and perspective. Future research may determine the specific neural mechanisms that drive these results by collecting neurophysiological data focusing on motor lateralization effects, which is currently underway.

### **REFERENCES**

"fpsyg-04-00957" — 2013/12/17 — 17:29 — page 7 — #7

Bekkering, H., Wohlschlager, A., and Gattis, M. (2000). Imitation of gestures in children is goal-directed. *Q. J. Exp. Psychol. A* 53, 153–164.

Bohlhalter, S., Hattori, N., Wheaton, L., Fridman, E., Shamim, E. A., Garraux, G., et al. (2009). Gesture subtype-dependent left lateralization of praxis planning: an event-related fMRI study. *Cereb. Cortex* 19, 1256–1262. doi: 10.1093/cercor/bhn168


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 03 October 2013; accepted: 05 December 2013; published online: 19 December 2013.*

*Citation: Kelly RL and Wheaton LA (2013) Differential mechanisms of action understanding in left and right-handed subjects: the role of perspective and handedness. Front. Psychol. 4:957. doi: 10.3389/fpsyg.2013.00957*

*This article was submitted to Cognition, a section of the journal Frontiers in Psychology. Copyright © 2013 Kelly and Wheaton. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

"fpsyg-04-00957" — 2013/12/17 — 17:29 — page 8 — #8

# The embodied mind extended: using words as social tools

# *Anna M. Borghi 1,2\*, Claudia Scorolli 1, Daniele Caligiore2, Gianluca Baldassarre2 and Luca Tummolini <sup>2</sup>*

*<sup>1</sup> EMbodied COgnition Lab, Department of Psychology, University of Bologna, Bologna, Italy*

*<sup>2</sup> Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy*

#### *Edited by:*

*François Osiurak, Laboratoire d'Etude des Mécanismes Cognitifs (EMC EA 3082), France*

#### *Reviewed by:*

*Martin H. Fischer, University of Potsdam, Germany Frank Garcea, University of Rochester, USA*

#### *\*Correspondence:*

*Anna M. Borghi, EMbodied COgnition Lab, Department of Psychology, University of Bologna, Viale Berti Pichat, 5, 40127 Bologna, Italy. e-mail: anna.borghi@gmail.com*

The extended mind view and the embodied-grounded view of cognition and language are typically considered as rather independent perspectives. In this paper we propose a possible integration of the two views and support it proposing the idea of "Words As social Tools" (WAT). In this respect, we will propose that words, also due to their social and public character, can be conceived as quasi-external devices that extend our cognition. Moreover, words function like tools in that they enlarge the bodily space of action thus modifying our sense of body. To support our proposal, we review the relevant literature on tool-use and on words as tools and report recent evidence indicating that word use leads to an extension of space close to the body. In addition, we outline a model of the neural processes that may underpin bodily space extension via word use and may reflect possible effects on cognition of the use of words as external means. We also discuss how reconciling the two perspectives can help to overcome the limitations they encounter if considered independently.

**Keywords: embodied cognition, extended cognition, tool-use, words as tools, language comprehension, social cognition, body schema, incorporation**

# **INTRODUCTION**

The embodied-grounded (EG) view and the extended mind (EM) view of cognition and language are typically considered as rather independent perspectives. Aim of this paper is to show how the two views can be integrated considering the case of words in their relationship with the bodily space. Specifically, we will propose that words are a very peculiar kind of tool.

According to embodied views of cognition, cognitive processes are constrained by our body, that is, human-like cognition cannot occur independently of a human-like body. In the embodied view, cognition is not for knowing; rather, "cognition is for action" (Wilson, 2002). Proponents of grounded views make a similar argument but posit that the involvement of the body is not exhaustive of cognition, which is grounded in many ways (Barsalou, 2008). In fact, while initially the label "embodied" was used in a more comprehensive way, in the recent literature a slight distinction between embodied and grounded approaches, and between the terms "embodied" and "grounded," is emerging (see Pezzulo et al., 2011; Fischer, 2012; Myachykov et al., in press). According to this view cognition can be grounded in multiple ways. These include not only bodily states, but also situations, actions, etc. (Barsalou, 2008; Pezzulo et al., 2011). In the following, we will use the term embodied and grounded cognition (EG) to refer to both approaches, since the distinction is not relevant for the proposal we will advance.

When it comes to language processing, EG views argue that language is grounded in perception and action systems (for reviews: Willems and Hagoort, 2007; Fischer and Zwaan, 2008; Gallese, 2008; Toni et al., 2008; Jirak et al., 2010; Borghi and Pecher, 2011, 2012; Glenberg and Gallese, 2012). Comprehending language would imply activating a simulation, consisting in a re-enactment of the previous interaction with objects, situations, etc., to which linguistic expressions refer.

In the last years another perspective on cognition, the EM view, is gaining credit, in particular in philosophy. The underlying idea, initially promoted by Clark and Chalmers (1998), is that the human mind is not wholly in our head/brain, but it is rather distributed in our brain, body, and external devices. These external devices (e.g., computers) have the power to complement and augment our internal cognitive processes (see Wilson, 2010).

In this paper, we will first discuss some general limitations of EG and EM views, then address some more specific limits of these views in understanding the role of language. We will then suggest that words can be understood as social tools, and explain why, in our opinion, this approach helps to reconcile EG and EM views of cognition and to overcome their limitations. Finally, we will discuss experimental evidence to support the Words as social Tools (WAT) proposal and we will outline a computational model to specify the neural mechanisms that might underlie the aforementioned processes.

# **EMBODIED-GROUNDED AND EXTENDED VIEWS**

"fpsyg-04-00214" — 2013/4/30 — 13:00 — page 1 — #1

Even though we favor an EG approach to cognition, we hold that EG theories have some problems (for critiques to aspects of the embodied approach, see Borghi and Cimatti, 2009, 2010; Chatterjee, 2010; van Elk et al., 2010; Wilson and Golonka, 2013). We will consider first some problems characterizing the EG approach in general, and then we will focus on the limitations of the EG approach to language, in particular to language comprehension. We will focus on content issues and not on methodological problems, as for example the problem of the lack of precise and unidirectional predictions, which in our opinion can be solved with a more extensive use of computational models (see for discussions on this problem Borghi et al., 2010; Chersi et al., 2010; Willems and Franken, 2012). Notice that our critiques might not

necessarily concern all versions of EG views, which are sometimes rather different (see Goldman and de Vignemont, 2009, for an analysis of this). One major problem of EG views is the high risk of adopting the view that Clark (2008) has called "brainbound." In this view, human cognition directly depends on neural activity, with the mind being modeled as inner and neurally realized. This position does not accept the idea that cognition might be distributed and extended beyond bodily borders. The brainbound view is not convincing for a simple reason, as explained by Noe (2009): "the subject of experience is not a bit of your body. You are not your brain. The brain, rather, is part of what you are" (pp. 7). In our opinion many versions of the EG view are too brainbound: they emphasize too much the role of the brain with respect to the body. This might seem paradoxical for an embodied approach: obviously no embodied view does fully neglect the importance of the body, but many EG approaches ascribe a too relevant role to the brain compared to the whole body, at the same time neglecting the possible role of body extensions. Similar critiques are expressed by Wilson and Golonka (2013) who claim: "The major problem with this research is that it again assumes all the hard work is done in the head, with perception and action merely tweaking the result." (Wilson and Golonka, 2013, p. 11). van Elk et al. (2010) further deepen this point, arguing that in cognitive neuroscience embodied approaches are still cognitivist. We report their own words: "In cognitive neuroscience the notion that concepts are embodied primarily means that there is a correspondence between the brain activations associated with processing the referent of a concept and the processing of the concept itself. For instance, seeing a car and thinking or reading about a car involves the activation in comparable visual areas. Thus, the dispute between modal and amodal theories of language comprehension is basically a discussion about the representational vehicle of concepts (i.e., whether the representational vehicle of concepts is shared with neural resources used for perception and action). Both modal and amodal theories of language thus share a cognitivist notion of cognition in terms of discrete internal representations of the world" (van Elk et al., 2010, p. 3).

The second problem with many EG theories is that they do not sufficiently consider and emphasize the fact that the sense of body might be plastically rearranged. Body boundaries are treated as rather static while some studies have revealed that they are flexible and can be modified, for example through the use of tools, changing with our sense of body (see for example the special issue on the sense of body by Tessari et al., 2010). We will further address this problem in the rest of the paper.

When they deal with language, one major limit of EG views is that language is mainly conceived in its referential aspects. This way of conceiving language relies on the classical notion that knowing the meaning of a word is knowing what it refers to. Accordingly, the meaning of a word like "hammer" consists in the re-enactment of past multimodal experiences with the word referent, i.e. hammers. For example, according to the indexical theory (Glenberg and Robertson, 2000) words would index their referents in the world, which would be represented in terms of perceptual symbols (Barsalou, 1999). This referential view of language has a number of merits. First, it provided the instruments to contrast the propositional view, which was dominant in psychology and cognitive sciences (see Lakoff, 2012, for a description of the times before the idea of embodied cognition). In this view concepts and word meanings were seen as the product of a transduction process from sensorimotor to abstract knowledge. Knowledge would be represented in terms of amodal symbols only arbitrarily related to their referents, organized through syntactic combinatorial rules (e.g., Fodor, 1975; Pylyshyn, 1984). More recent non-embodied views posit that word meaning is a consequence of the statistical distribution of words in language (for an influential version, see Landauer and Dumais, 1997). However, today the necessity to contrast the statistical and the embodied view is not so critical, and conciliatory approaches have been proposed (see for example Andrews et al., in press).

Second, the influential research program based on these premises has inspired many studies, which have led to important and sophisticated experimental results (for reviews see Barsalou, 2008; Fischer and Zwaan, 2008; Gallese, 2008; Toni et al., 2008; Jirak et al., 2010; Borghi and Pecher, 2011, 2012). However, an embodied referential view is probably not sufficient to provide a thorough account of word meaning.

While in psychology and cognitive science the propositional view has dominated for a long time and the referential view was introduced by EG theorists as an alternative to it, in philosophy the referential view of language has been widely criticized since at least the seminal work of Wittgenstein (1953; see Noe, 2009 for a contemporary statement): the most widespread view in philosophy holds that, for example, we can speak about fawns even if we have never seen them since we can rely on the expertise of our community. Words are compositional and we can access the meaning of words of which we do not know or cannot see the referent thanks to the expertise of other members of our community. As Noe (2009) nicely argues, "meaning depends on the practice" (p. 90), and being able to use words corresponds to knowing what they mean.

Curiously, while philosophical examinations have gravitated toward treating the practical nature of meaning, the referential view is still the predominant one in EG cognition theories. This has probably been due to the desire, on the part of EG proponents, to contrast the traditional propositional view, according to which words are arbitrarily linked to their referents. EG proponents have assumed that it was necessary to demonstrate that words are grounded, as their referents activate perception and motor systems.

Beyond the limit of the focus on referentiality, in our view the EG view of language has two further limitations given that it has neglected two other important aspects of words. The first concerns the social and public nature of words, the second the fact that words can be instruments for action. Words are social and public because, since they are a heritage of our speakers' community, to be effective they require someone else's presence, implicit or not. Indeed, speaking implies performing complementary actions in coordination with someone else (Clark, 1996). Words can be instruments for action since their use allows humans to modify the current state of the world, as it happens during tool-use. This point will be further developed in the course of the paper.

If EG approaches often tacitly assume a brainbound view of cognition, the most vigorous attack to this view derives from the

"fpsyg-04-00214" — 2013/4/30 — 13:00 — page 2 — #2

idea that cognition is not limited to the boundaries of body/skull but is extended. In other words, "minds like ours emerge from this colorful flux as surprisingly seamless wholes: adaptively potent mashups extruded from a dizzying motley of heterogeneous elements and processes" (Clark, 2008, p. 219). According to the EM view, tools complement our mental abilities: for example, a diary complements our memory. As a consequence of this relationship between brain-body system and external tools, our mind would be distributed (Hutchins, 1995) across a variety of bodily parts and non-bodily devices (Clark, 2003; Thompson and Stapleton, 2009). One potential limitation of EM views, and possibly one of the reasons why they have encountered resistance, is their appeal to functionalism (Kiverstein and Clark, 2009) which might conflict with the assumptions of an embodied view of cognition (but see Clark, 2008, for a different position, which does not put the two approaches in contrast).

The EM approach holds a peculiar view of the relation between words and cognition. Words themselves are considered as external devices and as cognitive tools capable of augmenting our computational abilities (Clark, 1998). This view (e.g., Clark, 1998) has its roots in the seminal work of Vygotsky (1962) who underlined the role played by inner language and its scaffolding function supporting actions. However, in our opinion, one of the most interesting aspects of Vygotsky's notion of inner language is that it involves the internalization of a phenomenon which is initially (and inherently) social and public and which augments our computational abilities. Such a social and public component is, however, underappreciated in the EM approach, which instead underlines the importance of language for developing thought and computational abilities.

Here we propose that EG and EM views can, and should, be integrated. Such integration will overcome their respective limitations when dealing with language: the limited focus of the EG view on the referential aspect of words and the neglect of the social dimension of words in the EM view.

# **THE INCORPORATION OF PHYSICAL TOOLS**

Even if it does not pertain to language, one line of research that may suggest how EG and EM views can be reconciled comes from recent work on the recoding of bodily space after tool-use. Below we will briefly review the behavioral, neural and computational literature on this topic and will then try to highlight why it is relevant for us.

Since Iriki's seminal work with monkeys (e.g., Iriki et al., 1996), neuroscientific studies with humans have revealed that active tooluse can change the representation of space, in particular inducing an extension of the near space (Berlucchi and Aglioti, 1997; Berti and Frassinetti, 2000; Maravita and Iriki, 2004; Farnè et al., 2005; Osiurak et al., 2012).

The neural mechanisms underlying the extension of body representation caused by the use of a tool have not yet been identified (Magosso et al., 2010; Stout and Chaminade, 2012). Recently, some attempts mainly using computational modeling approaches have been proposed with the aim of identifying such mechanisms. Each proposed model sheds light on some important aspect underlying the phenomena. Ursino et al. (2007) and Magosso et al. (2010), for example, point out the involvement of visual-tactile cortical

regions serving the representation of action affordances and action outcomes (including the parietal cortex, PC, and the pre-motor cortex, PMC) and Hebbian associative mechanisms to shape the body representation after using a tool. In particular, Ursino et al. (2007) claim that the enlargement of the peripersonal space after tool-use depends on an expansion of the visual receptive field of parietal bimodal neurons due to a strengthening of visual synapses through Hebbian mechanisms. In the same line the model proposed by Magosso et al. (2010) shows how different tool-use tasks lead to different re-sizing effects of the peri-hand space. The model also predicts that, after tool-use, a far visual stimulus acts as a near one, independently of whether the tool is present or absent in the subject's hand. The authors validate this prediction by an *in-vivo* experiment. Other models focus on the role of sub-cortical areas (such as the cerebellum, see Arbib et al., 2009, and Imamizu and Kawato, 2012) in learning and storing internal models of body and environment after the use of a tool. Other ones suggest that memory processes are responsible for the dynamical aspects of tool-use during tool-body assimilation (Nabeshima et al., 2007; Nishide et al., 2009).

An open issue in the literature on bodily extension concerns whether the characteristic recoding of spatial perception also determines a change the body schema. We will briefly focus on this discussion since it is important for our view of language. One interesting distinction is between bodily extension determined by successful tool-use and incorporation following successful prosthesis-use. According to De Preester and Tsakiris (2009), tool-use does not determine changes in the sense of bodyownership, but only in motor and perceptual capacities (Botvinick, 2004). A crucial difference is the experience of completion: a noncorporeal object can be incorporated if it replaces something that originally was present, and now is missing. If the object cannot be assimilated to the pre-existing body-model (Tsakiris, 2010), true incorporation cannot occur. Beyond incorporation and use, there might be different degrees of relationship between ourselves and the objects. Some objects are perceived as external, while other objects provoke effects in our own sense of body. However, even objects perceived as completely external evoke motor responses (affordances), if they are close enough to our own body (Costantini et al., 2011b; Ambrosini et al., 2012; for a comprehensive review on affordances see Thill et al., 2013).

The same distinction between incorporation and use can also be applied to language. The question we will address in the following pages was initially proposed by Clark (2008, p. 39) in the following formulation: "Could anything like this notion of incorporation (rather than mere use)... get a grip in the more ethereal domain of mind and cognition?" We will show how the notion of incorporation can be applied to the"ethereal" domain of language. Here words, and in particular their public and social dimensions, come into play.

# **WORDS AS SOCIAL TOOLS: THE CASE OF SPACE**

"fpsyg-04-00214" — 2013/4/30 — 13:00 — page 3 — #3

The idea that words can be conceived as tools is not completely new. Beyond Wittgenstein (1953), it has been proposed by a number of authors (Clark, 1998; Borghi and Cimatti, 2009, 2010, 2012; Mirolli and Parisi, 2009, 2011; Tylèn et al., 2010). However, different aspects of this idea have been stressed.

In *Philosophical Researches*,Wittgenstein (1953) highlighted the fact that words can have different and multiple functions, as tools in a toolbox. Clark (1998) spoke of the "magic" of words: words are external artefacts endowed with the power to augment and complement our computational abilities. According to him, while emphasis has been put on the communicative aspects of language, its computational role has been neglected, with the possible exception of Vygotsky who has underlined the role played by inner language and scaffolding to direct our actions.

The view we will present is slightly different. We agree that the computational role of inner language, intended as a guide for action, has not been considered enough. However, we intend to stress the role of other aspects of words that, despite the novel burst of interest for social neuroscience, have been neglected: the social and public role words possess. In order to be effective, words do not only need to refer correctly to objects or situations in the world. Language is also a powerful instrument for joint action. Words are tools, as they allow for the mental manipulation of information (Malt and Wolff, 2010). This in an individual and private use, as some authors have underlined. However, words have a peculiarity: to manipulate inner information we take advantage of a device that is social and public in its nature. For this reason we claim that words are "social tools." Specifically, in this paper we will consider a special case of similarity between words as social tools and physical tools, concerning the relationship between space and body.

Words and physical tools share an important feature: both can be used to accomplish goals via external means, respectively, other people and objects, resulting in a change of the current state of the world (Glenberg and Gallese, 2012) and in an extension of our capabilities. Consider the case of words as tools that can be used to reach for something. We can reach objects with a physical tool (e.g., a rake), but also by asking somebody to bring them to us. Thus, in certain contexts the same goal can be reached either through tools, or through words. In some cases, words are even more powerful than tools. For example, they might allow us to reach very distant objects.

However, words work as tools only under the condition that other people collaborate. Even if our proposal is in debt with the pragmatics literature (e.g., Levinson, 1983) and with Austin (1962)'s idea that we do things with words, here we intend to make a distinction between advancing a request for an object and performing an action with a tool. These two activities share many similarities, but are also clearly different. An action with an instrument can be planned but fail, for example due to problems of the instrument, etc. Similarly, a request can be disattended, either because of problems in its formulation, or due to disruptions in communication, or scarce compliance on the side of the addressee. But people can decide to use tools to reach a goal on their own, without the presence of other individuals. This is not possible with words. The referent of a word can be found, but if other individuals do not provide a support, i.e., if the social dimension implied in word use is absent, the request will not succeed. Thus words are a peculiar kind of instrument: they work effectively only if other people are available and respond positively to our implicit or explicit request. What counts is the dynamic interaction they are able to promote (see Cooke et al., 2013, on team cognition). When

performing activities which require coordination, such as lifting very heavy objects, we need to possess the sophisticated ability to understand others' action plans, others' willingness to collaborate, etc. (Marsh et al., 2009). Similarly, this ability should be present during language use as well, otherwise words, even if referentially correct, are not effective. In this respect, words constitute a bridge between ourselves, the environment and the others.

Here we propose that words and tools share a further similarity: we consider the possibility that when we use words to reach for something, word use expands the near space, modifying the representation of the relationship between our own body and the objects in space, similarly to what happens after tool use. The argument behind this hypothesis is the following: if words are similar to tools, then their use should lead to an extension of the bodily space, as it happens with real tools.

One could object that words and tools are substantially different, since tools are physical things in the world that we use with our bodies while words are not. We understand the objection, but the perspective we endorse is radically different: according to WAT (e.g., Borghi and Cimatti, 2009, 2012) not only tools but words as well can be considered as physical things. They are expressed through our bodies, be they spoken or written, and once pronounced or written they have a material and public existence, similarly to tools (Wittgenstein, 1953; Clark, 1998).

Now consider the relationship between words and body according to EG theories and the relationship between words and mind according to the EM view. EG theorists demonstrated that comprehending words activates the motor system. EM theorists propose that, as tools extend our body schema, "language extends our capacities for thought and therefore can be treated as extending our mind schema" (Noe, 2009). In fact, it has been shown that language modifies cognition, for example influencing perception and categorization (Wolff and Malt, 2010), in a flexible manner (Lupyan, 2012). But so far nobody has shown that word use might recode our bodily space with respect to objects, as it happens for physical tools. Notice that the parallel between words and tools is not only abstract and metaphorical; in contrast, we formulate the precise prediction, to be tested experimentally, that both words and physical tools have a specific effect on cognition, i.e., that their use determines an expansion of the bodily space representations. Demonstrating this would imply to apply the notion of incorporation to the "ethereal" domain of language. At the same time, it could help reconcile the EG and the EM view.

# **WORDS AS SOCIAL TOOLS AND SPACE: EXPERIMENTAL EVIDENCE AND A MODEL EXPERIMENTAL EVIDENCE**

Recent experimental evidence supports the idea that words can be considered as tools that extend the bodily space.

Scorolli et al. (2011; submitted) and Scorolli and Borghi (2012) demonstrated with a kinematics study that word use modifies spatial perception. Participants, children and adults, observed objects located in the peripersonal, extrapersonal or far and "border" space. For operational reasons we defined"peripersonal," or"near," as the space reachable extending the arm (but see the discussion on the problems of this definition due to the plasticity of the near space made by Longo and Lourenco, 2006; Lourenco and

"fpsyg-04-00214" — 2013/4/30 — 13:00 — page 4 — #4

Longo, 2009), "extrapersonal," or "far," as the non-reachable space, and "border" as the space reachable extending the arm and the back. Before and after training, subjects were asked to produce explicit verbal estimations on objects' distances, or to throw a toy-car toward objects' locations. During the training phase participants had to reach and grasp the "right" object and to put it in a box provided by different shaped holes. If the right object was too far, they could use a tool (a rake), press a button or use a linguistic label, pronouncing the object noun; all instruments were effective in reaching the goal. We introduced the button since we were interested in comparing the rake and the button, i.e., two instruments that, differently from words, do not imply a social context to be used. While participants hold the rake in their hands, the button has an arbitrary relation to the object, similarly to a word: once pressed, the object appears. In the last years, few studies have shown that even arbitrary relationships with a target can modify the perception of peripersonal space. Davoli et al. (2012) have shown that remote interactions with a target, for example illuminating the target object with a laser pointer, caused an extension of the perceived space. In the same vein, Bassolino et al. (2010) demonstrated that frequent use of a computer mouse determined a spatial extension. The difference between a button, i.e., a device that is arbitrary linked to the object to be reached, and a word is that the last one implies a social dimension.

The results of the study revealed that after training, even if the verbal estimations changed slightly, the car was thrown significantly closer than before the training. This indicated an extension of the reachable space, not modulated by the instrument kind.

As other studies on extended body, this work suggests that the distinction between near and far space is plastic and flexible. However, here the extension was brought about not only by physical tools but by immaterial ones as well, i.e. by words. The social dimension implicit in words made this possible: pronouncing an object name implies evoking somebody else performing a complementary action, helping us reach a distant object. Thus words, like tools, help us act in the world and influence our way of representing bodily space with respect to objects (Gianelli et al., 2013). However, with words, our operational space becomes larger because of the presence of others. Even if we propose that the social dimension is intrinsic in word use *per se*, we predict that the results will be stronger, i.e. the spatial extension with words will be more marked, in presence of another person. In particular, this extension should be particularly marked if the other person is close to the object, is looking directly at the participant and demonstrates through gestures and posture to be open to the interaction (see Innocenti et al., 2012; Scorolli et al., 2012). We predict, instead, that if the other person is not close to the object, and the body posture and the facial expression of the other are not expressing compliance, the effects of words will be reduced, given that the request is less likely to be attended. In sum, Scorolli et al. (2011; submitted) have shown that words alone are effective in modifying the bodily space. However, we predict that their effect will be more marked in a context in which the social dimension is emphasized, thanks to the real presence of another person.

These results are complementary with those obtained by Costantini et al. (2011b). Previous evidence demonstrated that objects afford actions only when presented in the peripersonal

"fpsyg-04-00214" — 2013/4/30 — 13:00 — page 5 — #5

space e.g., Costantini et al. (2011a). The novelty of the study by Costantini et al. (2011b) consisted in showing that when the object was outside subject's reaching space but within an avatar's reaching space, it evoked affordances as well. According to the authors, this indicates that an interpersonal body representation is formed in which one's own arm reaching space is mapped with that of others'. Notice that an avatar might evoke the presence of another person, but the effects it produces might not be as strong as those elicited by the presence of a real other.

However, these findings together with those by Scorolli et al. (2011; submitted) and Scorolli and Borghi (2012), in which words refer only implicitly to the presence of another person, suggest that the subject's representation of reaching space is actually extended. Importantly, in the study by Scorolli et al. (2011; submitted) the other person plays a complementary role as he/she is implicitly evoked to perform an action one cannot perform alone (Newman-Norlund et al., 2007).

In sum: it has been suggested that active tool-use determines a progressive incorporation of the tool within the body schema (Iriki et al., 1996; Povinelli et al., 2010). The analogous extension of the operational space found after the rake, the button and the word use suggests that the reaching space extension is not due to the possibility of the tool to be integrated into the body schema, but to the goal-directed character of the action (Hommel et al., 2001; Massen and Prinz, 2009). However, some issues remain open.

The studies discussed so far indicate that words, similarly to real tools, determine a plastic modification of the reaching space, even if they cannot be integrated into the body schema as tools do. However, the evidence we reported concerns concrete words, and specifically words with specific referents endowed with a precise spatial location. One could ask whether the claim that words are tools can be generalized, i.e., whether other kinds of words can determine variations in the bodily space. Even if we are not aware of any evidence, we can speculate that even words like "the" or like "freedom," which do not have a specific concrete counterpart, can expand our near space (for work on mapping between demonstratives such as "this" and "that" and near and far space, see Coventry et al., 2008, 2012; Bonfiglioli et al., 2009). As we say something to somebody else through words we somehow create a novel, shared space. This should happen with each word, as each word is pronounced to be heard by somebody else. However, while we reported evidence showing that concrete words expand the peripersonal space, the possibility that this is true for other kinds of words is currently a speculation, and further research is needed in order to demonstrate it.

A further question one could raise is the following: do intransitive gestures as well induce an extension of the near space, similarly to tools? Indeed, for communicative gestures to succeed, we need that others are available and ready to collaborate, as it happens with words. Compared to gestures, however, words have a number of advantages: (a) they are typically more specific than gestures (e.g., I can point to an object I would like to receive, but the context might not help you to identify the precise object: this potential problem can be easily solved using the appropriate word); (b) they are arbitrarily related to their referents, and this allows more freedom of action; (c) also thanks to b, they are less tightly anchored

to a specific context and situation. Normally gestures coexist with words, even if they can have a separate meaning (McNeill, 2000; Kendon, 2004). Furthermore, it has been shown that gestures do not develop imitating others, but emerge in an autonomous way and are integrated in speech, probably because they facilitate thinking (Bates, 1976; Capirci et al., 1996; Iverson and Goldin-Meadow, 1998). On this basis we can advance the prediction that combinations of gestures and words would increase the effect with respect to words alone. As to the sign language, where gestures directly substitute words, we predict a similar effect as the one obtained with words. But consider the case in which gestures are not coupled with words but used as substitutes for them. In this case our predictions are not so straightforward, and further research is needed to investigate this important issue (for relevant work, see De Stefani et al., under review).

### **TOWARD A COMPUTATIONAL MODEL OF WORDS AS SOCIAL TOOLS**

Although the models reviewed in the Section "The incorporation of physical tools"give important insights on the brain mechanisms underlying the adaptation of body representation after using a tool, they do not deal with the question of the possible neural mechanisms underlying the processes of words as tools. To address this problem, it is crucial to consider three key aspects not yet considered by previous models: (a) the brain has a hierarchically (soft)modular organization (Meunier et al., 2010; Houk, 2011; b) such organization pivots on anticipatory/goal-based representations of actions at multiple levels (Hamilton and Grafton, 2008; c) words are grounded on the same (or contiguous) neural representations sub-serving action (for reviews, see Martin, 2007; Jirak et al., 2010).

A bio-inspired neural architecture based on these points is sketched in **Figure 1**. The overall model architecture is built on the model of Caligiore et al. (2010), capturing important aspects of hierarchical brain organization. Even if the model proposed here is not computationally implemented, the discussion of its design features allows us to unveil important aspects overlooked by current models on tool-body assimilation. These aspects could be important to investigate the neural mechanisms underlying the notion of words as social tools.

**Figure 1A** represents the fact that the vision of an object in the peripersonal space evokes several potential affordances (encoded by the neurons of PC) and actions (encoded by the neurons of PMC) selected based on BG and local competitions (Cisek, 2007). Importantly, some neurons of these areas represent affordances and actions in terms of expected outcomes (Hamilton and Grafton, 2008) or goals ("distal goals," Umiltà et al., 2008), such as "reaching the object," rather than in terms of detailed movement commands encoded in the motor cortex (MC). The pre-frontal cortex (PFC), which encodes the agent's ultimate goals based on the internal and external context, exerts a top-down biasing effect on the formation of proximal goals and on the selection of different affordances and actions taking place in the PC and PMC and ultimately leading to perform specific movements (MC).

The mechanisms of affordance and action selection based on goals are crucial to explain the modulation of neural representations when a tool is used to reach far objects. The key idea is that the neurons of PC/PMC encoding affordances and actions in *terms of expected effects can allow the abstraction of the specific aspects of actions pertaining to the use of the limb or the tool*. For example, **Figure 1B** shows that, when using a tool to

"fpsyg-04-00214" — 2013/4/30 — 13:00 — page 6 — #6

**FIGURE 1 | A model of tool-body assimilation.** In the model, the visual cortex (VC) performs basic visual processing; the parietal cortex (PC) extracts affordances based on body/world relations and encodes abstract action goals; the pre-motor cortex (PMC) prepares actions based on more specific action goals; the pre-frontal cortex (PFC) encodes the agent's ultimate goals based on context and internal homeostatic regulations, and on this basis contributes to form action goals in the PMC and the PC; cortico-cerebellar (Cer) loops

simulate body-world, dynamics; cortico-basal ganglia (BG) loops underlie action and affordance selection processes. **(A–C)** represent a possible progression of development of representations from body actions, to tool-mediated actions and words-as-tool actions, all relying on the same macro brain areas and on partially overlapping local neural modules (cell assemblies). Multiple PC-PMC arrows represent multiple affordance-action options and the black arrow the most active within these.

reach the object, PC neurons might encode the salient features of both the target and the tool while PMC neurons might encode the "reach the target" goal: as these representations have many features in common (same object and context, similar effect, similar attentional focus on the object, etc.) with those activated when reaching without a tool, the neural populations encoding them might strongly overlap and form Hebbian associations. These might lead to change the representations related to space.

The effects of words as tool on space representation might be due to these mechanisms and to thefact that words are grounded in the same neural structures underlying perception and action (Caligiore et al., 2010). **Figure 1C** shows this with an example where the object is in the extrapersonal space but another person is close to it. In this case, the use of a phono-articulation of a word (e.g., the name of the target directed at a caregiver in childhood) might produce the same outcome of a direct reach. This and the similarities of context, intentions, target, or even (failed) reaching movement, might cause an overlap and association between the space-related representations active in the two conditions. The fact that heard words may further compact sensorimotor representations (Mirolli and Parisi, 2009) would strengthen this process. This might warp all representations of space incorporating "reachability" information and lead to effects such as those observed in our study.

Possible alternatives to our view could refer to the fact that the neural basis for language comprehension and tool-use might to some extent differ. As it is well known, the ventral stream plays a major role for semantics and language processing, whereas the dorsal stream is crucial for action preparation and execution (e.g., Chao and Martin, 2000; Johnson-Frey et al., 2005), processes very important for tool-use. There is also clear evidence of dissociations between language and praxis in neuropsychological patients (e.g., Buxbaum et al., 2001; Humphreys and Forde, 2001).

We do not think that our proposal is really weakened by these arguments, for at least two reasons. First, recent literature has smoothened the distinction between ventral and dorsal streams (see for example Goodale and Westwood, 2004). Some authors have shown the many interactions between the two routes (Gallese et al., 1999). Furthermore, a sub-distinction between a dorsodorsal and a dorso-ventral route has been proposed (e.g.,Rizzolatti and Matelli, 2003). Accordingly, words referring to action would be processed in the dorso-ventral rather than in the ventral stream (see proposals by Binkofski and Buxbaum, in press; Borghi and Riggio, 2009; Marino et al., 2013).

More generally, our aim is to show that in some conditions words can change some of our internal brain representations as is done by tools (for an analysis of shared brain mechanisms between complex tool-use and language, see Frey, 2008), but not that the caused changes are identical in the two conditions.

At a more basic level, here we do not intend to argue that language use equals tool-use in all respects. In line with theories of reuse (e.g.,Anderson, 2010) we think that language is grounded in the sensorimotor system, but that, being at a higher abstraction level, modifications and constraints are introduced (for developing this argument, see Borghi, 2012). In synthesis, our aim is to show that words are tools, but they are not only tools.

# **CONCLUSION**

Words are first encountered as objects. They are peculiar objects, though, because they implicitly refer to a social and public dimension and because they are immaterial ones. Later they become internalized (Vygotsky, 1962). The capability to use (inner) language modifies our internal processes; language is a powerful means to reconfigure our mental abilities and capability of control. Therefore words help us in "self-engineering" ourselves, to perform better in our ecological niche. But when we produce them, words are also objects outside from us. Differently from the physical tools that, when used, recode the spatial relationship between our body and the world, words are part of the ethereal world of cognition. Even if they are immaterial, we have suggested that words are both extended and embodied. They are both extended and embodied because their use determines a remapping of the relationships between our body, the objects and the space.

The evidence that EG theorists have collected shows that words are embodied and grounded in our sensorimotor system. However, so far EG research has been exceedingly focused on words' referents and on how their meaning is represented in the brain, while neglecting what can be achieved through words. Seeing words as tools that extend our near space allow us to overcome these limitations.

At the same time, EM theorists have shown that words can be used as tools that augment our computational potentialities, and that meaning is not limited to what is represented in the brain. However, the EM perspective has insufficiently explored the social and public role words play. As we have shown, the remapping of the bodily space we found with words is granted by the fact that words imply the presence of others: somehow our own space becomes larger as it incorporates the space of others. These implied others complement our abilities, and we call them into play by means of words.

In sum, we think that the idea that words work as social tools that extend our near space can help combining two very promising and sophisticated perspectives, the EG and the EM views.

We agree withClark (2008)when he invites us"to cease to unreflectively privilege the inner, the biological, the neural."(p. 218). Accepting this invitation does not imply avoiding to ascribe value to the inner, the biological, the neural. In contrast, it permits the combination of an EG and an extended perspective on cognition in which the mind emerges "at the productive interface of brain, body, and social and material world." Treating words as social tools highlights exactly this.

# **ACKNOWLEDGMENTS**

"fpsyg-04-00214" — 2013/4/30 — 13:00 — page 7 — #7

Funds: ROSSI project (Emergence of communication in Robots through Sensorimotor and Social Interaction (Grant agreement n. 216125) to Anna Borghi, Claudia Scorolli and Daniele Caligiore, MIUR funds to Anna Borghi, IM-CLeVeRproject "Intrinsically\_Motivated\_Cumulative\_Learning\_Versatile\_Robots," grant agreement No. ICT-IP-231722) to Daniele Caligiore and GianlucaBaldassarre, and SINTELNET FET-OPEN CA (grant agreement No. 286370) to Luca Tummolini.


M. Buscema, P. Cantucci, and B. D'Amore (Berlin: Springer).


*Interdisciplinary Themes*, eds P. Carruthers, and J. Boucher (Cambridge: Cambridge University Press).


"fpsyg-04-00214" — 2013/4/30 — 13:00 — page 8 — #8


**152**


*New Experience.* New York: Oxford University Press.


"fpsyg-04-00214" — 2013/4/30 — 13:00 — page 9 — #9

review. *Neurosci. Biobehav. Rev.* 37, 491–521.


*Translation*. Oxford: Blackwell Publishing (translated by Anscombe, G. E. M., 2001).

Wolff, P., and Malt, B. (2010). "The language-thought interface. An introduction," in *Words and the Mind: How Words Capture Human Experience*, eds B. Malt, and P. Wolf (Oxford: Oxford University Press), 3–15.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 30 January 2013; paper pending published: 28 February 2013;* *accepted: 08 April 2013; published online: 01 May 2013.*

*Citation: Borghi AM, Scorolli C, Caligiore D, Baldassarre G and Tummolini L (2013) The embodied mind extended: using words as social tools. Front. Psychol. 4:214. doi: 10.3389/fpsyg.2013.00214 This article was submitted to Frontiers in Cognition, a specialty of Frontiers in*

*Psychology.*

*Copyright © 2013 Borghi, Scorolli, Caligiore, Baldassarre and Tummolini. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any thirdparty graphics etc.*

"fpsyg-04-00214" — 2013/4/30 — 13:00 — page 10 — #10

# Tool use as distributed cognition: how tools help, hinder and define manual skill

# *Chris Baber 1\*, Manish Parekh1 and Tulin G. Cengiz <sup>2</sup>*

*<sup>1</sup> School of Electronic, Electrical and Computer Engineering, University of Birmingham, Birmingham, UK <sup>2</sup> Department of Industrial Engineering, Uludag University, Bursa, Turkey*

#### *Edited by:*

*François Osiurak, Université de Lyon, France*

#### *Reviewed by:*

*Blandine Bril, Ecole des Hautes Etudes en Sciences Sociales, France Lewis A. Wheaton, Georgia Tech, USA*

#### *\*Correspondence:*

*Chris Baber, School of Electronic, Electrical and Computer Engineering, University of Birmingham, B15 2TT Birmingham, UK e-mail: c.baber@bham.ac.uk*

Our thesis in this paper is that, in order to appreciate the interplay between cognitive (goal-directed) and physical performance in tool use, it is necessary to determine the role that representations play in the use of tools. We argue that rather being solely a matter of internal (mental) representation, tool use makes use of the external representations that define the human–environment–tool–object system. This requires the notion of Distributed Cognition to encompass not simply the manner in which artifacts represent concepts but also how they represent praxis. Our argument is that this can be extended to include how artifacts-in-context afford use and how this response to affordances constitutes a particular form of skilled performance. By artifacts-in-context, we do not mean solely the affordances offered by the physical dimensions of a tool but also the interaction between the tool and the object that it is being used on. From this, "affordance" does not simply relate to the physical appearance of the tool but anticipates subsequent actions by the user directed towards the goal of changing the state of the object and this is best understood in terms of the "complimentarity" in the system. This assertion raises two challenges which are explored in this paper. The first is to distinguish "affordance" from the adaptation that one might expect to see in descriptions of motor control; when we speak of "affordance" as a form of anticipation, don't we just mean the ability to adjust movements in response to physical demands? The second is to distinguish "affordance" from a schema of the tool; when we talk about anticipation, don't we just mean the ability to call on a schema representing a "recipe" for using *that* tool for *that* task? This question of representation, specifically what knowledge needs to be represented in tool use, is central to this paper.

**Keywords: distributed cognition, tool use, affordances, representation, extended mind, systems dynamics**

# **INTRODUCTION**

The central question for this paper is what representations are employed when using tools? In this paper, the term "representation"is taken to mean a set of parameters which describe an action (from goal to execution). In broad terms, one answer to this question might see the set of parameters as being specified prior to an action being performed, e.g., in the form of an action schema, or as being recruited in preparation of the action, e.g., in the form of activation of specific brain regions. In this case, the question becomes one of identifying what the representation might contain and where it might be stored. This is what we refer to as an "internal representation."Alternatively, the parameters might arise from the performance of the action in response to constraints imposed by the environment, e.g., in the dynamic behavior of a system. This is what we refer to as an "external representation." We argue that, while there is evidence to support the view that tool use can be guided by "internal representation," this only provides a partial view of such activity and that the use of "external representation" can provide a viable alternative account.

The position taken in this paper assumes that the physical behavior of the person can be viewed as part and parcel of their cognitive activity, and that there is a close coupling between a

person's action and their perception of features of objects in the world. However, neither assumption fully captures human activity when using physical objects for goal-directed activity (which is the broad definition of tool-use employed in this paper). Thus, we argue for a broader appreciation of Gibson's (1979) notion of *complimentarity* as an explanation of affordance at a"system"level. The notion of "system" here draws on Maravita and Iriki's (2004) idea of the "hand-tool body schema" but we extend this to cover person–environment–tool–object. For us, this requires the notion of Distributed Cognition to encompass not simply the manner in which artifacts represent concepts but also how they represent praxis. In other words, the design of the tool (as a human-made artifact) reflects not only the manufacturing process but also a set of assumptions about how that tool should be grasped and manipulated, and how activities involving that tool can be performed "correctly." This means that "tools" are distinct from other physical objects in the human environment because their use is defined not only by their appearance or the user's goals but also by cultural constraints that have influenced their production (Baber, 2003, 2006; Burghardt et al., 2011). While there are instances in which other physical objects, such as sticks or stones, can fulfil tool functions, and while the neurological evidence suggests that images

of these objects activate similar regions in the brain to images of tools, there is accumulating evidence that the pattern of brain activation for tools is somewhat different from that of physical objects *per se.*

*"Human beings, viewed as behaving systems, are quite simple. The apparent complexity of our behaviour over time is largely a reflection of the environment in which we find ourselves"* [Simon, 1969]

While Simon was not talking explicitly about Distributed Cognition, this quotation points to the need to understand human behavior in the environment in which it occurs. For us this implies a need to better understand how the environment makes an impact on our actions and decisions, and this suggests the benefit of an approach which studies human action as they occur in natural (or as near natural as possible) conditions. This raises challenges for "ecological validity" (Neisser, 1967) which takes us out of the laboratory (or, for that matter, the brain scanner) and into the settings in which activity is performed. A primary reason for this quest is the assumption that the relations between human, environment, tool, and object are fundamental to the study of perception and action (Gibson, 1979; Beek and Bingham, 1991; Newell, 1991). A study of the activities of tool use away from typical environments runs the risk of ignoring the constraints that the environment places on the performance of these activities. Thus, it is vital to ensure that enough of the characteristics of the person–environment–tool–object system are reflected in the design of studies (even if these are conducted in laboratories). We are interested in ways in which we might be able to capture data from the tool using actions of people in work environments, through analyzing video of their activity (and discussing these videos with them) or through putting sensors on the tools that they use. For this paper, the focus will be on the use of data collected from sensors on tools. Two areas of activity will be used in this paper: using hand-tools in jewelery and eating with cutlery. In both areas, the concern will be to compare experienced and less experienced users of the tools. The comparison will be qualitative rather than quantitative, i.e., examples of the data collected during our studies will be presented but more detailed analyses of these data will be found in other papers.

#### **WHAT NEEDS TO BE REPRESENTED IN TOOL USE?**

By way of a definition of the word "tool," we propose that a tool is a physical object which lends itself to manipulation by a human (or animal) in order to solve a problem presented by objects in the physical environment. This notion of tool-use as a form of problem-solving not only emphasizes the goal-directed aspect of using tools but also the need to respond to, and overcome, constraints. This definition allows us to combine both the physical action of manipulating the tool with the cognitive aspects of goal-directed, purposeful behavior. Following a similar line of argument (tool use as problem solving), Osiurak et al. (2010) suggest that the coordination of the physical actions involved in using tools represent a problem to be solved. They view cognition and physical activity in a dialectic in which a particular goal encourages the perception of particular affordances in the world and serves to influence the bodily action to perform, which, in turn,

moves the person towards their goal. This strikes us as an elegant reformulation of the notion of affordance as a goal-directed, physical response to the environment. The difference between this view and the one presented in this paper is simply (we believe) a matter of scale: rather than considering problem solving in the broad terms that Osiurak et al. (2010) offer, our focus is on the interface between tool and object (or, rather, we propose that the "problem" that concerns tool users is how to modify the object in ways that satisfies a goal, given the constraints that the tool (and the tool-users' ability to wield that tool) might impose on their action).

In order to explore further the question of representation in tool use, it is important to consider *what* needs to be represented in order to use a tool. Tool use is not only a matter of recognizing that an object is a tool but also of knowing how to hold and manipulate that particular tool. It is also a matter of understanding the consequences of a particular way of using a particular tool. Knowing that a piercing saw (used by jewelers to cut metals) is held vertically for cutting (with the wrist more or less locked and most of the motion about the elbow), and has teeth which cut in one direction, leads to an understanding that the cut is made on the downstroke (not the upstroke), and helps define a set of possible actions when using this tool. From this it might appear that we are arguing for (at least) some representations of the tool and the actions associated to be internal to the person. Does this mean that these representations are stored in the brain?

# **INTERNAL REPRESENTATION: NEURAL ACTIVATION IN TOOL USE**

The suggestion that the use of tools depends on "internal models" is nicely encapsulated in a recent paper by Imamizu and Kawato (2012). They review literature and report studies which indicate the existence of both a feed-forward model, taking efference copies of motor commands to enable motion dynamics, and inverse models used to manage these dynamics. During learning, changes in cerebellar activity indicate the acquisition and refinement of such models. As we argue in this paper, the notion that brain-based "internal models" are *causal* represents a particular view of tool use, and we are proposing that it is possible to explain much of the activity involved in tool use through a combination of Distributed Cognition and dynamics which might not be represented in the brain *per se*. However, before exploring this proposal further, we consider some of the neuropsychological evidence relating to tool use. Imamizu and Kawato (2012) review neuropsychological studies of tool use and suggest that, *"[A]lthough the brain regions related to each type of component cannot be uniquely determined*...*"* (p. 325) there are two distinct functional regions of the brain related to tool use: one related to the physical skills involved in dextrous tool manipulation, and one related to the semantic and conceptual knowledge relating to the functions of tools (see also Lewis, 2006; Higuchi et al., 2007, 2009). These distinct regions are discussed in more detail in the rest of this section.

In their now classic study, Chao and Martin (2000) used functional magnetic resonance imaging (fMRI) to show that viewing and naming of tools led to activation of the left ventral premotor cortex, suggesting a strong relationship between the physical appearance of objects and the fact these objects could be acted upon. Grafton et al. (1997) used positron emission tomography (PET) scanning of participants asked to observe or (silently) name tools and their use. Observation of tools resulted in strong activation of the left dorsal premotor cortex, and (silent) naming of these tools resulted in additional activation of Broca's area. However, naming the *use* of the tools led to activation in Broca's area, together with activation in left dorsal premotor cortex, left ventral premotor cortex, and left supplementary motor area. This implies that naming the use of a tool (even when the action is not performed with it) has motor valence which is additional to that obtained when looking at the tool. It also suggests that the physical appearance and name of a tool activates slightly different areas than the use of the tool. Taken together these, and related, studies imply that brain activation relates to specific properties of the tool-as-form and tool-as-function, and that these properties are not solely related to a tool's physical appearance but also to how it moves or how it is used (Johnson-Frey, 2004; Martin, 2007).

One suggestion is that representations of tools are held in specific regions of the brain and become activated during activities in which similar objects are used. According to Gallivan et al. (2013) the distributed coding of different actions associated with hand movement and tool use imply that these actions are represented separately and then integrated in the frontoparietal cortex. As Yee et al. (2013) show, in an ingenious experiment, asking people to think about manipulable objects when they are performing manual actions which are incompatible with those objects is difficult (but it easy to think about non-manipulable objects during the performance of such actions). This suggests that the meaning of objects (specifically in terms of their properties which support manipulation) is recruited during action, and that incompatible action interferes with this. Furthermore, work by Hoeren et al. (2013) points to the suggestion that the recognition of action (performed by other people) is processed using distinct streams: the dorso-dorsal stream focusing on movement determined by the properties of the objects being used, and the dorso-ventral stream focusing on functional appropriateness and dexterity of task performance.

#### **KNOWLEDGE OF (FAMILIAR) TOOL USE**

The discussion so far points to the need to draw on knowledge of the appropriateness of a given tool for a given task and how to wield that tool to achieve the most effective result. Riddoch et al. (2006) presented patients (manifesting visual extinction) with images of pairs of objects. The pairs showed objects which people are likely to have experienced being used together (e.g., a bottle and a glass), or objects which could plausibly be used together, although might not have been experienced as such (e.g., a bottle and a bucket), or were randomly paired in order to, as far as possible, produce pairs which had no association. The results showed that commonly paired objects were identified more quickly than plausibly paired objects which, in turn, were identified more quickly than the randomly paired objects (although this latter finding only held when the image showed the objects being used together rather than having them presented side by side). One implication of this work (which could be applied to normals as well as patients) is that

the common and plausible pairs activate familiar routines in tool use. In contrast with this observation, Vingerhoets (2008) found that presentation of images of "familiar" or "unfamiliar" tools activated the same brain regions, with "unfamiliar" tools generating more activation in the left hemispheric medial posterior occipital and inferior posterior temporal areas (in comparison to images of "familiar" tools) and more activation around the supramarginal gyrus for the familiar tools. While these results showed strong individual differences, they also imply that the activation in response to "familiar" tools can be associated with knowledge of the appropriate hand position for the *use* of the tool (as opposed to simply whether or not the tool *could* be grasped).

A similar line of argument comes from studies in which participants are asked to pick up handled objects (such as cups) when the handle faces either towards or away from the hand that they are instructed to use (Tucker and Ellis, 2001). For example, Bub et al. (2012) presented images of everyday objects together with images of hands in different orientations. The objects all had handles which were either oriented horizontally, e.g., pliers, frying pan, or vertically, e.g., beer mug, hairdryer. Participants were asked to name the object. Reaction (naming) time was significantly faster when both hand and wrist orientation matched the type of handle, or when neither hand and wrist orientation matched the handle, but much slower when either hand or wrist orientation was incongruent. Relating this to the previous discussion of neural imaging, one can assume that the photographs of the hands and the objects might have activated different regions, with a combination occurring *prior* to response.

The suggestion that there might be preparative neural activity which corresponds to different types of action (Rizzolatti et al., 1988) could provide evidence for the recruitment of a set of representations determining task performance. Certainly the movement-related cortical potential (MRCP) recorded from electroencephalography (EEG) begins 2–3 s before the onset of movement (Toma and Hallett, 2003; Wheaton et al., 2005). Furthermore, onset seems to be proportional to complexity of movement, with more complex movements having longer onset times. Such activity, typically in the left posterior parietal cortex, is taken to indicate the need to manage complex motor activity and, as Wheaton et al. (2005) propose may include *"*...*imagining executing such movements; the goal of the movement; determining the natural position and setting required for proper performance; sequence of motor acts and comprehension of the task."* (p. 535). While we have every reason to accept that complex movements involve recruitment of appropriate muscle groupings and specification of appropriate control parameters, we do not see why this necessarily involves the definition of specific representations of the task context. Thus, our debate is not with the neurological evidence *per se* but with the assumptions that these *must* point to internal representations which drive behavior.

What is interesting in the Bub et al. (2012) study is less the reinforcing of activation of congruent images (or, indeed, the effect of incongruence) than the problems caused when one of the hand images did not match the other image or the object. Bub et al. (2012) suggest that this reflected disruption of the plan being developed in working memory (with the images activating particular judgments about using tools). However, the images presented in these studies serve as the (external) representations about which people are asked to make judgments. As such the idea that they would need to create corresponding internal representations in order to make such judgments seems a little odd. The images that are presented provided sufficient information to make a judgment and the need is to determine whether these are "true" or "possible." On the one hand, it seems plausible to assume that prior experience provides the "grounding" (Mizelle and Wheaton, 2010) of a tool in terms of its usage, but on the other hand, it is equally plausible that this could be part of the person's action repertoire (e.g., in terms of Bernstein's (1967) idea of coordinative structures) as it is activation of specific regions of the brain.

For an action in which participants had to use different tools to touch a target, precuing the target had no benefit on performance, but precuing the tool to use had significant benefits (Massen and Prinz, 2007). We take this to suggest that the precuing of the tool enabled the recruitment of the appropriate "coordinative structure," to use Bernstein's (1967) phrase describing combinations of muscle enervation and limb movement, to perform the task with a given tool. What is interesting about this interpretation of their findings is that "representation" need not be same for different tasks (and, we would argue, shows how it can shift to outside the brain *per se*). Hermsdörfer et al. (2006) compared performance of apraxic patients with a control group of normals on a sawing task. Participants were asked to demonstrate sawing under three conditions: when they were shown a photograph of a saw and asked to pantomime sawing; when they were shown the photograph, given a piece of wood (the same size as the saw's handle) to hold and asked to pantomime sawing; when they were given the actual saw to hold. While the controls showed fairly consistent performance across the three conditions, apraxic patients showed motion errors (deduced from 3D motion tracking) in the first two conditions. Typically, these errors involved substituting mediolateral motion for the anteposterial motion expected. Interestingly, these errors were *not* apparent when the apraxic patients were given the actual saw to use. On the one hand, this supports a common finding in apraxic studies (that providing people with the physical object seems to enable them to perform tasks more effectively and reliably than when they do not have the object to hand). On the other hand, we believe it tells us something about the need for internal representation when using tools. Hermsdörfer et al. (2006) conclude that *"*... *pantomiming the use of a tool and actually using the tool are facilitated by largely different neural processes which differ in demands and goals."* [p. 1651]. We would argue further that these differences arise because the use of the tool involves the control of the person– environment–tool–object system and need *not* depend on internal representation.

#### **CONCLUSION**

Just because the tool-using behaviors have neural correlates does not mean that these are the only places in which representations for the behaviors exist. Clearly, the type of grasp is likely to be influenced by the action which one intends to perform with the tool. We have a repertoire of appropriate grasps for manipulable objects, and we adapt these grasps according to contextual demands. The adaptation often occurs with sufficient fluency and speed to make it unlikely that we have simply retrieved a particular piece of "motor schema" from memory and applied this; indeed, the very notion of a "motor schema" (with its attendant implication of stored sequences of action) has been called into question (Sherwood and Lee, 2003; Shea and Wulf, 2005)*.* Thus, we argue the tool user is, partly, using the tool to make changes to objects in the environment, but also partly using the tool to help create further opportunities in the environment for using the tool. In other words, tool use is an interplay between seeking a defined goal and managing the affordances arising from changes in the object in the environment (resulting from the ongoing use of tools). Before discussing the collection of data and their analysis, the next section describes the particular stance taken in this paper: Distributed Cognition.

# **EXTERNAL REPRESENTATION: DISTRIBUTED COGNITION AND THE EXTENDED MIND**

As the phrase implies, Distributed Cognition addresses situations in which the processing of information occurs outside the brain. For some writers, this is the proposal that the environment and the objects it contains can shape the way in which cognition is performed (Zhang and Norman, 1994; Hutchins, 1995a,b; Scaife and Rogers, 1996). While this position could be seen as paraphrasing the well-known observation that the representation of a problem space influences the strategy that problem solvers apply (Chase and Simon, 1973; Larkin et al., 1980; Chi et al., 1981), e.g., changing the layout of a puzzle can make it easier or harder to solve, it also points to the importance of interactivity in behavior. For example, people playing Tetris or Scrabble can benefit (in some situations) by being allowed to manipulate and rearrange the playing pieces (Kirsh and Maglio, 1994; Maglio et al., 1999). This points to the need to not simply focus on the arrangement and design of the problem representation, but also on the nature of the interaction between person and objects. From this point of view, "embodiment" becomes an essential feature of acting not only on the objects but also on the cognitive tasks involved in problem solving. In other words, rearranging the pieces is not simply performed in order to assist thinking, it *is* thinking. This is taken to mean that the relationships within the human–environment–tool–object system not only supports (or affords) different actions but also shapes cognition (Wilson, 2002). The reason for this is that activity within this system is often time-limited, in that the actions are performed at speed, in real-time and offer little opportunity for planning (what Clark, 1997, has termed "mind on the hoof"). From this, the main purpose of cognition (in tool use) is to support action in as situation-appropriate manner as possible. It also suggests that, rather than needing to construct "internal representations" of the environment, it is sufficient to respond to the appearance of the environment. From his work with robots, Brooks (1990) pointed out that robot performance could be more efficient if they spent less time "planning" and creating representations, and more

time "acting" because *"the world is its own best model"* [Brooks, 1990, 12].

#### **CONSIDERING AFFORDANCE**

Our reading of Gibson's (1977, 1979) concept of affordance lies in his notion of "complimentarity"in which the properties of objects in the environment are responded to by the animal. Turvey (1992) offers the term"effectivity"as a way of capturing these properties of the animal. So, one way of seeing affordance lies in the complimentarity between the object's properties and the animal's effectivity. One of the problems that the idea of "effectivity" and "properties" raises is the suggestion that these are separate aspects which are brought together during the performance of a task, which implies that they are independent, autonomous features which become coupled during task performance. Indeed, Gibson (1979) suggests that the "affordance" exists whether or not the observer perceives or attends to it. If this is the case, then it makes sense to assume that one aspect of the "effectivity" of the task performer would be the neural representations of the actions involved in performing the task (as well as morphological features and motor skills).

There are many situations in which the observer cannot but attend to the affordance, e.g., perseveration in the behavior of stroke patients, or response to "fake" cues by animals. Rather than implying (as Gibson seems to) that the "affordance"is an invariant property of the environment, the fact that perseveration is an unusual state of affairs suggests that humans (and some animals) are able to *choose* to respond to affordances (and by implication, to *see* affordances in different situations). This implies that what is essential to "affordance" is this combination of the property of the object in the environment and the effectivity of the specific individual (with specific knowledge, skills, abilities, and goals) in that environment. While the properties of the object in the environment may well be invariant, the actual affordance arises from the complimentarity of environment and actor. Affordance is partly a matter of perception-action coupling and partly a matter of intention (goal) – action coupling. Perhaps a better way of putting this is that perception-action coupling is mediated by the intentionality of the actor. However, as Chemero (2009) points out, the idea of separable components that can be coupled runs counter to the notion of complimentarity; taking affordance as the result of the system created by person–environment–tool–object (as we do in this paper) leads to the conclusion that this is a system which is non-decomposable and which exists only during the performance of the given task. From this, we suggest that the goal, in the person–environment–tool–object system is *partly* held by the person (in terms of the effect that they intend to produce on the object) and partly situated (Suchman, 1987) in the ongoing interactivity in the system. This assumption echoes the earlier assertion of van Leeuwen et al. (1994) that tool use can be *"*... *defined as performing an action on a target by performing an action on a tool. The action on the tool is embedded in the action on the target."* [van Leeuwen et al. (1994), p. 188–189]. For van Leeuwen et al. (1994) this embedding reflected a "higher-order affordance structure" of "mutually constraining complimentarities."

van Leeuwen et al. (1994) argued that it was important to understand the role of context in task performance in terms of a *sufficiency principle*, i.e., *"if an affordance has already been realized,* *there is no need to take it into account."* [van Leeuwen et al. (1994), p. 190]. To take this a little further, Turvey (1992) suggests that affordance might play a role in "predictive control" of activity and, while the analysis (and indeed use of the term), in this paper might differ from his, the idea that affordance refers not only to immediate action but to future actions is central to the ideas in this paper. Additionally, Mizelle et al. (2013) discuss the notion of *functional* affordance, in which there is an optimal manner in which a given object can be used to achieve a desired goal. For example, Mizelle et al. (2013) note that a hammer can be held a variety of grasps (some involving the handle, some involving the head, for instance) but that there is a grasp which *"*... *best affords the action of driving a nail*...*"* (p. 280). This can be seen as taking the predictive control further, in that there is a goal state against action can be optimized. While these notions of affordance *could* be represented internally (in terms of specific neural correlates of functional affordance that can be adapted to contextual demands), the notion of complimentarity followed in this paper offers a more parsimonious explanation. In other words, the Gibsonian notion of affordance is taken in this paper to describe a particular form of complimentarity in the person–environment–tool–object system, and it is the "system" as a whole which can be said to optimize the tool-using activity.

#### **TASKONOMY AND HOW THE ENVIRONMENT AFFORDS SKILLED ACTION**

One way in which the environment can be created to provide affordances for future action is in the ways in which experts lay out their workspace. In their discussion of blacksmiths, Keller and Keller (1996) use the term "taskonomy" to refer to the ways in which an expert's knowledge of the tasks to be performed help create the arrangement (taxonomy) of tools in their space. This arrangement is not simply a matter of having particular types of tools kept near each other, but arises through a combination of tools and actions. A similar pattern can be seen in the workspace of the jeweler (**Figure 1**).

As the jeweler performs a particular task, so a tool is picked up, used and then laid down in the workspace; as work progresses so tools are either reused or new ones introduced. However, the expert is often able to describe what work had been completed in a particular workspace by looking at the collection of tools in the immediate vicinity. In some cases, specialized tools will be brought to the workspace with the intention of supporting a particular goal. Thus, the workspace becomes managed to provide particular affordances (in terms of available tools and the position in which these tools are placed to support particular types of grasp). This suggests the anticipation of tasks and the arrangement of the workspace in line with these anticipations. In these ways, the movement of tools in the workspace (as the result of deliberately selecting these in preparation of a specific job, or as the result of picking up and putting down the tools during the performance of the job, or as the result of moving tools which are no longer needed further away from the central point of reaching) becomes part of the structuring of the workspace. Rather than simply reflecting the ebb and flow of actions in the workspace, we argue that this reflects the management of potential affordances and, as such, is a form of Distributed Cognition. The suggestion

that moving tools around the workspace is a form of "cognition"is logically the same as the suggestion that presentation of a problem will "frame" the approach to the solution and that manipulating pieces in a puzzle might be a form of thinking. In other words, layout of the workspace will frame the actions which are most likely to be performed and this framing is the result of deliberate choices made to retain, discard or move a tool after it has been used (rather than merely a consequence of moving tools around).

# **WORKING WITH TOOLS**

In his discussion of craftwork, David Pye draws the useful distinction between "certainty" and "risk" in craftwork. He argues that in "the workmanship of certainty" there is an impetus to design work to ensure consistency, repeatability, and minimize variation or ambiguity. Such work involves heavily proscribed procedures and measures of quality and could be interpreted in terms of industrialized production processes. In this approach, the artifact being produced will be tightly specified prior to production and the resulting artifact will be considered in terms of this specification. Anyone who has constructed flat-pack or selfassembly furniture will have encountered a situation in which the manufacturer has sought to encourage workmanship of certainty. However, anyone who has built self-assembly furniture will also recognize the challenges that this poses. Misreading the instructions or believing that you know what you are doing so don't need to read the instructions *can* lead to results which differ from the goal. This could be quite minor (a handful of left over components) or quite major (the door which doesn't open, the shelf which drops out when the unit is stood up). This variation illustrates the workmanship of risk. This, in turn, reflects the variability in outcome which can arise from decisions made by the worker during the performance of the tasks. The decisions could reflect a choice of tool, or knowledge/skill in the use of the tool, but they could equally reflect responses to the opportunities presented (or constraints created) by the materials being used. For example, the knot in a piece of wood, or the finish on one side of the self-assembly wardrobe, could constraint the actions which are possible or could suggest an appropriate action to perform. In contrast, the "workmanship of risk" does

not involve such tight specification, i.e., *"*... *the quality of the result is not predetermined, but depends on the judgment, dexterity and care which the maker exercises as he works."* (Pye, 1968, p. 20). Rather than the intent or purpose being predetermined, it is now something which crystalizes through the developing interaction between craftworker, tools, and materials being worked. This is something which we noted in our study of jewellery making (Baber and Saini, 1995): the jeweler worked to very sketchy "plans" but adapted these plans to suit the resulting state of the material, often modifying a particular ring or brooch to capitalize on a particular facet that they noticed as the metal was being worked.

*"First, the experienced worker usually employs "smoother*" *and more consistent movements…Secondly, the experienced worker operates more rhythmically, indicating that a higher degree of temporal organization has been achieved. Thirdly, the experienced worker makes better use of the sensory data*...*Fourthly, the experienced worker reacts in an integrated way to groups of sensory signals, and makes organized grouped responses to them"* [Seymour, 1972, 35–36]

The quotation from Seymour (1972) indicates how the output of the human–environment–tool–object system is being optimized, but not necessarily how the dynamics of the system relate changes in input to output. In order to consider this, we turn our attention to series of studies conducted by Bril and her colleagues, focusing on tasks involving hammering (either stone hammers to knap flint or metal hammers to shape stone or glass beads).

### **SYSTEM DYNAMICS: TRANSFORMATIONS IN TOOL USE**

Our actions, when using tools, involve the coordination of a set of transformations (Biryukova and Bril, 2012). We transform kinetic energy into tool motion – but need to appreciate how much energy to exert in order to produce the desired motion of the tool (and in order to produce the desired effect on the object from the tool's motion). We manage dynamic transformations, balancing the movement of the tool in the air and on the object with our own motions and with the outcome of the tool's activity. We anticipate what effect the tool's motions will produce and relate these to the outcomes that we desire. As Ingold

(2000) points out, *"Intentionality and functionality are* ... *immanent in the activity itself, in the gestural synergy of human being, tool and environment."* (Ingold, 2000, p. 352.) The ability to both anticipate the outcome of the tool's action and manage the functionality of the tool are an integral part of the use of the tool. The dynamics of using the tool thus becomes far more important than might be implied by the neurological imaging work which concentrates on the form and function of the tool. Given that these (and related) transformations need to be managed during the use of tools, it is worth asking *where* these transformations might be represented? If they are "represented" simply during the performance of an action, and arise from the moment-to-moment correction of the action, then one might not expect to see anticipatory effects. On the other hand, if there is evidence of anticipation (of the consequences of any of these transformations) then this implies a need to represent the consequence and the question remains, where does this representation reside and what form does it take?

Before considering the questions of transformations, it is worth repeating some of the observations from these studies regarding expertise. For example, Roux et al. (1995) showed that expert craftsmen (making stone or glass beads) showed significantly less inter- and intra-individual variations in performance than less experienced workers. Similarly, Biryukova and Bril (2008) showed that expert knappers used a larger repertoire of joint angle combinations than their less qualified colleagues (who tended to demonstrate more rigid behavior), and Bril et al. (2010) showed that experts showed a lower variability in kinetic energy compared to intermediates and novices. In related work, Vernooij et al. (2012) explored learning in a task involving the use of a 300-g hammer-stone. Analysis of motion tracked during the performance of this task showed inter-individual differences in the ways in which joint angles were combined to strike a particular type of blow and that these combinations changed during the course of the study. The analysis of learning to use such a hammer suggested that participants were only able to modify one parameter (relating to joint angles or impact force) but not both at the same time, until they had gained proficiency in the task.

In a series of experiments comparing expert, intermediate and novice users of stone hammers (in flint-knapping tasks conducted in the laboratory), Bril et al. (2010) identify three primary parameters that seem to contribute to the dynamics of tool use in this context. The first are Control Parameters, such as the velocity with which the hammer stone approached the target. The study showed that, in general, novices appreciated the need to control velocity but were not able to control this efficiently (this finding is supported by the work on Vernooij et al., 2012, discussed above). Thus, we would expect greater variability in the novice performance on these control parameters; as Seymour (1972) put it, the expert actions would be performed in a "smoother,""more consistent,""more rhythmical[ly]"manner. The second set of parameters considered by Bril et al. (2010) are regulatory parameters, such as the trajectory followed by the hammer stone and the potential energy applied. Experts tended to show shorter trajectories and smaller ratios between parameters. In Seymour's (1972) terms, this shows how experts are able to use a "higher degree of temporal

organization" and also to make "better use of the sensory data" in managing their actions. As Bril et al. (2010) note, *"In the present task, the velocity of the hammer had to be controlled to produce the required kinetic energy in relation to the mass of the hammer. This was achieved by concurrently changing the trajectory, the amplitude of the movement, and the muscular force. In this perspective, the movement became meaningful only in relation to the production of functional parameters at the level of the task, which allowed for movement flexibility as long as the task requirements were fulfilled."* (Bril et al., 2010, p. 837). This quotation introduces the third parameter, the Functional parameter, such as kinetic energy, which experts appear to hold constant and aim to apply the lowest kinetic energy that is sufficient for the task. As the experiments involved presenting participants with hammers of different weights and requiring them to produce flakes of different sizes, one can assume that all participants would be able to discern changes in hammer weight or task demands (in terms of flake size), but the results suggest that a characteristic of expertise (which was not available to the novices) was the ability to respond to "nested relationships" (Wagman and Carello, 2003) between weight of hammer and size of flake to produce. The ability to appreciate these "nested relationships" allowed the experts to interpret the constraints placed on them by the person–environment–tool–object relationships and respond to these in ways that the novices could not. So, we return to the question of *where* these constraints might be represented? One possibility (implied by Seymour, 1972 and mooted by Bril et al., 2010) is that the initial representations involve Functional parameters which are learned and then adapted to changes in context.

In his discussion of dexterity, Bernstein (1967) highlighted that the main determinant was not bodily movement so much as the capability to respond to changes in the conditions surrounding the person. Bernstein's (1967) notions of tool use, in terms of dexterity, relate to the quotation from Simon (1969) at the start of this paper. The manipulation of tools is rarely an end in itself but is performed with the intention of shaping objects in the environment. The actions performed lead to changes in the objects but also indicate the intentionality of the tool user (providing they have sufficiently dexterity in their use of the tools). The expert tool user thinks through the tools that are used because the actions performed with the tools shape the environment in such a way as to solve the problems that it presents and in such a way as to produce the results that the tool user desires. The action performed with the tool also creates the opportunities for the next action; and this, in turn, reflects the type of grip and posture which the tool user adopts. In this way, grip and posture (in holding and using a tool) indicate the chosen solution to the problem that the tool user is solving.

In much the same way that Rosenbaum et al. (2012) speaks of end-state comfort (and the ways in which a posture anticipates a particular end-state following the movement), so we can think of the ways in which the tool user is continually seeking to adapt their current motions in anticipation of subsequent motions and states of the object. For this paper, we take this to mean that the skilled tool-user is better able to coordinate the person–environment–tool–object system and to anticipate how changes in this system require adaptation of activity. This realtime adaptation need not imply internal representation of either

task dynamics or some form of "motor program." Rather, the expert is able to produce movements which are coordinated to task goals (being more efficient and economical in terms of energy use). In a sense, expertise is the practiced adaptation of intrinsic dynamics to task dynamics (where task dynamics are defined by the person–environment–tool–object system) so that changes in task constraints and affordances can be appropriately responded to through subtle tuning of actions. This implies that experts are able to modify the pattern of activity without necessarily impairing the

functional impact of the activity. We do not believe that experts need to possess, or even represent, these various patterns of activity but rather these arise on-the-fly during the coordinated control of limbs holding and controlling tools.

# **STUDIES USING SENSORS FITTED TO THE HANDLES OF TOOLS**

In order to explore these questions of dynamics, we have been exploring ways in which to capture behavior in the field (or, at

**the "paint removal" tasks.** The spectrogram of frequencies 0–20 Hz over time for *y*-movement (i.e., back and forth motion of the file across the wood), moving between the 1st and 3rd dots on the file with the goal of removing two layers of paint.

least, in laboratory and workshop settings which are as close to the field as possible). This has involved designing and developing handles which combine different types of sensor to capture the actions a person performs. Often such data are collected from using camera systems with markers on the person. While these can be very accurate, they are not easy to use in the field. Thus, it makes more sense to instrument the person or their tools in order to collect data *in* situ. We have taken the lead from Bril and her colleagues (discussed in the previous section) to instrument our tools (**Figure 2**). In our work, strain gages are used to capture force applied to the handle and a three-axis accelerometer is used to capture motion (Parekh and Baber, 2010 for a description of the design of these handles).

In order to appreciate how experience in using a given tool can shape activity, **Figure 3** presents an extract of recordings (from a three-axis accelerometer and strain gages integrated into the handle of a jeweler's file) of an experienced silversmith filing the edge of a metal strip. **Figure 3** shows three filing strokes over the course of 2.5 s. Each stroke (occurring at approximately 11.6, 12, and 12.9 s) is indicated by an increase in the *y*-velocity data. There are two types of stroke here: rapid (at 11.6 and 12.9 s) in which the file in moved rapidly across the metal, and slow (12 s) in which the file is drawn more slowly over the metal. During each stroke, there is downward pressure on the file (indicated by the decrease in *z*-velocity data and increase in "top grip" force applied to the top of the handle). Immediately following the stroke, the file is lifted up (increase in *z*-velocity) and brought back to the starting point. Prior to the next stroke the file is adjusted and aligned with the metal (which takes around 1 s), which involves little change in grip force applied and z-velocity. The top grip loosens as the file position is reset for lifting the file off the object (movement in the *z* direction); the expert user only applies force on the forwards motion. This action is partly dictated by the file being used and partly by the results that the tool user intends. As the expert said, you can remove metal easily enough but you can't put it back. So filing is about removing sufficient (but not too much) of the metal. Furthermore, the metal being worked (copper in this instance) could easily be dulled if too much of the upper surface was removed, and so filing was also a matter of retaining the luster of the metal. Such knowledge can affect the way in which the tool is wielded and influence the outcomes that one might expect when using the tools.

In another study, we asked novice users of a file to remove paint from a piece of wood. **Figure 2B** shows the task being performed. There are three dots painted on the top of the file and participant was instructed to ensure that the file was kept between the first and second, or the first and third dots.

Contrasting three people performing the filing task (**Figure 4**), we can see that while the main activity (yellow on the spectrographs) occurs at similar frequencies, the harmonics vary. These variations might reflect differences in strategy. We would expect to see harmonics from these data due to the periodicity of the repetitive motions employed. This also suggests that differences in performance can be captured through a better appreciation of dynamics and, potentially, following the lead of Bernstein (1967), can be reflected in the conservation of energy of the tool users.

The raw accelerometer data were integrated to produce velocity, on which we applied a Fourier transform to determine the fundamental frequency of the filing motion. **Table 1** suggests that the main determinant of this fundamental frequency is not the tool-specific goal to keep the two dots inside the wood, but the task-specific goal to remove one or two layers of paint.

#### **CULTURAL AFFORDANCES**

In this section, we turn our attention to the broader question of cultural effects in tool use. For the sake of the discussion, we restrict ourselves to the simple assumption that cultural constraints can



have a bearing of the experiences that people might have with specific types of tools and, in particular, can serve to define acceptable or proper ways in which particular artifacts are used. Thus, one question that can be used to address the issue of "culture" in tool use is to ask how should one *properly* use cutlery, such as a spoon, knife or fork?

In their study of eating (kale or water) with a spoon, van der Kamp and Steenbergen (1999) used video-based motion tracking to record arm motion. The likelihood of spilling the contents of the spoon (kale or water) when it was moved from bowl to mouth increased the number of corrective sub-movements made during the action which affected the kinematic profile of the movement. The contents of the spoon also affected head motion. Participants were more likely to move their head towards the spoon when it contained water which, in turn, shows how the coordination of the motion system (i.e., contents–spoon–hand–arm–head) changes in response to task demands. Interestingly, the study also hinted at variation in"eating styles"which reflected individual differences in performance. We are interested in how these "eating styles" might also reflect cultural responses to cutlery and how culture defines the "proper"way to use an item of cutlery. Of course, the use of the word "properly" is deliberately provocative and culturally loaded. At one level, "proper" use could simply mean that food is moved from plate to mouth in a controlled manner, in sufficient quantities to make it easy to eat. At another level, "proper" use could relate to various social mores and rules of etiquette in terms of how the knife and fork are held and moved, and how much food is held on the fork or put into the mouth. For example, in her discussion of using forks, Visser (1991) contrasts the "English" style (of eating from the back of the fork tines and holding the knife in the other hand) with the "American" style (of eating from the bowl of the fork and swapping, or "zig-zagging" fork and knife).

We asked participants, using a knife and a fork (fitted to our instrumented handles), to perform a somewhat unusual version of "eating." The task goal required participants to lift a forkful of sweet-corn to their chin. This breaks down into: "load fork', "lift fork', and "terminate" (e.g., most participants simply tipped the fork to drop the sweet-corn back onto the plate). The "English" or "American" styles outlined above are illustrated by **Figure 5**. In order to consider variation, we selected one participant who was familiar with the "English" style (**Figure 6**) and one of the participants who had never used cutlery in this manner (**Figure 7**).

**Figures 6** and **7** show that variability in the data from the inexperienced user are consistently higher than the experienced user for both grip and accelerometer data. This echoes the earlier findings relating the variability in "skilled" performance. Rather than the "skill', in this case, being the result of instruction, training and practice (as one might expect in the use of hand-tools), these results hint that enculturation and exposure to particular beliefs about appropriate use of cutlery can have an impact on the ease with which these artifacts are manipulated in different ways.

# **DISCUSSION**

We use tools to solve the problems that objects in the environment present to us. This is an obvious statement but hides a couple of points which are worth noting. The first is that intention which underlies the use of the tool combines a task goal with the affordances of the tool–object interface, and the constraints of the person–environment–tool–object system. This means that "cognition" becomes the active response to the affordances of the interaction between tool and object in terms of the task goal that the user is seeking to achieve. Taking Gibson's notion of complimentarity, we can say that the dynamic aspect of this activity continually shapes the actions of the person as much as it shapes the state of the object. In other words, the states of the object, environment, tool, and person become combined to form the focus of action and, by implication, to help frame and reframe the task goal. One might expect the task goal to be kept constant during the performance of the task. However, our discussions with, and observations of, expert jewelers suggests that this not entirely the case. While the high-level objective might remain the same (e.g., produce a ring of a particular size set with a particular stone), the development of the "plan" to achieve this goal adapts to the state of the metal and the performance of the task. Thus, the task goal would appear to follow the notion of "situated action" (Suchman, 1987) which changes with context. This raises the second point, that, the focus of action is context-dependent and the context is continually changing. So, tool use is enactive, embedded and embodied.

The comparisons of experienced and inexperienced users of tools (and cutlery) considered in this paper show that expertise not only involves less variability in physical performance but also better control of energy expended in the performance of a given task with a given tool. We believe that this points to the well-known assertion that the expert develops a "feel" for the tool, and often prefer to use their own tools for particular tasks because these have become very familiar to them. Indeed, a potential problem that we face with the instrumented handles that we use is that these feel different from those that the experienced tool users prefer. Anecdotally, only the experienced tool users commented on the feel (weight, balance, material) of these handles during the data collection.

The skilled craft-worker will often speak of the tool becoming part of the body, and the feeling of manipulating the tool being akin to simply moving the hand in which the tool held. For some writers, this implies that the tool can be considered as a physical

**FIGURE 5 | Comparing English (left) and American (right) cutlery use.**

**FIGURE 6 | Consistent "English" use (over six separate attempts as indicated by thick lines between each attempt).** The pattern of grip force applied (particularly to the fork handle) and the

smoothness of the fork's accelerometer trace show how the experienced participant's repetitions are consistent and reflect a well-practiced motion.

**FIGURE 7 | Variable "English" use (over six separate attempts).** Th*e inexperienced* participant shows large variation in grip force and accelerometer trace for the fork. During the task, his preferred approach was to tilt the fork on its side and move it towards the kernels, using the knife as a stop. He then held the fork at an angle and used the knife to keep the kernals

pressed to the tines as he lifted both knife and fork. There is less correlation shown between grip and activity from the knife, which is being pushed on to the top of the fork, and, particularly towards the right of the graphs, the fork is held with force primarily only on two sides of the handle as opposed to a full grip.

extension of the person and that, therefore, motor control becomes a matter of adapting to the added potential of the"extended-limb." However, rather than simply being a matter of planning movement with the addition of the tool, it is plausible to suggest that the tool changes the perception of space around the tool user (Maravita et al., 2002). *"People who use tools*...*build an increasingly rich implicit understanding of the world in which they use the tools*...*"* [Cutler, 1994, p. 80]. In her discussion of representations in tooluse, Massen (2013) emphasizes the need to appreciate how tools become part of the peripersonal space of the user, such *that "there is no need to distinguish between external goal locations to which the tool has to be moved and the locations to which the bodily effector has to be moved."* (p. 2). While this makes sense when considering movements with tools, it overlooks an equally important aspect of the skilled craft-worker. The reason that the tool feels as if it is part of the person is because it "disappears" from attention which becomes more and more focused on the object being worked on. This suggests that, rather than the tool being an extension of the body, it makes more sense that the tool creates a focus of attention – with the sense that the tool's movement becomes so central to attention that the control of the limb operating it becomes less important. This suggests that, rather than considering the tool-hand combination, it is more important to consider the toolobject combination because this is where the skilled practitioner is attending.

The use of tools, by experts, seems to involve anticipatory, feedforward control of movement (as well as rapid and efficient use of feed-back through all of their senses) in which subtle adjustments in the manipulation of the tool are performed in order to effect desired changes in the object being worked on. Not only does this explain the minimal variability but also highlights the central question of this paper; if so much of the activity of the expert tool user is anticipatory, how are these anticipations represented? We propose that it is not sufficient to only look in the brain of the expert tool user to discover these representations. Even if there are regions which are active under specific conditions, the skill of the expert tool user comes from the ability to control their activity with sufficient spare capacity to cope with future demands and to respond to the changing context in which they are using the tools to effect changes in the object being worked on. The idea that the environment (and the objects it contains) can be interpreted in different ways, suggests that these become "external representations" to which the person responds. Response is partly a matter of knowledge, skill and ability of the person, partly a matter of fit between action and environment and partly a matter of the nature of the environment and the objects it contains. As the person focuses on specific aspects, which are relevant to the task (of shaping a piece of metal or arranging tools in a workspace) so these aspects become the cognitive space in which subsequent decisions are made. Tool use, as a form of problem solving, becomes a matter of making these decisions as the cognitive space changes; and a means of acting upon the cognitive space to create new opportunities. This further suggests that much of the activity which is assumed to be "feed-forward" (in the sense that there needs to be a model which guides behavior) could be explained by fast-acting, negative feedback loops (integrated across several sensory modalities) which support moment-by-moment correction through solving the inverse kinematics problems of positioning a given tool in a given position in order to effect change in the object being worked on.

We believe that much of the "representation" drawn upon in the use of tools can be in the form of external representations (the objects and tools in a given environment, particularly in support of the situated action of ongoing planning in tool use) and in the form of coordinative structures (the control and management of physical activity, particularly in terms of feed-forward control of movement and use of feed-back from the results of the movement). In other words, following the lead of Riccio (1993) and the more detailed arguments of Chemero (2009) an internal representation is not necessary for the control, coordination and (we propose) planning of tool use because it is sufficient for the tool user to have the ability to perceive the state of the object on which she works and to manipulate the tool in order to produce a particular pattern of perceptions (and, in this case, we suggest that these patterns are equally as likely to be olfactory, haptic, and auditory as visual). This ability becomes manifest only during the performance of a person–environment–tool–object system (echoing Butler's claim that "strictly speaking, nothing is a tool except during use") and this system can be described using System Dynamics, in which the systems goal is the optimization of specific movement parameters in order to produce an effect on a given object. This reduces the need for there to be internal representations *per se* (see also Barrett, 2011). Furthermore, any"representation" that the tool user employs is likely to spread across the entire nervous system rather than solely in regions of the brain. From this, the strong and compelling evidence accumulated from the activation of specific regions in the brain is taken to indicate the result rather than the cause of tool using behavior (whether observed, imagined, or performed) which arises from the recruitment and activation of coordinative structures (Bernstein, 1967) through task-specific devices (Beek and Bingham, 1991). While our paper has not sought to present evidence in support of this claim, we believe that this statement helps to bring together the ongoing work that we have reviewed and raises the opportunity to develop testable hypotheses for future exploration of the ways in which people use tools.

#### **ACKNOWLEDGMENTS**

An early draft of this paper was much improved through comments from Anthony Chemero and two anonymous reviewers.

#### **REFERENCES**


to task constraints. *J. Exp. Psychol. Hum. Percept. Perform.* 36, 825–839. doi: 10.1037/a0018171


Hutchins, E. (1995a). *Cognition in the Wild*. Cambridge, MA: MIT Press.


*Cognitive Science Society* (Englewood Cliffs, NJ: Lawrence Erlbaum Associates), 326–330.


Simon, H. (1969). *The Sciences of the Artificial*. Cambridge, MA: MIT Press.


Visser, M. (1991). *The Rituals of Dinner*. London: Penguin.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 28 November 2013; accepted: 27 January 2014; published online: 24 February 2014.*

*Citation: Baber C, Parekh M and Cengiz TG (2014) Tool use as distributed cognition: how tools help, hinder and define manual skill. Front. Psychol. 5:116. doi: 10.3389/fpsyg.2014.00116*

*This article was submitted to Cognition, a section of the journal Frontiers in Psychology. Copyright ©2014 Baber, Parekh and Cengiz. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*