A commentary has been posted on this article:
Affordances, Adaptive Tool Use and Grounded Cognition
- School of Applied Physiology, Georgia Institute of Technology, Atlanta, GA, USA
Choosing how to use tools to accomplish a task is a natural and seemingly trivial aspect of our lives, yet engages complex neural mechanisms. Recently, work in healthy populations has led to the idea that tool knowledge is grounded to allow for appropriate recall based on some level of personal history. This grounding has presumed neural loci for tool use, centered on parieto-temporo-frontal areas to fuse perception and action representations into one dynamic system. A challenge for this idea is related to one of its great benefits. For such a system to exist, it must be very plastic, to allow for the introduction of novel tools or concepts of tool use and modification of existing ones. Thus, learning new tool usage (familiar tools in new situations and new tools in familiar situations) must involve mapping into this grounded network while maintaining existing rules for tool usage. This plasticity may present a challenging breadth of encoding that needs to be optimally stored and accessed. The aim of this work is to explore the challenges of plasticity related to changing or incorporating representations of tool action within the theory of grounded cognition and propose a modular model of tool–object goal related accomplishment. While considering the neuroscience evidence for this approach, we will focus on the requisite plasticity for this system. Further, we will highlight challenges for flexibility and organization of already grounded tool actions and provide thoughts on future research to better evaluate mechanisms of encoding in the theory of grounded cognition.
Introduction
The theory of grounded cognition has offered a fascinating window into mechanisms of storage and recall of concepts. Generally, the act of simulation of past events is an important aspect of current theories of grounded cognition (Decety and Grezes, 2006). For a given object-related interaction, information from the modal senses for perception (e.g., vision, somatosensation), action (e.g., kinethesis, proprioception), and introspection (e.g., affect), are blended into a representation of that experience (Barsalou, 2008). As knowledge is needed to later represent that object, this multimodal information (perception, action, introspection) is recalled to simulate the brain states associated with that object (Barsalou, 2008). Through this process, both cognitive and motoric functions can take advantage of the ideas offered by this theory. Here, we will focus on a specific aspect of motoric function: implementing tools and objects in various actions. For the purposes of this work, we will use the word “tool” to define non-animate artifacts that operate on something by a user (e.g., spoon). “Object” will be that which is operated on by a tool (e.g., mug). “Action” will generally involve mutual use of the tool and object to accomplish a goal (e.g., stir coffee). We will go further to differentiate knowledge of tool manipulation from knowledge of tool function, where manipulation is “how” to use a tool and function is “why” to use a tool (Buxbaum et al., 2000). Some of the principles presented can be applicable for many types of movements, such as communicative gesture, but we are limiting the scope of this to tool use alone.
Grounded Cognition Relevant to Tool Use
An idea that is being pursued is that acts relevant to tool use are grounded, where tool knowledge is grounded based on past acts. Thus, hammer does not come with an operant functional definition, but more precisely is based on how we have used a hammer in our lives. This will “ground” a representation of the tool in a state that encodes usage. Of use is also to note that this grounding may differ from person to person, although possibly in subtle ways. For example, the two co-authors of this work have slightly distinct histories of a hammer in multiple obvious and atypical contexts. Further, actions can become encoded deeper in context, based on situational rules and tendencies. For example, occasionally a spoon is more likely/useful than a plastic stirrer for coffee (more will follow of this example). The act of stirring coffee is grounded with multiple potential items based on typical contexts.
Basic aspects of tools and objects have a clear foundation, making the idea of grounding tool use attractive. Related to grounded views of tool usage, vital work in this field was performed that demonstrated how objects and actions are coupled based on perception of how the object is held (Tucker and Ellis, 1998). Such an idea has been advanced by suggesting that seen objects can activate motor representations as a result of functional knowledge of the seen object (Anderson et al., 2002). Many other studies have established this important idea (Grezes et al., 2003; Handy et al., 2003; Tucker and Ellis, 2004; Rice et al., 2007). A strong basis for the argument of grounded cognition in tool and object-related action is seen in a review by Grafton (2009) which clearly illustrates that action can be recognized using a system where subjects can effectively match seen actions with their own representations for action understanding (Grafton, 2009). Further, theories have been well proposed to suggest how vision and action knowledge are integrated to allow for the fusing of perception and action (Buxbaum and Kalenine, 2010). Similar ideas are given in the context of action. It is plausible that a key basis for our ability to act based on prior experiences through simulation (Barsalou, 1999). Here, we can recall our prior experiences and apply them as we are carrying out an act (Barsalou, 2008). A similar mechanism can be used when we have to represent a seen action, by recalling the states of the observer in tool use (Kellenbach et al., 2003; Jarvelainen et al., 2004), object grasping (Pierno et al., 2009; Valyear and Culham, 2010), and action (Frey and Gerry, 2006; Evangeliou et al., 2009) studies.
A potential Limitation? – “Plasticity” of Grounded Cognition
While grounded cognition is attractive for the ideas above, a common thread is that sensorimotor experience drives this process for tools/object action knowledge. However, two aspects of this theory should be addressed related to this idea of how we actually ground tool/object/action knowledge.
First, what if we have limited or no prior experience with a tool? How well is it grounded? The recall of a familiar tool (e.g., hammer) for action or recognition is clearly left parietofrontal involved (Lewis, 2006). However, there is suggestion that tool representations differ based on our relative history with a tool (Vingerhoets et al., 2009). Our questions regarding grounded cognition come in consideration of the process and neural architecture used to encode a tool/object representation, then how this grounded representation may change over time, particularly to contract into a limited-use tool representation.
To our second question, how can a grounded tool/object/action be modified? As another example, how might we come to the idea that a hammer can be used to re-seal a paint can (without any sensorimotor experience of this), when we likely first learned it as an implement to drive in a nail? Similar examples are provided in other work, where grounded views may result in a need to break out of “functional fixedness” on a regular basis (Wilson, 2002). Thus, we focus on the idea of the “plasticity of grounded cognition.” That is, how do these parietofrontal tool–object representations develop, grow, and change over time based on our experiences and creativity? Further, what is the neuroscience of this process? As will be seen, these two questions are not fully segregated from each other.
Creation/Maturation of a Grounded Representation
In consideration of the first question (progression of novel tool/object/action representations), the study of Vingerhoets (2008) offers an intriguing point. Here, unlike familiar tools, recognition of unfamiliar and rarely used tools heavily engages left temporo-occipital areas. As unfamiliar tools natively have no clear action (specifically sensorimotor) representation, this will clearly prevent simulation-based action knowledge, and should fail to generate clear action–observation knowledge related to such tools (as in Grafton, 2009). However, rarely used tools likely would have some limited action (sensorimotor) knowledge, and may engage action–observation related networks. Yet, the recognition of both classes of tools falls outside of the suggested tool-related network for knowledge (that codes recognition and action). Similarly, others have recently shown that prefrontal and mediotemporal areas of the left hemisphere are important for understanding the use of a new tool (Menz et al., 2010).
Although the exact length of time required for a tool or object to become grounded is unknown, we can look to the motor control and learning literature for insight. It is well known that motor learning in humans occurs in three distinct phases: Initial stage, where trial-and-error is required to establish new sensory information with correct motor commands (e.g., derivation of a novel sensorimotor association); Intermediate stage, where the newly acquired sensorimotor association is learned through practice (e.g., consolidation of the sensorimotor map); Advanced stage, where the working memory access of the sensorimotor association is no longer required and movements can be performed with less reliance on sensory feedback processing and attention (Halsband and Lange, 2006). These stages of learning, although variable, begin immediately and may continue to develop for days to weeks (Karni et al., 1998), at which point the skill is retained (Karni and Sagi, 1993). Some interesting work suggests that, in humans, development of tool-specific functional knowledge and skilled usage (for previously non-tool objects) occurs within the first training session and continues over successive sessions (Weisberg et al., 2007). However, at which stage an object becomes grounded as a tool remains an open question.
Clearly, sensorimotor knowledge is gained (albeit in a limited degree) in rarely used tools and one might presume that it became grounded. One could argue that the process of grounding the tool was “incomplete” for the rarely used tools, as action encoding and grounding seems to reflect left parietofrontal activation (Buxbaum and Kalenine, 2010). Though, if unfamiliar or very rarely used tools are not grounded and activate temporo-occipital regions, perhaps this is a site of initial storage of new/atypical tools before grounded in a sensorimotor experience. Here, a storage area is defined that represents an object that remains low level (perhaps visual) and shares some tool features, while a clear tool/action representation (left parietofrontal) has yet to be defined. This effect was seen in the study of Weisberg et al. (2007) on training the use of novel tools in actions. This study nicely demonstrated that before training, object–action matching for novel tools generated activity limited to occipital cortex followed by left temporo-parieto-frontal areas after training.
The spirit of this theory is seen in studies of recognition of visually presented objects. A study demonstrated that the occipitotemporal areas (lateral occipital–posterior fusiform) process recognized pictures with more activity than non-recognized images (Grill-Spector et al., 2000). Further, a more concise review demonstrates that such areas do not only respond to presence of stimuli, but to our perception of the stimuli (Grill-Spector and Malach, 2004). This garners some speculation that our perception (perhaps including familiarity, environment, etc…) may affect the meaningful processing of a tool or object (Lin et al., 2008). Similarly, activity within primary visual cortex can be modulated by crossmodal sensory processing during visual–auditory illusion (Watkins et al., 2006) and in the visual–tactile domain (Ramos-Estebanez et al., 2007), suggesting a complex link between low-level visual structures and perception. Closer to the current topic, studies have shown that perceptual awareness of two objects is increased when the objects form a familiar tool–object action pair and are positioned, within a visual array, in a plausible way for action (Riddoch et al., 2003; Humphreys et al., 2006). More recently, Roberts and Humphreys (2010) observed specific activations of lateral occipital areas (e.g., early visual processing areas) for familiar objects positioned correctly for action as compared to those positioned incorrectly for action (Roberts and Humphreys, 2010). This object-orientation/object-position effect seems to be independent of visual attention (Riggio et al., 2008), and suggests that low-level visual processes directly contribute to our understanding of action-related object features.
Higher-level visual representations of tools and objects have been the subject of study for many years, and the neural mechanisms of tool identification and basic tool use are well known. Previous work has identified class-specific mechanisms of object recognition in the brain (Martin and Chao, 2001), and it is known that viewing tools activates inferior regions of the left intraparietal sulcus and ventral premotor cortex (Chao et al., 2002). Extensive tool-specific activation is also commonly seen in temporal regions (Beauchamp and Martin, 2007). Although commonly active in response to tool stimuli, there does appear to be some specialization in the type of information processed at these regions based on familiarity. Posterior parietal and premotor activation in response to tools may be specialized to convey information related to the motor affordance of a tool rather than its identity (Jeannerod et al., 1995; Johnson-Frey, 2004), while posterior and inferior temporal activation seems to be of particular importance in tool identification rather than understanding motoric qualities (Martin, 2007). Further, the anterior frontal regions, especially the ventrolateral aspect of the prefrontal cortex (Ranganath et al., 2004), are thought to form a network with regions of the temporal cortex (Mayes et al., 2007) to mediate the semantic and associative memory used in object representations (Martin, 2007). This concept is supported by lesion studies, which have shown that disrupted communication between the prefrontal and temporal regions impairs the recall of visual associative information (Tomita et al., 1999). As such, specialized mechanisms seem to link the identification of manipulable objects with information about the actions and context associated with their use, an idea that is well supported by previous literature. Grounding of action associated with a tool should engage extensive areas that affect perception. This infers that a strong test of tool grounding is more action based, and should be reflected in low and high level visual areas.
Modifying What’s Grounded
In regards to the second question (modifying already grounded tools/objects/action), this speaks to a consideration that the parieto-temporo-frontal system related to tool action is very plastic. In this construct, new functions are attributed to existing tools or a task action is learned using a new set of tools. Such learning is typical of our daily lives, as rarely do we use a tool in only the capacity in which we have learned it. Routinely, tools are used to accomplish a task by the nature of basic aspects that the tool can afford. Here, if one needs to stir paint in absence of a paint stirrer, objects can be used that bear some similarity to the necessary tool. The issue of tools being grounded based on prior action history does not preclude one from using them apart from their intended purpose. Though, certain tools may be favored. The handle of a hammer could be used to stir paint, though this might make latter use of the hammer more difficult. An alternative may be a disposable pencil to accomplish the same task. The important fact is that neither the pencil nor the hammer was grounded for stirring a can of paint.
This issue bears similarities to ideas proposed in the study of retaining and recalling motor memories, based on our ability to store, recall, and modifying motor plans for varying environmental contexts. One theory, MOSAIC (modular selection and identification for control), takes into account our ability to operate in the world in multivariate environments (Wolpert and Kawato, 1998). Interestingly, this work introduces the idea of object use to promote how behavior depends on many internal and external variables. Hence, we are not scripted to perform movements only in the state that we learned the movement (e.g., only moving a full can from a neutral arm posture). In the MOSAIC model, modules of “controllers” for motor behavior account for some number of potential contexts. When the motor context is realized, the appropriate control is implemented. Thus, once context is realized, we can modify our movements based on existing modular states. Similarly, the vast array of tools and objects (and the possible interactions between them) creates a tremendous number of potential contexts that would have to be learned. Here, when a tool is needed but unavailable, other known tools may be imported to replace the needed one to afford action.
Herein, we propose a Modular Selection for Action Goals (MSAG) model. While the MOSAIC model is focused at the controller level and primarily describes the adaptive nature of sensorimotor control based on multiple pairs of forward and inverse models, we approach motor control from a more global level and propose a modular structure for the realization of action goals. In our model, an action goal is fixed (grounded), while the selection of an appropriate tool, context of use (of the tool), and neurobiomechanical parameters are contextually dependent on the action goal. A modular organization of tool/object action affords similar flexibility for rules in tool/object usage that allow deviation from established norms into new contexts, just as the MOSAIC model affords similar optimization of “situational” motor control. In this construct, once a tool is learned for a given task, that tool can be moved with relative ease in and out of possible contexts and become potentially “wired” to related/alternative tasks. In essence, a tool can be grounded to a particular sensorimotor and conceptual experience, while being embedded into a larger module accessible for many potential uses, which may or may not share similarities with past actions. Thus, we learn to use a knife to cut something in particular, then to cut in general, and, last, we extend the representation of a knife to a myriad of other potential uses (e.g., stir). This theory is represented in Figure 1. Three distinct modules are represented: a Tools Module, a Usage Contexts Module, and a Potential Neurobiomechanics Module. Each module provides unique information to a controller, which then derives a motor plan for the execution of behavior. Here, an array of canonical and available tools is identified for a given action, along with the expected neurobiomechanics. In this array, the spoon is available and selected, paired with the motoric demands of the action for successful accomplishment. As will be discussed later (Section “Relevance in Apraxia”) the modular organization of MSAG helps to tentatively explain unique aspects of tool action impairment.
FIGURE 1
Figure 1. Proposed MSAG model of tool, neurobiomechanical, and action representations for driving appropriate tool–object behaviors. In this example, canonical tools are available and spoon is selected for achieving the action goal of stirring coffee.
Existing theories of the structure of the encoding of tools/objects may afford this. The proposed “superordinate” representations of tool knowledge relate to how we categorize tools/objects etc. around themes. Superordinate classes of tools may reflect an organization that allows us to group things based on similarities, which may be functional (Murphy and Wisniewski, 1989), and has been used to address developmental theories (see next section). Behavioral evidence indicates that there is a potentiating of judgment when a current image bears similarities with a previous one (Myung et al., 2006). Similarly, behavioral evidence would also suggest that action can be afforded when presented with a superordinate category by way of activating a myriad of potential actions related to that category (Heit and Barsalou, 1996; Borghi and Caramelli, 2003; Marques, 2006). This last evidence may be important to the MSAG theory of defining novel tool uses (of familiar tools) for an action.
So, consider if a tool is desired but not available (e.g., Figure 2; no spoons or stirrers). Seeking the desired tool may, in effect, “prime” a supraordinate class of tools that are the best-fit plausible alternatives, if any are available. Here, eye glasses may not be primed, but the knife would be (Figure 2) as it would be a part of a “utensil” supraordinate class. Once selected, the knife may become grounded for that new action upon use, while not impairing tool knowledge of the former action. Here, selection of knife is driven by functional similarity (e.g., used when eating) to the canonical (but unavailable) tool. Similarly, key to the MOSAIC model is that learning a new movement repertoire does not impede modules for other actions (Wolpert and Kawato, 1998). Further, interference from other potential usage contexts is avoided (slice, spread, chop), as they automatically do not meet the action goals (stir coffee). We highlight that at this stage, we are back to the first question, as actually grounding the “best-fit” alternative tool to a new action may require more than just one exposure. Under MSAG for tool selection and action, the “plasticity” of tools for many potential tasks is optimized.
FIGURE 2
Figure 2. Proposed MSAG model of tool, neurobiomechanical, and action representations for driving appropriate tool–object behaviors. In this example, canonical tools are not available and the novel usage context of stir is applied to a best-fit-tool (knife) to achieve the action goal of stirring coffee.
It is important to emphasize that we are not fully regarding the MSAG model to tool–hand action alone (i.e., kinematic strategies for hammers of different weights). Much previous work has identified a potential role for the MOSAIC model in this capacity (Wolpert and Kawato, 1998; Haruno et al., 2001; Imamizu et al., 2003, 2007a,b; Imamizu and Kawato, 2009; Shah and Barto, 2009). This MSAG proposal extends our theory into conceptual tool use in variable situational states where a known tool takes on a “new” function. We will further advance theories related to MSAG into the neurological disorder of apraxia (see Section “Relevance in Apraxia”), offering a potential mechanistic description of deficits common to motor and conceptual apraxias.
Ideas from Developmental Studies
Adolescent behavioral neuroscience literature offers insight into the development of tool/object/action representations. Behavioral studies have demonstrated the development of functional associations between items in the formation of concepts related to manipulation knowledge (Kalenine and Bonthoux, 2008). Further, action priming is shown to enable object recognition (Mounoud et al., 2007). Particularly at a young age, knowledge of tool-based relationships is critical above more abstract, categorical relationship knowledge (Perraudin and Mounoud, 2009). Age plays a dramatic role in establishing how we categorize objects, where at very young ages (7 years old) object categorization is best at a basic level than at a superordinate level, which is no longer noticeable by age 9 (Kalenine et al., 2009a). Even infants display basic categorizations of visually presented objects at anterior–posterior brain areas (Grossmann et al., 2009). In keeping with this evidence, it is suggested that learning categories of tools and objects based on a mutual sense of action between the two is possible (Gershkoff-Stowe and Rakison, 2005; Smith, 2005) and may reflect similar, ongoing process in adulthood. Observational-based tool learning is robust in children (Hopper et al., 2010), though with age this capability likely introduces more motor error than necessary (McGuigan et al., 2010) and is suggestive of a need for sensorimotor experience for learning. Nevertheless, these developmental data represent a basis that the acquisition of tool concepts is heavily related to binding with objects or action. Thus, a spoon is a type of item that matches other spoons (belongs to a class of spoons based on structural characteristics), but is also associated with bowls (based on functional characteristics). Interestingly, knowledge that “spoon” is a member of the larger class of “utensil” requires more time. Evidence in adults suggests that such categorization knowledge of tool with a functional unit is represented, as expected, in pareito-temporal areas that have a high correspondence to action knowledge areas (Kalenine et al., 2009b). Thus, to learn new tool/object/action representations, or particularly in modifying existing ones, we heavily utilize pre-established, grounded concepts. The advantage of this concept would be in using existing templates that can be modified over time (see Section “Modifying What’s Grounded”). To our knowledge this is still speculative, although further work considering developmental models may provide greater insight into the mechanisms of formation, storage, and modification of tool/object representations.
Relevance in Apraxia
Of interest to the theory of grounded cognition (relevant to tool use) is apraxia. Apraxia is a deficit commonly arising after stroke that will impair tool-related behavior. This can include performance of tool movements, selection and ordering of tools for a task, and the general understanding of tools. Apraxia also extends to communicative gesture impairments, though we will not consider this feature in the current discussion. Here, we will focus on two forms of apraxia to highlight the implications of grounded cognition in explaining apraxia-related deficits, and we will offer an alternative theory based on the MSAG model.
We will consider two types of apraxia in this new model: ideomotor apraxia and conceptual apraxia. The first form, ideomotor apraxia, is commonly characterized by deficits in pantomime of tool use (i.e., demonstrating tool use without the actual tool). In this case, a patient is deficient at pantomiming tool usage out of context (e.g., “Show me how to use a hammer.”) (Wheaton and Hallett, 2007). The second form, conceptual apraxia, is manifest as the inability to select tools adequate for a task. Thus, selecting tools in task-driven ways is impaired though it is possible to retain functional knowledge of the tool even when removed from natural settings (Heilman et al., 1997). Here, appropriate knowledge of tools and objects for a given task may become disrupted.
If we consider the nature of performing a praxis motor task, say driving a nail, there are many potential solutions to the goal. These solutions vary based on a number of factors, including the exact tool used (e.g., hammer vs. other possible implements), neurobiomechanical constraints to human motion (e.g., kinetics and kinematics, neuromuscular synergies, etc.), and contextual parameters (e.g., roofing vs. a more delicate task). The neurologically intact person is capable of deriving an appropriate set of motor commands from the combination of all of these factors. We first identify that a hammer is a “better” tool for driving a nail as compared to a shoe, for example. Then, we integrate the properties of the hammer with our sensorimotor system to develop and execute a motor plan, thus allowing us to drive the nail. Although seemingly automatic for many of us, patients with ideomotor and conceptual apraxia seem to show failures at different levels of this type of task.
In the case of ideomotor apraxia, there seems to be a disruption in translating the correctly identified tool (Figure 1, left side, many tools to one tool solution remains intact) into a single sensorimotor control parameters set (resulting in a one tool – appropriate sensorimotor parameters failure). In this case, the MSAG controller may be given the correct tool-related input, yet these parameters fail to converge at an appropriate, single set of sensorimotor tool-task control parameters. For example, a patient may be able to identify a fork as the most appropriate tool to eat a meal, but is unable to derive the neurobiomechanical control strategy to utilize the fork in the correct manner. In this framework, ideomotor apraxia is related to a disruption of the input or output of the Neurobiomechanical Module on the right side of Figure 1, while the Available Tools Module (left side of Figure 1) functions normally.
Conceptual apraxia, however, seems to be a disruption of the opposite phenomenon. There is a failure in deriving the many tools to one tool solution, yet the one tool – appropriate sensorimotor parameters solution is intact. Interestingly, here the MSAG controller may be given faulty tool-related information, yet appropriate tool-task sensorimotor control parameters are often realized which results in successful motoric (although contextually incorrect) use of a tool to achieve the desired behavior. For example, a patient may be able to successfully eat a meal using a toothbrush instead of a fork. In our proposed framework, conceptual apraxia is related to a disruption of the Available Tools Module (left side of Figure 1), while the Neurobiomechanical Module (right side of Figure 1) functions normally based on the action goals.
According to MSAG, the detriments in apraxia are based on separately damaged modules. Hence, a modular organization of MSAG is plausible. Thus, tool selection module fails in conceptual apraxia, while the tool action module fails in ideomotor apraxia. At the same time, the tool action module survives in conceptual apraxia while the tool selection module survives in ideomotor apraxia. Lesion profiles for the two types of apraxia potentially support this notion of multiple modules, as primary damage to left parietofrontal regions is seen in ideomotor apraxia, while conceptual apraxia tends to result from bilateral damage to temporoparietal areas (Heilman and Gonzalez Rothi, 2003).
Aging
Aside from apraxia, evidence suggests specific processes related to tool knowledge may be affected by aging. It is important to note that these studies identify praxis impairments in a non-neuropathological aging population. Early work into the effect of healthy aging on praxis function revealed age-specific reductions in performance of pantomimes executed to verbal command. In healthy subjects, Ska and Nespoulous (1987) evaluated self-oriented (e.g., brushing the teeth) and external (e.g., tearing a piece of paper) pantomimes. One-handed repetitive and non-repetitive pantomimes were most affected, where older subjects often committed Body Part as Object (BPO) errors (e.g., instead of shaping the hand as if to use the absent object, the hand becomes the acting object).
More recently, Rodrigues Cavalcante and Caramelli (2009) evaluated gesture production to verbal command and imitation in healthy older subjects. As before, older subjects showed reduced performance of pantomime to verbal command, primarily due to commission of BPO errors. Further, Mozaz et al. (2009) evaluated arm–hand postural knowledge of tool use and communicative gestures in healthy older adults. Although knowledge related to both types of postures were reduced at a similar rate with increasing age, tool-related knowledge showed greater overall declines. This is in line with previous reports also suggesting dissociations between tool use and communicative gestures in apraxia (Bartolo et al., 2001; Villarreal et al., 2008; Bohlhalter et al., 2009). Mozaz et al. (2009) went further to suggest that loss of tool-use postural knowledge was unrelated to deficits in basic visual function, associative agnosia or semantic processing, but may be accounted for by the domain-specific representational hypothesis (Buxbaum and Kalenine, 2010). Evidence does not suggest that older subjects have “apraxia” per se, but that domain-specific processes are impaired in these subjects similar to stroke patients with apraxia, where tool-use knowledge is largely affected. Hence, aging may play a role in the grounded tool knowledge state. The mechanisms of what is changing are of importance and have a high relevance to understanding neural disease states common in advancing age. As well, it is clear that normal aging alters many neuroanatomical, cognitive and physiological brain processes (Lu et al., 2002; Davis et al., 2009; Hutton et al., 2009; Park and Reuter-Lorenz, 2009; Zahr et al., 2009) which may affect the neural architecture and function of the grounded cognition model as well. Adjustments under advancing age may represent another aspect of essential plasticity of tool knowledge states.
Are We for, or Against, Grounded Cognition?
This section is of importance to correctly represent the above discussion. We do not feel that the grounded theory is invalid, but that there are challenging issues that must be addressed when considering whether and how something is “grounded.” The main issue we hope to make clear is in the essentials of plasticity of the system. That is, how flexible are the functional representations of grounded tool/object/action to multiple variables (e.g., environment)? To make the point clear, we rely on our previous example of Figure 2, stirring coffee. If you need to prepare coffee but no stirrers or spoons are readily available, what do you do? In strict form, you may only have two alternatives (spoon or stirrer) with which to complete the action based on your sensorimotor history. Their absence would end in a “fault” and thus you would not be able to prepare your coffee. However, we know that other stirrer-like or spoon-like objects may be available which can be used in the place of a spoon or stirrer, and so we are able to advance and finish preparing our coffee. These alternatives may exist based on functional or structural similarity.
This adaptability presents a computational challenge to our neural architecture. How can we have nearly limitless adaptability of committing an act regardless of setting and the most basic tools at hand? While behaviorally robust, the neural representation of this is worth understanding. We propose a modular organization (MSAG) of tool/object ordinates as a method to accomplish the goal of importing and applying tools/objects in variable action states which afford for the plasticity of tool/object based decisions for action, and also utilize MOSAIC-like principles for tool-related motor behavior. Although seemingly oxymoronic at first, we propose this “plasticity” of grounded elements is an essential characteristic. Our neural systems for tool use must have the powerful ability to adapt, incorporating new acts and goals of various tools and objects into our motor repertoire. Such a system would strengthen our ability to map familiar tools to new actions, and afford importing new tools into the same neural architecture.
This proposal stems from the theory of ad hoc categorization (Barsalou, 1983) and is supported by recent literature in grounded and embodied cognition. It has been proposed that a “strong” and “weak” version of embodiment exists, with the latter described as a form of “graded” grounding (Chatterjee, 2010). Chatterjee (2010) goes further to suggest that “Referring to graded grounding invites consideration of continua and trade-offs between what is lost and what is gained. Representations by virtue of being less grounded in sensory and motor details lose some of their referential power. But, by virtue of being less grounded they also gain generative and flexible power.” Other evidence specifically suggests that a distributed brain network abstracts action representations away from actors (Kable and Chatterjee, 2006), thereby diminishing sensory and motor details of specific actors and objects and allowing for relational plasticity. Our model is in agreement with this “graded grounding” conceptual approach, where tool–object relational processing may be removed from strict sensorimotor grounding and instead be governed by the context-dependent abstracted (e.g., ad hoc) category “Action Goal.” Here, the original tool and object percepts remain intact, yet novel, context-driven relational interactions can form. In keeping with our knife-stir example, a knife is capable of adopting the function of a stirring implement with respect to a cup of coffee, yet the knife remains a knife (e.g., it is not perceived as a spoon).
Points for Further Refinement
Here we suggest a few thoughts to consider in support of the advancement of the theories that pertain to the storage of action information relevant to tools and objects:
1. Consideration of how new tools/objects are incorporated into our motor repertoire. Understanding of this “entry mechanism” will enable us to uniquely observe grounding taking place. Further, the time-course of this process can help to directly elaborate on the creation/maturation of grounded acts.
2. When might the action–observation network suggest that grounding has occurred? Based on the review of Grafton (2009) and considering the compelling evidence of action–observation network in matching a template of seen to understood action, this would suggest that this network would be a strong marker to detect grounding.
3. Address how tool/object relationships are weighted (i.e., is one tool/object more grounded than another?). Further understanding as to whether there is differential weighting of tools/objects to action may help to understand the development and structure of tools/objects to actions.
4. Computational models. Such models have been well used to drive improved theories of motor control, such as the MOSAIC model (Haruno et al., 2001) and our proposed MSAG model. These models have also been well argued for advancing theories of motoric aspects tool use (Krakauer and Shadmehr, 2007).
Conclusions and Summary
While an appealing notion, the mechanism of grounded cognition in tool use still remains an unclear. Of note, we demonstrate two main ideas that remain to be explored. Work on the mechanisms of grounding from a new tool/object and modifying grounded tools/objects to new actions perhaps sheds light on the mechanisms of this system. Consideration of computational and human research may suggest an alternative strategy, where the MSAG processes allow for behavioral optimization and (chiefly) operational flexibility without being computationally daunting. Fortunately, there are promising ideas to pursue that will help in better formulating ideas of how the system functions and offer insights to understand the neuroscience of this process from pediatric and adolescent development, adulthood, aging, and neural pathology.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Anderson, S. J., Yamagishi, N., and Karavia V. (2002). Attentional processes link perception and action. Proc. Biol. Sci. 269, 1225–1232.
Barsalou, L. W. (1999). Perceptual symbol systems. Behav. Brain Sci. 22, 577–609; discussion 610–560.
Bartolo, A., Cubelli, R., Della Sala, S., Drei, S., and Marchetti, C. (2001). Double dissociation between meaningful and meaningless gesture reproduction in apraxia. Cortex 37, 696–699.
Beauchamp, M. S., and Martin, A. (2007). Grounding object concepts in perception and action: evidence from fMRI studies of tools. Cortex 43, 461–468.
Bohlhalter, S., Hattori, N., Wheaton, L., Fridman, E., Shamim, E. A., Garraux, G., and Hallett, M. (2009). Gesture subtype-dependent left lateralization of praxis planning: an event-related fMRI study. Cereb. Cortex 19, 1256–1262.
Borghi, A. M., and Caramelli, N. (2003). Situation bounded conceptual organization in children: from action to spatial relations. Cogn. Dev. 18, 49–60.
Buxbaum, L. J., and Kalenine, S. (2010). Action knowledge, visuomotor activation, and embodiment in the two action systems. Ann. N. Y. Acad. Sci. 1191, 201–218.
Buxbaum, L. J., Veramonti, T., and Schwartz, M. F. (2000). Function and manipulation tool knowledge in apraxia: knowing ‘what for’ but not ‘how’. Neurocase 6, 83–97.
Chao, L. L., Weisberg, J., and Martin, A. (2002). Experience-dependent modulation of category-related cortical activity. Cereb. Cortex 12, 545–551.
Davis, S. W., Dennis, N. A., Buchler, N. G., White, L. E., Madden, D. J., and Cabeza R. (2009). Assessing the effects of age on long white matter tracts using diffusion tensor tractography. Neuroimage 46, 530–541.
Decety, J., and Grezes, J. (2006). The power of simulation: imagining one’s own and other’s behavior. Brain Res. 1079, 4–14.
Evangeliou, M. N., Raos, V., Galletti, C., and Savaki, H. E. (2009). Functional imaging of the parietal cortex during action execution and observation. Cereb. Cortex 19, 624–639.
Frey, S. H., and Gerry, V. E. (2006). Modulation of neural activity during observational learning of actions and their sequential orders. J. Neurosci. 26, 13194–13201.
Gershkoff-Stowe, L., and Rakison, D. H. (eds) (2005). Building Object Categories in Developmental Time. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Grafton, S. T. (2009). Embodied cognition and the simulation of action to understand others. Ann. N. Y. Acad. Sci. 1156, 97–117.
Grezes, J., Tucker, M., Armony, J., Ellis, R., and Passingham, R. E. (2003). Objects automatically potentiate action: an fMRI study of implicit processing. Eur. J. Neurosci. 17, 2735–2740.
Grill-Spector, K., Kushnir, T., Hendler, T., and Malach, R. (2000). The dynamics of object-selective activation correlate with recognition performance in humans. Nat. Neurosci. 3, 837–843.
Grill-Spector, K., and Malach, R. (2004). The human visual cortex. Annu. Rev. Neurosci. 27, 649–677.
Grossmann, T., Gliga, T., Johnson, M. H., and Mareschal, D. (2009). The neural basis of perceptual category learning in human infants. J. Cogn. Neurosci. 21, 2276–2286.
Halsband, U., and Lange, R. K. (2006). Motor learning in man: a review of functional and clinical studies. J. Physiol. Paris 99, 414–424.
Handy, T. C., Grafton, S. T., Shroff, N. M., Ketay, S., and Gazzaniga, M. S. (2003). Graspable objects grab attention when the potential for action is recognized. Nat. Neurosci. 6, 421–427.
Haruno, M., Wolpert, D. M., and Kawato, M. (2001). Mosaic model for sensorimotor learning and control. Neural Comput. 13, 2201–2220.
Heilman, K. M., and Gonzalez Rothi, L. J. (2003). “Apraxia,” in Clinical Neurophysiology, eds K. M. Heilman and E. Valenstein (New York: Oxford University Press), 215–235.
Heilman, K. M., Maher, L. M., Greenwald, M. L., and Rothi, L. J. (1997). Conceptual apraxia from lateralized lesions. Neurology 49, 457–464.
Heit, E., and Barsalou, L. W. (1996). The instantiation principle in natural categories. Memory 4, 413–451.
Hopper, L. M., Flynn, E. G., Wood, L. A. N., and Whiten, A. (2010). Observational learning of tool use in children: Investigating cultural spread through diffusion chains and learning mechanisms through ghost displays. J. Exp. Child. Psychol. 106, 82–97.
Humphreys, G. W., Riddoch, M. J., and Fortt, H. (2006). Action relations, semantic relations, and familiarity of spatial position in Balint’s syndrome: crossover effects on perceptual report and on localization. Cogn. Affect. Behav. Neurosci. 6, 236–245.
Hutton, C., Draganski, B., Ashburner, J., and Weiskopf, N. (2009). A comparison between voxel-based cortical thickness and voxel-based morphometry in normal aging. Neuroimage 48, 371–380.
Imamizu, H., Higuchi, S., Toda, A., and Kawato, M. (2007a). Reorganization of brain activity for multiple internal models after short but intensive training. Cortex 43, 338–349.
Imamizu, H., Sugimoto, N., Osu, R., Tsutsui, K., Sugiyama, K., Wada, Y., and Kawato, M. (2007b). Explicit contextual information selectively contributes to predictive switching of internal models. Exp. Brain Res. 181, 395–408.
Imamizu, H., and Kawato, M. (2009). Brain mechanisms for predictive control by switching internal models: implications for higher-order cognitive functions. Psychol. Res. 73, 527–544.
Imamizu, H., Kuroda, T., Miyauchi, S., Yoshioka, T., and Kawato, M. (2003). Modular organization of internal models of tools in the human cerebellum. Proc. Natl. Acad. Sci. U.S.A. 100, 5461–5466.
Jarvelainen, J., Schurmann, M., and Hari, R. (2004). Activation of the human primary motor cortex during observation of tool use. Neuroimage 23, 187–192.
Jeannerod, M., Arbib, M. A., Rizzolatti, G., and Sakata, H. (1995). Grasping objects: the cortical mechanisms of visuomotor transformation. Trends Neurosci. 18, 314–320.
Johnson-Frey, S. H. (2004). The neural bases of complex tool use in humans. Trends Cogn. Sci. 8, 71–78.
Kable, J. W., and Chatterjee, A. (2006). Specificity of action representations in the lateral occipitotemporal cortex. J. Cogn. Neurosci. 18, 1498–1517.
Kalenine, S., and Bonthoux, F. (2008). Object manipulability affects children’s and adults’ conceptual processing. Psychon. Bull. Rev. 15, 667–672.
Kalenine, S., Bonthoux, F., and Borghi, A. M. (2009a). How action and context priming influence categorization: a developmental study. Br. J. Dev. Psychol. 27, 717–730.
Kalenine, S., Peyrin, C., Pichat, C., Segebarth, C., Bonthoux, F., and Baciu, M. (2009b). The sensory-motor specificity of taxonomic and thematic conceptual relations: a behavioral and fMRI study. Neuroimage 44, 1152–1162.
Karni, A., Meyer, G., Rey-Hipolito, C., Jezzard, P., Adams, M. M., Turner, R., and Ungerleider, L. G. (1998). The acquisition of skilled motor performance: fast and slow experience-driven changes in primary motor cortex. Proc. Natl. Acad. Sci. U.S.A. 95, 861–868.
Kellenbach, M. L., Brett, M., and Patterson, K. (2003). Actions speak louder than functions: the importance of manipulability and action in tool representation. J. Cogn. Neurosci. 15, 30–46.
Krakauer, J. W., and Shadmehr, R. (2007). Towards a computational neuropsychology of action. Prog. Brain Res. 165, 383–394.
Lin, Z., Lin, Y., and Han, S. (2008). Self-construal priming modulates visual activity underlying global/local perception. Biol. Psychol. 77, 93–97.
Lu, L. H., Crosson, B., Nadeau, S. E., Heilman, K. M., Gonzalez-Rothi, L. J., Raymer, A., Gilmore, R. L., Bauer, R. M., and Roper, S. N. (2002). Category-specific naming deficits for objects and actions: semantic attribute and grammatical role hypotheses. Neuropsychologia 40, 1608–1621.
Marques, J. F. (2006). Specialization and semantic organization: evidence for multiple semantics linked to sensory modalities. Mem. Cogn. 34, 60–67.
Martin, A. (2007). The representation of object concepts in the brain. Annu. Rev. Psychol. 58, 25–45.
Martin, A., and Chao, L. L. (2001). Semantic memory and the brain: structure and processes. Curr. Opin. Neurobiol. 11, 194–201.
Mayes, A., Montaldi, D., and Migo, E. (2007). Associative memory and the medial temporal lobes. Trends Cogn. Sci. 11, 126–135.
McGuigan, N., Makinson, J., and Whiten, A. (2010). From over-imitation to super-copying: adults imitate causally irrelevant aspects of tool use with higher fidelity than young children. Br. J. Psychol. doi: 10.1348/000712610X493115
Menz, M. M., Blangero, A., Kunze, D., and Binkofski, F. (2010). Got it! Understanding the concept of a tool. Neuroimage 51, 1438–1444.
Mounoud, P., Duscherer, K., Moy, G., and Perraudin, S. (2007). The influence of action perception on object recognition: a developmental study. Dev. Sci. 10, 836–852.
Mozaz, M. J., Crucian, G. P., and Heilman, K. M. (2009). Age-related changes in arm–hand postural knowledge. Cogn. Neuropsychol. 26, 675–684.
Murphy, G. L., and Wisniewski, E. J. (1989). Categorizing objects in isolation and in scenes: what a superordinate is good for. J. Exp. Psychol. Learn Mem. Cogn. 15, 572–586.
Myung, J. Y., Blumstein, S. E., and Sedivy, J. C. (2006). Playing on the typewriter, typing on the piano: manipulation knowledge of objects. Cognition 98, 223–243.
Park, D. C., and Reuter-Lorenz, P. (2009). The adaptive brain: aging and neurocognitive scaffolding. Annu. Rev. Psychol. 60, 173–196.
Perraudin, S., and Mounoud, P. (2009). Contribution of the priming paradigm to the understanding of the conceptual developmental shift from 5 to 9 years of age. Dev. Sci. 12, 956–977.
Pierno, A. C., Tubaldi, F., Turella, L., Grossi, P., Barachino, L., Gallo, P., and Castiello, U. (2009). Neurofunctional modulation of brain regions by the observation of pointing and grasping actions. Cereb. Cortex 19, 367–374.
Ramos-Estebanez, C., Merabet, L. B., Machii, K., Fregni, F., Thut, G., Wagner, T. A., Romei, V., Amedi, A., and Pascual-Leone, A. (2007). Visual phosphene perception modulated by subthreshold crossmodal sensory stimulation. J. Neurosci. 27, 4178–4181.
Ranganath, C., Cohen, M. X., Dam, C., and D’Esposito, M. (2004). Inferior temporal, prefrontal, and hippocampal contributions to visual working memory maintenance and associative memory retrieval. J. Neurosci. 24, 3917–3925.
Rice, N. J., Valyear, K. F., Goodale, M. A., Milner, A. D., and Culham, J. C. (2007). Orientation sensitivity to graspable objects: an fMRI adaptation study. Neuroimage 36(Suppl. 2), T87–T93.
Riddoch, M. J., Humphreys, G. W., Edwards, S., Baker, T., and Willson, K. (2003). Seeing the action: neuropsychological evidence for action-based effects on object selection. Nat. Neurosci. 6, 82–89.
Riggio, L., Iani, C., Gherri, E., Benatti, F., Rubichi, S., and Nicoletti, R. (2008). The role of attention in the occurrence of the affordance effect. Acta Psychol. (Amst) 127, 449–458.
Roberts, K. L., and Humphreys, G. W. (2010). Action relationships concatenate representations of separate objects in the ventral visual system. Neuroimage. 52, 1541–1548.
Rodrigues Cavalcante, K., and Caramelli, P. (2009). Evaluation of the performance of normal elderly in a limb praxis protocol: influence of age, gender, and education. J. Int. Neuropsychol. Soc. 15, 618–622.
Shah, A., and Barto, A. G. (2009). Effect on movement selection of an evolving sensory representation: a multiple controller model of skill acquisition. Brain Res. 1299, 55–73.
Ska, B., and Nespoulous, J. L. (1987). Pantomimes and aging. J. Clin. Exp. Neuropsychol. 9, 754–766.
Tomita, H., Ohbayashi, M., Nakahara, K., Hasegawa, I., and Miyashita, Y. (1999). Top-down signal from prefrontal cortex in executive control of memory retrieval. Nature 401, 699–703.
Tucker, M., and Ellis, R. (1998). On the relations between seen objects and components of potential actions. J. Exp. Psychol. Hum. Percept. Perform. 24, 830–846.
Tucker, M., and Ellis, R. (2004). Action priming by briefly presented objects. Acta Psychol. (Amst) 116, 185–203.
Valyear, K. F., and Culham, J. C. (2010). Observing learned object-specific functional grasps preferentially activates the ventral stream. J. Cogn. Neurosci. 22, 970–984.
Villarreal, M., Fridman, E. A., Amengual, A., Falasco, G., Gerscovich, E. R., Ulloa, E. R., and Leiguarda, R. C. (2008). The neural substrate of gesture recognition. Neuropsychologia 46, 2371–2382.
Vingerhoets, G. (2008). Knowing about tools: neural correlates of tool familiarity and experience. Neuroimage 40, 1380–1391.
Vingerhoets, G., Acke, F., Vandemaele, P., and Achten, E. (2009). Tool responsive regions in the posterior parietal cortex: effect of differences in motor goal and target object during imagined transitive movements. Neuroimage. 47, 1832–1843.
Watkins, S., Shams, L., Tanaka, S., Haynes, J. D., and Rees, G. (2006). Sound alters activity in human V1 in association with illusory visual perception. Neuroimage 31, 1247–1256.
Weisberg, J., van Turennout, M., and Martin, A. (2007). A neural system for learning about object function. Cereb. Cortex 17, 513–521.
Wolpert, D. M., and Kawato, M. (1998). Multiple paired forward and inverse models for motor control. Neural Netw. 11, 1317–1329.
Keywords: grounded cognition, MOSAIC, apraxia, aging, development, tools, action
Citation: Mizelle JC and Wheaton LA (2010) The neuroscience of storing and molding tool action concepts: how “plastic” is grounded cognition? Front. Psychology 1:195. doi: 10.3389/fpsyg.2010.00195
Received: 14 July 2010;
Accepted: 21 October 2010;
Published online: 15 November 2010.
Edited by:
Anna M. Borghi, University of Bologna, ItalyReviewed by:
Binkofski Ferdinand, University of Luebeck, Germany;Maurizio Gentilucci, University of Parma, Italy
Copyright: © 2010 Mizelle and Wheaton. This is an open-access article subject to an exclusive license agreement between the authors and the Frontiers Research Foundation, which permits unrestricted use, distribution, and reproduction in any medium, provided the original authors and source are credited.
*Correspondence: Lewis A. Wheaton, School of Applied Physiology, Georgia Institute of Technology, 281 Ferst Drive, Room 104, Atlanta, GA 30332-0356, USA. e-mail: lewis.wheaton@ap.gatech.edu