Flexibility in Embodied Language Understanding

Willems, Roel M.; Casasanto, Daniel

doi:10.3389/fpsyg.2011.00116

REVIEW article

Front. Psychol., 03 June 2011

Sec. Psychology of Language

volume 2 - 2011 | https://doi.org/10.3389/fpsyg.2011.00116

Flexibility in Embodied Language Understanding

Roel M. Willems^1,2*

Daniel Casasanto^1,3,4

¹ Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, Netherlands
² Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, CA, USA
³ Neurobiology of Language Group, Max Planck Institute for Psycholinguistics, Nijmegen, Netherlands
⁴ The New School for Social Research, New York, NY, USA

Do people use sensori-motor cortices to understand language? Here we review neurocognitive studies of language comprehension in healthy adults and evaluate their possible contributions to theories of language in the brain. We start by sketching the minimal predictions that an embodied theory of language understanding makes for empirical research, and then survey studies that have been offered as evidence for embodied semantic representations. We explore four debated issues: first, does activation of sensori-motor cortices during action language understanding imply that action semantics relies on mirror neurons? Second, what is the evidence that activity in sensori-motor cortices plays a functional role in understanding language? Third, to what extent do responses in perceptual and motor areas depend on the linguistic and extra-linguistic context? And finally, can embodied theories accommodate language about abstract concepts? Based on the available evidence, we conclude that sensori-motor cortices are activated during a variety of language comprehension tasks, for both concrete and abstract language. Yet, this activity depends on the context in which perception and action words are encountered. Although modality-specific cortical activity is not a sine qua non of language processing even for language about perception and action, sensori-motor regions of the brain appear to make functional contributions to the construction of meaning, and should therefore be incorporated into models of the neurocognitive architecture of language.

Introduction

Do people use parts of the brain involved in perception and action to understand language? The idea that meaning is grounded in perceptuo-motor states, which traces its origins at least to the philosophy of Aristotle (De Anima, Book II–III), flourished in the writings of the British Empiricists (e.g., Locke, 1690/1979). In the twenty-first century, this proposal has been reborn in the context of theories of embodied cognition (e.g., Barsalou, 1999, 2008; Prinz, 2002; Anderson, 2003; Zwaan, 2004; Pulvermuller, 2005; Feldman, 2006; Gibbs, 2006). One difference between the current incarnation of this ancient idea and its previous instantiations is that scientists now have tools to test it in unprecedented ways, probing how the neural substrates of language and thought relate to neural systems for perception and action.

Here we review neurocognitive studies of language embodiment in healthy adults and evaluate their possible contributions to theories of linguistic meaning. We start out by sketching the minimal predictions that an embodied theory of language understanding makes for empirical research (see Embodied Language Understanding). We then survey cognitive neuroscience studies that have been offered as evidence for embodied semantic representations (see Neural Investigations of Embodied Language Understanding), and explore four debated issues. First, does activation of sensori-motor cortices¹ during action language understanding imply that action semantics relies on mirror neurons (see Does Motor System Activation During Language Understanding Entail Mirror Neurons?). Second, what is the evidence that activity in sensori-motor cortices plays a functional role in understanding language (see Activation of Sensori-Motor Cortices: Functional or Epiphenomenal?). Third, to what extent do responses in perceptual and motor areas depend on the linguistic and extra-linguistic context (see Context-Dependence of Sensori-Motor Cortex Activation During Language Processing). And finally, can embodied theories accommodate language about abstract concepts (see Processing Non-Literal Language).

Embodied Language Understanding

What does it mean for language to be “embodied”? Embodied cognition is understood in various ways, making more or less radical claims about how cognition, perception, and action interact. Proposals range from attributing some role for perception and action systems in grounding cognition (Barsalou, 2008) to extending the human mind to include parts of the environment (Clark and Chalmers, 1998; see Wilson, 2002 for discussion), leading Anderson (2008) to conclude that “in fact there are nearly as many theories of [embodied] grounding – what it is, and what it means – as there are theorists” (Anderson, 2008, p. 423, our addition).

What unites the seemingly disparate notions of embodied cognition is what they argue against. Here we narrow our scope to the study of linguistic meaning, or embodied semantics. Theories of embodied semantics argue against semantic representations as purely amodal and symbolic in nature, such as the symbols manipulated by a Turing machine². Put more positively: Embodied cognition argues that meaning is grounded in our interactions with the world, and that this grounding is reflected in a concept’s neural representation. On embodied accounts, when people understand language about perception and action they use parts of our brain that are also involved in perception and action. On another assumption shared by embodied semantic theories, neural systems that support perception and action also play a functional role in language processing. Sensori-motor cortex activation is not epiphenomenal, nor is it a down-stream consequence of processing language that triggers explicit perceptual or motor imagery. To summarize, the minimal predictions of an embodied theory of language understanding are that: (a) parts of our action and perception systems are involved in language understanding, and (b) this involvement is functional, constituting or contributing to the construction of meaning.

The embodied framework is closely related to the theory of distributed semantics, as formulated by Allport (1985; see Saffran and Scholl, 1999). According to Allport (1985), semantic representations are distributed along the perceptual features of concepts. For instance, color words would be partially represented by parts of the visual system involved in color perception, nouns describing objects would be partially represented by the visual system involved in perceiving objects, etc. On embodied theories, semantic content is distributed over modality-specific systems because concepts are at least partly constituted by mental simulations of our physical interactions with the world.

Our review will focus on results from experiments that use functional magnetic resonance imaging (fMRI), event-related potentials (ERPs) derived from electroencephalography (EEG), magnetoencephalography (MEG), and transcranial magnetic stimulation (TMS). We will also focus on language understanding rather than language production, since at present the great majority of the available data test predictions about neural activity in comprehenders (in part due to the technical challenges of studying the neural correlates of language production).

We acknowledge that there is a rich literature on embodied language understanding using behavioral techniques (see e.g., Glenberg, 2007; Barsalou, 2008; Fischer and Zwaan, 2008; Zwaan and Kaschak, 2008 for review). Although relevant, these studies lie beyond the scope of this review. The central claim of embodied semantics concerns the functional activation of neural systems for perception and action during language use. We will therefore focus on neural data, which bear on this claim most directly.

Neural Investigations of Embodied Language Understanding

Action-Related Language

In one line of research, researchers have used the somatotopic representation of movements performed with different effectors in the motor system to study the neural correlates of semantics (see Jirak et al., 2010 for review). Somatotopy means that neighboring parts of motor cortex are involved in movement of particular effectors such as the hands, feet, tongue, etc., (Woolsey, 1963; Woolsey et al., 1979; but see Graziano and Aflalo, 2007 for a modern perspective). The strategy has been to test whether understanding action-related language leads to somatotopic activation of motor areas. For instance, in one fMRI study subjects read verbs describing actions performed with the feet, hands, or face (e.g., “kick,” “pick,” “lick”; Hauk et al., 2004). Motor production areas were mapped by having participants move their foot, finger, or tongue. Reading action verbs led to a similar pattern of activation in premotor cortex as actually moving the effectors for two of the three effectors (the hands and the feet).

Tettamanti et al. (2005) similarly showed somatotopy in response to sentence comprehension. Participants listened to action-related sentences (“I bite an apple”) and abstract sentences (“I appreciate sincerity”). The action sentences included three types: those describing mouth actions (“I bite an apple”), hand actions (“I grasp the knife”), or leg actions (“I kick the ball”). All action sentences activated left inferior frontal cortex more strongly than abstract sentences did. Moreover, sentences describing actions with different effectors activated the premotor cortex in an effector-specific manner (Aziz-Zadeh et al., 2006).

Postle et al. (2008) did not replicate the somatotopic activation pattern in premotor cortex in response to the reading of hand, foot or mouth verbs found in other studies. They speculated that this may have been due to the fact that their control stimuli were concrete nouns (e.g., “moon,” “hill”) with similar levels of imageability as the action verbs, whereas in some previous studies hash marks or abstract words/sentences were used as baseline. However, a study from our group showed effector-specific premotor activation in hand areas even though the stimuli we contrasted were matched for imageability, along with other psycholinguistic factors (Willems et al., 2010b). For now it is unclear why Postle and colleagues did not observe an effector-specific activation of premotor cortex in response to action verbs.

One factor that could give rise to discrepant findings across action verb studies is the method of analysis. Creating subject-specific regions of interest in motor areas appears to be a more powerful analysis strategy than employing whole brain analysis or using regions of interest at the group level (Aziz-Zadeh et al., 2006; Willems et al., 2010b,c). Studies show that, across participants, there is considerable variation in the exact location of premotor activation during action verb reading (Aziz-Zadeh et al., 2006; Kemmerer and Gonzalez-Castillo, 2008; Fernandino and Iacoboni, 2010; Jirak et al., 2010; Willems et al., 2010b,c). This variation might lead to null-findings after averaging at the group level.

Action-related nouns have been found to activate motor areas, as well as verbs. An experiment by Rueschemeyer et al. (2010) shows that the amount of action associated with an object word drives activation of the motor system during word reading. Participants performed a lexical decision task while they read words denoting objects such as “bookend,” “clock,” “cup,” and “hammer.” These words differ in how much action is needed to use the object effectively. Although they are all concrete, manipulable objects, one does not need to actively manipulate a bookend or a clock in order to use it. This is different for a cup or a hammer, which one needs to manipulate to use them effectively. This distinction was reflected in increased activation levels in premotor and inferior parietal cortex, areas known to be involved in action and object use (see Beauchamp and Martin, 2007; Martin, 2007 for an overview of conceptual representations of objects).

In studies by Kable et al. (2002, 2005), participants performed a semantic matching task on triads of pictures or words describing action or objects. In one of these studies increased activity to action words was observed in the motor cortex just as in the studies described above (Kable et al., 2005). However, additionally, increased activation was found in the vicinity of human motion area MT when subjects had to judge actions compared to when they had to judge objects (Kable et al., 2002, 2005). The finding that motion areas are activated during action word understanding is in line with results from Kemmerer et al. (2008) who replicated the somatotopic activation in response to action verbs in premotor cortex. Moreover they showed that different verbs which share the semantic feature “ACTION” all activate premotor structures and that verbs sharing the semantic feature “MOTION” consistently activated more posterior areas known to be involved in movement processing (see Kemmerer and Gonzalez-Castillo, 2008 for discussion). This indicates that in addition to motor properties, perceptual (visual) properties can also be activated when processing language.

Similarly, Saygin et al. (2010) observed increased BOLD levels in motion sensitive area MT when participants read sentences like “The wild horse crossed the barren field” versus “The black horse stood in the barren field” (see also Desai et al., 2010). In contrast, Bedny et al. (2008) did not observe sensitivity of area MT to amount of implied motion in nouns or verbs. Participants judged pairs of words which implied motion (animals, e.g., “the horse” and “the dog”), had intermediate implied motion (tools, e.g., “the sword,” “the axe”), or had little implied motion (natural kinds, e.g., “the bush,” “the pebble”). The main finding was that motion area MT was not sensitive to this parametric variation. An area slightly more anterior to the visual motion area did show such sensitivity. But because this area was not activated in a motion localizer, this neighboring activity cannot be interpreted as support for the hypothesis that understanding motion language reuses brain areas involved in motion perception (see Simmons, et al., 2007 for discussion). It is not clear, however, to what extent the lack of overlap between activation during the localizer task and language task in the Bedny et al. (2008) data should be interpreted as evidence against embodied semantics, and to what extent these data simply show a mismatch in the neurocognitive demands of the two tasks. In principle, two different motion perception tasks could also activate non-overlapping cortical regions in or around MT. In defense of the Bedny et al. (2008) interpretation it should be noted that also posterior superior temporal sulcus did not show sensitivity to the amount of implied motion expressed by the verbs. Superior temporal sulcus is believed to respond rather invariantly to biological motion (e.g., Grossman et al., 2010). There may be a simple explanation for the contradictory findings from Bedny et al. (2008) and Saygin et al. (2010). Participants in the Saygin et al. (2010) study were presented with full sentences describing motion scenes, which may have cued them to create richer mental models than participants in the Bedny et al. (2008) study, who judged de-contextualized pairs of words.

How might one’s own action experience influence the neural correlates of action language understanding? Willems et al. (2010b) investigated whether premotor cortex activation in response to action verbs is body-specific (Casasanto, 2009), meaning that activation should be different for individuals who habitually perform actions differently. We measured brain activation (with fMRI) in healthy left- and right-handed participants during a lexical decision task. Participants read verbs that name hand actions (“to throw”) and verbs that name actions performed with other effectors (“to kneel”). Right- and left-handers showed contrasting patterns of activity in cortical motor regions when reading manual action verbs like grasp and throw, as compared to reading non-manual action verbs. Each group preferentially activated premotor areas in the hemisphere contralateral to their dominant hand: the hand they would normally use to perform the actions named by the manual action verbs. Understanding an action verb involves creating a representation of how the comprehender would perform the action with his or her own body (see also Beilock et al., 2008; Lyons et al., 2010).

It appears that the way people act in the world influences their semantic representations (see also Aglioti et al., 2008). However, people can also understand language about actions they have never performed. This creates a potential tension between embodiment and abstraction. We suggest that although some constituents of word meaning are body-specific, it is likely that other constituents are abstracted away from people’s physical experiences. It remains an open question whether people can understand words referring to actions they have never performed as completely as they understand words for actions they have performed themselves.

Does Motor System Activation during Language Understanding Entail Mirror Neurons?

Discussions of motor simulation and embodiment often invoke mirror neurons. Motor mirror neurons, which have been observed directly in monkeys, fire both when an animal performs an action and when it observes the same action (di Pellegrino et al., 1992; Rizzolatti and Craighero, 2004). Do human mirror neurons – if they exist in the human – contribute to language understanding? According to Glenberg et al. (2008b):

The mirror neuron system has been implicated in the understanding of language. […] Thus, observed actions are understood by engaging in motor simulation, and language about actions is understood by engaging in a substantially similar type of activity. An important fact supporting this interpretation is that a part of the human mirror neuron system is located in Broca’s area which has long been associated with language. (p. 917)

Invoking mirror neurons here is potentially misleading, for two reasons (see also Hickok, 2009; Turella et al., 2009). First, it gives the impression that the specific “mirroring” properties of mirror neurons are needed to support the modality-specific simulation postulated by embodied theories. But mirroring (i.e., firing similarly during action observation and execution) is not required for cells involved in action execution to be reused during language comprehension. As an analogy, consider the proposal that we reuse visual areas during visual imagery. Does it follow from this that cells in visual cortex have mirror neuron-like properties? It is sufficient to posit that visual imagery reuses parts of the brain involved in visual perception, and analogously, that action language understanding reuses parts of the brain involved in performing actions.

Second, bringing in mirror neurons seems to put special emphasis on the coupling between language and the motor system. Although the motor system appears to have special significance for understanding language about actions, there may be nothing special about the role of the motor system in understanding language, in general (see also Willems and Hagoort, 2007; cf. Corballis, 2003; Arbib, 2005). As is evident from Sections “Does Motor System Activation During Language Understanding Entail Mirror Neurons?” and “Perceptual Language” of this review, the data relevant to embodied theories of semantics come primarily from studies of the motor system, and secondarily, of the visual system. But these neural systems have no special status theoretically; in principle, modality-specific simulation should contribute to comprehension of language about any sensory, motor, or affective state (see Barsalou, 2008).

Perceptual Language

Does language understanding also involve activations of perceptual areas when the semantic content includes perceptible features? This was tested in an fMRI study by Simmons et al. (2007) in which participants were presented with written words (e.g., “eggplant”) which would be followed by a color word depicting either the typical color of the object (e.g., “purple”) or an atypical color (Simmons et al., 2007). On control trials, an object-word (e.g., “screwdriver”) would be followed by a motor property (e.g., “turning”). Participants’ task was to respond as quickly as possible to the color/motor word, indicating whether it described an actual property of the object named by the first word. Color sensitive areas in extrastriate cortex were localized using a perceptual color task. The main finding was that parts of extrastriate cortex sensitive to color perception were also more strongly activated when participants judged a word’s color property, as compared to when they judged a motor property³.

Moscoso del Prado Martin et al. (2006) measured EEG while participants passively read color-related words, form-related words, or non-words. They observed that ERP responses to words started to differ from non-words around 150 ms after stimulus presentation. ERPs for color-related words and form-related words differed later, around 200 ms. The neural generators of the ERP effects were localized using minimum norm current estimates. The word versus non-word difference localized mainly to left fusiform gyrus, perhaps corresponding to the “visual word form area” (Cohen et al., 2002). The color-related versus form-related words difference localized to left middle temporal gyrus, whereas the difference between form-related and color-related words peaked in left anterior frontal areas. The authors interpret the difference in localization of form- and color-related words to differences in the processing of color and form. This conclusion should be taken with caution though, since the areas activated during word processing were not the particular extrastriate areas that are usually involved in processing color and form.

Aziz-Zadeh et al. (2008) performed an fMRI study in which participants listened to sentences related to faces (e.g., “The farmer has freckles on his cheeks”) or related to places (e.g., “The house has a couch near the fireplace”). Previous research indicates that seeing faces and places elicits activation increases in confined parts of cortex, often labeled fusiform face area (FFA) and parahippocampal place area (PPA), respectively (Kanwisher et al., 1997; Aguirre et al., 1998; Epstein and Kanwisher, 1998; Epstein et al., 1999). Aziz-Zadeh et al. (2008) localized FFA and PPA in their participants using a perceptual task and subsequently looked at activation in these areas during listening to face- and place-related sentences. Face-related sentences selectively modulated FFA activity, and place-related sentences PPA activity. Somewhat surprisingly, however, this modulation was present in the form of a decrease to sentences of the type the extrastriate area was sensitive to: FFA was less activated by face-related sentences as compared to place-related sentences, and vice versa for PPA. This effect was only observed in the left hemisphere and only to sentences involving famous people or places (e.g., “George Bush has wrinkles around his eyes” and “The Taj Mahal faces a long thin reflecting pool”).

In a related study, Landau et al. (2010) presented participants with the same place- and face-sentences, which were now followed by the picture of a face. EEG was measured and time locked to the presentation of the face picture. Interestingly, it was found that the face-sensitive ERP component N170 to a face preceded by a face-related sentence, was larger compared to faces that were preceded by a place-related sentence. On the contrary, priming a face picture by another face picture (visual–visual priming), instead led to a decreased N170 amplitude. In short this study shows that linguistic priming leads to an opposite amplitude modulation of the N170 as compared to visual priming.

The decreased BOLD activations (Aziz-Zadeh et al., 2008), and opposite amplitude modulations of the ERP (Landau et al., 2010) are explained as resulting from inhibition of content-specific information during language understanding, at least partially in order to prevent interference from the actual perceptual content. The idea is that when we talk with someone face-to-face, activating a face representation in response to language about faces may interfere with our perception of our speech partner’s face, and that hence listening to language about faces leads to suppression of activity in the FFA.

This explanation may be correct, but it is somewhat surprising that suppression would persist even when comprehenders are alone, lying in the scanner, not talking with anyone. Furthermore, an embodied theory of language such as Pulvermuller’s (2005) clearly predicts language-related activation in FFA and PPA; not suppression. Still, a skeptic who wished to interpret Aziz-Zadeh et al.’s (2008) data as evidence against an embodied theory would be obligated to explain why content-specific perceptual areas were selectively modulated by linguistic stimuli, at all.

Section Summary

Together, these studies provide evidence that parts of the cortical motor and perceptual systems can be activated (or deactivated) during language understanding, although it should be noted that this modulation is not always observed. Moreover, they show that this activation can be selective. Verbs naming actions performed with a certain effector elicit activation in the areas involved in using that effector. This effector-specific activation also reflects the way comprehenders tend to perform actions with their particular bodies. Likewise, assessing the color properties of words activates visual areas involved in color perception, whereas understanding sentences about faces and places modulates activity in face- and place-specific visual areas.

Results like these have been interpreted as evidence for an embodied view of language understanding (e.g., Gallese and Lakoff, 2005; Feldman, 2006; Glenberg, 2007). Classical models of language in the brain primarily include peri-sylvian “language areas,” and do not predict that perceptual and motor areas should be involved in language understanding (Cutler and Clifton, 1999; Levelt, 1999; Perfetti, 1999). The research that we reviewed is teaching us something new, but from the studies reviewed so far it is not clear what lesson we should take away. In showing involvement of modality-specific perceptuo-motor areas during language comprehension, these studies satisfy the first minimal requirement of embodied semantics that we suggested above (see Embodied Language Understanding), but they do not address the second: how do we know the observed activation plays any functional role in language understanding?

Activation of Sensori-Motor Cortices: Functional or Epiphenomenal?

The data reviewed above are consistent with embodied theories of semantics, but these data could also be explained by a traditional amodal view of meaning. Modality-specific activation could be a down-stream consequences of “true” language processing (Mahon and Caramazza, 2008). For example, upon reading a verb like “to throw,” the amodal semantic representation of the verb is activated in the mental lexicon, corresponding to activation in left temporal lobe (see Indefrey and Levelt, 2004; Indefrey and Cutler, 2005). Activating this amodal representation leads to a cascade of spreading activation, and may cue the generation of an explicit motor image of throwing (see also Weiskopf, 2010). On this account, the activations in premotor cortex do not contribute to language understanding; they result from it.

To address this skeptical possibility, some researchers have attempted to establish the functional significance of modality-specific activity by showing how rapidly it is cued by language. MEG and EEG studies show that differences between leg, arm or face words emerge around 200 ms (Hauk and Pulvermuller, 2004; Pulvermuller et al., 2005b). Results showed a dissociation between leg words in dorsal premotor cortex and arm and face words in more ventral premotor regions, consistent with the expected somatotopy. The authors argue that the latency of this effect is too fast to be explainable in terms of general strategic effects or post-comprehension mental imagery, and must therefore be part of the semantic representation of the word. Indeed, such rapidity argues against interpreting this language-related premotor activity as a down-stream consequence of explicit imagery (Farah, 1989). However, these data do not speak to the functional significance of this activity for meaning construction: not without a host of assumptions about information flow within and between neural systems that may be difficult to substantiate (Mahon and Caramazza, 2008; Willems et al., 2010c).

In another effort to establish the functional significance of modality-specific activity, Hauk et al. (2008) looked at how lexical frequency influenced neural processing of action-related words (e.g., “kick”) and vision-related words (e.g., “sun”). By their reasoning, parametric variation of activity in motor and visual areas with the lexical frequency of motor and visual words would be indicative of lexico-semantic processing: not strategic task-related responding or explicit imagery. Furthermore, they reasoned that observing separable effects of frequency for action- and visual-words in motor and visual areas, respectively, would provide evidence of modality-specific lexical representations. The main finding was a correlation of lexical frequency for visually related words with activation in left fusiform gyrus and of lexical frequency for action-related words with activation in left middle temporal gyrus. This was interpreted as reflecting “true” lexico-semantic processing and not epiphenomenal or down-stream activation. While compelling in their specificity, these data are, once again, equivocal with respect to the functional significance of activation in visual and motor areas cued by language; these correlational data do not establish any causal role for modality-specific activity in the construction of linguistic meaning.

Researchers have used TMS to investigate a causal role for motor cortex activation in understanding action-related language. In one study, hand/arm and leg words were presented while stimulating the “hand/arm area” and “leg area” of the motor cortex (Pulvermuller et al., 2005a). Stimulation was single-pulse TMS and subjects’ task was to make a lexical decision to the visually presented words. Reponses to target words were given by means of a mouth movement. Faster reaction times were observed to hand/arm words after stimulation of the “hand/arm area” as well as to leg words after stimulation of the “leg area.” Papeo et al. (2009) tried to replicate this finding using a more elaborate stimulation scheme, applying TMS pulses at several time points after verb presentation. The main finding was that hand motor evoked potentials (MEPs) were increased to reading of action verbs when TMS was applied 500 ms after stimulus presentation, but not when TMS was applied 170 and 340 ms after stimulus presentation. The finding that only the latest post-stimulus TMS stimulation increased MEPs makes it difficult to interpret the MEPs Papeo et al. (2009) observed as evidence that cortical motor areas play a role in language understanding, per se; the timing of the N400 ERP component suggests that important aspects of word meaning are typically computed in less than 500 ms. Moreover, the MEP increases were only present when Papeo et al. (2009) participants performed a semantic categorization task, but not during a syllabic categorization task. This raises the question of how automatically (pre)motor representations are constructed in response to action verbs, a question we will return to in Section “Context-Dependence of Sensori-Motor Cortex Activation During Language Processing.”

Further complicating interpretation of these TMS data, Buccino et al. (2005) found a pattern of results opposite to the pattern found in the studies described above, using a similar paradigm. Participants passively listened to sentences describing leg and arm actions, while the leg/foot and arm areas of motor cortex were stimulated with single-pulse TMS. MEPs were recorded from the participants’ hands and feet. The main finding was that single-pulse TMS led to decreased hand MEPs when participants were listening to hand-related sentences as compared to foot-related sentences, and vice versa. One notable difference across studies is the relative timing of stimulation. TMS pulses were delivered during reading in the Buccino et al. (2005) paper, 150 ms after word presentation in the Pulvermuller et al. (2005a) study, and at later time points in the Papeo et al. (2009) study.

A related technique to study the involvement of premotor cortex in action language understanding is theta burst TMS. Theta burst TMS modulates cortical activity in a targeted region for a relatively short (up to 1 h) time (Huang et al., 2005). Willems et al. (in press) used this technique to modulate excitability of left or right premotor cortex of right-handed participants, who performed a lexical decision to manual and non-manual action verbs. The main finding is that reaction times to manual and non-manual verbs were differentially modified by TMS over left, but not right premotor cortex. In particular, participants were faster to manual verbs after left premotor TMS than after right premotor TMS. Again, as in the discussion of single-pulse TMS studies, it is essentially unclear why theta burst TMS led to faster reaction times.

Together, these studies indicate there is some causal relationship between motor cortex activation and action verb processing, but based on the TMS results to date, this relationship remains unclear.

A few studies have tested for functional relationships between motor activity and action language processing in patients with motor system impairments. For example, Boulenger et al. (2008b) performed a masked priming study in patients with Parkinson’s disease, a neurological condition that impacts the motor system. Compared to age-matched controls, Parkinson’s patients showed weaker masked priming effects for verbs compared to nouns. Strikingly, there was no verb–verb priming in patients who were off dopaminergic medication; when disease symptoms were strongest, patients showed no trace of verb–verb priming, yet noun–noun priming in these patients was similar to controls. There are several methodological issues that make these data less than straightforward to interpret (see Mahon and Caramazza, 2008 for discussion). Despite these limitations, however, this study suggests that impairment to the motor system may selectively impair processing action verbs.

Arevalo et al. (2010) found some effect of lesions in primary and premotor cortex on word–picture matching of actions, but the impairment was only observed for foot-related actions. Strikingly, no somatotopic relation between lesion site in the motor system and understanding of mouth, foot, or hand words was observed and the authors argue against a necessary role for motor cortex in understanding action concepts.

A complete overview of potentially relevant neuropsychological data is beyond the scope of this paper (see Saffran and Scholl, 1999; Gainotti, 2000; Mahon and Caramazza, 2009). For the current purposes, we note that there are some indications that impairments to brain structures specialized for perception and motor action lead to corresponding problems in language understanding. Upon skeptical evaluation, however, these patient-based studies do not appear to be conclusive (see Mahon and Caramazza, 2008; Arevalo et al., 2010).

In summary, EEG and MEG studies demonstrate that language can activate brain areas that subserve perception and action very rapidly, challenging the skeptical position that modality-specific activity is only an artifact of strategic responding or post-comprehension mental imagery. TMS and patient-based studies provide initial evidence that motor cortex activation plays some causal role in understanding action-related language, but the available evidence does not support strong conclusions, in particular since TMS can lead to faster as well as slower reaction times in seemingly comparable experimental situations. (See also Processing Non-Literal Language for discussion of a behavioral study by Glenberg et al. (2008a) that provides evidence that motor representations play a causal role in language understanding).

Context-Dependence of Sensori-Motor Cortex Activation During Language Processing

How automatically are sensori-motor cortices activated when people understand language about perception and action? According to some proposals, word forms are linked to (somatotopic) representations in sensori-motor cortices via Hebbian cell assemblies (Gallese and Lakoff, 2005; Pulvermuller, 2005). On the most extreme view, activation of motor cortex in response to action words is reflexive: merely perceiving a spoken or written word like “grasp” should activate cortical areas involved in planning and executing actions, even if the perceiver is not attending to the word (see Pulvermuller, 2005, p. 578; Gallese and Lakoff, 2005). Although some studies suggest that sensori-motor areas are activated by language highly automatically (e.g., Hauk and Pulvermuller, 2004; Pulvermuller et al., 2005b), the majority of studies suggest that modality-specific activation depends on the context – linguistic and extra-linguistic – in which words referring to precepts and actions occur.

Experimental task appears to influence how motor cortex responds to action-related language. Sato et al. (2008) found that hand-action verbs interfered with button presses, but only when participants performed an explicit semantic judgment task, not during lexical decision. A similar task effect was observed by Papeo et al. (2009), who found modulation of hand MEPs during reading of hand-action verbs when single-pulse TMS was applied only during an explicit semantic categorization task (“Is this word action-related or not?”, but not during a syllable detection task (“How many syllables are there in this word?”). Some studies show effector-specific activity even during tasks with shallow processing demands, such as lexical decisions on action verbs (e.g., Willems et al., 2010b), but in general tasks that require deeper “semantic-level” processing (Craik and Lockhart, 1972) are more likely to produce evidence of modality-specific simulation during language processing.

The linguistic context can also cause motor areas to respond differently to the same action-related words. Tettamanti et al. (2008) showed that negation of actions modulates activation levels in premotor cortex. Participants read sentences like “Now I push the button” versus “Now I don’t push the button.” For comparison, they read sentences unrelated to concrete actions like “Now I appreciate sincerity” and “Now I don’t appreciate sincerity.” The main finding was an interaction between sentence content (action versus no action) and negation (affirmative versus negative) in part of dorsal premotor cortex, with higher activation for affirmative action sentences compared to negative action sentences. In a further analysis of these data it was shown that connectivity between auditory (temporal) and motor regions was increased during listening to action-related sentences, as compared to when listening to the more abstract sentences (Ghio and Tettamanti, 2010).

van Elk et al. (2010) manipulated whether an action mentioned in a sentence was performed by an animal or by a human. Participants read sentences like “The duck swims in the pond” versus “The woman swims in the pond,” during EEG measurements. A stronger decrease in power in the beta and mu frequency bands was observed in response to the verb (“swims”) when the action was performed by an animal compared to when it was performed by a human. The neural generators of these power differences were localized to primary and premotor cortex. Decreases in beta power have been interpreted as indicating increased motor simulation (Caetano et al., 2007; van Elk et al., 2008). Therefore, it is surprising that beta decreases were greater when the sentences referred to an animal’s action as opposed to a human’s, since presumably people can simulate human actions more easily and in greater detail than animal actions. It is difficult to interpret these data as clear evidence for or against embodied theories of semantics. For the present purposes, this study provides further evidence that the motor cortex does not respond to action verbs indiscriminately. Whether it actually does respond depends upon the linguistic context the language is perceived in, as well as the task or intention of the perceiver.

Moody and Gennari (2010) showed that the amount of effort expressed in a sentence modulates activation levels in premotor cortex. “The delivery man is pushing the piano” led to stronger activation as compared to “The delivery man is pushing the chair,” which in turn led to stronger activation levels compared to the control condition “The delivery man has forgotten the piano.” Along similar lines, van Dam et al. (2010) found that part of inferior parietal cortex is sensitive to the implied specificity of single action verbs (e.g., “to wipe” versus “to clean”). These studies are further evidence for the context-specificity of activation in the motor system upon reading action-related language.

Finally, few studies have looked at how sensori-motor areas are engaged in story comprehension, specifically when the story describes actions. Speer et al. (2009) for instance found that parts of motor cortex were specifically activated when participants read about physical actions, such as manipulations of objects. Interestingly, this was the case in the absence of an explicit task and the materials were not biased toward actions, making it less likely that explicit imagery was invoked by participants (see also Xu et al., 2005).

Thus in summary, the available evidence weighs against the view that merely perceiving a perception or action word necessarily activates perceptuo-motor areas (Pulvermuller, 2005), while still showing that these areas can be activated with a surprising degree of automaticity, even during shallow language processing.

Processing Non-Literal Language

According to theories of metaphorical mental representation, people understand abstract words and concepts, in part, by activating representations in more concrete domains, particularly the domains of space and motion (Lakoff and Johnson, 1980). There is now abundant behavioral evidence that spatio-motor representations contribute to people’s understanding of many abstract domains, including time (Boroditsky, 2000), preference (Casasanto, 2009), goodness (Meier and Robinson, 2004), intimacy (Williams and Bargh, 2008), social dominance (Schubert, 2005), kinship (Enfield, 2005), musical pitch (Rusconi et al., 2006), and similarity (Casasanto, 2008). Some of these studies show connections between bodily action and abstract mental states that may seem, at first, to directly support the claim that abstract concepts and word meanings are embodied. For example, a study by Casasanto and Dijkstra (2010) shows that simple motor actions affect how people produce language with positive and negative emotional valence. Participants were asked to recount positive or negative autobiographical memories while moving marbles upward or downward between two cardboard boxes. In one experiment, participants were faster to tell positive memories while moving marbles up and negative memories while moving them down, consistent with linguistic metaphors for positive and negative emotions (e.g., “her spirits rose”; “her hopes fell”). In a second experiment, upward marble movements caused participants to recount more positive events and downward movements more negative events from their store of personal experiences. Beyond influencing how efficiently stories could be produced, irrelevant marble movements partly determined the emotional content of the memories participants retold.

Yet, it would be premature to interpret data like these as evidence that modality-specific simulations support abstract language and thought. Together, this body of behavioral data provides support for metaphor theory, showing for instance that people activate spatio-motor representations when they talk and think about abstract domains. But evidence for metaphor theory is not necessarily evidence for embodiment. Although these behavioral experiments show that some sort of spatio-motor representations contribute to abstract thoughts, they are not informative about the nature of these representations. In principle, motor actions could activate mental metaphors that consist of associative links between amodal representations in “source domains” like motion and space and “target domains” like emotion, similarity, or time (see Casasanto, 2009, for discussion). Although behavioral data that support metaphor theory are consistent with embodied semantics, they are also consistent with disembodied alternatives.

In order to determine whether the spatio-motor components of metaphorical representations are embodied simulations, it is necessary to test predictions about neural activity more directly. Several studies have investigated whether motor areas are activated when people understand non-literal uses of action language. They have produced mixed results, perhaps because these studies have tested a mixed bag of non-literal language: action metaphors, action idioms, and non-action verbs derived (diachronically) from action verbs.

Rueschemeyer et al. (2007) used fMRI to compare neural activation to action verbs and verbs that have an action verb as their stem, but do not have any action-related meaning (e.g., the German verbs “greifen” – to grasp – versus “begreifen” – to understand). In principle, there are two reasons why “begreifen” might activate parts of premotor cortex even though its meaning does not signal a literal action. First, this verb could be morphologically decomposed by the comprehender: the “greifen” in “begreifen” could be sufficient to activate motor areas. Second, “begreifen” could be understood as a metaphorical extension of literal grasping. Results showed that, compared to abstract verbs without action stems, the literal action verbs activated premotor areas, as in previous studies. By contrast, no premotor activation was found for verbs like “begreifen.” When action verbs function as stems for more complex non-action verbs, they do not appear to activate motor areas.

Aziz-Zadeh et al. (2006) also used fMRI to compare premotor activity during literal and non-literal uses of action verbs. They found effector-specific premotor responses to phrases describing literal hand, foot, and mouth actions (“grasping the scissors,” “pressing the piano pedal,” “biting the peach”), but not to phrases that used action verbs non-literally (“handling the truth,” “chewing over the details,” “kicking off the year”; Aziz-Zadeh et al., 2006). Similarly, in an fMRI study by Raposo et al. (2009), motor regions were activated when participants heard isolated action verbs (“kick”) and literal action sentences (“kick the ball”), but not when they heard action verbs used idiomatically (“kick the bucket,” meaning “to die”; Raposo et al., 2009). By contrast, Boulenger et al. (2008a) reported somatotopic activation in motor cortex when participants read idiomatic sentences like “He kicked the habit” (meaning “to quit”). However, this effect was only observed when neural activity was modeled 3 s after the end of each sentence, not when the neural response was modeled based on the presentation of the word that disambiguated literal from idiomatic sentences (e.g., “habit” in this example). The authors explain this late activation as resulting from the later metabolic changes allegedly associated with abstract sentences. However, as discussed in Section “Activation of Sensori-Motor Cortices: Functional or Epiphenomenal?”, this late engagement of motor areas could also be due to explicit mental imagery (e.g., of literal kicking) that occurs when experimental participants are given time to reflect on the stimuli.

So far, these studies comparing literal to non-literal uses of action verbs further demonstrate that the link between action language and motor cortex is context-sensitive. But what do they say about the metaphorical representation of abstract concepts and word meanings? In our view: very little. Metaphor theories differentiate active metaphors (e.g., “grasp the idea”) from etymologically derived forms (e.g., “begreifen”) and frozen idioms (e.g., “kick the bucket”). Unlike etymological relics and idioms, active metaphors are systematic and productive (see Casasanto, 2009). “Grasping an idea” is part of a system of expressions that can indicate different levels of understanding, and the new variants of these expressions can be readily understood (Lakoff and Johnson, 1980). When people are struggling with an idea they are “trying to grasp it” or “trying to get a hold of it.” When they understand something well they have a “firm grip” or a “good grasp.” When they are trying to forget an idea, they are “letting go of it.” By contrast, frozen idioms are neither systematic nor productive. “Kicking the bucket” means to die, but “kicking the cup” does not mean to merely get injured. Whereas metaphors are flexible, idioms a fragile; changing the way they are worded destroys their idiomatic meaning.

On most theories, only metaphors should be understood via activation of representations in the source domain (e.g., in case of “grasp the idea,” the source domain is literal grasping, and the target domain is the abstract notion of understanding). Frozen idioms and etymological roots should not necessarily activate source domain representations (Casasanto, 2009; cf. Gibbs, 2006). For the studies reviewed so far in this section, stimuli in the non-literal condition were partly (or entirely) composed of etymological derivations and frozen idioms. It is not surprising, therefore, that understanding them did not activate motor areas. The studies by Rueschemeyer et al. (2007), Aziz-Zadeh et al. (2006), Raposo et al. (2009), and Boulenger et al. (2008a) provide valuable information; they tell us that action verbs embedded in morphologically complex verbs and in frozen idioms do not activate motor areas (at least not on a timecourse relevant to early stages of language comprehension). But these studies do not bear on the question of whether metaphors with perceptuo-motor source domains are understood via activity in sensori-motor cortices.

A few cognitive neuroscience studies have tested comprehension of expressions which, according to the criteria of systematicity and productivity, should be processed via active mappings from metaphorical source domains. In one TMS study by Glenberg et al. (2008b), participants listened to sentences that described transfer (movement) of a concrete object (“Andrea carries the pizza to you”) or sentences describing non-literal transfer (“Arthur presents the argument to you”). By hypothesis, “presenting” an argument is understood metaphorically, and uses the same motor circuits that support understanding literal transfer scenarios like “presenting a gift.” Consistent with the authors’ predictions, there was an increase of hand MEPs to transfer sentences compared to non-transfer control sentences (e.g., “You smell the pizza with Andrea”). Importantly, there was no difference between concrete transfer sentences and abstract transfer sentences. In a related study, Glenberg et al. (2008a) asked participants to move hundreds of beans either toward or away from their bodies before making sensibility judgments on concrete and abstract transfer sentences, responding with button presses that required movements toward or away from them. Participants were slower to judge transfer sentences when they described concrete or abstract “motion” in the same direction in which they had been moving beans, presumably because the same neural circuits that subserve physically transferring objects with the hands also support understanding transitive constructions in language.

Critics of embodied cognition often point to the existence of abstract concepts as prima facie evidence that some knowledge lies beyond the scope of theories that foreground modality-specific representation (e.g., Mahon and Caramazza, 2008). But this argument is not well justified. Metaphor theory is one example framework in which to pursue testing roles for modality-specific neural activity in the representation of abstract concepts and language (see Barsalou, 2008 and Prinz, 2002 for alternatives). Here we have focused on activation of the motor system as a metaphorical source domain, but representations in abstract target domains could be neurocognitive simulations, as well. For example, thinking about emotions or understanding affective language could involve partial reenactments of emotional states, in the regions of the brain that give rise to emotional experiences, and there is indeed evidence to suggest that this is the case (e.g., Isenberg et al., 1999; Strange et al., 2000; Herbert et al., 2009; Willems et al., 2010a). To the extent that abstract concepts are constituted by metaphorical source- and target-domain representations, modality-specific activity in sensory, motor, and interoceptive (e.g., affective) systems could contribute to the semantics of language about abstract things that people can never experience directly through perception or motor action.

Summary and Directions for Future Research

As is now evident from numerous studies, when people hear or read language about percepts and actions they reliably activate brain areas that also subserve perceiving and acting. Activation of sensori-motor cortices is specific: processing language about hues activates visual areas involved in discriminating chromatic as opposed to achromatic colors (Simmons et al., 2007); processing language about a bodily action performed with one effector or another engages somatotopic representations in the motor system (Pulvermuller, 2005); the neural representation of action language reflects the ways that individuals tend to perform actions with their particular bodies (Willems et al., 2010b). Engagement of sensori-motor cortices has been shown to be rapid and automatic (Pulvermuller et al., 2005a), but also dependent on the linguistic and extra-linguistic context in which words occur.

Perceptuo-motor representations are not a sine qua non of language processing, even for action words; rather, modality-specific representations are more likely to be activated in some contexts than in others (Aziz-Zadeh et al., 2006; Sato et al., 2008; Papeo et al., 2009). Yet, sensori-motor cortex activity appears to have some functional consequences for understanding language about actions, and even for understanding language about abstract ideas (Glenberg et al., 2008b). Thus, the evidence reviewed here satisfies the minimum requirements for a theory of “embodied semantics,” namely that (a) parts of our action and perception systems are involved in language understanding, and (b) this involvement is functional, constituting or contributing to the construction of meaning.

The flexibility with which sensori-motor representations are activated during language processing is consistent with a flexible view of conceptual representations, more broadly. The assumption that conceptual representations are stable dates back at least to Plato’s Ideas (Copleston, 1946/1993) and has been the dominant position – at least implicitly – for large part of the twentieth century. This has led to the general assumption that concepts are stable, with a specific pattern of brain activation being the representation of the concept DOG (see Weiskopf, 2009 for discussion). This is an extreme position, and there is ample evidence against it (Barclay et al., 1974; Zwaan, 2004; Taylor and Zwaan, 2008; Zwaan and Kaschak, 2008; Zwaan et al., 2010). At the other extreme, it could be that context determines concepts and word meanings to such an extent that no neurocognitive representation is ever the same twice (Elman, 2004, 2009; Casasanto and Lupyan, under review). The studies reviewed here, while demonstrating flexibility in sensori-motor activity during language processing, do not speak to the limits of this flexibility, or explore its consequences. For example, although several studies show that language-related RTs or MEPs are modulated when motor areas are perturbed using TMS, it remains to be determined in what ways people’s understanding of the linguistic stimuli changes as a consequence. When a perception or action word cues modality-specific cortical activity, does this result in a qualitatively different understanding of the word than when it does not? Is there a phenomenal difference for the comprehender? Does a representation of a word that includes modality-specific activity produce different behaviors (beyond affecting RTs in a laboratory task), or support different inferences than a representation that does not comprise any modality-specific activity? Such questions await further research.

Yet, on the basis of what has been shown so far, the literature on embodied language understanding calls for a reappraisal of what should be counted among the core neural systems involved in language processing. At the end of the twentieth century, leading psycholinguists drafted “blueprints” of the neurocognitive architecture for producing speech (Levelt, 1999) and for understanding spoken language (Cutler and Clifton, 1999) and written text (Perfetti, 1999), based on the initial decades of neurocognitive research on language. None of these blueprints suggests any important role for perceptuo-motor representations in the construction of linguistic meaning, or predicts any activation in sensori-motor cortices corresponding to semantic processing (only during perception and production of words’ forms). Thus, the hypotheses pursued by embodiment theorists in the early twenty-first century constitute a different way of thinking about language. The considerable amount of evidence for meaning-related activation of perceptuo-motor systems suggests that the blueprints for the neurocognitive architecture of language may need to be redrawn.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

Supported by grants from the Dutch Science Foundation (NWO Rubicon 446-08-008) and the Niels Stensen foundation to Roel M. Willems and a James S. McDonnell Foundation Scholar Award to Daniel Casasanto. Publication costs were paid for via an Open Access publication grant from NWO.

Footnotes

^We choose to use the term “sensori-motor cortex” since this is accepted usage in the field, even though most experiments investigate brain regions involved in perception and action, not sensation and action.
^A young Alan Turing interestingly foreshadowed this disembodied perspective: “As regards the question of why we have bodies at all; why we do not or cannot live free as spirits and communicate as such, we probably could do so but there would be nothing whatever to do. The body provides something for the spirit to look after and use.” (in Hodges, 1983, p. 64).
^Note that the dorso-medial of the brain including motor areas was not covered during scanning. Hence it was not possible to investigate the response in the motor system to words in the motor property task (e.g., “screwdriver”).

References

Aglioti, S. M., Cesari, P., Romani, M., and Urgesi, C. (2008). Action anticipation and motor resonance in elite basketball players. Nat. Neurosci. 11, 1109–1116.