Exploring the neural basis of real-life joint action: measuring brain activation during joint table setting with functional near-infrared spectroscopy

Introduction

Many every-day situations require joint actions of at least two individuals. Tasks like lifting a heavy object, folding a large blanket, or setting a dinner table with a partner are all complex, joint action movements that require interindividual temporal and spatial coordination of alternating or simultaneous movements. Interest has recently been focused on methods to investigate this type of social interaction in the laboratory and disclose the relevant cognitive processes.

Examining the neural processes underlying joint actions demands that social context of these actions also be considered (Sebanz et al., 2006; Hari and Kujala, 2009). Creating a believable social task within the constraints posed by common imaging modalities is a methodological challenge. Using inventive paradigms, the limited number of published neuroimaging studies on joint action have demonstrated that despite the restrictions of an fMRI setting, relevant aspects of joint action can be examined (Sebanz et al., 2007; Newman-Norlund et al., 2008; Kokal et al., 2009; Kokal and Keysers, 2010). Tasks in these studies have been limited to simple motor tasks that can be completed easily by participants who are required to lay down in an fMRI scanner during imaging, e.g., pressing a button (Sebanz et al., 2007), balancing a virtual bar by squeezing force pads (Newman-Norlund et al., 2008), or moving sticks (Kokal et al., 2009; Kokal and Keysers, 2010). Even though these relatively simple motor tasks share several aspects and affordances with natural joint action settings (e.g., task sharing, common goals), it is not known whether the findings of previous studies are generalizable to real-life joint action tasks demanding the coordination of more complex movements in a more social context. For example, an every-day life task like setting a dinner table in cooperation with another person consists of distinct object-related grasping and displacing movements with more spatial and temporal degrees of freedom than any of the simple motor tasks mentioned. For assessing brain activation during a complex, joint action motor task that is relevant to every-day situations, a functional brain imaging methodology is needed which allows the assessment of brain activation in a more natural social setting.

Functional near-infrared spectroscopy (fNIRS) is a relatively new imaging technique, which is non-invasive and highly flexible. The fNIRS optodes are fastened directly onto the head, which enables the imaging of brain activation of participants in more natural positions. Furthermore, it is robust against movement artifacts. For example, it has successfully been applied to investigate neural correlates of rather complex actions like walking and running on a treadmill (Suzuki et al., 2004), peeling an apple (Okamoto et al., 2004b), or executing a knot-tying task (Leff et al., 2008). A recent study also applied it to the investigation of face-to-face conversation (Suda et al., 2010). The success of these studies suggest that fNIRS is an appropriate method for measuring brain activation during naturalistic, real-life joint action tasks in which the joint action partners act in a face-to-face situation without severe movement restrictions and with directly observable action sequences. Comparing brain activation during the execution of a real-life task alone to brain activation measured during the same real-life task executed with a partner could reveal brain areas which are more activated during real-life joint action tasks. On the basis of the functional properties of these areas, cognitive processes could be specified which are thought to be preferentially or specifically engaged in real-life joint action (Newman-Norlund et al., 2007).

Current theories suggest that action simulation processes are essential for interindividual action coordination (Bekkering et al., 2009). Joint action partners have to coordinate the timing of their actions in a shared common space. Motor simulation of another person’s action plan could enable one to predict the intentional, temporal, and spatial aspects of the other’s action, and this information could be used to adjust one’s own action plan (Sebanz and Knoblich, 2009). More specifically, according to Wilson and Knoblich (2005), we use our own motor representations as internal models to “emulate” the trajectories of an observed conspecific’s action. This “emulator” could immediately provide information about actual movements and probable future events. Predictive processes should play an especially important role in real-life, complex joint action tasks. For instance, jointly acting in turns without delays demands that the interaction partners adjust their movements to the temporal aspects of the other’s actions. Therefore, simulating the other’s actions to anticipate the end of the partner’s movements seems to be a relevant process in real-life joint action. Simulation processes are proposed to take place in brain regions which are involved both in action execution and action observation and can therefore directly link observed motor actions with internal models of action (Gallese, 2006; Grafton, 2009); two such areas that have been identified are the inferior frontal gyrus (IFG) and the inferior parietal lobule (IPL; for a review see Iacoboni and Dapretto, 2006). In an fMRI study on joint action, Newman-Norlund et al. (2008) found increased activation in a network including the IFG and the IPL when participants acted together compared to when they acted alone. They suggested that this activation pattern may reflect motor simulation of the other’s action and its integration into the own action system (for a critical discussion of the integration aspect, see Kokal et al., 2009; Kokal and Keysers, 2010). To differentiate action simulation from other processes which take place in the IFG and IPL, it has been suggested to map areas which are activated both during action execution and action observation (Kokal et al., 2009). So far, assessing brain activation in real-life joint action situations and scrutinizing if processes like action simulation can be uncovered during naturalistic tasks has remained a methodological challenge.

In the present study, we introduce a novel joint action paradigm that enables the assessment of brain activation during real-life joint action tasks. We aimed to explore if fNIRS is capable of revealing cerebral effects of joint action in natural tasks. Moreover, we explored whether our new paradigm would allow us to depict the involvement of action simulation in real-life joint action and to uncover its role in different types of every-day joint action tasks.

We used fNIRS to measure participants’ brain activation while they performed table setting tasks, either alone or together with a partner sitting opposite. To determine if it is generally possible to reveal activation differences between solo action and joint action in a complex, real-life setting with fNIRS, we compared the brain activation elicited by these tasks. In the solo action task, participants had to arrange tableware alone. In the joint action task, participants had to take turns with their partner to arrange the tableware. The number and duration of sub-actions, the motor effectors (right hand and arm), and the movement path of the object-related grasping and displacing were identical in the solo action and joint action tasks. In the Section “Materials and Methods” we describe in detail how this was achieved. We predicted increased cerebral activation during the joint action condition compared to the solo action condition, a result that would indicate that effects of real-life joint action can be assessed with fNIRS.

Secondly, we wanted to examine if the activation differences between joint action and solo action identified in this real-life setting reflect simulation processes. For this purpose, we included an observation task, in which participants watched the partner arrange the tableware set. Assuming that action simulation processes are necessary for interindividual action coordination, we predicted that areas relevant to real-life joint action would be activated during both the observation condition and the solo action condition. This outcome would support the involvement of action simulation processes during real-life joint action.

Finally, we aimed to compare various types of real-life joint action which differed in their demand on simulation processes. Consistent with the motor simulation hypotheses described above, we assumed that action types with high demand in temporal–spatial coordination, e.g., our alternating table setting, would be indicative of simulation processes. However, in real-life, not all joint action situations demand the coordination of alternating movements, which involves the simulation of action plans differing in time and space. For example, a task like lifting, carrying, and placing one and the same object with a partner requires simultaneous movement of the interaction partners. The trajectories of one’s own movements and the partner’s movements are comparable and may therefore demand less action simulation than the coordination of actions with different trajectories. Furthermore, by manipulating the same object, joint action partners receive immediate proprioceptive feedback of the partner’s execution of the simultaneous movement, which additionally may reduce simulation affordances. In such joint action tasks, simulation of the other’s action may take place to a lesser extent than during joint action tasks involving alternating movements. To investigate the involvement of action simulation processes in real-life tasks requiring simultaneous movements, we introduced an additional joint action condition, in which participants had to set the table by manipulating each object together with a partner. This joint action task was identical to the solo action task and the alternating joint action task with regard to the number and duration of sub-actions and with regard to the motor effectors of the participant. Furthermore, in the simultaneous joint action task and in the alternating joint action task, the movements of the interaction partner were identical. Therefore, it served as a control condition. If the activation differences in simulation areas exclusively reflect observation of the experimenter’s movements (which were identical in the two joint action conditions but absent in the solo action condition) then we would expect no activation differences between the two types of joint action conditions. If the activation differences in simulation areas reflect the involvement of simulation processes then we would expect that the demand for action simulation (solo action < simultaneous joint action < alternating joint action) would be reflected by the intensity of activation in the brain areas identified as relevant for action simulation during joint action.

Materials and Methods

Participants

Seventeen healthy participants (eight female and nine male) with a mean age of 23.5 years (SD = 3.3) participated in this study. All participants were right-handed and had normal or corrected-to-normal vision. They were recruited via advertisements placed in a local Internet portal and were compensated at the rate of 10 Euros per hour. All subjects were informed of the nature of the experiment and of the operating mode of the NIRS instrument before giving their written consent. No participant had a known history of any neurological or psychiatric disorder. The study was approved by the local Ethics Committee and all procedures involved were in accordance with the latest version of the Declaration of Helsinki. All participants gave written informed consent after comprehensive explanation of the experimental procedures.

Procedure

The participant and the interaction partner (experimenter) executed object-related grasping and displacing movements with their right hands; they arranged tableware sets that consisted of five objects on a table. The brain activation of the participant was measured by fNIRS. The participant was seated at a table with a black surface opposite the experimenter. On the table were two paper tableware sets, a yellow one on the participant’s side of the table and a white one on the experimenter’s side. White marks on the tabletop assigned the potential positions of the tableware (for details, see Figure 1A). Every tableware set consisted of five objects: a large plate, a small plate, a napkin, a fork, and a cup. This was also the order in which the tableware set was to be moved to the opposite side of the table. Returning the tableware set to its original side, the cup was moved first and the reverse order was followed. The order in which the objects had to be manipulated and the resulting target arrangement conformed to table setting conventions (where the large plate, small plate, napkin, and fork are stacked, and the cup is placed on the side) and was therefore intuitive and easy to perform.

FIGURE 1

Figure 1. Schematic illustration of the experimental setting. Displayed are the possible locations of the tableware after one joint action block (A) and an example section of the experimental pseudo-randomized block design (B).

The four experimental conditions (solo action condition = SA; joint action condition with alternating movements = JA; observation condition = OBS; joint action condition with simultaneous movements = JA_sim) were identical with regard to the number of movement sequences for the participant (five object-related movements/condition), the objects manipulated (five different pieces of tableware/condition), the types of movement (grasping and displacing of the objects with the right hand), the units each movement was composed of (1: reaching, 2: grasping, 3: lifting, 4: transporting, 5: placing the object), and the setting order of tableware (1: large plate, 2: small plate, 3: napkin, 4: fork, and 5: cup and vice versa). In the SA condition, the participant had to manipulate the yellow tableware set alone. This meant that the participant first had to move the large yellow plate, then the small yellow plate, then the yellow napkin, etc., until all five objects of the tableware set were at the marked positions on the other side of the table, resulting in five object-related grasping and displacing movements executed by the participant. In the JA condition, the participant and the experimenter had to manipulate the yellow and white tableware sets together by alternating movements. That meant that the participant first had to move the large yellow plate, and the experimenter then had to move the small yellow plate, followed by the participant moving the yellow napkin, etc. The process was then repeated with the white tableware set, until all pieces of both tableware sets were moved across the table, resulting again in five object-related grasping and displacing movements executed by the participant. The movement coordination had to be executed as smoothly as possible without any delays in arranging the tableware so that the five movements of the participant had the same timing and duration as the movements required in the SA task. This was trained and controlled in practice trials (see below). In the OBS condition the experimenter arranged the white tableware set and the participant was instructed to observe each single actions of the experimenter as accurately as possible. In the JA_sim condition, the participant and the experimenter had to manipulate the yellow tableware set together by simultaneous movements. This meant that the participant and the experimenter had to reach, grasp, lift, transport, and place the yellow large plate cooperatively, and then they had to reach, grasp, lift, transport, and place the yellow small plate cooperatively, etc., until all five objects of the yellow tableware set were at the marked positions on the other side of the table, resulting in five object-related grasping and displacing movements executed by the participant. As in the JA task, prior training ensured that the five movements of the participant had the same timing and duration as the movements required in the SA task. Furthermore, video recordings of the SA, the JA, and the JA_sim conditions were spot-checked with regard to the comparability of the grasping and displacing movements which had to be executed by the participant.

We used a pseudo-randomized block design. Each of the four conditions was repeated eight times, resulting in 32 blocks. After the instruction of the tasks, each subject completed eight practice trials before fNIRS measurements began. During the practice trials, it was ensured that the grasping and displacing movements executed by the participant in the SA, JA, and JA_sim conditions had similar timing. This was done by checking that the movement phase of each block had the same length (about 11 s) in every condition. Before each block, there was a 15-s resting phase in which the participants were instructed to sit calmly and relax while focusing on the middle of the table. An example of the pseudo-randomized design is shown in Figure 1B. Randomization was restricted since every condition was immediately repeated once. The first block consisted of moving the tableware set across the table, the second block consisted of returning the tableware set to its original side. The start and type of condition were announced with cues provided by the experimenter’s voice.

Data Acquisition

Presentation Software 12.2 (Neurobehavioral Systems, Albany, CA, USA) was used to cue the participants at the beginning of each trial and to send triggers to the NIRS machine. The monitor displaying the cues was visible only to the experimenter, who then relayed the cues to the participants.

Cerebral oxygenation changes were recorded by a Hitachi ETG-4000 (Hitachi Medical. Co., Kashiwa, Japan) with a sampling frequency of 10 Hz and a continuous wave system using two different wavelengths (830 ± 20 nm for [oxy-Hb] and 695 ± 20 for [deoxy-Hb]). The basics of fNIRS are described in detail elsewhere (e.g., Hoshi, 2003; Obrig and Villringer, 2003). A lock-in technique was applied to differentiate between wavelengths. From the measured wavelengths the concentrations of [oxy-Hb] and [deoxy-Hb] were calculated based on a modified Beer–Lambert Law (see e.g., Villringer and Chance, 1997; Obrig and Villringer, 2003).

Functional near-infrared spectroscopy with 52 channels is not able to map the whole brain. We chose to measure the prefrontal and parietal areas of the left hemisphere because brain activation in response to executed and observed right-handed reaching and grasping movements is stronger in the left hemisphere (Filimon et al., 2007). Furthermore, the right hemisphere is known to be dominantly involved in spatial and attentional processes. Since attentional and spatial aspects are difficult to control in a real-life task, measuring the left hemisphere ensured the highest possible reduction of the influence of such confounds.

With a 3 × 11 optode probe set consisting of 16 photo-detectors and 17 light emitters, 52 channels were measured. The optodes were affixed to a probe set with an inter-optode distance of 3 cm covering an area of ~6 cm × 30 cm. The probe set was fastened to the participant’s head by elastic straps. As illustrated in Figure 2A, the probe set was mounted on the scalp with its lowest-row center optode at EEG position T3 of the international 10–20 system (Jasper, 1958). Thus, the probe set extended toward the EEG positions Fz and Oz. For horizontal fixation, the lower edge of the probe set was fixed 1 cm above the nasion.

FIGURE 2

Figure 2. Probe set definition (A) and t-plots for the contrast JA > SA for [oxy-Hb] and [deoxy-Hb] separately (B). Gray shaded channels denote the channels excluded from interpretation for methodological reasons (see Section “Results”).

Data Analyses and Statistics

In each individual data set, the high-frequency portion of the signal was removed by calculating a moving average with a time window of 5 s. To remove baseline drifts, a baseline correction was conducted based on the 10-s before each block. The linear trend between the baseline of one block and the baseline of the following block was then calculated for each trial, and thereafter, the linear trend of the measured data was corrected. As a result, data reached the same level. Afterward, the data were averaged across all blocks for each condition. This method of preprocessing and the following statistical analyses have been applied in previous fNIRS studies (Herrmann et al., 2006, 2007, 2008a).

Neural activation typically leads to an increase in oxygenated hemoglobin [oxy-Hb] and a decrease in deoxygenated hemoglobin [deoxy-Hb] (e.g., Villringer and Chance, 1997). To reveal activation differences between conditions (JA > SA, JA_sim > SA, JA > JA_sim), we first averaged the concentrations of [oxy-Hb] and [deoxy-Hb] for the baseline (10 s right before the start of the block) and the activation phase (the first 13 s of the block, as hemodynamic response of an 11-s movement sequence is typically inside this time window, which was also proved by inspection of the time curves of the conditions) for each channel and condition separately. Then we calculated the relative concentrations of [oxy-Hb] and [deoxy-Hb] by subtracting the baseline from the active periods for each channel and condition. These relative data sets were further analyzed using one-tailed t-tests for paired samples. To depict the hemodynamic response, positive t-values were assigned to an increase in the chromophores and negative t-values were assigned to a decrease in the chromophores.

For all tests consisting of multiple comparisons, we corrected the 5% alpha level according to the false discovery rate method used in fMRI and NIRS studies (FDR_BH according to Benjamini and Hochberg, 1995; Singh and Dan, 2006).

Probabilistic Registration of FNIRS Channels to MNI Space

We applied a virtual registration method developed by Tsuzuki et al. (2007) to register fNIRS data to MNI standard brain space. This method has been successfully applied in other fNIRS studies (Herrmann et al., 2008b; Hofmann et al., 2008). It uses structural information from an anatomical database (Okamoto et al., 2004a; Jurcak et al., 2005) to provide estimates of the channel positions in a standardized stereotaxic 3D brain atlas (Montreal Neurological Institute coordinate system, MNI, Collins et al., 1994; cf. Tsuzuki et al., 2007). It also estimates the spatial uncertainty due to inter-subject variability in the channel locations. The estimated locations were anatomically labeled by means of a Matlab function using anatomical labels from the brain atlas of Tzourio-Mazoyer et al. (2002).

Results

To determine if it is generally possible to reveal activation differences between solo and joint actions in a complex, real-life setting with fNIRS, we first had to identify areas that were more activated during joint action than during solo action. Thus, we first calculated the contrast between the JA and SA conditions. Results revealed substantial activation differences between joint and solo actions: a significantly higher [oxy-Hb] increase during the JA condition in comparison to the SA condition was found in 15 channels, t(16) > 2.431; p < 0.0136; t_max = 4.5. For t-plots showing the statistical differences, see Figure 2B. These channels corresponded to areas in the left IPL, the orbitofrontal cortex, the medial and superior temporal gyrus, and the occipital cortex. The analyses of [deoxy-Hb] for the JA < SA contrast did not reveal any significant decreases (t_min = −3.9, p = 0.0006 at channel 33) that survived the FDR correction. Instead, there were five channels in occipital regions (channels 31, 40, 41, 42, and 52), which revealed a higher [deoxy-Hb] concentration for the SA condition than for the JA condition (t > 1.0) while also depicting significant effects for [oxy-Hb] (for details see Table 1). Because neural activation typically leads to an increase in [oxy-Hb] and a decrease in [deoxy-Hb] (e.g., Villringer and Chance, 1997), channels showing such inconsistencies should be interpreted carefully (Obrig and Villringer, 2003). A channel with a positive difference between the conditions concerning [oxy-Hb] should demonstrate a corresponding negative difference for [deoxy-Hb]. Irregularities may be a sign of motion artifacts caused by loose positioning of the optode. A well-known origin of this problem is too much hair at the probe area, which can hinder securing the position of the optode. Channels at the problematic positions were excluded from interpretation (channels shaded gray in Figure 2). In this study, only the 10 channels that showed a significant positive effect for [oxy-Hb] and a negative or at least neutral (t < 1.0) difference concerning the [deoxy-Hb] concentration were interpreted (channels 7, 8, 17, 19, 28, 30, 34, 38, 39, 49; for detailed anatomical locations see Table 1).

TABLE 1

Table 1. Channel t-values and anatomical labels for the contrast (JA > SA).

Secondly, we examined which of the channels that were more active during joint action than during solo action were additionally activated during both action execution and action observation. To this end, we analyzed which of the relevant channels also showed statistically significant activation in the SA and OBS conditions compared to baseline. Since the examination of [deoxy-Hb] did not reveal any significant results in the statistical comparisons between conditions, we conducted the analysis for SA and OBS based on the [oxy-Hb] concentration. We first compared the average concentration of the OBS baseline with the average concentration of the OBS activation phase for all channels by using one-tailed t-tests. The same procedure was then applied to the SA condition. This procedure allowed us to identify joint action areas that were additionally sensitive to action observation and action execution. Four channels in the IPL (channels 8, 17, 19, 39; for detailed anatomical locations see Table 1) fulfilled the three criteria [JA > SA], [OBS > baseline], and [SA > baseline], p_FDR < 0.05 (see Figure 3).

FIGURE 3

Figure 3. Time course of the hemodynamic responses for OBS, SA, and JA exemplified for those areas that were involved in real-life joint action and that demonstrated the functional properties of action simulation areas [(JA > SA), (OBS > baseline), and (SA > baseline), p_FDR < 0.05]. The red dots mark the anatomical location of the relevant channels.

Finally, we analyzed whether the intensity of activation in the four simulation channels revealed in the preceding analyses (channels 8, 17, 19, 39) varied according to the action type and its potential demand on action simulation. As shown in Figure 4, the mean concentration was highest in the JA condition, followed by the JA_sim condition, and lowest in the SA condition in all four of these channels. Thus, a gradual increase in activation was observed from solo action to alternating joint action. The JA_sim condition did not differ significantly from either of the other two conditions, such that neither the JA_sim > SA contrast (t_max = 2.11, p = 0.025 at channel 17) nor the JA_sim < JA contrast (t_max = 2.28, p = 0.018 at channel 17) obtained significant [oxy-Hb] concentration differences which survived the FDR correction. This suggests that the JA_sim condition evoked an intermediate level of activation in the action simulation channels.

FIGURE 4

Figure 4. Mean values (activation phase – baseline) with SD for [oxy-Hb] over all subjects for the four channels in the IPL which show the properties of action simulation areas. Displayed are the conditions SA, JA_sim, and JA.

Discussion

The main goal of this study was to introduce a novel joint action paradigm capable of exploring the cerebral basis of joint action in real-life situations. Brain activation was measured using fNIRS while participants were involved in real-life tasks (table setting), which they had to perform alone or in coordination with a partner sitting opposite to them. We wanted to know if this novel fNIRS paradigm demonstrated effects of real-life joint action by contrasting brain activation elicited by joint action with brain activation elicited by solo action. Furthermore, we were interested in determining if this paradigm would enable us to identify the involvement of action simulation processes in real-life joint action and whether it could uncover the role of these processes different types of real-life joint action.

Assessing Brain Activation during Real-Life Joint Action

The comparison of the joint action condition with the solo action condition revealed areas showing greater activation (higher [oxy-Hb]-concentration) during joint action. Independently of the precise locations of these areas (see next Section), this finding demonstrates – to our knowledge – for the first time that it is possible to identify effects of every-day joint action tasks in a real-life setting using fNIRS. Due to the characteristics of fNIRS, we were able to assess brain activation of people sitting opposite one another and directly interacting with each other. The jointly acting dyads coordinated their movements in time and space to reach a common goal, which in our case, was the arrangement of tableware sets. This task consisted of a larger series of complex sub-actions, including distinct object-related grasping and displacing movements. With this study, we extend the usability of fNIRS from the investigation of single individuals executing complex motor actions (Okamoto et al., 2004b; Suzuki et al., 2004; Leff et al., 2008) or being involved in a face-to-face conversation (Suda et al., 2010) to the investigation of individuals who directly coordinate their complex actions with those of another individual. The practicability of this new approach is important because it provides the basis for future studies proving if joint action effects of previous fMRI studies with simple motor tasks (Sebanz et al., 2007; Newman-Norlund et al., 2008; Kokal et al., 2009) can be generalized to situations more comparable to real-life.

Action Simulation Processes during Real-Life Joint Action

Among the areas that were more strongly activated during joint action than during solo action, areas located in the IPL showed an overlap of activation during the action observation condition and the solo action condition. It has been proposed that this close link between the processing of observed and of executed actions occurs in areas that are associated with action simulation processes (Gallese and Goldman, 1998). Thus, our findings confirm the notion that action coordination in real-life situations is supported by simulating the action of the partner.

Notably, our conclusion is not derived from the mere localization of an activation difference between joint action and solo action. Rather, consistent with Kokal et al. (2009), we mapped the functional properties of relevant areas by analyzing the activation overlap during solo action and action observation. In our study, the areas denoting this functional overlap were localized to the IPL. This overlap is consistent with other neuroimaging studies, which showed that areas including the IPL are associated both with action observation and action execution (for a review see Iacoboni and Dapretto, 2006).

Moreover, the brain areas in the IPL identified in the present study as relevant for action simulation during joint action are in accordance with the finding and conclusion of a previous fMRI study on joint action: Newman-Norlund et al. (2008) also found increased activation in areas including the IPL when comparing joint action with solo action in a virtual bar-balancing task. Participants interacted with a person outside of the scanner by squeezing force pads to control a bar displayed on a computer screen. In our joint action task, participants had to take turns to arrange a tableware set. Our findings demonstrate that the involvement of action simulation processes in joint action can be generalized from the coordination of simplified and non-naturalistic motor tasks to the coordination of more complex movements in real-life settings.

A limitation of the method used in this study is that an exact localization of the activation differences between joint action and solo action, as would be possible with fMRI, is not achievable (Obrig and Villringer, 2003). Nevertheless, due to a probabilistic mapping method (Tsuzuki et al., 2007), it is possible to determine the most likely anatomic locations of the changes in cortical activation in a standardized, stereotaxic 3D brain atlas (Montreal Neurological Institute coordinate system, MNI, Collins et al., 1994), which enhance the comparability of our results to those of fMRI studies.

Action Simulation Processes in Different Types of Real-Life Joint Action

In this study, we also explored if types of real-life joint action that differ according to their respective demand on simulation processes elicited a corresponding activation pattern in simulation areas. We focused on the channels that are supposed to be involved in action simulation during joint action with alternating movements and analyzed the degree to which they were also involved in the joint action task with simultaneous movements. The magnitude of activation increased numerically with the respective demand for motor simulation (SA < JA_sim < JA). Statistical differences in activation were demonstrated between the JA and the SA condition. However, the overall magnitude of changes in activity between the three conditions was rather small (see Figure 4), so it is not surprising that activation in the intermediate condition, the simultaneous joint action task, did not differ significantly from that elicited by the solo action task or by the alternating joint action task.

One might hypothesize that the activation differences between solo action and joint action in simulation areas merely reflect the observation of the experimenter’s actions, since the experimenter moves in the joint action condition, but not in the solo action condition. The simultaneous joint action condition served as a control condition for this issue. In both joint action tasks, the visible movements of the experimenter were identical but the tasks differed with regard to the simulation affordances. Therefore the finding of an activation gradient (SA < JA_sim < JA) indicates that the movement of the experimenter cannot solely be responsible for an activation difference between the joint action condition and the solo action condition. Hence, in our study, the activation difference JA > SA does not merely reflect direct action observation, but rather action simulation processes.

The finding of an activation gradient is in agreement with the suggestion that we simulate the trajectories of an observed conspecific’s action to predict its future effects (Wilson and Knoblich, 2005). This was most important in the alternating joint action condition. Coordinating the alternating movements as smoothly as possible without any delays demanded the adjustment of one’s movements to the temporal aspects of the other’s actions. Therefore, the highest degree of activation in simulation areas in the alternating joint action task may reflect the demand of simulating the trajectories of the other’s actions to anticipate the end of the partner’s movements. This aspect was less involved in the simultaneous joint action task. The interaction partners did not have to combine the spatial and temporal properties of two different action plans as they did in the alternating joint action task. The trajectories were more comparable concerning temporal and spatial aspects. Additionally, by manipulating the same object, the joint action partners received immediate proprioceptive feedback from the partner’s actions which further reduced the simulation affordances. Lastly, simulation demands were lowest in the solo action task, as one’s own movements did not depend on the temporal and spatial aspects of the partner’s actions. Revealing a corresponding activation gradient, the present results suggest that the joint action areas specified in the present study are sensitive to action simulation demands.

Conclusion

To date, due to the restrictions of common imaging methods, neurocognitive studies on joint action have been limited to the measurement of brain activation during simple motor tasks executed under artificial circumstances. The current experiment was conducted to explore the cerebral basis of interindividual action coordination in an experimental setting comparable to real-life. The interaction partners were sitting opposite to each other and coordinated complex movements in table setting tasks. We showed that relevant processes like action simulation can be identified with this new paradigm. The plausibility of our results and their concordance with previous joint action studies and theories indicate that fNIRS is a valid approach for exploring functional correlates of joint action in real-life settings. Our approach offers new possibilities for research on joint action. Because fNIRS is a non-invasive method with minimal constraints for the experimental environment, it has the potential to assess brain activation in children (Schroeter et al., 2004; Shimada and Hiraki, 2006). Recent literature has focused on developmental aspects of joint action coordination (Carpenter, 2009; Meyer et al., 2010, 2011). Using fNIRS to measure brain activation in children involved in joint action tasks may reveal new insights. In addition, due to the high temporal resolution of fNIRS, future studies may also shed light on the temporal aspects of interpersonal coordination, for example, by simultaneously measuring brain activation of two jointly acting individuals.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This work was partially funded by the DFG, center of excellence 277 “Cognitive Interaction Technology” (CITEC, authors Johanna Egetemeir, and Prisca Stenneken).

References

Bekkering, H., de Bruijn, E. R. A., Cuijpers, R. H., Newman-Norlund, R. D., Schie, H. T. V., and Meulenbroek, R. G. J. (2009). Joint action: neurocognitive mechanisms supporting human interaction. Top. Cogn. Sci. 1, 340–352.

CrossRef Full Text

Benjamini, Y., and Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. B 57, 289–300.

Carpenter, M. (2009). Just how joint is joint action in infancy? Top. Cogn. Sci. 1, 380–392.

CrossRef Full Text

Collins, D. L., Neelin, P., Peters, T. M., and Evans, A. C. (1994). Automatic 3D intersubject registration of MR volumetric data in standardized Talairach space. J. Comput. Assist. Tomogr. 18, 192–205.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Filimon, F., Nelson, J. D., Hagler, D. J., and Sereno, M. I. (2007). Human cortical representations for reaching: mirror neurons for execution, observation, and imagery. Neuroimage 37, 1315–1328.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gallese, V. (2006). Intentional attunement: a neurophysiological perspective on social cognition and its disruption in autism. Multiple perspectives on the psychological and neural bases of understanding other people’s behavior. Brain Res. 1079, 15–24.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gallese, V., and Goldman, A. (1998). Mirror neurons and the simulation theory of mind-reading. Trends Cogn. Sci. 2, 493–501.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Grafton, S. T. (2009). Embodied cognition and the simulation of action to understand others. Ann. N. Y. Acad. Sci. 1156, 97–117.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hari, R., and Kujala, M. V. (2009). Brain basis of human social interaction: from concepts to brain imaging. Physiol. Rev. 89, 453–479.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Herrmann, M. J., Langer, J. B., Jacob, C., Ehlis, A. C., and Fallgatter, A. J. (2008a). Reduced prefrontal oxygenation in Alzheimer disease during verbal fluency tasks. Am. J. Geriatr. Psychiatry 16, 125–135.

CrossRef Full Text

Herrmann, M. J., Woidich, E., Schreppel, T., Pauli, P., and Fallgatter, A. J. (2008b). Brain activation for alertness measured with functional near-infrared spectroscopy (fNIRS). Psychophysiology 45, 480–486.

Herrmann, M. J., Walter, A., Ehlis, A. C., and Fallgatter, A. J. (2006). Cerebral oxygenation changes in the prefrontal cortex: effects of age and gender. Neurobiol. Aging 27, 888–894.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Herrmann, M. J., Walter, A., Schreppel, T., Ehlis, A. C., Pauli, P., Lesch, K. P., and Fallgatter, A. J. (2007). D4 receptor gene variation modulates activation of prefrontal cortex during working memory. Eur. J. Neurosci. 26, 2713–2718.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hofmann, M. J., Herrmann, M. J., Dan, I., Obrig, H., Conrad, M., Kuchinke, L., Jacobs, A., and Fallgatter, A. J. (2008). Differential activation of frontal and parietal regions during visual word recognition: an optical topography study. Neuroimage 40, 1340–1349.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hoshi, Y. (2003). Functional near-infrared optical imaging: utility and limitations in human brain mapping. Psychophysiology 40, 511–520.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Iacoboni, M., and Dapretto, M. (2006). The mirror neuron system and the consequences of its dysfunction. Nat. Rev. Neurosci. 7, 942–951.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Jasper, H. H. (1958). Report of the committee on methods of clinical examination in electroencephalography. Electroencephalogr. Clin. Neurophysiol. 10, 371–375.

Jurcak, V., Okamoto, M., Singh, A., and Dan, I. (2005). Virtual 10-20 measurement on MR images for inter-modal linking of transcranial and tomographic neuroimaging methods. Neuroimage 26, 1184–1192.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kokal, I., Gazzola, V., and Keysers, C. (2009). Acting together in and beyond the mirror neuron system. Neuroimage 47, 2046–2056.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kokal, I., and Keysers, C. (2010). Granger causality mapping during joint actions reveals evidence for forward models that could overcome sensory-motor delays. PLoS ONE 5, e13507. doi: 10.1371/journal.pone.0013507

CrossRef Full Text

Leff, D. R., Elwell, C. E., Orihuela-Espina, F., Atallah, L., Delpy, D. T., Darzi, A. W., and Yang, G. Z. (2008). Changes in prefrontal cortical behaviour depend upon familiarity on a bimanual co-ordination task: an fNIRS study. Neuroimage 39, 805–813.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Meyer, M., Bekkering, H., Paulus, M., and Hunnius, S. (2010). Joint action coordination in 2 1/2- and 3-year old children. Front. Hum. Neurosci. 4:220. doi: 10.3389/fnhum.2010.00220

CrossRef Full Text

Meyer, M., Hunnius, S., Elk, M., Ede, F., and Bekkering, H. (2011). Joint action modulates motor system involvement during action observation in 3-year-olds. Exp. Brain. Res. 211, 581–592.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Newman-Norlund, R. D., Bosga, J., Meulenbroek, R. G. J., and Bekkering, H. (2008). Anatomical substrates of cooperative joint-action in a continuous motor task: virtual lifting and balancing. Neuroimage 41, 169–177.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Newman-Norlund, R. D., Noordzij, M. L., Meulenbroek, R. G. J., and Bekkering, H. (2007). Exploring the brain basis of joint action: co-ordination of actions, goals and intentions. Soc. Neurosci. 2, 48–65.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Newman-Norlund, R. D., Schie, H. T. V., van Zuijlen, A. M. J., and Bekkering, H. (2007b). The mirror neuron system is more active during complementary compared with imitative action. Nat. Neurosci. 10, 817–818.

CrossRef Full Text

Obrig, H., and Villringer, A. (2003). Beyond the visible – imaging the human brain with light. J. Cereb. Blood Flow Metab. 23, 1–18.

Pubmed Abstract | Pubmed Full Text

Okamoto, M., Dan, H., Sakamoto, K., Takeo, K., Shimizu, K., Kohno, S., Oda, I., Isobe, S., Suzuki, T., Kohyama, K., and Dan, I. (2004a). Three-dimensional probabilistic anatomical cranio-cerebral correlation via the international 10-20 system oriented for transcranial functional brain mapping. Neuroimage 21, 99–111.

CrossRef Full Text

Okamoto, M., Dan, H., Shimizu, K., Takeo, K., Amita, T., Oda, I., Konishi, I., Sakamoto, K., Isobe, S., Suzuki, T., Kohyama, K., and Dan, I. (2004b). Multimodal assessment of cortical activation during apple peeling by NIRS and fMRI. Neuroimage 21, 1275–1288.

CrossRef Full Text

Schroeter, M. L., Zysset, S., Wahl, M., and Cramon, D. Y. V. (2004). Prefrontal activation due to Stroop interference increases during development – an event-related fNIRS study. Neuroimage 23, 1317–1325.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sebanz, N., Bekkering, H., and Knoblich, G. (2006). Joint action: bodies and minds moving together. Trends Cogn. Sci. 10, 70–76.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sebanz, N., and Knoblich, G. (2009). Prediction in joint action: what, when, and where. Top. Cogn. Sci. 1, 353–367.

CrossRef Full Text

Sebanz, N., Rebbechi, D., Knoblich, G., Prinz, W., and Frith, C. D. (2007). Is it really my turn? An event-related fMRI study of task sharing. Soc. Neurosci. 2, 81–95.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Shimada, S., and Hiraki, K. (2006). Infant’s brain responses to live and televised action. Neuroimage 32, 930–939.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Singh, A., and Dan, I. (2006). Exploring the false discovery rate in multichannel NIRS. Neuroimage 33, 542–549.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Suda, M., Takei, Y., Aoyama, Y., Narita, K., Sato, T., Fukuda, M., and Mikuni, M. (2010). Frontopolar activation during face-to-face conversation: an in situ study using near-infrared spectroscopy. Neuropsychologia 48, 441–447.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Suzuki, M., Miyai, I., Ono, T., Oda, I., Konishi, I., Kochiyama, T., and Kubota, K. (2004). Prefrontal and premotor cortices are involved in adapting walking and running speed on the treadmill: an optical imaging study. Neuroimage 23, 1020–1026.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Tsuzuki, D., Jurcak, V., Singh, A., Okamoto, M., Watanabe, E., and Dan, I. (2007). Virtual spatial registration of stand-alone fNIRS data to MNI space. Neuroimage 34, 1506–1518.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., Mazoyer, B., and Joliot, M. (2002). Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage 15, 273–289.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Villringer, A., and Chance, B. (1997). Non-invasive optical spectroscopy and imaging of human brain function. Trends Neurosci. 20, 435–442.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Wilson, M., and Knoblich, G. (2005). The case for motor involvement in perceiving conspecifics. Psychol. Bull. 131, 460–473.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text