**A MULTIDISCIPLINARY APPROACH TO MOTOR LEARNING AND SENSORIMOTOR ADAPTATION**

**Topic Editors Rachael D. Seidler and Sean Kevin Meehan**

#### *FRONTIERS COPYRIGHT STATEMENT*

© Copyright 2007-2014 Frontiers Media SA. All rights reserved.

All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.

The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.

Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.

Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.

As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.

All copyright, and all rights therein, are protected by national and international copyright laws.

The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.

Cover image provided by Ibbl sarl, Lausanne CH

**ISSN** 1664-8714 **ISBN** 978-2-88919-365-3 **DOI** 10.3389/978-2-88919-365-3

## *ABOUT FRONTIERS*

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

## *FRONTIERS JOURNAL SERIES*

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing.

All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

## *DEDICATION TO QUALITY*

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view.

By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

## *WHAT ARE FRONTIERS RESEARCH TOPICS?*

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area!

Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

## **A MULTIDISCIPLINARY APPROACH TO MOTOR LEARNING AND SENSORIMOTOR ADAPTATION**

Topic Editors: **Rachael D. Seidler,** University of Michigan, USA **Sean Kevin Meehan,** University of Michigan, USA

The plasticity of the living matter of our nervous system, in short, is the reason why we do a thing with difficulty the first time, but soon do it more and more easily, and finally, with sufficient practice, do it semi-mechanically, or with hardly any consciousness at all." --William James, 1899. It is over 100 years since James described the acquisition of skill. How much, or how little, have recent advances in science changed the way we think about skill learning? What theories and ideas do we still hold dear and which have we discarded?

Advances in neuroimaging over the past 20 years have provided insight into the dynamic neural processes underlying human motor skill acquisition, focusing primarily on brain networks that are engaged during early versus late stages of learning. What has been challenging for the field is to tightly link these shifting neural processes with what is known about measureable behavioral changes and strategic processes that occur during learning. The complex nature of behavior and strategy in motor learning often result in a trade-off between experimental control and external validity.

The articles assembled for this special issue cut across a number of related disciplines and investigate skill learning across multiple domains. The broad range of theoretical, analytical and methodological approaches offer complementary approaches that can be exploited to develop integrated models of skilled learning. It is our hope that this collection inspires innovation and collaboration amongst researchers, and thereby, accelerates development of societally relevant translational paradigms.

# Table of Contents


*126 Graph Network Analysis of Immediate Motor-Learning Induced Changes in Resting State BOLD*

S. Sami and R. C. Miall

*140 Cerebellar Contributions to Visuomotor Adaptation and Motor Sequence Learning: An ALE Meta-Analysis*

Jessica A. Bernard and Rachael D. Seidler


Masaki O. Abe and Dagmar Sternad

*179 Mind Wandering and Motor Control: Off-Task Thinking Disrupts the Online Adjustment of Behavior* Julia W. Y. Kam, Elizabeth Dao, Patricia Blinn, Olav E. Krigolson, Lara A. Boyd and

Todd C. Handy


## Introduction to the special topic: a multidisciplinary approach to motor learning and sensorimotor adaptation

#### *Rachael D. Seidler <sup>1</sup> \* and Sean K. Meehan2*

*<sup>1</sup> Psychology, Kinesiology, Neuroscience, Neuromotor Behavior Laboratory, University of Michigan, Ann Arbor, MI, USA*

*<sup>2</sup> Kinesiology, University of Michigan, Ann Arbor, MI, USA*

*\*Correspondence: rseidler@umich.edu*

#### *Edited by:*

*Hauke R. Heekeren, Freie Universität Berlin, Germany*

#### **Keywords: motor learning, sensorimotor adaptation, sequence learning, motor cortex, consolidation**

"The plasticity of the living matter of our nervous system, in short, is the reason why we do a thing with difficulty the first time, but soon do it more and more easily, and finally, with sufficient practice, do it semi-mechanically, or with hardly any consciousness at all." –William James, 1899.

Advances in neuroimaging over the past 20 years have provided insight into the dynamic neural processes underlying human motor skill acquisition, focusing primarily on brain networks that are engaged during early versus late stages of learning. What has been challenging for the field is to tightly link these shifting neural processes with what is known about measureable behavioral changes and strategic processes that occur during learning. The complex nature of behavior and strategy in motor learning often result in a trade-off between experimental control and external validity. Researchers in different disciplines have employed varying approaches to understand motor learning but with relatively little crosstalk. Here, we bring together a set of papers which investigate skill learning spanning multiple domains.

There are several striking and unique features about the papers assembled for this special issue. One is the broad range of investigative techniques brought to bear on the problem of understanding skill acquisition, including cutting edge analytical approaches (Abe and Sternad, 2013; Sami and Miall, 2013), metrics of brain structure and function (Kam et al., 2012; Steele et al., 2012; Bernard and Seidler, 2013; Gentili et al., 2013; Wadden et al., 2013), behavioral experiments with carefully crafted conditions (Armstrong et al., 2013; Kitago et al., 2013; Leow et al., 2013; Nemeth et al., 2013; Taylor and Ivry, 2013), and comprehensive reviews which put forth new theories and novel viewpoints for interpretation (Abrahamse et al., 2013; Bock, 2013; Heuer and Sülzenbrück, 2013; King et al., 2013; Ruddy and Carson, 2013; Vadakkan, 2013). We expect that motor scientists will find inspiring new ideas, techniques, approaches, and theories in this collection of articles.

Another important aspect of these papers is that they report on differing types of skill acquisition including practice of a new skill, adaptation to visuomotor distortions, and acquiring new action sequences. For example, Heuer and Sulzenbruck review their findings evaluating how subjects learn the transformation of a sliding first-order level. This has highly practical implications as this tool type is used in minimal access surgery. The sliding first-order level is a type of tool often used in laproscopic surgery; a fulcrum effect at the skin insertion site results in forward hand movements producing backward tool movements. Moreover, linear hand motions result in curved tool tip paths. Taylor and Ivry leverage comparisons of subjects adapting to visuomotor rotations and to visual translational shifts, and report an interaction between the type of perturbation applied and whether targets are presented in a circular or rectilinear arrangement. Interestingly, they observed that generalization of adaptation across the workspace was linked more to the environmental context than to the perturbation type. Steele and colleagues report findings from a multimodal neuroimaging study using their well-characterized temporal motor sequence task, which requires participants to learn both spatial response locations and a temporal rhythm, similar to playing a musical instrument. They report complementary structural and functional changes with learning; the rate of learning was positively correlated with gray matter volume in cerebellar lobules HV and VI. These same regions exhibit decreases in functional activation with training. Finally, Kitago et al. focus on unlearning in an effort to determine whether it represents forgetting of acquired representations or just reverting back to habitual performance. Their findings support that unlearning is not just forgetting, but is rather an active process. This has important implications for individuals who need to learn new ways of performing everyday skills after suffering from injury or neurological insult.

Several of the papers in this special issue also highlight the differing contributions of neurocognitive mechanisms across learning, consolidation and retention. For example, Nemeth et al. assessed skill learning in healthy adults and those with mild cognitive impairment to investigate the role of the hippocampus and medial temporal lobe (MTL) structures in skilled learning. Using the alternating serial response task (ASRT) they report that individuals with MCI, and likely compromised hippocampal/MTL structures, demonstrate a reduced ability to reactivate/recall learned sequences in subsequent blocks of practice. Interestingly, they report that differences in learning disappeared during the second half of a practice block suggesting a differential role for hippocampus/MTL structures across practice even within a block. In a second paper, Wadden et al. evaluated individual variability in the neural networks underlying motor sequence learning in middle aged adults. Comparing initial task performance to that at a delayed retention test following 5 days of continuous tracking practice they report variability in overall measures of implicit sequence specific learning. However, when learning was decomposed into temporal and spatial elements to account for individual variation, improvement in temporal elements were associated with a network of cortical, sub-cortical and cerebellar areas tied to performance instruction stressing speed over accuracy. In a third paper, Abe and Sternad highlight time dependent changes in learning parameters across six days of a virtual ball throwing task. Analyzing both the distribution and temporal structure of variability they demonstrate and model the importance of time scales. These papers demonstrate that understanding changes across the time course of learning, consolidation and retention is crucial to evaluating the contributions of

## **REFERENCES**


(2013). Functional near-infrared spectroscopy-based correlates of prefrontal cortex dynamics during a cognitive-motor executive adaptation task. *Front. Hum. Neurosci.* 7:277. doi: 10.3389/fnhum.2013.00277


neurocognitive mechanisms and needs to be investigated despite the difficulty in undertaking such work.

It is our belief that this assemblage of papers will facilitate an integrative view of motor learning, foster discussion across disciplines, and stimulate collaboration. Such a cross disciplinary focus will help to elucidate the neural and cognitive processes underlying skill learning, and may serve to further accelerate translational paradigms that are grounded in skill learning theory.


*Received: 31 July 2013; accepted: 19 August 2013; published online: 09 September 2013.*

*Citation: Seidler RD and Meehan SK (2013) Introduction to the special topic: a multidisciplinary approach to motor learning and sensorimotor adaptation. Front. Hum. Neurosci. 7:543. doi: 10.3389/fnhum.2013.00543*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2013 Seidler and Meehan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Neural correlates of the age-related changes in motor sequence learning and motor adaptation in older adults

## *Bradley R. King, Stuart M. Fogel , Geneviève Albouy and Julien Doyon\**

*Functional Neuroimaging Unit, Centre de Recherche de l'Institut Universitaire de Gériatrie de Montréal, University of Montreal, Montreal, QC, Canada*

#### *Edited by:*

*Rachael D. Seidler, University of Michigan, USA*

#### *Reviewed by:*

*Stephan P. Swinnen, KU Leuven, Belgium Jordan A. Taylor, Princeton University, USA*

#### *\*Correspondence:*

*Julien Doyon, Functional Neuroimaging Unit, Centre de Recherche de l'Institut Universitaire de Gériatrie de Montréal, University of Montreal, 4545 Chemin Queen Mary, Montreal, QC H3W 1W5, Canada.*

*e-mail: julien.doyon@umontreal.ca*

As the world's population ages, a deeper understanding of the relationship between aging and motor learning will become increasingly relevant in basic research and applied settings. In this context, this review aims to address the effects of age on motor sequence learning (MSL) and motor adaptation (MA) with respect to behavioral, neurological, and neuroimaging findings. Previous behavioral research investigating the influence of aging on motor learning has consistently reported the following results. First, the initial acquisition of motor sequences is not altered, except under conditions of increased task complexity. Second, older adults demonstrate deficits in motor sequence memory consolidation. And, third, although older adults demonstrate deficits during the exposure phase of MA paradigms, the aftereffects following removal of the sensorimotor perturbation are similar to young adults, suggesting that the adaptive ability of older adults is relatively intact. This paper will review the potential neural underpinnings of these behavioral results, with a particular emphasis on the influence of age-related dysfunctions in the cortico-striatal system on motor learning.

**Keywords: aging, motor learning, consolidation, adaptation, procedural memory, neuroimaging, striatum, cerebellum**

## **INTRODUCTION**

The learning of new motor skills, as well as the modification of previously learned skills, is necessary for both the performance of everyday activities and the implementation of neurorehabilitative training programs following brain injury (i.e., stroke). As the average age of the world's population continues to rise, an increased comprehension of the relationship between aging and motor learning will be fundamental to both our understanding of how the motor system functions and how to treat motor deficits. Accordingly, the overarching purpose of this paper is to provide a review of the extant literature investigating motor learning, as well as the associated neural underpinnings, in older adults. To achieve this aim, we will examine the results from research investigating the behavioral and neural correlates of the two most frequently studied motor learning paradigms: motor sequence learning (MSL) and motor adaptation (MA).

MSL involves integrating the temporal structuring of a series of actions into a coherent unit, whereas MA requires the modification of previously learned movements in response to changes in the organism, task or environment. Both MSL and MA have been extensively studied in young subjects and are thought to follow several distinct phases: (1) a fast initial, within-session learning phase where the magnitude of the behavioral improvements is substantial; (2) a slow, across-session phase in which smaller behavioral improvements are evident over days, weeks, or months of practice; and, (3) an intermediate phase that occurs between practice sessions in which the motor memory is transformed from an initial labile trace to a more stable and resistant form (e.g., Karni et al., 1995, 1998; Doyon et al., 2003; Krakauer et al., 2005). Although the behavioral and neural correlates of

MSL and MA are relatively similar during early learning, there is ample evidence indicating that they differ when performance becomes asymptotic and motor memory consolidation begins (for reviews, see Doyon et al., 2003, 2009a; Doyon and Benali, 2005). Indeed, the initial fast learning phase of both MSL and MA elicits widespread activation in cortical and subcortical structures, including the basal ganglia, cerebellum, the supplementary motor area (SMA) as well as the primary motor (M1), premotor (PM), and prefrontal (PFC) cortices. However, consolidation and retention of learned motor sequences is thought to be dependent on the cortico-striatal network, whereas consolidation and retention following MA is predominantly considered a function of the cortico-cerebellar system (Krebs et al., 1998; Penhune and Doyon, 2002; Ungerleider et al., 2002; Doyon et al., 2003, 2009a; Doyon and Benali, 2005; Galea et al., 2010; Landi et al., 2011).

Behavioral studies examining the influence of aging on MSL and MA have consistently reported the following pattern of results: (1) the initial, fast learning phase of MSL appears to be relatively spared by the aging process except under conditions of increased task complexity (e.g., Curran, 1997; Feeney et al., 2002; Howard et al., 2004; Bennett et al., 2007; Rieckmann and Bäckman, 2009); (2) older adults demonstrate impairments in the consolidation of learned motor sequences (e.g., Spencer et al., 2007; Brown et al., 2009; Nemeth and Janacsek, 2010; Nemeth et al., 2010; Fogel et al., 2012; Wilson et al., 2012); and, (3) older adults demonstrate deficits during the exposure phase of MA paradigms; however, the magnitude of the aftereffects in the post-exposure phase is comparable to that of young adults (e.g., McNay and Willingham, 1998; Fernandez-Ruiz et al., 2000; Bock, 2005; Bock and Girgenrath, 2006; Seidler, 2006, 2007a; Heuer and Hegele, 2008; Hegele and Heuer, 2010; Anguera et al., 2011). Although seemingly distinct, these behavioral results may be manifestations of common age-related degradations in the structure and functioning of relevant neural substrates and networks. This paper will discuss the influence of the aging brain on the impairments highlighted above, with a particular emphasis on the cortico-striatal networks critical for the different phases of MSL and MA.

This review is organized into four sections. Following this introductory section, we provide a brief overview of MSL and MA, emphasizing behavioral results and neural correlates from research in young adults. The third section highlights motor learning in older adults <sup>1</sup> , and discusses evidence linking the behavioral deficits to age-related changes in relevant neural substrates; specifically the cortico-striatal network. The fourth section will then provide general conclusions.

## **MOTOR LEARNING IN YOUNG ADULTS: AN OVERVIEW MOTOR SEQUENCE LEARNING (MSL)**

## *Behavioral results*

MSL refers to the process by which simple, stereotyped movement elements come to be performed effortlessly as a unitary wellrehearsed sequence. This type of procedural learning has been investigated with a variety of different laboratory-based protocols; the most common requires participants to use the fingers of the right or left hand to either press buttons on a keyboard, or to lightly touch one's own thumb in a precise and sequential order. The sequence of movements may be explicitly (e.g., Karni et al., 1995; Korman et al., 2003) or implicitly learned (e.g., Robertson et al., 2004b), self-initiated (e.g., Karni et al., 1995), cued by visual or acoustic stimuli (e.g., Nissen and Bullemer, 1987), or interleaved with random movements (e.g., Howard and Howard, 1997). Despite these methodological differences, participants typically increase the velocity of their finger movements and decrease the interval between successive key presses with practice, resulting in a decrease in the duration to complete the repeated sequence (a measure of speed) and the number of errors made (a measure of accuracy). These behavioral improvements are indicative of learning the sequence and can also be used as indices of memory consolidation when performance is subsequently retested.

Although a detailed characterization of the initial acquisition of movement sequences is critical to our understanding of motor learning, it is equally important to understand how the retention of these newly acquired memories occurs over longer periods of time. In the context of implementing interventions designed to ameliorate age-related declines in motor performance or to increase functional mobility following neurological injury, improvements in motor functioning must be maintained beyond the conclusion of the training session. Experimental protocols typically assess retention by having participants return to the laboratory after a period of no practice to perform the same motor sequence. Retention is then quantified by making various comparisons across the different experimental sessions. In the interest of clarity, this review will adopt the following terminology that is used in the extant literature to characterize retention. The term "savings," although more commonly used in the MA literature, refers to significantly better performance (i.e., reduced errors or faster rate of learning) during the early portion of the retention session as compared to the early portion of initial training (Krakauer, 2009). "Off-line gains" refers to better performance in the early portion of the retention session as compared to the end of the initial training session (e.g., Robertson et al., 2004a)<sup>2</sup> . And finally, the term "consolidation" refers to the process by which an initially labile memory trace becomes transformed into a more stable, enduring memory (McGaugh, 2000; Walker et al., 2003; Krakauer and Shadmehr, 2006). Consolidation may be reflected by off-line gains, maintenance of a trace across testing sessions as well as resistance to interference from competing memories (Robertson et al., 2004a; Walker, 2005). Critically, previous research in young adults has demonstrated substantial savings and off-line gains following periods of non-practice of a motor sequence for several hours up to 1 year (Karni et al., 1995, 1998; Penhune and Doyon, 2002; Walker et al., 2002; Romano et al., 2010).

The magnitude of the savings and off-line gains in young adults is enhanced by a period of sleep during the interval between initial training and retention. More specifically, both nighttime sleep and a daytime nap result in significant increases in off-line learning and resistance to interference from a competing memory trace as compared to an equivalent period of wakefulness (Walker et al., 2002, 2003; Walker and Stickgold, 2006; Korman et al., 2007; Nishida and Walker, 2007; Doyon et al., 2009b; Debas et al., 2010). There is also growing evidence to suggest that stage 2 sleep, and sleep spindles in particular, are involved in this consolidation process (Fogel et al., 2007; Nishida and Walker, 2007; Morin et al., 2008; Barakat et al., 2011, 2012). Sleep spindles are short synchronous bursts of neuroelectrical activity between 12 and 15 Hz that propagate through the thalamocortical loop (Steriade, 2006; Bonjean et al., 2011). Perhaps most importantly for the context of this review, sleep spindles are thought to be involved in long-term synaptic plasticity, providing an explanation for their role in the consolidation of learned motor sequences (for review, see Fogel and Smith, 2011).

Sleep-dependent consolidation has consistently been reported in explicit MSL paradigms where the sequence of elements to be performed is explicitly provided to the participants either prior to or throughout training (e.g., Korman et al., 2007; Debas et al., 2010; Albouy et al., 2013a). Conversely, implicit sequence learning paradigms typically employ some variant of the serial reaction time (SRT) task where participants press a button with the appropriate finger that corresponds to a specific visual stimulus presented on a computer screen. Unbeknownst to the participants, the sequence of stimuli (and thus corresponding finger movements) follows a repeating pattern or an underlying structure.

<sup>1</sup>Our discussion of age-related behavioral deficits will be limited to motor *learning*. For a detailed discussion on age-related deficits in motor *performance*, please see Seidler et al. (2010).

<sup>2</sup>The notion of spontaneous, off-line enhancements has recently been a topic of debate. It has been suggested that off-line gains are manifestations of fatigue effects during the end of the initial training session (Brawn et al., 2010). However, recent results (Albouy et al., 2013a,b) demonstrated off-line gains even after controlling for fatigue.

The role of sleep in the consolidation of implicit motor sequence memories remains controversial as some studies have reported no influence of sleep (Robertson et al., 2004b; Song et al., 2007; Nemeth et al., 2010) whereas others have demonstrated sleepdependent benefits (e.g., Albouy et al., 2008). The reasons for these inconsistent findings remain unknown, although some insights have been offered based on the recruitment of relevant neural substrates, a topic that is a focus of the subsequent section.

## *Neural correlates*

The neural substrates underlying MSL in young adults have been extensively characterized (Grafton et al., 1995; Penhune and Doyon, 2002; Ungerleider et al., 2002; Doyon et al., 2003, 2009a; Doyon and Benali, 2005; Penhune and Steele, 2012) and are thus briefly summarized here. The initial acquisition phase of MSL elicits widespread activation, including, but not limited to, the basal ganglia, cerebellum, hippocampus as well as relevant cortical areas (e.g., SMA, M1, PFC, and PM cortex). However, the relative contributions of these different structures change as a function of learning. Activity in the striatum collectively increases while activity in the cerebellum decreases with practice, especially when behavioral performance is asymptotic (Grafton et al., 1995; Doyon et al., 2002; Penhune and Doyon, 2002). Within the fronto-striatal networks, it has been suggested that the caudate-DLPFC circuit as well as the rostrodorsal (associative) regions of the putamen are involved early in the learning process and are critical for acquiring an accurate sequence representation (Jueptner et al., 1997; Lehericy et al., 2005). By contrast, activity in the caudoventral (sensorimotor) areas of the putamen increases as a function of practice, suggesting that this region is involved in the execution of well-learned or automatic sequences (Jueptner et al., 1997; Lehericy et al., 2005). Independent of its role in motor execution, the cerebellum is especially critical for early sequence learning, not only for error detection and correction, but also in the acquisition of sequence knowledge (Seidler et al., 2002; Orban et al., 2010; Steele and Penhune, 2010). Last, the long-term storage of the motor memory is thought to be dependent on a distributed cortico-striatal network (Karni et al., 1995, 1998; Penhune and Doyon, 2002; Penhune and Steele, 2012).

The hippocampus has traditionally received very little attention in MSL and other procedural memory tasks as its function has been considered limited to declarative memory or tasks involving explicit learning mechanisms. More recently, however, the hippocampus has been implicated in both the initial learning and memory consolidation phases regardless of whether the sequences are implicitly or explicitly learned (Schendan et al., 2003; Albouy et al., 2008; Fernández-Seara et al., 2009; Gheysen et al., 2010). More particularly, activity in both the striatum and hippocampus during initial MSL (Albouy et al., 2008), as well as their functional interactions (Albouy et al., 2013b) have been described to predict subsequent consolidation processes. Rather than a distinction based on the implicit or explicit nature of the learning, recruitment of the hippocampus appears to depend on the *type* of information learned. Rose et al. (2011) demonstrated that bilateral hippocampal activation was evident only during learning of the perceptual, but not motor, component of a sequence. This result is analogous to recent research in our own laboratory suggesting that the hippocampus appears to be critical for the learning and consolidation of an allocentric, spatial representation of a sequence whereas the striatum is more involved in the learning and consolidation of an egocentric, motor representation (Albouy et al., 2012, 2013a).

Interestingly, consolidation of the allocentric, and presumably hippocampal-dependent, representation was enhanced by sleep whereas consolidation of the egocentric representation was not (Albouy et al., 2013a), suggesting that the recruitment of the hippocampus may be critical for sleep-dependent consolidation. This link between the hippocampus and sleep-dependent consolidation has also been used to explain the conflicting results investigating the role of sleep in *implicit* sequence learning (Section Behavioral results) (Song et al., 2007). Specifically, explicit, as compared to implicit, sequence learning is thought to rely more heavily on the hippocampus; thus, increasing the probability of sleep-dependent consolidation. It should be emphasized that this hypothesis certainly warrants further investigation because: (1) implicit sequence learning results in significant hippocampal activation (Schendan et al., 2003; Albouy et al., 2008; Gheysen et al., 2010); and, (2) sleep-dependent effects have been previously observed in implicit learning paradigms (Albouy et al., 2008).

Collectively, these results from neuroimaging research indicate that the hippocampus and both the cortico-cerebellar and cortico-striatal systems are involved in the initial learning of a movement sequence; however, consolidation and long-term retention are functions of the hippocampus and cortical-striatal network.

## **MOTOR ADAPTATION (MA)** *Behavioral results*

Movements need to be modified in response to changing conditions, such as when muscles are fatigued, when the dynamics of the end effector have changed as a result of growth or development or in response to bodily or brain injury. This adaptation process is typically examined by manipulating conditions in the environment in which participants move, specifically, by introducing visuomotor distortions (e.g., Kagerer et al., 1997) or mechanical perturbations (e.g., Shadmehr and Mussa-Ivaldi, 1994) during the execution of goal-directed movements. During initial exposure to a perturbation, participants typically make within-trial, feedback-dependent corrections (Thoroughman and Shadmehr, 1999). However, with continued exposure, these corrective responses are utilized in a feed-forward process, altering the initial motor commands of subsequent movements (Shadmehr and Mussa-Ivaldi, 1994; Thoroughman and Shadmehr, 1999). This feed-forward update becomes apparent when the perturbation is abruptly removed and subsequent movement paths are distorted in the direction opposite to that of the imposed perturbation (i.e., a clockwise visuomotor distortion would result in counter-clockwise movement trajectories). These distorted trajectories, in the absence of external perturbations, are referred to as aftereffects and provide a measure of the level of adaptation acquired during the exposure conditions (Shadmehr and Mussa-Ivaldi, 1994; Kagerer et al., 1997).

If young adults are re-exposed to the same perturbation after a time delay, the magnitude of the errors is decreased and the rate of adaptation is substantially increased, indicating savings in performance (Brashers-Krug et al., 1996; Shadmehr and Brashers-Krug, 1997; Shadmehr and Holcomb, 1997; Krakauer et al., 2005; Krakauer and Shadmehr, 2006; Krakauer, 2009). Yet in contrast to memory consolidation following MSL, the influence of sleep on consolidation following MA is less clear. Tononi and colleagues have demonstrated that sleep not only enhances MA consolidation in young adults, but the magnitude of the off-line improvements is correlated to the amount of slow wave activity (*<*4 Hz) in the right parietal region (Huber et al., 2004; Landsness et al., 2009). Conversely, research from our own group has demonstrated equivalent savings following periods of sleep and wake (Doyon et al., 2009b; Debas et al., 2010). These data are consistent with previous literature indicating that the passage of time, with or without sleep, is sufficient for MA savings (Brashers-Krug et al., 1996; Shadmehr and Brashers-Krug, 1997; Shadmehr and Holcomb, 1997, 1999; Krakauer et al., 2005). Last, sleep deprivation in young participants has been shown to have no detrimental influence on savings in performance but does deteriorate stabilization of the memory trace (Donchin et al., 2002; Albouy et al., 2013c). Although further research is certainly necessary, the majority of the evidence to date suggests that time in the wake state is necessary, but sufficient for MA consolidation to occur, and that sleep does not offer additional benefits for consolidation.

## *Neural correlates*

Adapting or modifying movements in response to sensorimotor perturbations has largely been considered a function of the cerebellum. The cerebellum generates predictions of future states computed based on efferent copies of descending motor commands (Barto et al., 1999; Bastian, 2006; Miall et al., 2007; Nowak et al., 2007; Tseng et al., 2007; Miall and King, 2008). Discrepancies between actual and predicted states are then used as error signals that drive the adaptation process by altering the synaptic weights between the posterior parietal cortex (PPC), critical for specifying spatial information about both the end effector and desired target, and M1 (Tanaka et al., 2009). These error signals are ideal for supervised learning algorithms, a type of learning thought to be implemented in the cerebellum (Doya, 2000). Additional support for the role of the cerebellum in MA comes from both patient and imaging studies. Patients with cerebellar damage have demonstrated substantial deficits in sensorimotor adaptation (Martin et al., 1996; Smith and Shadmehr, 2005; Rabe et al., 2009; Criscimagna-Hemminger et al., 2010; Werner et al., 2010; Donchin et al., 2012) and studies using PET and fMRI have repeatedly shown extensive cerebellar activation during MA in healthy adults (Krebs et al., 1998; Imamizu et al., 2000; Nezafat et al., 2001; Seidler et al., 2006; Albouy et al., 2013c). Cerebellar activation can even predict the amount of subsequent savings in performance (Debas et al., 2010; Albouy et al., 2013c) and is also thought to be involved in delayed recall assessments, suggesting that the cerebellum is involved in the acquisition, consolidation and long-term retention of MA (Shadmehr and Holcomb, 1997; Imamizu et al., 2000; Nezafat et al., 2001; Della-Maggiore and McIntosh, 2005; Debas et al., 2010).

The basal ganglia also contribute to MA as research in Parkinson's disease (PD) has revealed that patients demonstrate substantial performance deficits, particularly when the magnitude of the movement errors is large as in abruptly introduced visuomotor perturbations (Contreras-Vidal and Buch, 2003; Messier et al., 2007; Paquet et al., 2008; Venkatakrishnan et al., 2011; Mongeon et al., 2013). Similarly, results from neuroimaging research has indicated that the contribution of the basal ganglia, and the striatum in particular, appears to be greatest during the initial adaptation stage (Seidler et al., 2006; Albouy et al., 2013c), and then progressively decreases as a function of training (Shadmehr and Holcomb, 1997; Krebs et al., 1998). One explanation for the increased activation during initial adaptation is that the striatum functions as an adaptive search mechanism that selects new sensorimotor representations that may be more appropriate for moving in the novel sensorimotor environment (Contreras-Vidal and Buch, 2003; Grosse-Wentrup and Contreras-Vidal, 2007; Scheidt et al., 2012). Successful selections are subsequently rewarded whereas unsuccessful selections are penalized, resulting in a reward-based learning algorithm thought to be implemented in the basal ganglia circuitry (Doya, 2000). In addition to increased striatal activation, the initial adaption also results in increased activation in frontal cortical areas, including the PFC (Shadmehr and Holcomb, 1997; Della-Maggiore and McIntosh, 2005; Anguera et al., 2007; Gentili et al., 2011). While the striatum may be involved in finding sensorimotor mappings suited for the novel, perturbed environment, the frontal cortex appears to inhibit previously learned or established sensorimotor mappings that are no longer appropriate (Shadmehr and Holcomb, 1999; Gentili et al., 2011).

## **AGING AND MOTOR LEARNING**

## **MOTOR SEQUENCE LEARNING**

#### *Initial acquisition of motor sequences*

During the fast learning phase of MSL paradigms, older adults demonstrate significant improvements in performance as a function of practice, suggesting that they can learn novel motor sequences (Howard and Howard, 1989, 1992; Daselaar et al., 2003; Shea et al., 2006; Spencer et al., 2007; Brown et al., 2009; Fraser et al., 2009; Rieckmann and Bäckman, 2009; Nemeth and Janacsek, 2010; Nemeth et al., 2010; Romano et al., 2010; Fogel et al., 2012; Wilson et al., 2012). However, under certain task conditions such as increased task complexity or explicit knowledge of the sequence, older adults, as compared to young adults, have demonstrated deficits in learning rate and magnitude (Curran, 1997; Howard and Howard, 2001; Feeney et al., 2002; Howard et al., 2004, 2008; Bennett et al., 2007, 2011; Rieckmann and Bäckman, 2009). For example, the complexity of the learned sequence can be increased when random movements are interleaved with the to-be-learned repeated elements (i.e., a movement sequence of*r4r1r3r2* where *r* represents a random element and the numbers represent components of the repeated finger sequence to be learned). Such an increase in sequence complexity has revealed a disproportionately negative influence on older adults (Curran, 1997; Feeney et al., 2002; Howard et al., 2004; Bennett et al., 2007). Similarly, providing explicit information about a repeating sequence, particularly when the sequence is long, appears to impede MSL in older adults, whereas it has a negligible or even facilitative influence on sequence learning in younger participants (Willingham and Goedert-Eschmann, 1999; Howard and Howard, 2001; Willingham et al., 2002).

It has been proposed that these deficits are the result of age-related decreases in cognitive functioning (Salthouse, 1996; Howard and Howard, 2001; Howard et al., 2004; Rieckmann and Bäckman, 2009). For example, performing the alternating serial reaction time (ASRT) task (i.e., *r4r1r3r2*) requires that nonadjacent elements of the sequence be linked as part of a repeating sequence. Decreases in cognitive processing speed will interfere with linking the non-adjacent elements, effectively hindering the learning process (Salthouse, 1996; Howard et al., 2004). Similarly, providing explicit information about a repeating sequence is thought to negatively influence learning because this information consumes additional cognitive/neural resources. The additional resources allocated to the explicit learning of the motor sequence may result in reaching the ceiling of cognitive processing capacity in older but not younger adults (Frensch and Miner, 1994; Howard and Howard, 2001; Rieckmann and Bäckman, 2009). In addition, Seidler and colleagues have reported a significant correlation between explicit sequence learning and working memory in both young and older adults (Bo et al., 2009). This suggests that age-related decreases in working memory contribute to the age-related deficits in the initial acquisition of motor sequences.

As both the frontal cortex and the striatum are heavily involved in the initial learning of motor sequences, these task-dependent behavioral deficits may be attributed to age-related degradations in cortico-striatal networks (Rieckmann and Bäckman, 2009; Rieckmann et al., 2010). Indeed, there are several pieces of evidence to support this explanation. First, substantial age-related structural changes are evident in both the frontal cortex and striatum, including reductions in volume (**Figures 1A,B**) (Gunning-Dixon et al., 1998; Raz et al., 2003, 2005; Hedden and Gabrieli, 2004; Allen et al., 2005; Kennedy and Raz, 2005). Second, aging is associated with significant decreases in dopamine (the prominent neurotransmitter acting in the basal ganglia), the presence of which has been shown to facilitate sequence learning and motor memory formation (**Figure 1C**) (Kaasinen and Rinne, 2002; Floel et al., 2005, 2008; Bäckman et al., 2006, 2010; Simon et al., 2011). Third, the integrity of the white matter tracts connecting the caudate nucleus and the dorsolateral PFC is decreased in older, as compared to younger, adults (**Figure 1D**) (Bennett et al., 2011). The caudate-DLPFC circuit is not only thought to be involved in forming associations between repeated elements that are necessary for early MSL (Jueptner et al., 1997; Poldrack et al., 2005), but degradations in this tract have also been related to age-related declines in sequence learning (Bennett et al., 2011). Fourth, implicit sequence learning in older adults is associated with decreased activation in the right putamen (Aizenstein et al., 2006). Interestingly, decreased activation in the putamen has also been observed in older adults during an interlimb coordination task (Van Impe et al., 2009) and proprioceptive stimulation (Goble et al., 2012), the latter of which was the result of age-related structural deficits. This decreased activation in the putamen is particularly surprising given that widespread agerelated and task-dependent *increases* in activation are frequently

**FIGURE 1 | Age-related volumetric declines in the (A) lateral prefrontal cortex and (B) caudate nucleus.** Reprinted from Raz et al. (2005), by permission of Oxford University Press. Similar volumetric decreases are also evident in the putamen (Raz et al., 2003). **(C)** Transaxial planes at the level of the caudate nucleus depicting decreased dopaminergic activity (relative uptake of a dopamine D2-like receptor ligand) in a representative older subject compared to a young participant. Reprinted from Neuroscience and Biobehavioral Reviews, 26, Kaasinen and Rinne (2002), with permission from Elsevier. **(D)** FA values for three different white matter tracts in younger and older adults. Reprinted from Neurobiology of Aging, 32, (Bennett et al., 2011), with permission from Elsevier. <sup>∗</sup>Significant age group differences (*p <* 0*.*01). reported (Mattay et al., 2002; Ward and Frackowiak, 2003). And, fifth, the pattern of brain activation during sequence learning in older adults suggests that the hippocampus may be compensating for disrupted striatal functioning (Rieckmann and Bäckman, 2009; Rieckmann et al., 2010). More specifically, in young adults, hippocampal activity decreases and striatal activity increases as a function of sequence learning (Schendan et al., 2003; Albouy et al., 2008), whereas in the older adults, activity in both the MTL, including the hippocampus, and the striatum *increases* (Rieckmann et al., 2010). The increased MTL activity may serve a compensatory function in order to maintain similar levels of performance despite age-related decreases in the structure and function of the striatum (Rieckmann and Bäckman, 2009; Rieckmann et al., 2010).

Collectively, these findings suggest that age-related degradations in the striatum contribute to the age-related deficits in the acquisition phase of MSL. When learning is implicit or when sequence complexity is relatively low, MSL is comparable to young adults due to compensation from other relevant neural structures, including the MTL, and the hippocampus in particular. However, in task conditions with an increased cognitive load (i.e., greater sequence complexity or explicit nature of the MSL task), the performance of older adults during the initial learning is *not* maintained, potentially due to an inability of the MTL and other neural substrates to compensate for age-related degradations in the striatum (Rieckmann and Bäckman, 2009).

It should be emphasized that degradations in the frontal cortico-striatal system are likely not the only neural correlate of impaired sequence learning in older adults. The initial phase of sequence learning is dependent on a widespread network of cortical and subcortical structures, including the frontal cortex, striatum, cerebellum and hippocampus. It is thus likely that agerelated changes in these additional structures, particularly the hippocampus (e.g., Allen et al., 2005; Raz et al., 2005), contribute to the behavioral deficits. Interestingly, the pattern of brain activation during sequence learning in older adults (e.g., increased activation in *both* the hippocampus and striatum) (Rieckmann et al., 2010) was identical to that of a sub-group of participants in the experiment of Albouy et al. (2008) that demonstrated a decreased rate of sequence learning, suggesting that the interaction between the striatum and hippocampus may also contribute to sequence learning deficits in older adults.

## *Consolidation of motor sequences*

Although older adults demonstrate significant savings in performance across multiple sessions, indicating retention of newly acquired motor memories for a period of up to 1 year (Shea et al., 2006; Spencer et al., 2007; Fraser et al., 2009; Nemeth et al., 2010; Romano et al., 2010; Wilson et al., 2012), the magnitude of the savings is less than that demonstrated by young subjects (Spencer et al., 2007; Brown et al., 2009; Nemeth and Janacsek, 2010; Nemeth et al., 2010; Wilson et al., 2012). Moreover, older adults fail to demonstrate the off-line gains in the absence of additional practice that are typically observed in young adults (Spencer et al., 2007; Wilson et al., 2012), suggesting that the consolidation process of motor memories following MSL is impaired in older adults.

Similar to the deficits in the initial acquisition of movement sequences, the deficits in MSL consolidation demonstrated by older adults can likely be attributed, at least partially, to agerelated degradations in the striatum and/or hippocampus. In addition to the substantial age-related volumetric and dopaminergic declines discussed above (Gunning-Dixon et al., 1998; Kaasinen and Rinne, 2002; Raz et al., 2003, 2005; Hedden and Gabrieli, 2004; Floel et al., 2005, 2008; Kennedy and Raz, 2005; Bäckman et al., 2010), recent research in our lab investigating the role of sleep in MSL consolidation provides further evidence linking age-related changes in striatal activity to the motor memory consolidation deficits observed in older adults (Fogel et al., 2012).

In comparison to younger adults, older adults experience disrupted sleep, including increased sleep fragmentation and decreased sleep time and efficiency (Myers and Badia, 1995; Landolt and Borbely, 2001; Phillips and Ancoli-Israel, 2001; Huang et al., 2002). Despite spending more time in sleep stages 1 and 2, older adults have decreased amplitude, duration and number of sleep spindles (Landolt et al., 1996; Wei et al., 1999; Landolt and Borbely, 2001; Nicolas et al., 2001; Crowley et al., 2002). There is also growing evidence to suggest that spindles are involved in procedural memory consolidation (Fogel and Smith, 2006, 2011; Fogel et al., 2007; Nishida and Walker, 2007; Morin et al., 2008; Barakat et al., 2011, 2012). Moreover, a recent study has shown that in young subjects, activity in the putamen was increased following MSL and the increased activity was correlated with sleep spindles (Barakat et al., 2012). It is thus likely that impaired motor sequence consolidation demonstrated by older adults can be attributed to their disrupted sleep architecture as well as the interaction between sleep and the neural substrates subserving MSL consolidation (i.e., the corticostriatal system and hippocampus). In support of this hypothesis, a recent study in our lab examined the consolidation of an explicit motor sequence following a retention period containing either a 90-min daytime nap or equivalent period of wake. The aim was to investigate the associated changes in functional brain activity in young and older adults (Fogel et al., 2012) to better understand the neural correlates of the age-related deficit in MSL consolidation (Spencer et al., 2007; Brown et al., 2009). Results demonstrated that while young adults revealed enhanced behavioral performance following an afternoon nap, older adults did not. Moreover, spindles in the young group were related to increased changes in activation in the putamen from training to the post-nap retest. By contrast, sleep spindles in the older adults were related to increased activation in regions in the cortico-cerebellar loop, a neural network that, although involved in the initial acquisition of motor sequences, is not essential for motor sequence memory consolidation. Critically, these data provide a link between sleep spindles, the cortico-striatal system and enhanced consolidation in younger adults. No such beneficial relationship was evident in older adults, a finding that is likely the result of agerelated degradations in both the cortico-striatal system and sleep architecture.

Similar to the discussion on the initial learning of motor sequences, the age-related declines in the hippocampus (Allen et al., 2005; Raz et al., 2005) may also contribute to deficits in sequence consolidation. Whereas "fast-learning" young adults demonstrate increased and decreased activation in the striatum and hippocampus, respectively, as a function of practice, both "slow-learning" young adults and older adults demonstrate increased activation in both substrates (Albouy et al., 2008; Rieckmann et al., 2010). These "slow-learning" young adults in the experiment of Albouy et al. (2008) also demonstrated impaired overnight consolidation, suggesting that the altered dynamics between the hippocampus and striatum may, at least partially, underlie the age-related deficits in motor sequence memory consolidation.

### **MOTOR ADAPTATION**

## *Initial adaptation session in older adults*

During exposure to various sensorimotor perturbations, older adults have demonstrated gradual reductions in movement errors, indicating that they can adapt to manipulations in the sensorimotor environment. However, results have consistently shown that the rate of adaptation and final level of performance are significantly worse in older adults, as compared to younger individuals (McNay and Willingham, 1998; Fernandez-Ruiz et al., 2000; Buch et al., 2003; Bock, 2005; Bock and Girgenrath, 2006; Seidler, 2006; Heuer and Hegele, 2008; Hegele and Heuer, 2010; Anguera et al., 2011). Despite the age-related differences *during exposure* to sensorimotor perturbations, older adults demonstrate equivalent or even larger aftereffects, as well as similar levels of transfer across behavioral tasks as compared to young adults, suggesting that aging does not result in impaired sensorimotor adaptation (Fernandez-Ruiz et al., 2000; Buch et al., 2003; Bock, 2005; Bock and Girgenrath, 2006; Seidler, 2007a; Heuer and Hegele, 2008; Hegele and Heuer, 2010).

The dissociation between the exposure and post-exposure phases appears paradoxical. But, it has been postulated that the performance during the post-exposure phase reflects the ability to adapt implicitly, or recalibrate, to novel changes in the environment, whereas performance during exposure to the perturbation reflects both implicit adaptation as well as the implementation of strategies utilized in response to the movement errors caused by the perturbation (Bock and Schneider, 2002; Buch et al., 2003). Within this context, implicit adaptation or sensory recalibration does not degrade with age. Conversely, the age-related differences evident in the exposure phase would result from deficits in cognitive, strategic control (McNay and Willingham, 1998; Fernandez-Ruiz et al., 2000; Bock and Schneider, 2002; Bock, 2005; Bock and Girgenrath, 2006; Heuer and Hegele, 2011; Heuer et al., 2011). Several pieces of evidence are used to provide support for this explanation. First, older, as compared to younger, adults fail to acquire equivalent explicit information about the nature of the sensorimotor perturbations, and this explicit information is correlated to performance during the exposure phase, but not to the magnitude of the aftereffects (Bock, 2005; Heuer and Hegele, 2008). This suggests that younger adults benefit from acquired explicit information during the exposure phase. Second, deficits during the exposure phase in older adults are related to degradations in measures of cognitive functioning, suggesting a role for cognitive processes during the exposure phase (Bock, 2005; Heuer and Hegele, 2008; Anguera et al., 2011; Langan and Seidler, 2011). For example, Seidler and colleagues have indicated that the inability to appropriately engage spatial working memory processes are correlated to the MA deficits observed in older adults (Anguera et al., 2011). This result is similar to their findings indicating that working memory deficits contribute to difficulties in the initial acquisition of movement sequences (Bo and Seidler, 2009; Bo et al., 2009). Third, when the potential use of explicit strategies is minimized by introducing the sensorimotor perturbation in gradual increments, age-related deficits during the exposure phase disappear (Buch et al., 2003; Cressman et al., 2010). These results thus suggest that age-related deficits in cognitive, strategic control, and not necessarily implicit MA, underlie the behavioral difficulties observed in older adults during exposure to sensorimotor perturbations.

The underlying neural substrates may help elucidate the dissociation described above between performance during the exposure and post-exposure phases. As discussed earlier, results from both patient and neuroimaging studies have implicated the cerebellum and striatum as key contributors to MA. Specifically, the cerebellum is thought to generate predictions of future states, and discrepancies between actual and predicted states are then used as error signals that drive the adaptation process by altering the synaptic weights between the PPC and M1 (Tseng et al., 2007; Tanaka et al., 2009). This cortico-cerebellar network would then be considered responsible for the implicit adaptation or sensory recalibration process that is reflected by the magnitude of the aftereffects when the perturbation is suddenly removed. The cortico-striatal network would operate in parallel, particularly during the early portions of the exposure phase when the magnitude of the movement errors is large (Venkatakrishnan et al., 2011; Mongeon et al., 2013). Specifically, the striatum is thought to function as an adaptive search mechanism that attempts to retrieve sensorimotor representations more appropriate for the perturbed environment (Contreras-Vidal and Buch, 2003; Grosse-Wentrup and Contreras-Vidal, 2007; Scheidt et al., 2012). The frontal cortex and the PFC in particular, would inhibit previously learned, established sensorimotor mappings that are no longer appropriate (Shadmehr and Holcomb, 1999; Gentili et al., 2011). This cortico-striatal network would then contribute, along with the cortico-cerebellar network, to the reduction of movement errors in the exposure phase. Within this context, the decreased performance observed during the exposure phase demonstrated by older adults would appear to be the result of impaired functioning of the cortico-striatal networks. In addition to the age-related decreases in dopamine and striatal volume that were highlighted in the section *Initial Acquisition of Motor Sequences* (Kaasinen and Rinne, 2002; Raz et al., 2003, 2005; Kennedy and Raz, 2005; Bäckman et al., 2006, 2010), the frontal cortex, and the PFC in particular, shrink substantially with age; and, there are robust degradations in the white matter tracts connecting the caudate nucleus and DLPFC (Allen et al., 2005; Hedden and Gabrieli, 2005; Kennedy and Raz, 2005; Raz et al., 2005; Bennett et al., 2011). Altogether, the present findings suggest that similar to the initial learning and consolidation of motor sequences, the age-related changes in the frontal cortico-striatal network likely contribute to the performance deficits evident in the exposure phase of MA paradigms.

As the magnitudes of the aftereffects are generally comparable in young and older adults, this would suggest that age-related degradations in the functioning of the cortico-cerebellar system are relatively minimal. However, the cerebellum does exhibit similar age-related declines as the striatum, at least with respect to reductions in volume (Luft et al., 1999; Raz et al., 2005). In addition, such degradations in the cortico-cerebellar system are thought to substantially contribute to age-related deficits in motor and cognitive functioning (e.g., Hogan, 2004). Thus, this raises the following question: why do older adults demonstrate comparable aftereffects despite substantial age-related declines in the cortico-cerebellar system? There are two potential, and certainly not mutually exclusive, possibilities. First, there is evidence to suggest that different types of MA depend on different regions of the cerebellum. Specifically, research on cerebellar patients suggests that the posterior lobe of the cerebellum is more involved in visuomotor adaptation, whereas the anterior lobe is more involved in force field paradigms (Rabe et al., 2009; Donchin et al., 2012). This finding is consistent with activation, as measured with PET, in the posterior lobe during visuomotor adaptation (Krakauer et al., 2004). There is also evidence suggesting that while there are significant age-related degradations in the cerebellum as a whole, the anterior lobe experiences substantial changes with age, including reductions in volume as well as granule and Purkinje cell numbers (Andersen et al., 2003). Age-related declines in the posterior lobe were less robust and tended to not reach significance. As the majority of MA research in older adults has employed visuomotor paradigms, the lack of substantial agerelated deficits in the magnitude of the aftereffects is consistent with the notion that age-related degradations in the posterior lobe appear to be relatively minimal, effectively resulting in similar aftereffects in young and older adults. This explanation would then predict age-related differences in the magnitude of the aftereffects following force field adaptation, as this paradigm is more dependent on the anterior lobe of the cerebellum.

A second potential explanation is that as the majority of MA paradigms employ sensorimotor perturbations during the execution of goal-directed reaching movements, it could be argued that the adaptive processes underlying the traditional reach adaptation paradigm are relatively "simple" and are robust to the age-related degradations in cortico-cerebellar functioning. If task difficulty were increased, then age-related changes in the cortico-cerebellar system would result in more robust deficits at the behavioral level. Support for this hypothesis comes from a recent study in which older adults demonstrated reduced aftereffects in an adaptive locomotion task (e.g., split-belt paradigm) (Bruijn et al., 2012). The authors suggested that gait adaptation necessitates the reorganization of all body segments and that this increased task complexity, as compared to reaching adaptation paradigms, reveals age-related deficits in MA that are likely the result of degradations in the cortico-cerebellar networks (Bruijn et al., 2012). However, it should be emphasized that gait and posture are also more dependent on the anterior lobe of the cerebellum; thus, the age-related differences in Bruijn et al. (2012) may not be the result of task complexity *per se*, but may also be manifestations of the age-related degradations in the anterior cerebellar lobe noted above (Andersen et al., 2003). Regardless, the explanations presented above are speculative and additional research investigating the relationship between age-related degradations in cortico-cerebellar pathways and MA is necessary.

### *Motor adaptation retention*

Surprisingly, retention following MA has not been as extensively examined in older adults. In a 5-year follow-up of the mirrortracing task, older adults demonstrated savings in performance, although the magnitude of the savings was less than that of middle-aged and young adults (Rodrigue et al., 2005). However, older adults demonstrated significant transfer across different adaptation tasks (i.e., visual gain and rotation adaptation) and perturbation magnitudes when transfer was assessed 1–2 days after the initial training (Seidler, 2007a,b; Bock and Schneider, 2001). This facilitative effect was even more pronounced in the older subjects (Bock and Schneider, 2001). Collectively, this previous research potentially suggests that retention following MA is not impaired in older subjects. A lack of age-related behavioral deficits in MA retention, predominantly considered a function of the cortico-cerebellar network, would further suggest that the deficits observed in older adults during the exposure phase of MA paradigms are the result of age-related cortico-striatal, and not cortico-cerebellar, degradations. Again, however, a more in-depth investigation of this hypothesis is certainly necessary.

## **CONCLUDING REMARKS**

The extant aging and motor learning literature has consistently reported that older adults have deficits in: (1) the initial acquisition of movement sequences under conditions of increased task complexity; (2) the consolidation of learned motor sequences; and, (3) the exposure, but not post-exposure, phase of MA paradigms. This review discussed evidence linking the behavioral deficits to age-related changes in relevant neural substrates. Specifically, the behavioral results are, at least partially, manifestations of age-related dysfunctions in the structure and functioning of the fronto-striatal networks subserving the different phases of the two motor learning paradigms.

An open question is what are the specific changes within the cortico-striatal network that result in the behavioral deficits discussed above? We have reviewed evidence indicating that the aging process is associated with decreased volume in the frontal cortex as well as the caudate and putamen (Raz et al., 2003, 2005; Allen et al., 2005), disruptions in the dopaminergic system (Kaasinen and Rinne, 2002; Bäckman et al., 2010, 2006) and degradations in the white matter tracts connecting the striatum to the frontal cortex (Bennett et al., 2011). Although these age-related neural changes have been associated with learning deficits in older adults (Kennedy and Raz, 2005; Paquet et al., 2008; Bennett et al., 2011), the specific influence of each of these neural changes on MSL and MA is not fully understood. Future research should attempt to disentangle the relative contributions of these age-related neural changes on motor learning, a task that is difficult as these changes occur in parallel.

Future research should also investigate conditions or interventions in which the potential for motor learning in older adults is facilitated. For example, given that the evidence reviewed here suggests that age-related changes in sleep may underlie the MSL consolidation deficits observed in the elderly, interventions to improve sleep quality in older adults may have a therapeutic benefit for motor learning. Research in young adults has also indicated that motor learning and consolidation is enhanced if participants avoid potentially interfering tasks immediately following training (Krakauer et al., 2005; Korman et al., 2007). This suggests that it may be possible to structure a training regimen that maximizes the probability of enhanced motor learning in older adults. A second potential avenue to enhance motor learning is noninvasive brain stimulation, such as transcranial direct current stimulation (tDCS). It has proven effective in facilitating motor learning, consolidation and retention in young adults across a range of tasks including MSL and MA (Galea et al., 2009, 2010; Reis et al., 2009; Nitsche et al., 2010). Last, action observation training has also contributed to motor memory formation (e.g., Stefan et al., 2005). Optimizing the potential for motor learning and experience-dependent brain plasticity in older adults will not only enhance the effectiveness of interventions aimed to mitigate age-related declines in motor performance, but can also be used to improve neurorehabilitative interventions for individuals with movement disorders or neurological injuries (e.g., Celnik and Cohen, 2004; Ertelt et al., 2007; Celnik et al., 2008, 2009).

In sum, we reviewed substantial evidence demonstrating degradations in neural structure and function associated with

## **REFERENCES**


resonance imaging study. *J. Sleep Res.* 22, 144–154.


aging. It should be emphasized that these dysfunctions are not the result of passive processes that simply unfold as a function of age. Future research should continue to investigate potential experiences or therapeutic interventions, such as physical and mental activity regimens that may minimize the age-related neural degradations associated with the aging process (for review, see Seidler et al., 2010). Such investigations will promote the importance of specific experiences as an effective avenue to address a subset of the challenges introduced by our aging society.

#### **ACKNOWLEDGMENTS**

We would like to thank Melissa M. Pangelinan for her insightful comments and suggestions on the manuscript. Financial support provided by the Canadian Institutes of Health Research (CIHR; Identifying # MOP-97830), Ministère du Développement, de l'innovation et de l'exportation (MDEIE; PSR-SIIRI-704) and the European Commission—ERA Net Neuron (Identifying # 26659). Postdoctoral fellowship support for author SF provided by the Natural Sciences and Engineering Research Council of Canada (NSERC) and by the Fonds de la Recherche en Santé du Québec (FRSQ). Postdoctoral fellowship support for author GA provided by the Fyssen Foundation and the Canadian Institutes of Health Research (CIHR).

data and future avenues. *Neurosci. Biobehav. Rev.* 34, 670–677.


Corticothalamic feedback controls sleep spindle duration *in vivo*. *J. Neurosci.* 31, 9124–9134.


sequence learning. *Proc. Natl. Acad. Sci. U.S.A.* 99, 1017–1022.


activations in learning rotations and gains for reaching: a PET study. *J. Neurophysiol.* 91, 924–933.


and Acker, J. D. (2003). Differential aging of the human striatum: longitudinal evidence. *AJNR. Am. J. Neuroradiol.* 24, 1849–1856.


sensorimotor adaptation. *Brain Res. Bull.* 70, 337–346.


global decreases: a functional magnetic resonance imaging investigation of five days of motor sequence learning. *J. Neurosci.* 30, 8332–8341.


word-pair learning but not motor sequence learning in healthy older adults. *Neurobiol. Aging* 33, 991–1000.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 20 December 2012; accepted: 01 April 2013; published online: 17 April 2013.*

*Citation: King BR, Fogel SM, Albouy G and Doyon J (2013) Neural correlates of the age-related changes in motor sequence learning and motor adaptation in older adults. Front. Hum. Neurosci. 7:142. doi: 10.3389/fnhum.2013.00142 Copyright © 2013 King, Fogel, Albouy and Doyon. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

## Neural pathways mediating cross education of motor function

## *Kathy L. Ruddy1,2 and Richard G. Carson1,2\**

*<sup>1</sup> School of Psychology, Queen's University Belfast, Belfast, UK*

*<sup>2</sup> Trinity College Institute of Neuroscience and School of Psychology, Trinity College Dublin, Dublin, Ireland*

#### *Edited by:*

*Rachael D. Seidler, University of Michigan, USA*

#### *Reviewed by:*

*Lara Boyd, University of British Columbia, Canada Chiara Bozzacchi, Istituto Italiano di Tecnologia, Italy*

#### *\*Correspondence:*

*Richard G. Carson, Trinity College Institute of Neuroscience and School of Psychology, Lloyd Building, Trinity College Dublin, Dublin 2, Ireland e-mail: richard.carson@tcd.ie*

Cross education is the process whereby training of one limb gives rise to enhancements in the performance of the opposite, untrained limb. Despite interest in this phenomenon having been sustained for more than a century, a comprehensive explanation of the mediating neural mechanisms remains elusive. With new evidence emerging that cross education may have therapeutic utility, the need to provide a principled evidential basis upon which to design interventions becomes ever more pressing. Generally, mechanistic accounts of cross education align with one of two explanatory frameworks. Models of the "cross activation" variety encapsulate the observation that unilateral execution of a movement task gives rise to bilateral increases in corticospinal excitability. The related conjecture is that such distributed activity, when present during unilateral practice, leads to simultaneous adaptations in neural circuits that project to the muscles of the untrained limb, thus facilitating subsequent performance of the task. Alternatively, "bilateral access" models entail that motor engrams formed during unilateral practice, may subsequently be utilized bilaterally—that is, by the neural circuitry that constitutes the control centers for movements of *both* limbs. At present there is a paucity of direct evidence that allows the corresponding neural processes to be delineated, or their relative contributions in different task contexts to be ascertained. In the current review we seek to synthesize and assimilate the fragmentary information that is available, including consideration of knowledge that has emerged as a result of technological advances in structural and functional brain imaging. An emphasis upon task dependency is maintained throughout, the conviction being that the neural mechanisms that mediate cross education may only be understood in this context.

#### **Keywords: interlimb, bilateral, transfer, motor learning, interhemispheric**

## **INTRODUCTION**

#### **GENERAL CONTEXT**

The capacity for activity of one limb to influence the subsequent performance of its opposite counterpart has been documented for more than a century. As early as 1894, Scripture and colleagues employed a simple manometer to demonstrate that unilateral strength training gives rise to enhanced performance of the same task by the untrained opposite limb. This effect—for which the term "cross education" was coined, has been reproduced in a plethora of research investigations, encompassing both the transfer of strength and motor skill (Laszlo et al., 1970; Parlow and Kinsbourne, 1989; Imamizu and Shimojo, 1995). Despite longstanding interest in the phenomenon, there is, however, little consensus concerning the mediating neural mechanisms.

Why is this knowledge deficit of more general significance? As a case in point, a significant risk associated with the fractures that arise from falls by older adults, is that the loss of specific muscle strength or general capacity resulting from limb immobilization will leave the person below the level of capability necessary to perform everyday tasks, and thus maintain independent living. Even in younger persons with extensive functional reserves, 3 weeks of immobilization leads to declines of strength in the order of 50% of initial capacity (Hortobagyi et al., 2000). If, however, the opposite limb is trained during the period of immobilization, the loss of functional capacity is attenuated (Farthing et al., 2009; Magnus et al., 2010; Pearce et al., 2012). Given this therapeutic potential, there is an obvious need to provide a principled basis upon which to design interventions and tailor these appropriately to address individual requirements.

## **SCOPE OF THE REVIEW**

While originally cross education was deemed to encompass the transfer of muscle strength following a period of unilateral resistance training, and the transfer of skill following unilateral skill training (Scripture et al., 1894), the majority of contemporary empirical studies have treated strength transfer and skill transfer as separate entities (Farthing, 2009). The conviction that the two facets of cross-education are intimately related underpins the present review. Specifically, the transfer of strength or vigor following a period of unilateral resistance or ballistic training, and the transfer of skill following a period of unilateral skill training appear to be mediated by shared mechanisms. That which is at issue is the precise nature of these mechanisms, and the degree to which their respective contributions vary in accordance with specific task demands.

With regard to the extant literature, two principal theoretical models can be delineated (**Figure 1**). The first is derived from observations that the execution of many unilateral tasks is associated with increased excitability of both contralateral and ipsilateral cortical motor areas. The principal tenet of the "crossactivation" model is that bilateral cortical activity generated during unilateral training drives concurrent neural adaptations in both cerebral hemispheres. Accordingly, unilateral training induces task specific changes in the configuration of cortical motor networks that normally control the muscles of the opposite (quiescent) limb (Hellebrandt, 1951). Since the magnitude of cross-activation is contingent on the intensity of the unilateral contraction (Perez and Cohen, 2008), the degree of transfer is predicted to scale with the level of neural drive required to perform the training task. The "bilateral access" model (Laszlo et al., 1970; Taylor and Heilman, 1980; Imamizu and Shimojo, 1995) holds that motor "engrams" elaborated during unilateral training are not specific to the control of trained limb. Rather they are encoded in a more abstract fashion, at a locus that is also accessible for the control of the opposite untrained limb (Anguera et al., 2007). In this scheme, the degree of transfer is predicted to vary with the complexity of the training task (Farthing, 2009).

The primary goal of the present review is to elucidate the neural pathways that mediate cross education of motor function. In principle, one might also aim to assess the degree to which the structural and functional properties of the brain networks thus identified are commensurate with the respective theoretical models. To do so, however, it is first necessary to identify grounds upon which the models can be considered distinct. By Reductio ad Absurdum, transfer via cross-activation would be restricted to homologues of the effectors engaged during training (Davis, 1898). Conversely, transfer via bilateral access would be effector independent (Latash, 1999). We argue that the mechanisms that mediate the cross education of motor function are most profitably considered in relation to such factors as the characteristics of the training task (e.g., Sosnik, 2010). While the terms "Cross Activation" (section) and "Bilateral Access" (section) are retained for presentation purposes, we consider the empirical findings not only in relation to the eponymous models, but also ask whether it is possible to achieve a deeper appreciation of the mediating neural pathways through means other than their contrast. By and large our analysis is focused upon upper limb movements.

## **CROSS ACTIVATION**

### **THEORETICAL FRAMEWORK AND EMPIRICAL EVIDENCE**

In proposing that "some grooving of the neuronal pattern thought to be essential to motor learning must take place on the ipsilateral side, while the main stream of descending impulses flows to the contralateral limb," Hellebrandt (1951) was making appeal to two lines of evidence. The first was derived from observations that in a large cohort of school children, the developmental increase in strength, rate, and precision of movement exhibited by the preferred right hand was not markedly larger than that of the non-preferred hand—in spite of the much greater use presumed of the former (Bryan, 1892). The original author was led to conclude "the effects of use on the right side have been shared by the corresponding joints on the left side" (page 201). The second was the report by Welch (1898) that when a maximum intensity grip was generated by one hand, there was activation of

the muscles of the opposite hand, and indeed muscles in other parts of the body. This is the phenomenon of motor irradiation. When considered specifically in relation to effectors or muscles (i.e., of the opposite limb) that are homologues of those engaged in focal contractions, such terms as associated movements (Todor and Lazarus, 1986), mirror movements (Mayston et al., 1999) and contralateral irradiation (Cernacek, 1961) have been applied. Hellebrandt was perhaps the first to draw an explicit link between the presence of this phenomenon and the cross education (or bilateral transfer) of motor function. It was, however, presaged by Davis (1898) more than a century ago. While the potential origins of cross education may appear obvious in circumstances in which training movements of one limb give rise to associated movements of the other limb, cross activation may be latent and yet still have functional consequences in relation to the transfer of strength or skill.

As a result of technological advances in recent years, it has become possible to probe the nature of such latent interactions. Bilateral variations in the excitability of corticospinal projections during movements that are by intention unilateral, have been demonstrated using transcranial magnetic stimulation (TMS). Motor evoked potentials (MEPs) induced by TMS are increased in amplitude by isometric contractions of the homologous muscles in the opposite forearm (Hortobagyi et al., 2003). The amount of potentiation, or "crossed-facilitation," is positively correlated with the amount of force that is generated by the contractions of the opposite limb (Perez and Cohen, 2008). In the case of rhythmic movements, MEPs evoked in the quiescent muscles of a static limb vary in accordance with the phase of motion of the opposite (moving) limb (Carson et al., 1999, 2004). The MEP is maximally potentiated during the phase in which the homologous muscle in the opposite (moving) limb is maximally activated. Since corresponding changes in response amplitude are not obtained when potentials are evoked by stimulating the corticospinal pathway at the level of the cervico-medullary junction (Hortobagyi et al., 2003; Carson et al., 2004), it has been concluded that the phenomenon of crossed-facilitation has inter-hemispheric interactions between cortical motor areas as its primary physiological basis.

While it is evident that these interactions find expression via corticospinal output from M1, it cannot be assumed that direct interactions between the primary motor cortices represent the source of crossed facilitation. In monkey, mirror movements are abolished by the temporary inactivation (through injection of muscimol) of M1 ipsilateral to the actively moving limb, whereas they are largely preserved, or indeed enhanced, in circumstances in which the opposite M1 (i.e., contralateral to the moving limb) is injected (Tsuboi et al., 2010). This pattern of outcomes suggests that crossed facilitation arises from common drive to both primary motor cortices from other centers in the motor network. In a related vein, it has been noted (Cisek et al., 2003) that in non-human primates, there is a strong correlation between the directional tuning of cells in dorsal premotor cortex (PMd) during reaching movements made by the ipsilateral and contralateral limb, whereas for primary motor cortex the degree of association is markedly lower. During rhythmic contractions of a finger muscle performed by humans, connectivity from the contralateral (to movement) PMd to ipsilateral M1– as assessed by pairedpulse TMS techniques, is modulated by variations in contraction frequency (Uehara et al., 2013).

Human neuroimaging provides complementary evidence. Although activity in ipsilateral M1 is elevated during unilateral movements (Singh et al., 1998b; Dai et al., 2001; Kobayashi et al., 2003; van Duinen et al., 2008), greater increases are typically registered in areas anterior, lateral and ventral to the primary motor cortex, in a region on the precentral gyrus that most likely corresponds to premotor cortex (Kawashima et al., 1997; Singh et al., 1998a; Cramer et al., 1999; Kobayashi et al., 2003; Stancák et al., 2003; Koeneke et al., 2004; Hanakawa et al., 2005; ´ Verstynen et al., 2005; Horenstein et al., 2009; Verstynen and Ivry, 2011; Diedrichsen et al., 2013). The firing rate of neurons in this region, when recorded directly in primate models, relates to movement parameters such as acceleration and velocity (Kubota and Hamada, 1978), extent of movement, direction and amplitude (Fu et al., 1993; Kurata, 1993). Similarly in humans, activity in ipsilateral premotor cortex is modulated by task parameters that dictate the level of neural drive that must be directed to the muscles of the active limb. Elevations in cerebral blood flow related to movement velocity have been reported for ipsilateral premotor cortex, anterior cerebellum, superior parietal lobule and basal ganglia (Turner et al., 1998), and corresponding to movement frequency in ipsilateral premotor cortex and cerebellum (Jenkins et al., 1997). During repetitive key tapping movements, as the level of force necessary to depress the key is increased from 5 to 60% maximal voluntary contraction (MVC), there is a pronounced increase in regional cerebral blood flow (rCBF) in primary motor cortex ipsilateral to the active hand (Dettmers et al., 1996). Similarly, BOLD signal intensity registered in ipsilateral M1 scales with the applied level of force (Dai et al., 2001; van Duinen et al., 2008). In this regard, it is notable that when comparisons are made within individual studies (e.g., Walters, 1955), or across studies (Zhou, 2000), the degree of cross education appears to be contingent upon the level of voluntary drive generated during training.

In seeking to establish whether activity generated during unilateral training drives concurrent adaptations in both cerebral hemispheres that are sufficient to increase the functional capacity of the untrained limb, it is thus necessary to assess the totality of neural pathways and mechanisms that may play a causal role. Recognizing that in all natural tasks control is achieved through the balanced modulation of inhibitory and facilitatory processes, it is also important to consider whether specific variations in this balance may arise through training, be subject to chronic adaptation over varying time courses, and exert a functional effect upon movements of the opposite limb.

## **THE CONCEPT OF CROSSED SURROUND INHIBITION**

Studies in cat indicate there are facilitatory connections with the homotopic area of the opposite motor cortex that are surrounded by a more extensive zone in which inhibitory responses to transcallosal stimulation may be obtained (Asanuma and Okuda, 1962). Single unit studies further reveal that there is wider dispersion of inter-spike intervals in the peripheral (inhibitory) zone, suggesting a greater number of interceding synaptic relays (Kogan and Kuraev, 1976). This is consistent with the conjecture that callosal neurons are glutamatergic (Werhahn et al., 1999) and facilitatory to their immediate targets (Houzel and Milleret, 1999). Thus, the extent to which the output of primary motor cortex invokes crossed inhibition is contingent on neural interactions that converge upon circuits local to the opposite hemisphere (Bianki and Shrammapril, 1985; Berlucchi et al., 1990; Daskalakis et al., 2002; Carson, 2005).

It has been proposed that this mode of organization provides a means of focusing activity in thalamocortical relays—via surround inhibition (Beck and Hallett, 2011), an effect that is attenuated markedly by callosal section (Bianki, 1981). Importantly in the present context, the narrowing of the excitatory focus that occurs through this means is thought to be reciprocal in nature (Bianki and Makarova, 1980). Increases in surround inhibition in one hemisphere give rise to a reverse (i.e., symmetrical and selective) influence on the contralateral hemisphere (Bianki and Shrammapril, 1985). If the modulation of intracortical inhibition by means of callosal projections (**Figure 2**) is indeed reciprocal (e.g., Pal, 2005), the changes that occur in the organization of the homologous representation of the muscles engaged in training can be conceived of as being functional and adaptive, rather than simply incidental. From a broader phylogenetic perspective, it would appear likely that the mechanisms underlying cross education have bestowed fitness beyond the range of circumstances that are the subject of contemporary interest.

It has long been supposed that inhibition in general (Welch, 1898), and surround inhibition in particular (e.g., Denny-Brown, 1967), plays a crucial role in the selective recruitment of the focal muscles engaged in a task, and (i.e., with training) the disengagement of muscles with actions that might otherwise interfere with the desired movement outcome (Carson, 2006). Nonetheless, it remains to be determined whether surround inhibition arises

**FIGURE 2 | Crossed reciprocal surround inhibition.** The black filled circles represent excitatory nodes within intracortical circuits (i.e., in both hemispheres) that project to the focal muscles engaged by the training task. These are surrounded by a more extensive inhibitory zone. Following Bianki (e.g., Bianki and Makarova, 1980; Bianki and Shrammapril, 1985), Panel **(A)** represents interactions that occur during unilateral training. Panel **(B)** illustrates the altered state that results from unilateral training: the excitatory foci are enlarged, and surround inhibition is reduced.

from interactions local to cortex, or is mediated by inputs from other nodes of the motor network (Beck and Hallett, 2011). Duque et al. (2008) have previously raised the possibility that changes in surround inhibition may be one factor mediating the bilateral neuroplastic adaptation that results from unimanual training. The empirical literature concerning changes in surround inhibition that may occur as a consequence of training is, however, somewhat sparse. It is not even clear that surround inhibition can be revealed in all individuals using current TMS-based measures (e.g., Kang et al., 2012). Repetitive movements requiring the selective engagement of a single effector lead to diminution of MEPs and increased intracortical inhibition (inferred using paired-pulse TMS techniques) in other hand muscles (Liepert et al., 1998; Bütefisch et al., 2005). Conversely, following tasks that require synchronized movements of two fingers the opposite effect is obtained (Kang et al., 2012). We are not aware of any studies that have directly examined variations in surround inhibition in the context of cross education.

## **EXPERIMENTAL INDICES OF INTERHEMISPHERIC INHIBITION AND FACILITATION**

The possibility has been raised elsewhere that unilateral practice exerts its chronic effects on the functional capacity of the opposite limb through modification of the focal excitatory relationship between the primary motor cortices and/or the characteristics of the inhibition engendered by intracortical circuits (Hortobagyi, 2005). Task- and practice-dependent variations in these processes, and their balance, are however, challenging to resolve using the tools currently available in human electrophysiology.

It is well-documented that if an initial magnetic (conditioning) stimulus is applied to one primary motor cortex shortly (6–15 ms) before a second (test) stimulus is directed to the other M1, the magnitude of the response to the test stimulus is reduced (Ferbert et al., 1992). If however, the test response is evoked by transcranial electrical stimulation (TES), magnetic conditioning stimuli applied to the opposite hemisphere have no such effects (Ferbert et al., 1992; Hanajima et al., 2001), thus corroborating the assumption that inhibitory effects of M1 inter-hemispheric projections are mediated by local interneurons. While such interhemispheric inhibition (IHI) is more readily elicited in the laboratory, facilitation may also be obtained (Ugawa et al., 1993; Salerno and Georgesco, 1996; Hanajima et al., 2001; Baumer, 2006). The interval between the conditioning stimulus and the test stimulus is typically shorter than that required for IHI (Salerno and Georgesco, 1996), and the effect is most readily precipitated when the conditioning is either by TES or medially directed magnetic stimulation (Hanajima et al., 2001).

In seeking to use these techniques as a means of elucidating the neural basis of cross education, it is necessary to consider whether they are capable of discriminating changes in the excitatory balance between the primary motor cortices caused by unilateral training, from alterations in excitatory-inhibitory interactions within local interneuron circuits in the hemisphere ipsilateral to the training limb. It would also be advantageous to first demonstrate that they are capable of resolving the basis of acute variations in corticospinal excitability, such as those obtained *during* contractions of muscles in the ipsilateral limb. In this regard, tonic or pulsed isometric contractions are typically used as experimental paradigms (for a review see Perez, 2012).

In a condition in which the sizes of the conditioned and test MEP were matched across torque levels, Perez and Cohen (2008) reported that IHI measured in the resting flexor carpis radialis muscle (FCR) during isometric flexion of the opposite wrist was lower when torque was generated at 30% and 70% maximum voluntary contraction (MVC) than at 10% MVC (see also Chiou et al., 2013). In contrast, when the MEPs generated by the conditioning stimulus (CS) were not matched across conditions, an increase in IHI was obtained. This accords with the outcomes of other studies in which matching of the conditioning stimulus intensity was not performed (Ferbert et al., 1992; Vercauteren et al., 2008; Talelli et al., 2008a). Hinder et al. (2010b) reported a similar upwards scaling of IHI during pulsed applications of force (5% to 30% MVC).

A small number of studies concern changes in IHI arising as a result of short-term practice. Bologna et al. (2012) required that individuals maximize the initial acceleration of ballistic abduction movements of the (right—dominant) index finger, while attempting to maintain constant the level of activity (at 5–10% MVC) recorded from the homologous [first dorsal interosseus (FDI)] muscle of the opposite limb. Practice consisted of 100 repetitions of the movement at a rate of ∼ 0.2 Hz. Prior to and following these movements IHI (adjusted CS and TS intensities) was assessed at rest using interstimulus intervals of 12 ms (prototypical short-latency) and 30 ms (long-latency). Although the practice-related improvements in performance were accompanied by an increase in the excitability of corticospinal projections from the contralateral M1 (i.e., to the training limb), there were no corresponding changes in IHI (i.e., from the "trained" to "non-trained" hemisphere). In contrast, in the context of a task that required modulation of precision pinch (index finger and thumb) grip to acquire a sequence of five force targets, improvements in the speed and accuracy of performance of the (nondominant) left hand were observed following 180 training trials performed by the right hand (over 30 min). This positive transfer of learning was accompanied by a decrease in IHI ("trained" to "non-trained" hemisphere)—estimated using adjusted CS and TS intensities (Camus et al., 2009).

In one of the only studies conducted thus far in which potential variations in IHI have been assessed in the context of chronic training protocols, Hortobagyi et al. (2011) engaged volunteers to participate in 20 training sessions, conducted over 8 weeks, during which 1000 submaximal (80% MVC) applications of (abduction) force by the right index finger were undertaken. The maximum force applied by the trained finger was elevated by 49.9% as a consequence of the intervention, and the untrained finger exhibited an increase of 28.1%. Measures of IHI ("trained" to "non-trained" M1) were obtained at rest prior to the intervention and after every fifth session, using CS intensities that were fixed, and TS intensities that were adjusted (within and across sessions) for each participant. Similar estimates were also recorded at the beginning and end of these specific training sessions. It was reported that IHI decreased by 30.9% over the course of the entire intervention, and acutely by 8.9% on average during single sessions. In addition when the degree of cross education was correlated (across participants) with changes in IHI, the level of covariation was observed to increase over the course of the intervention.

The findings of Hortobagyi et al. (2011) provide a strong indication that the chronic effects of unilateral training upon movements of the opposite limb are mediated, at least in part, by processes manifested via TMS derived IHI (assessed at rest). This interpretation is not without some caveats. The conclusions that are drawn on the basis of the IHI technique can depend profoundly on the control of conditioning stimulus intensities. For example, in circumstances in which both the CS and TS are fixed, an increase in IHI is obtained with elevations in contraction (ipsilateral to TS) intensity (e.g., Ferbert et al., 1992; Perez and Cohen, 2008; Talelli et al., 2008b; Vercauteren et al., 2008). If however, the stimulation intensity is adjusted to maintain constant the amplitude of the conditioning MEP, experimentally elicited (short-latency) IHI is attenuated with increases in the level of contraction (Perez and Cohen, 2008; Chiou et al., 2013).

A more general challenge is thereby illustrated—that of relating measures of interhemispheric interaction obtained from conscious humans using non-invasive (e.g., magnetic) brain stimulation to those derived from reduced animal preparations. The ipsilateral silent period (iSP) provides another index of interhemispheric inhibition. It is obtained when TMS is delivered at high intensity to the M1 ipsilateral to contracting muscles (Wassermann et al., 1991). The spread of activation at these intensities appears to mask any excitatory effects, thus giving rise to net inhibition of the opposite motor cortex. As with IHI, the initial portion of the iSP appears to be mediated, at least in part, by the fibers of the corpus callosum (Meyer et al., 1995, 1998). Nonetheless, short-latency (e.g., 8 ms interval) IHI and iSP do not vary equivalently in response to a number of experimental manipulations (Chen et al., 2003; Giovannelli et al., 2009). The greater covariation observed for the iSP and long-latency (e.g., 40 ms interval) IHI suggests that these respective effects may be subserved by overlapping subpopulations of neurons (Chen et al., 2003).

If considered in relation to the variations that are manifested at different levels of isometric contraction, the area of the iSP (i.e., the degree of inhibition of EMG activity ipsilateral to the stimulation) is greater during both minimal (5% MVC) and maximum engagement of the homologous (FDI) muscle (i.e., opposite limb) than when it is quiescent (Giovannelli et al., 2009). These outcomes are thus consistent with those obtained for short-latency IHI, when fixed CS and TS intensities are employed, since both measures of inhibition scale with the intensity of the contraction performed by the homologous muscle. Notably in respect of Giovannelli et al. (2009), increases of iSP area were also obtained during (maximal) contraction of the opposite extensor indicis proprius (EIP), but not with contraction of more proximal upper limb muscles or lower limb muscles, suggesting that the effect is topographic but not entirely focal (see also Hinder et al., 2010b). To the best of our knowledge, the iSP technique has not yet been used to investigate interlimb transfer of training.

How is the ostensible elevation in inter-hemispheric inhibition (i.e., fixed CS intensity IHI; iSP) that occurs during the course of unilateral ballistic (e.g., Duque et al., 2007) and isometric contractions (Vercauteren et al., 2008), and rises with the intensity of contraction (e.g., Perez and Cohen, 2008; Giovannelli et al., 2009), to be reconciled with the decreases that are measured acutely (at rest) during the course of a unimanual training session (e.g., Camus et al., 2009; Hortobagyi et al., 2011) and chronically over multiple sessions (Hortobagyi et al., 2011), and which may be related to the cross education of motor function that is observed in such circumstances? One possibility is that *during* voluntary contractions, the excitability of transcallosal projections is modulated in parallel with that of corticospinal neurons (Avanzino et al., 2007). While this will give rise to increases in both inter-hemispheric facilitation and inhibition, it may simply be the case that the experimental techniques that are typically employed (e.g., IHI and iSP) do not provide an adequate representation of variations in the local balance between excitation and inhibition that occur in the ipsilateral M1 during voluntary movement. We are, for example, unaware of any attempts to apply paired-pulse TMS techniques to examine levels of inter-cortical facilitation in these contexts. Alternatively, the decreases in IHI that are registered when the CS intensity is adjusted (downwards) during unilateral contractions may be gauged more representative of processes that are implicated in cross education of function. It is also noteworthy that experimentally elicited (short-latency) IHI is abolished when forces greater than 50% MVC are generated by the muscle in which the test MEP is recorded (Chen et al., 2003). It seems likely that the decreases in IHI observed as a result of training—in contexts in which cross education is obtained, express alterations in the excitatory-inhibitory balance within interneuron circuits local to the hemisphere ipsilateral to the training limb, rather than changes in the characteristics of the projections between the primary motor cortices that are recruited at rest by magnetic stimulation. Such acute (i.e., within a single training session) and chronic (i.e., across multiple training sessions) alterations may arise in association with, for example, increases in surround inhibition induced during training by reciprocal interactions between the hemispheres (Bianki and Shrammapril, 1985). In summary, experimental indices of interhemispheric inhibition (and facilitation) provide only a partial indication of the relationship between the physiological processes that are operative during the execution of the training movements, and thus of the neural pathways that mediate their cumulative functional consequences.

## **THE NATURE AND ROLE OF CROSSED FACILITATION**

The evidence that voluntary contractions of one limb—at the intensities employed in training regimes, give rise to increases in the excitability of descending projections to the homologous muscles of the opposite limb is incontrovertible. Furthermore, many of the factors that modulate this crossed facilitation are also those that, when manipulated, alter the level of cross education that is brought about by unilateral training. Perhaps the strongest indication that the two phenomena are functionally related is provided by the recent report that crossed facilitation registered during background contractions (20% and 80% MVC) of the trained (homologous) muscle of the opposite limb, increased over the course of 20 training sessions. These changes were correlated highly with the level of cross education (Hortobagyi et al., 2011).

When repetitive movements are performed using a distal effector of one limb, the frequency with which clearly distinguishable EMG activity is registered in the homologous muscles of the opposite limb increases when the focal movements are subject to external resistance (Cernacek, 1961), or performed with greater effort (Hopf et al., 1974). Similarly, if the muscles of the active limb are progressively fatigued, there is a corresponding increase in the EMG activity recorded from the opposite limb (Arányi and Rösler, 2002). These data are consistent with the proposal that the extent of irradiation to the opposite limb is contingent upon the level of neural drive directed to the muscles engaged in the focal movement (Todor and Lazarus, 1986). In the absence of voluntary drive, when a limb is moved passively, functional neuroimaging techniques generally fail to reveal signal change in ipsilateral cortex, despite the registration of activity in the hemisphere contralateral to movement (Francis et al., 2009; Yu et al., 2011; Szameitat et al., 2012). There is also at least one proposal that, at low levels of force, unilateral contractions suppress ipsilateral motor cortical activity (Liepert et al., 2001). In this regard, it has been suggested that lower levels of crossed facilitation during low force tasks, particularly when these are bimanual, may serve to prevent interference between the limbs. Conversely, the presence of motor irradiation during high force movements is ostensibly advantageous in carrying heavy loads when bilateral cooperation is desirable (e.g., Liepert et al., 2001). This conjecture is, however, inconsistent with the widely noted bilateral force deficits expressed in circumstances in which maximal levels of motor output are demanded simultaneously (Ohtsuki, 1983; Archontides and Fazey, 1993).

It has long been recognized that cross facilitation effects persist beyond the period of training. The phenomenon, which was first dubbed the "aftercontraction effect" (Craske and Craske, 1986), may also be detected on the basis of changes in corticospinal output in response to TMS (e.g., Carson et al., 2008). In the context of tasks in which short-term unilateral practice engenders bilateral improvements in performance, sustained increases in the excitability of corticospinal projections to the muscles of the untrained limb (recorded at rest) have been reported in acute (Carroll et al., 2008; Lee et al., 2010; Hinder et al., 2011; Poh et al., 2013) and chronic (Koeneke et al., 2006) ballistic training protocols. Corresponding outcomes have been obtained during (acute) practice of precision grip force modulation (Liang et al., 2007). It cannot be assumed, however, that such changes are of adaptive functional significance, since they are obtained not only for homologues of the muscles engaged in the training task, but also for homologues of muscles that do not make a *direct* mechanical contribution to the action that is trained (Carroll et al., 2008). In addition, there is evidently no relationship across participants between the degree of cross education and increases in the excitability of corticospinal projections to the homologues of muscles engaged in training, when these are assessed at rest in the context of either acute (Carroll et al., 2008; Hinder et al., 2011) or chronic (Hortobagyi et al., 2011) training protocols. On these grounds, it would appear reasonable to consider whether the functional adaptations that underpin interlimb transfer of gains in performance either occur in areas upstream of the primary motor cortex, or via changes in the effectiveness of synaptic transmission through projections from these areas onto M1 targets.

It has been reported that short- interval intracortical inhibition (SICI) increases during isometric contractions of the ipsilateral limb (Perez and Cohen, 2008) and the modulation of precision pinch grip force (Camus et al., 2009), whereas corresponding effects have not been obtained for ballistic movements (Hinder et al., 2010a). The observation that the effect of IHI conditioning on SICI invoked within the M1 ipsilateral to contractions is stronger during efforts at 70% MVC than at rest, suggests either that at least some of the modulation of intracortical circuits mediating SICI occurs via direct input from the opposite hemisphere (Perez and Cohen, 2008) or that these circuits are interposed with interneurons (i.e., in the target hemisphere) that are engaged in the expression of IHI. Nonetheless, neither of these putative mechanisms provide a direct account of the influence of factors such as vision on levels of cross facilitation (Carson et al., 2005; Garry et al., 2005; Carson and Ruddy, 2012) in circumstances in which the descending output from the active M1 does not vary across conditions (see also Avanzino et al., 2007). Furthermore, measures of intracortical inhibition (SICI) and facilitation (ICF) do not change systematically within (McCombe Waller et al., 2008) or across training sessions, and thus these measures do not correlate with the induced levels of transfer (Hortobagyi et al., 2011). This lack of association suggests that the processes that mediate the expression of SICI and ICF (when assessed at rest) are incidental to those that underlie cross education. While it remains to be resolved whether a local recasting of the inhibitory-excitatory balance as characterized by variations in ICF or SICI, is promoted directly by variations in the state of transcallosal neurons projecting from the homologous "active" M1 during unilateral contractions, the alterations in intracortical excitability thus expressed do not appear functionally related to changes that are instrumental in relation to the interlimb transfer of gains in performance realized through repeated training.

As far as we are aware, only a single study has used a perturbation approach to gain insight in relation to the locus of adaptations that underlie cross education. As the results of this study are amenable to a number of alternative interpretations, it is worth considering in some detail. Lee et al. (2010) asked their participants to perform 300 ballistic movements with a view to maximizing acceleration of the right index finger. The peak acceleration of the trained finger increased by 93%, and that of the untrained (left) finger increased by 62%. When rTMS (15 min at 1 Hz) was applied subsequently to the right M1—contralateral to the untrained limb, the peak acceleration of the left index finger was attenuated by 15.5% (i.e., relative to the value obtained immediately following the cessation of training). In contrast, the performance of the trained right finger was unchanged. In the complementary condition, in which rTMS was administered to the (left) M1 contralateral to the training limb, the peak acceleration of the right index finger was attenuated by 13.1%. There was no corresponding diminution of performance for the left index finger. For the groups that performed active training movements, there was marked elevation of MEPs evoked in the target muscle of the trained and untrained limbs following the 300 movements (prior to rTMS), suggesting increases in the excitability of the output circuits from both primary motor cortices. As noted previously, such changes do not appear instrumentally related to levels of transfer (Carroll et al., 2008; Hinder et al., 2011). It seems unlikely therefore that the disruptive impact of the rTMS upon performance was realized through the M1 circuits that are recruited in generating a corticospinal volley in response to single pulse TMS. It has been highlighted recently that the state of the cortex at the time of stimulation (e.g., rTMS) both determines the overall neuronal response of the stimulated cortex, and shapes the responsiveness of distinct subpopulations of cortical neurons (Siebner et al., 2009). The functional consequences of rTMS on the output of M1 are therefore likely to be quite different if delivered at rest (or in control conditions in which no preceding movements are performed), or in circumstances in which the stimulated neurons have been preconditioned by movements of the contralateral or ipsilateral limb (as in the key experimental conditions of the Lee et al. study). Thus, it is possible that the performance decrements observed by Lee and colleagues following rTMS reflected disturbance of interneuronal networks other than those directly engaged in generating corticospinal output. Conceivably these networks include the intracortical circuitry that is engaged in IHI protocols, and which exhibits adaptation related in extent to the level of cross education (Camus et al., 2009; Hortobagyi et al., 2011). Alternatively, the effects of the rTMS on task performance may simply reflect attenuation of the net M1 response to synaptic input from other brain areas, rather than disruption of mechanisms acting within M1 that are specifically related to cross education.

The interpretation that the output circuits of the primary motor cortex ipsilateral to the training limb are the conduit rather than the wellspring of cross-limb transfer is likewise consistent with reports that unilateral strength training increased the capacity of the motor cortex to drive the homologous untrained muscles (Lee et al., 2009). These observations serve to illustrate the more general point that, at least with respect to resistance training and other "maximal output" paradigms, chronic adaptations are often only revealed in circumstances in which output circuits of the primary motor cortex receive synaptic drive (i.e., during voluntary contractions). This appears to be the case both for the untrained (Hortobagyi et al., 2011) and the trained limb (Griffin and Cafarelli, 2007; Carroll et al., 2009). Indeed, since it is by no means established that resistance training engenders adaptations in M1 output circuitry contralateral to the limb that is directly engaged (Carroll et al., 2002; Jensen et al., 2005), even in circumstances in which transmission via this area necessarily approaches maximum levels during training, it would appear counterintuitive if the crossed effects were to be mediated by this means.

### **SOURCES OF FUNCTIONAL CONNECTIVITY**

It is necessary to consider whether there are other sources of bilateral functional connectivity within the motor network that have the potential to mediate cross activation, and provide a mechanism for cross education of function (Farthing et al., 2007). During unilateral movement, ipsilateral activation has been reported not only for M1 and premotor cortex, but also in regions including supplementary motor cortex (SMA) (Dai et al., 2001; Diedrichsen et al., 2013), primary sensory cortex (S1) (Dai et al., 2001; Kobayashi et al., 2003), cerebellum (Dai et al., 2001; van Duinen et al., 2008; Horenstein et al., 2009), parietal lobe (Dai et al., 2001; Hanakawa et al., 2005; van Duinen et al., 2008; Horenstein et al., 2009), and cingulate cortex (Dai et al., 2001).

For example, activity in the cingulate motor area (CMA) which forms part of the anterior cingulate cortex and is thought to be a strategic entry point for limbic influence on the voluntary motor system, is closely associated with the amount of effort demanded by a motor task (Winterer et al., 2002). The observation that a high proportion of CMA neurons exhibit activity that is modulated when the ipsilateral hand is engaged (Kermadi et al., 2000), is consistent with the widespread finding that crossed facilitation is accentuated with increased effort or volition (Hopf et al., 1974). In primates, the cingulate motor area (CMA) is very densely connected with its homologue in the other hemisphere via fibers of the corpus callosum (Rouiller et al., 1994). In addition, functional connectivity between the caudal ACC and the primary and supplementary motor areas in humans is now clearly established (Koski and Paus, 2000). Thus, it is conceivable that the bilateral activity registered in elements of the motor network during unilateral movement, arises first in the cingulate cortex of the contralateral hemisphere, extends through callosal fibers to the ipsilateral CMA, and subsequently to other (ipsilateral) motor areas before influencing M1 output (Carson et al., 2005) (**Figure 3**). This conjecture is supported by the observation that in neurologically healthy human subjects, the activity registered in cingulate cortical areas during unimanual movements is correlated positively with the size of the posterior truncus of the corpus callosum (Stancák et al., 2003 ´ ). Taken together, these findings suggest that the level of input from regions such as anterior cingulate

cortex that appear to act as neural mediators of the central command (e.g., Chefer et al., 1997) may also determine the bilateral distribution of activity across elements of the cortical motor network that arises during the effortful engagement of a single limb.

### **THE STRUCTURAL BASIS OF CROSS ACTIVATION**

What are the possible grounds for evaluating the proposal that cross education of motor function is mediated by mechanisms acting via neural pathways projecting from areas upstream of the primary motor cortices? A necessary but not sufficient step is to assess bilateral structural connectivity with a view to delineating the routes by which such functional interactions might occur.

In non-human primates, the density of callosal connections exhibits a rostrocaudal gradient for the M1, SMA-proper, and pre-SMA (Liu et al., 2002), whereby the hand representation in primary motor cortex is relatively sparsely connected with its contralateral counterpart (Jenny, 1979; Rouiller et al., 1998). The pre-SMA, which is believed to be involved in early phases of motor preparation and planning, exhibits much denser callosal connectivity than the SMA-proper or primary motor cortex. The scope for direct inter-hemispheric interactions via callosal pathways thus decreases progressively along a functional gradient that culminates in those regions that have the most prominent role in generating motor output. In the context of bimanual movement, it has been proposed previously that this organization is consistent with the requirement that inter-hemispheric interference at the level of execution is minimized, while mutual "cross-talk" in relation to movement planning is promoted (Liu et al., 2002). The endeavor of extending this approach to humans has been facilitated in recent years through new technologies that complement and extend anatomical studies undertaken using classical postmortem techniques and animal models. A key advantage of these new approaches is the facility to obtain measurements of structural connectivity *in-vivo*, and relate these both to indices of brain activity and to behavior.

Positioned directly below the gray matter cortex, cerebral white matter forms a dense network of communication cables that connect distant brain regions, and composes half of the human brain, a percentage much greater than in any other animal (Fields, 2008). The integrity, density and structural connectivity of the white matter pathways can be measured and imaged using diffuse tensor imaging (DTI), which allows the tracking of water diffusion in tissues in the brain, using output measures such as fractional anisotropy (FA). This quantifies the diffusion of water molecules, the movement of which are constrained by cellular structures such as the walls of the axons. Molecular motion is limited further by layers of lipidic cover that constitutes the myelin sheath. The measure thus derived is largest in regions that are assumed to be heavily myelinated or that have densely packed axons, although the precise nature of the link between FA (i.e., derived *in vivo*) and human histology (i.e., myelination) remains elusive (see Alba-Ferrara and de Erausquin, 2013, for a review). Whether the measure expresses constrained diffusion caused primarily by the structure of the axon itself, or the ultrastructure of myelin sheaths, the assumption nonetheless is that higher FA reflects greater structural connectivity.

While recent DTI derived evidence suggests that interhemispheric callosal projections are largely homotopic (Fling et al., 2011), within the corpus callosum there are marked differences in the quantity and strength of fibers projecting from the different components of the motor network that are involved in voluntary movement (**Figure 4**). In terms of quantity, there are significantly more fibers connecting homologous SMA regions than connecting M1, primary sensory cortices (S1), pre-SMA or dorsal premotor cortices (PMd) (Fling et al., 2011). In contrast to the conclusions drawn on the basis of retrograde tracing in primates, it appears that in human there are fewer interhemispheric fibers from Pre-SMA than from SMA, but more than for M1, S1 or PMd. In relation to number of fibers, there are more homotopic projections for M1 than for S1 or PMd.

Panel **(A)** represents fractional anisotropy (FA) of transcallosal fibers connecting homologous motor regions. Thicker tubes indicate higher FA values. SMA fibers exhibit greater FA values than pre-SMA, M1 and S1. Transcallosal PMd fiber FA values are greater than those connecting pre-SMA, M1 and S1. Fibers connecting homologous M1 and homologous pre-SMA exhibit higher FA values than S1 fibers. Panel **(B)** represents the quantity of interhemispheric fiber tracts connecting homologous motor regions. There are more fiber tracts connecting homologous SMA regions than M1, PMd and pre-SMA. There are more fibers connecting pre-SMA than M1, S1 or PMd. Transcallosal M1 fibers are more numerous than those connecting S1 or PMd. (See text for details).

It is important to note, however, that this metric is not necessarily paramount in relation to the functional implications of white matter connectivity. PMd-PMd interhemispheric connections, along with SMA-SMA fiber tracts, display the highest microstructural integrity (FA) values (**Figure 3**). Lower FA values are obtained for homotopic projections from S1, M1 and pre-SMA, whereas in relation to PMv, direct interhemispheric connections have not been identified using DTI (Fling et al., 2011).

Two critical considerations are thereby highlighted. In the first instance, summary measures of connectivity calculated for the entire bundle of fibers passing across the corpus callosum are unlikely to be revealing with respect to the mediation of cross education, when these are considered in relation to variations in behavioral outcomes exhibited within or between groups of individuals. Furthermore, specific metrics of fiber orientation and strength—such as FA values (derived for projections between clearly delineated nodes in the motor network), will probably bear a more direct relation to functional interactions between the limbs than global or local estimates of fiber number.

As case in point, Bonzano et al. (2011a) reported that in a group of patients with corpus callosum pathology due to multiple sclerosis (MS), levels of interlimb transfer in a reaction time task were not associated with FA values derived for the entire callosal body. A positive relationship was, however, obtained when FA values were calculated for a subregion (CC3), following de Lacoste et al. (1985), presumed to encompass fibers projecting to primary motor and sensory areas (but not for a subregion (CC2) deemed to contain fibres projecting to premotor and supplementary motor areas).

### *Structural connectivity between the primary motor areas*

It has been demonstrated that the microstructural integrity of the white matter in transcallosal pathways projecting into the M1 hand area correlates positively with levels of interhemispheric inhibition, measured both using paired pulse techniques (Wahl et al., 2007), and the ipsilateral silent period (Koerte et al., 2009; Fling et al., 2011). Given that greater microstructural integrity as indexed by FA, may be reflective of either the dense packing of many fibers, or their myelination quality (expressed as lower signal degradation), the finding of elevated interhemispheric inhibition between motor cortices in individuals with higher FA values may be indicative of an excitatory signal transmitted via the corpus callosum that results in proportionately greater activation of the inhibitory interneuron network in the target hemisphere. As emphasized previously, however, the net balance between inhibition and facilitation that results from transcallosal input (i.e., from the opposite M1) is also subject to task-dependent modulation by areas upstream of M1, such as premotor cortex, which assume a focusing role by regulating the activity of interneurons in primary motor cortex (Münchau et al., 2002).

The relevance of this general point in relation to the interpretations that might be drawn concerning the structural pathways that mediate interhemispheric inhibition on the one hand, and cross education of motor function on the other, cannot be overstated. While performing index finger to thumb opposition movements, individuals with MS exhibit higher levels of BOLD response in ipsilateral M1, and decreased levels of interhemispheric inhibition (registered using iSP). These variables correlated (negatively and positively, respectively) with FA values calculated for the body of the corpus callosum (Lenzi et al., 2007). Nonetheless, the capacity for intermanual transfer appears largely unaffected in this population (Bonzano et al., 2011a).

In a related vein, there is an age-related dissociation in the relationship between IHI and M1 callosal tract microstructural integrity. Young adults with relatively larger FA values also exhibit greater (iSP derived) interhemispheric inhibition, whereas for older adults the opposite relationship is obtained (Fling and Seidler, 2012). There is mounting evidence to suggest that both the quantity and quality of cerebral white matter diminishes with age (e.g., Seidler et al., 2010; Sullivan et al., 2010). It is also well-established that experimentally derived measures of interhemispheric inhibition diminish overall with advancing age, and that these changes are related to the level of ipsilateral activity that is present during the performance of single limb tasks (Talelli et al., 2008a,b). Furthermore, commensurate elevations of crossed facilitation (e.g., Fling and Seidler, 2012) and contralateral irradiation of motor output throughout the lifespan have been thoroughly documented (see Addamo et al., 2007 for a review). At first glance it might therefore appear paradoxical that levels of cross education are diminished in older adults in comparison to younger counterparts (Bemben and Murphy, 2001) in some cases markedly so (Hinder et al., 2011). Taken as a whole however, these lines of evidence serve to emphasize that the structural factors that directly influence levels of interhemispheric inhibition between the primary motor cortices may not to be those that assume a principal role in mediating the crossed transfer of functional capacity.

## **BILATERAL ACCESS APPROACH**

#### **THEORETICAL CONTEXT AND SCOPE OF THE PRESENT ANALYSIS**

In seeking to account for the mechanisms that give rise to cross education of motor function, there have been numerous advocates of the view that neuroplastic changes occurring in conjunction with unilateral training are amenable to utilization (subsequently) when the untrained limb is engaged. A point of contrast with cross activation models is that task and effector specific changes in the state of neural circuits projecting to the muscles of the quiescent limb are not necessarily anticipated for the period of training. The integrity of any such distinction necessarily depends on the facility to demarcate brain regions that assume a *functional* role in relation to movements performed on one side of the body, but not on the other. As highlighted previously, it is not even clear that primary motor cortex can be categorized in this manner (Bianki and Shrammapril, 1985). Although some proponents of the bilateral access approach have emphasized the role of the corpus callosum as a means for information transfer from a single hemisphere in which the "motor engram" has been elaborated (e.g., Taylor and Heilman, 1980), it is not necessarily apparent that such lateralization is a logical necessity. On a priori grounds alone, bilateral representation (e.g., Parlow and Kinsbourne, 1989) of a capability acquired unilaterally cannot be excluded. The possibility has also been highlighted (Nadel and Buresova, 1968) that transcallosal "read-out" of a lateralized memory trace may initiate an active process in the "trained" hemisphere which precipitates transcallosal information flow in the opposite direction that is to say—from the trained to the untrained hemisphere (**Figure 5**). Through active "write in," which may occur over the course of just a few trials or on even a single trial (Fenton and Bures, 1994), a duplicate "motor engram" is formed in the untrained hemisphere—a mode of transfer that has been termed *imperative*. Direct "read-out" of a lateralized engram that does not require an equivalent active process has been designated *facultative* transfer (Bureš et al., 1988).

Necessarily therefore, the patterns of neural activity that are instrumental in enhancing execution during training, and their relationship to those present during the ensuing performance of the opposite untrained limb, are an empirical matter. It is our argument that when considered on this basis, there emerge few grounds for distinction between the cross activation and bilateral access models. Rather, we contend that the degree to which there is bilateral engagement of various elements of the motor network, and the extent of cross education that accrues from unilateral training, is contingent upon specific task parameters. In seeking to illustrate this point in the sections that follow, we restrict our attention to a relatively small subset of exemplars. Consideration is not, for example, extended to interlimb transfer in the context of prism (e.g., Martin et al., 1996), visuomotor (e.g., Sainburg and Wang, 2002) or force-field adaptation (Criscimagna-Hemminger et al., 2003). In addition, factors that might influence asymmetries of transfer between the dominant and non-dominant limb are not considered at length.

Rather, we focus our attention upon variants of sequential motor tasks. Typically these require that buttons or keys be pressed by the fingers of one hand—often by means of isometric

**FIGURE 5 | Two phases of transfer.** In each instance the "X" represents the putative locus of training related adaptations. White circles indicate lateralized motor networks in their entirety. Following Nadel and Buresova (1968), Panel **(A)** represents the 'read-out' phase (solid arrow) whereby training related adaptations in motor networks projecting to the training limb are accessible during movements made subsequently by the untrained limb. Panel **(B)** depicts the possibility that, during the course of such access (i.e., "read out"), a "write-in" phase (dashed arrow) occurs, whereby a parallel duplicate motor engram is formed.

contractions, in accordance with a memorized or perceptually cued sequence. In some variants (e.g., Hicks et al., 1982), the participant is instructed to repeat the sequence as many times as possible within a fixed interval. In others, such as the serial reaction time task (SRTT), (Nissen and Bullemer, 1987), participants respond repeatedly to a fixed sequence of stimuli, which is not typically made explicit. Learning is inferred on the basis of decreases in reaction time that accord with the probabilities governing transitions between successive stimuli in the sequence. The demands associated with actuation nominally remain fixed. That is, there is no overt stipulation for the keys or buttons to be depressed with increasing force or rapidity. Thus, since it is generally assumed that such tasks place *minimal* requirements on motor execution, progressive changes in their performance are typically interpreted as evidence of motor sequence learning (Hardwick et al., 2013).

## **TASK PARAMETERS: A CASE IN POINT**

In a study that engaged a large cohort of young adults in a five-key sequential tapping task performed with either the left or the right hand, Parlow and Dewey (1991) required that a subset of participants undertake the training phase (ten 15 s trials) while simultaneously engaging the opposite ("untrained") limb in the production of sustained (i.e., 15 s) sub-maximal (Experiment 1) or maximal (Experiment 2) isometric grip force. It was noted that these groups exhibited positive transfer of performance from the trained to the untrained limb that was comparable to that obtained for (control) groups that did not engage in the secondary task. On the basis of the assumption that the generation of maximum grip force by the "untrained" limb during practice of the sequential tapping task (i.e., by the opposite limb) prevented training task-specific "motor overflow" from engaging brain circuits that might in principle become adapted, these findings are commonly considered support for bilateral access models of cross education. Rather than it being the case that homologous circuits were engaged by this manipulation, the markedly different demands imposed by the sequence generation and isometric grip force tasks, coupled with the observation that dual task deficits were not necessarily present during acquisition, suggest an alternative interpretation—that the engagement of somewhat distinct brain circuits was required in each instance. In this light, the fact that the (secondary) task did not impede the transfer of acquired competence on all variants of the primary task reflects a *lack* of interference between the patterns of motor network activity associated with each form of action. The more general point is thereby illustrated. The specific neural pathways that mediate cross education of motor function are likely to be strongly contingent upon the dimensions of the task. These dimensions might include, for example, the degree to which maximal motor output is demanded, the extent to which the action goals can be represented in an internal or external coordinate scheme (e.g., Hikosaka et al., 1999), or the relevance of procedural knowledge (Obayashi, 2004).

## **NEURAL CORRELATES OF SEQUENCE LEARNING**

Empirical evidence derived from functional neuroimaging has reinforced the appreciation that distributed brain networks necessarily function in concert during the learning of motor sequences. Nonetheless, it does not inevitably follow that all of the constituent regions within these networks assume an equivalent role in the mediation of cross education. In the following sub sections, we adopt a pragmatic approach, whereby the individual brain regions that may be implicated are discussed individually. This should not be taken to imply that they function in an isolated fashion. Rather, there is unquestionably an integrated dynamic interplay between these regions, with their relative contributions to cross education having more or less emphasis depending upon factors such as task type, stage of learning, and task complexity.

## *Dorsal premotor cortex*

In primates, stimulation of both dorsal and ventral premotor cortices results in observable twitch responses, suggesting that they may play an instrumental role in the generation and control of movement (Dum and Strick, 2005). Furthermore, in addition to direct descending spinal projections, PMd has reciprocal connections with ipsilateral M1 (Dum and Strick, 1991, 2002; He et al., 1993, 1995), rendering it well-placed to provide neuromodulatory control of M1 output. Within PMd, there appears to be a rostrocaudal continuum of activity, with rostral (anterior) locations implicated in sensory and working memory style tasks, and caudal (posterior) aspects of PMd more prominently engaged in motor learning. More generally, given the direct projections that exist between rostral PM and prefrontal areas, and between caudal PM and M1 (and spinal cord) respectively, it has been proposed that rostral PM fields may represent a functional node of a prefrontal network, whereas caudal PM may be regarded as a true motor area engaged primarily during movement execution (Schubotz and von Cramon, 2003).

In a recent meta-analysis, Hardwick et al. (2013) reported that the degree to which dorsal premotor cortex (PMd) activity was lateralized depends critically upon the characteristics of the task. It was noted that variants of the SRTT elicited bilateral PMd activity, whereas tasks that required the learning of novel movement kinematics and dynamics gave rise primarily to left PMd activity (i.e., independently of the side of execution). This finding may be interpreted in light of the conjecture that the left PM is engaged during the acquisition of new motor patterns –whether performed with the dominant or nondominant hand, whereas the right PM is involved to a greater degree in the storage of sequences (e.g., Schubotz and von Cramon, 2003). The latter supposition is supported by the observation that levels of right PM activation co-vary with length of motor sequence (Sadato et al., 1996). In a perceptual counterpart of the SRTT, the requirement for serial prediction *per se* was associated with elevated activity in right PM. Increases in the number of elements in the sequence resulted in prominent increases in the levels of activation registered in PMd within both hemispheres (Schubotz and von Cramon, 2002). Thus, while the task-specific factors that determine the lateralization of PMd activity remain to be resolved, there is consensus that this brain region is a pivotal structure for motor learning in general, and for motor sequence learning in particular. Additionally, given that interhemispheric PMd-PMd connections are among the strongest of all motor regions (Fling et al., 2011), it may also be implicated in the cross education of performance in this context.

## *Supplementary motor area*

The finding of activity in the supplementary motor area (SMA) is common to all neuroimaging studies that have investigated SRTT learning (Hardwick et al., 2013). This is unsurprising given that which is known about the role of SMA and its interactions with adjacent cingulate motor cortical regions, both of which contribute to the initiation of voluntary movement (Deecke and Kornhuber, 1978; Hoffstaedter et al., 2012). There is also evidence that the SMA plays distinct functional roles at different times during the performance and acquisition of a new movement sequence. In the gaps between the generation of individual elements, SMA serves the function of encoding and planning the next movement in the sequence (Tanji and Shima, 1994; Gerloff et al., 1997; Shima and Tanji, 1998), whereas during the execution of these elements, SMA assumes an additional role in relation to online monitoring and control (Seitz and Roland, 1992; Tanji and Shima, 1994; Shima and Tanji, 2000; Padoa-Schioppa et al., 2002; Lee and Quessy, 2003). A task specific distinction may, however, be drawn between the two composite regions of SMA: pre-SMA and SMA proper. It has been revealed that only SMA proper is activated during tasks requiring novel movement kinematics or dynamics, whereas during variants of the SRTT, both SMA proper and pre-SMA are involved (Hardwick et al., 2013). Human neuroimaging data indicating a specific functional role for pre-SMA during variants of the SRTT are also consistent with evidence derived from single cell recordings in non-human primates (Tanji and Shima, 1994; Clower and Alexander, 1998; Shima and Tanji, 2000). Aside from sequence learning, the pre-SMA appears to have functions that are predominantly cognitive in nature, serving a minimal role in other forms of motor learning. Stimulation of SMA proper appears to enhance motor learning in a task with a sequential learning component, whereas pre-SMA stimulation has no such effect (Vollmann et al., 2012). Analogous to the rostrocaudal continuum of cognitive-motor function within the PMd, the SMA, which shares with PMd a cytoarchitecturally defined location on Broadman's area 6, is similarly subdivided, with the more rostral region (pre-SMA) assuming a role in cognitive functions, and the caudal SMA proper having undisputed motor properties (Hardwick et al., 2013).

## *Primary motor cortex*

While there is widespread evidence that M1 is integral to a network of brain regions involved in the learning and retention of motor skills, the extent of its contribution varies in a task and time-dependent fashion. In the initial stages of acquiring skills for which a significant degree of cognitive involvement is required, there are relatively high levels of activity in prefrontal, bilateral sensorimotor, and parietal cortices. It has been proposed that for tasks of this nature, the initial phases of skill acquisition are mediated via regions of a cortical network specialized for executive function, motor planning/execution and the processing of somatosensory feedback, and that sub-cortical circuits in the cerebellum and basal ganglia assume a commensurately greater role as automaticity of performance is achieved (Floyer-Lea and Matthews, 2004). It is also the case that during early learning of a "fast-as-possible" ballistic motor task—for which few cognitive demands might be assumed (Rosenkranz et al., 2007; Carroll et al., 2008; Hinder et al., 2011), and in visuomotor adaptation tasks following a perceptible state transition (Riek et al., 2012), there are increases in the excitability of corticospinal projections from M1.

The relative contribution of M1 to the most rapid phase of performance adaptation, as opposed a slow repetition-dependent component, in tasks requiring modified movement kinematics or dynamics, and the significance of this demarcation with respect to acquisition and retention, remains a matter of considerable debate (e.g., Richardson et al., 2006; Galea et al., 2011; Orban de Xivry et al., 2011; Riek et al., 2012). During unilateral motor sequence learning, elevated activity is registered in M1 ipsilateral to the training limb (Daselaar et al., 2003; Bischoff-Grethe et al., 2004; Verstynen et al., 2005). This is thought more likely to be reflective of excitatory rather than inhibitory neural activity (Waldvogel et al., 2000). There is also some evidence to suggest that left M1 is activated regardless of the limb that is the focus of training. In contrast, right M1 is not engaged prominently during right hand execution (Hardwick et al., 2013). While this pattern suggests that the left primary motor cortex performs a specialized function in this form of task (Jueptner et al., 1997b; Seidler et al., 2005; Bapi et al., 2006), it appears likely that the activity is more closely related to effector aspects than to serial prediction *per se* (Sanes and Donoghue, 2000; Hardwick et al., 2013).

## *Superior parietal lobule*

The parietal cortex has traditionally been considered as the bridge between vision and movement (Critchley, 1953; Milner and Goodale, 1993), with the superior parietal lobule (SPL) in particular assuming a significant role in relation to actions involving the hands (Mountcastle et al., 1975; Rizzolatti et al., 1998; Connolly et al., 2003; Glover et al., 2005; Battaglia-Mayer et al., 2007). This area is activated consistently during all motor variants of the SRTT, but not necessarily in tasks that require the acquisition of novel limb kinematics or dynamics. As most often the SRTT includes the requirement to respond to visual stimuli, and given the centrality of its relationship with PMd in visuomotor integration and control (Wise et al., 1997), it is perhaps unsurprising that the SPL is engaged during this type of motor sequence learning. In a learning task in which an auditory metronome was used to pace movements, and visual feedback was not provided, significant levels of SPL activation were not obtained (Jantzen et al., 2002). In the context of a network in which PMd represents the "hub" of sequence learning, the SPL thus appears to perform a relatively specific role in the transformation of sensory input into motor output (e.g., Hardwick et al. (2013).

#### *Thalamus, striatum, and cerebellum*

The role of the striatum has been emphasized as a critical component for the planning, acquisition and execution of new motor skills (Doyon et al., 2009). It receives major afferent inputs from cortical areas, from the midbrain, and from the thalamus (Delong and Wichmann, 2007). Its principal role is thought to lie in encoding motor programs, and it is activated consistently during both implicit and explicit sequence learning (see Doyon et al., 2009 for a review). Neuroimaging data suggests that there exists a dynamic functional interplay between the striatum and cerebellum while subjects are acquiring a motor skill—up to the point of asymptotic levels of performance. Once the behavior is extremely well-learned, activity in the cerebellum becomes barely detectable (Friston et al., 1992; Grafton et al., 1994; Seitz et al., 1994; Jueptner et al., 1997a; Doyon et al., 2002), whereas activation in the striatum persists (Grafton et al., 1994; Doyon et al., 1996; Jueptner et al., 1997b). This has led to the view that striatal activity is associated with the long-term retention of motor skill. Similarly the thalamus: a multi-nucleus "relay station," receiving inputs from an array of brain sub-systems, and conducting them onwards to their appropriate destinations, also shows "sustained" activation after asymptotic levels of performance have been achieved (Duff et al., 2007). In the SRTT studies assessed by Hardwick et al. (2013) in their recent meta-analysis, the (left) thalamus was prominently engaged, an effect that was most apparent when in contrast with tasks that require the acquisition of novel dynamics or kinematics.

One of the key roles ascribed to the cerebellum in motor learning is that of "state estimation," whereby the actual sensory consequences of actions are compared to the predicted sensory consequences. It is upon the basis of the prediction errors thus derived that improvements of performance, in relation to parameters such as speed and accuracy, are thought to develop (Manto et al., 1994; Miall et al., 2007; Tseng et al., 2007; Miall and King, 2008). As SRTT variants have actuation demands that nominally remain fixed, it is perhaps not surprising that this form of task acquisition is associated with lower levels of cerebellar activity than other forms of motor learning. With respect to regional variation, engagement of the right lateral cerebellum in the SRTT appears to be a consistent finding (Hardwick et al., 2013).

## **BILATERAL TRANSFER OF SEQUENCE LEARNING—FUNCTIONAL BRAIN IMAGING**

Very few empirical studies have used neuroimaging techniques to investigate bilateral transfer of sequence learning. Perez et al. (2007a) reported that following right limb SRTT training, areas of (fMRI registered) activation during left hand task execution included bilateral SMA, PMd, striatum, extrastriate visual cortex, cerebellum, thalamus, and also the right M1. It should be noted in this context that as projections to the cerebellum are double-crossed, activation registered in this region is generally associated with movements of the ipsilateral limb. Additionally, activity in some regions was correlated with behavioral measures of intermanual transfer of performance. Pre-training activity in the right ventrolateral posterior (VLp) thalamic nucleus was predictive of the amount of interlimb transfer that would be observed following training, and post-training activity in the (bilateral) ventrolateral anterior (VLa) thalamic nucleus and SMA correlated positively with the amount of interlimb transfer that had occurred. Importantly, activity in these areas was not correlated with performance changes in a control movement sequence.

The areas of activation detected in sequence learning tasks depend, at least in part, upon whether the transfer task requires that the sequence is executed in the original spatial format (i.e., defined with respect to an external coordinate scheme) or in a mirrored layout (that preserves the internal (anatomical) coordinate mapping). Instances of the latter type would require the use of the corresponding effectors on the opposite (untrained) hand, and generation for the homologous muscles of the same motor output patterns as those that required during training. In the case of handwriting, similar patterns of brain activation are noted in right-handed subjects when the right hand is writing normally, and the left hand is required to write in a mirrored format. Many additional brain regions are, however, engaged when the left hand is required to write such that the "normal," (with respect to the right hand) spatial pattern is preserved, presumably as additional transformations are required to generate the novel muscle synergies (Halsband and Lange, 2006). In this task context at least, the paucity of additional brain activity suggests that mirrored performance by the untrained limb is subserved by the same engram that is utilized by the trained limb (Grafton et al., 2002).

## **BILATERAL TRANSFER OF SEQUENCE LEARNING—ELECTROPHYSIOLOGICAL INDICES**

Although neuroimaging techniques are invaluable for localizing variations in cerebrovascular demand, they cannot be used easily to assess inhibitory neural processes (Waldvogel et al., 2000). We have highlighted previously that for tasks that require maximum levels of motor output, these processes may assume a functional role in relation to cross education. Is there evidence that they are implicated in bilateral transfer of sequence learning? Perez et al. (2007b) reported that following unilateral SRTT training, there was a decrease in IHI from the M1 contralateral to the training limb, to the M1 contralateral to the transfer limb. The extent of this decrease was correlated with the amount of non-specific performance transfer to the untrained limb. In the SRTT, this is typically expressed as decreased reaction times in all aspects of the task, including random blocks that have no sequential component (Robertson, 2007). The level of sequence-specific transfer of learning was not, however, correlated with IHI measures. This pattern of outcomes accords with that reported by Hortobagyi et al. (2011), and suggests that the non-specific transfer observed for the SRTT may be similar in nature to the cross education observed for maximal output training tasks. The results of Perez et al. (2007b) further imply that experimentally derived measures of interhemispheric inhibition between the primary motor cortices are insensitive to the neural adaptations that mediate the interlimb transfer of elements specific to sequence learning. Following SRTT training, SICI is reduced in both the trained and untrained M1, a finding that is consistent with the proposal (Bianki and Makarova, 1980) that a narrowing of excitatory focus in the primary motor cortex contralateral to the training limb emerges from reciprocal interhemispheric interactions. Furthermore, the observation that elevations in the net excitability of corticospinal projections from M1 were present only for the hemisphere contralateral to the training limb (Perez et al., 2007b), is consistent with the conclusion highlighted previously that the functional adaptations that underpin cross education are either mediated by interneuronal networks within primary motor cortex- other than those directly engaged in generating corticospinal output, or via changes in the effectiveness of synaptic transmission through projections from other areas of the motor network onto M1 targets. These two possibilities are not exclusive.

Various forms of non-invasive brain stimulation, including TMS, have been used to disrupt processing in a region of interest during classical motor learning tasks. In noting any associated behavioral effects, the usual intent is to draw causal inferences. An important caveat holds, however, when M1, or indeed any other area having descending projections to the spinal cord, is the region of interest. The motor system may accommodate this challenge by altering the activity in other brain areas involved in movement planning and execution in a manner that preserves motor output (e.g., Touge et al., 2001; Shemmell et al., 2007; Ortu et al., 2009). Thus, it may not be possible to determine whether the effect of M1 stimulation upon motor learning is attributable to an altered contribution of the target region, or due to compensatory changes occurring elsewhere. The problem is particularly acute when a limited range of measures is employed to assess the impact of the intervention. For example, in a recent investigation, Riek et al. (2012) demonstrated that following the administration of theta burst rTMS prior to initial learning in a visuomotor adaptation task, the overt characteristics of performance (as assessed by trajectory error and movement time) were maintained. There was, however, a profound impact upon the latency of response preparation—a measure not obtained typically in adaptation paradigms. There are more general implications. The brain region that is of critical functional importance in relation to the behavior under consideration may be one that receives (excitatory or inhibitory) inputs from the stimulation target. Thus, the effects of such interventions upon learning can rarely be considered profitably without additional controls, and corroborating evidence derived from other investigative techniques.

With these qualifications in mind, we turn to one of the few studies in which this general approach has been applied to investigate the contribution of a specific region—in this case SMA, to intermanual transfer. As aforementioned, (see section Supplementary Motor Area), the contribution of SMA to sequence learning is thought to be phase dependent. It is engaged in encoding and planning the next movement in a sequence, and in controlling and monitoring movements once they are initiated. Perez et al. (2008) reported that in a SRTT variant, applying 1 Hz rTMS to SMA along the sagittal midline in the intervals between successive movements reduced levels of intermanual transfer. Conversely, applying rTMS to SMA during movement execution had no such effect. On this basis, the authors concluded that the contribution of SMA to the interlimb transfer of sequence learning occurs primarily in the intervals between movements (Perez et al., 2008). Given the poor temporal resolution of fMRI and PET, which hampers the use of corresponding experimental (i.e., imaging) designs, it is difficult to determine the degree to which a mediating role of SMA, as opposed to pre-SMA is implied by these data. In light of the considerations noted above, it may be noted that the midline SMA stimulation had no impact upon the rate at which performance improved for the training limb. It has been suggested that this may indicate that distinct mechanisms mediate the increases in performance manifested by the training limb, and the concurrent increase in capability exhibited by the opposite limb (Perez et al., 2008).

## **BILATERAL TRANSFER OF SEQUENCE LEARNING—STRUCTURAL CORRELATES**

White matter structural integrity is thought to impinge directly upon motor performance, as the quality of myelin and axon diameter impact upon the propagation speed of neural impulses (Fields, 2011). These in turn contribute to the larger scale synchronization of signals across distributed components of the functional networks that are required for skilled task execution and learning (Fields, 2008). In the present context, interhemispheric callosal pathways and intrahemispheric association fibers are of particular interest.

Neuroimaging studies have demonstrated that a distributed network of frontal, parietal and motor regions, are activated intrahemispherically during explicit (visuomotor) sequence learning (Jenkins et al., 1994; Schlaug et al., 1994; Honda et al., 1998; Sakai et al., 1998). The superior longitudinal fasciculous (SLF) is a pair of fiber bundles that connects these regions *intra*hemispherically (Makris et al., 2005; Koch et al., 2010), and provides the structural basis for their interaction. Even when training is unilateral, there appears to be bilateral engagement of this network (Honda et al., 1998; Müller et al., 2002), thus suggesting that the fibers of the corpus callosum are also essential for this type of motor learning to proceed. In accordance with this view, Bonzano et al. (2011b) reported that the integrity of transcallosal fibers had a much greater bearing on an individual's capacity for unilateral (explicit) sequence specific learning, than similar indices derived for the fibers of the SLF. No such association was found for non-specific sequence learning (i.e., a decrease in reaction time obtained when stimuli are presented randomly). These outcomes imply that the involvement of transcallosal pathways is crucial, at least for this form of sequence learning.

It is almost certainly the case that distinct sub-portions of the corpus callosum subserve different functions. The degree to which there is interlimb transfer of non-specific learning in a SRTT context, correlates positively with fractional anisotropy (FA) values for the posterior midbody of the corpus callosum (Bonzano et al., 2011a). This may accord with the finding of Perez et al. (2007b) that variations in IHI are related specifically to disparities in the transfer of non-specific motor sequence learning. The differentiated roles of the corpus callosum in relation to cross education in this class of tasks is further emphasized by findings that anterior callosotomy produces deficits in intermanual transfer in circumstances in which sequence-specific learning is exhibited by the training limb

(de Guise et al., 1999; Peltier et al., 2012). In suggesting that the anterior body of the corpus callosum is essential for the effective transfer of sequence specific motor learning, the outcomes are complementary to those showing that microstructural characteristics of the posterior midbody of the corpus callosum determine levels of transfer of non sequence-specific learning (Bonzano et al., 2011a) (**Figure 6**). They are also consistent with the more general assumption that these facets of SRTT learning are processed by different brain networks (Hikosaka et al., 1999; Bischoff-Grethe et al., 2004). Fibers passing through the posterior midbody of the corpus callosum may mediate transfer of non sequence-specific learning, whereas, interhemispheric projections between homologous regions of SMA appear a more likely waypoint for transfer of sequence specific learning. Transcallosal SMA-SMA connections are more plentiful and have greater structural integrity than those connecting any other motor region (Fling et al., 2011), however, we are not aware of any direct investigation of the relationship between the structural characteristics of transcallosal SMA projections and expressions of cross education.

## **CONCLUSIONS**

The credo motivating the present review is that the transfer of strength or vigor accruing from a bout of unilateral resistance or ballistic training, and the transfer of skill following a period of unilateral skill training are mediated by common mechanisms. In seeking to illustrate the origins of this conviction, we elected to present empirical findings principally in the context of the experimental paradigms in which they were derived. This approach was driven by a number of key considerations. The relevant research literature is fragmented. There are remarkably few instances in which the engagement of specific neural pathways has been studied by applying the same analytic approach to multiple paradigms that bring forth the expression of cross education. Similarly, it has seldom been the case that the dimensions of single tasks have been manipulated systematically with a view to altering the level of cross education. By and large therefore, the necessary inferences cannot be drawn directly. The intercession of common mechanisms can, however, be deduced through synthesis and assimilation. In the preceding sections we have sought to highlight the findings that are critical in relation to this integration. In these closing sections, we provide an explicit summary of that which can reasonably be concluded as a consequence, and present a number of conjectures—for which resolution may await experimental designs beyond those that have thus far been customary in the study of cross education.

## **SUMMARY**

During the course of unilateral training, both in tasks that demand maximal levels of motor output, and in those that require the learning of action sequences, there is augmentation of activity (registered by neuroimaging) in the primary motor cortex ipsilateral to the training limb (e.g., Dai et al., 2001), and an accompanying elevation in the excitability of corticospinal output projections, as revealed by increases in the amplitude of motor potentials evoked by TMS. The latter changes persist beyond the cessation of training, and extend beyond homologues of the muscles engaged in the training task (Carroll et al., 2008). With respect to both indices, the extent of the induced variation is contingent upon the level of efferent drive required to perform the training movements. This factor is also a determining influence on the level of cross education exhibited by the opposite ("untrained") limb. Nonetheless, there is no apparent association between the excitability of corticospinal output pathways projecting to the untrained limb—when these are assessed at rest, and the level of contemporaneous (i.e., acute) or deferred (i.e., chronic) transfer of performance (e.g., Hinder et al., 2011; Hortobagyi et al., 2011). At least two possibilities are thus admitted. Cross education of motor function may be mediated by mechanisms acting via neural pathways projecting from areas upstream of the primary motor cortices. If this is the case, the elevations in the excitability of corticospinal projections observed during and immediately following training of the opposite limb, may simply reflect crossed facilitation that is not instrumentally related to transfer of performance. An alternative and not exclusive possibility is that cross education effects are mediated, at least in part, by adaptations in interneuronal networks within M1 other than those directly engaged in generating corticospinal output. In this conception, changes in the state of these interneuronal networks, which may play a role in narrowing of the excitatory focus of motor output, will be expressed in circumstances in which they receive synaptic drive, but not necessarily when the output circuits of the primary motor cortex are activated by low intensity single pulse TMS. In this regard, synaptic drive to these networks may occur not only during voluntary contractions, but also in non-physiological contexts, including paired pulse TMS paradigms, such as those employed to obtain measures of interhemispheric inhibition. It is therefore notable that decreases in IHI ("trained" to "untrained" hemisphere) are expressed acutely during the course of a single unimanual training session and chronically over multiple sessions, and that these changes can be related to the degree of cross education (Hortobagyi et al., 2011). It is likely that such decreases in IHI express alterations in the excitatory-inhibitory balance within interneuron circuits local to the hemisphere ipsilateral to the training limb, rather than adaptive changes in the characteristics of the transcallosal volley.

These considerations in relation to M1 notwithstanding, evidence derived from the functional and structural neuroimaging literature suggests that there is greater scope for inter-hemispheric interactions between other elements of the motor network during the production of unilateral movements. On an a priori basis alone, it would appear likely that both in the context of tasks that require maximal levels of motor output, and in those emphasizing the learning of action sequences, there are demands imposed upon the functional capacities of specific brain regions that will be subject to adaptive pressure during training that brings about marked improvements in performance. For example, the cingulate motor area (CMA), which is strategic entry point for limbic influence on the voluntary motor system, is closely associated with the amount of effort demanded by a motor task (Winterer et al., 2002), and exhibits activity that is modulated when the ipsilateral hand is engaged (Kermadi et al., 2000). Involvement of the SMA would be anticipated in tasks that impose requirements for movement planning, both in relation to the totality of an action sequence, and with respect to the individual elements of that sequence. Similarly, the dorsal premotor cortex (PMd) is a pivotal structure for motor learning in general, and for motor sequence learning in particular. The relative contributions of these regions will vary not only in accordance with specific task parameters, but also over time as the adaptations that form the basis of cross education are initiated and consolidated. In so much as activity in the striatum (e.g., Doyon et al., 1996) and thalamus (e.g., Duff et al., 2007) exhibit sustained activation after asymptotic levels of performance have been achieved, it is probable that these regions are associated with the long-term persistence of the transfer effects.

#### **CONJECTURES**

Following Bianki (e.g., Bianki and Makarova, 1980; Bianki and Shrammapril, 1985), we propose that reciprocal interactions between the primary motor cortices are an obligatory facet of unilateral training, and that these serve to narrow the excitatory

## **REFERENCES**


focus of cortical output to the principal muscles engaged in the task via modifications of surround inhibition. In addition to being specific to homologues of the muscles engaged in training, the concurrent and contingent adaptations induced ipsilateral to the training limb are functional rather than incidental. While the surround inhibition is instantiated in circuits local to M1, its modulation during training is mediated by inputs from other nodes of the motor network. Synaptic drive directed subsequently to these adapted circuits results in patterns of efference characterized by greater specificity in recruitment of the focal muscles engaged in a task, and in disengagement of muscles with actions that might otherwise interfere with the desired movement action. To the extent that the remodeling of motor output resembles that which is exhibited by the trained limb, cross education will be demonstrated. Necessarily the relative contribution of this mechanism to the behavioral effects will be greatest in those tasks for which enhancements in performance outcomes do not require the composition of novel synergies or the execution of novel action sequences.

Similarly, the comparative contributions of inter-hemispheric interactions between other elements of the motor network to the interlimb transfer of performance are task dependent. In circumstances in which increased effort or volition results in superior execution of the training movements, bilateral adaptations in neural circuits receiving projections from the cingulate motor areas, including targets within M1, are to be anticipated. Variations with respect to other (orthogonal) task dimensions, such as the requirement that new activation profiles be generated in refashioning muscle synergies, or that a fixed sequence of actions be reproduced, will lead to consequential changes in the state of projections from disparate regions of the network. Common to all such task-contingent variations is their consolidation over the course of extended training in thalamic and striatal relays, thereby providing the substrate for retention of cross education.

### **ACKNOWLEDGMENTS**

UK Biotechnology and Biological Sciences Research Council (ID: BB/I008101/1). Northern Ireland Department for Employment and Learning. Richard Carson thanks Atlantic Philanthropies for their generous support of his research through their funding of the NEIL (Neuroenhancement for Independent Lives) programme at Trinity College Institute of Neuroscience.

suggested underlying mechanisms. *J. Sports Sci.* 11, 145–158. doi: 10.1080/02640419308729978


investigation of cortical and subcortical networks in the learning of abstract and effector-specific representations of motor sequences. *Neuroimage* 32, 714–727. doi: 10.1016/j.neuroimage.2006.04.205


cortex produces interhemispheric facilitation through distinct pathways. *J. Physiol.* 572, 857–868.


*Hum. Mov. Sci.* 18, 307–343. doi: 10.1016/S0167-9457(99)00013-5


in the human brain. *Exp. Brain Res.* 153, 140–145. doi: 10.1007/s00221- 003-1587-1


*J. Neuropathol. Exp. Neurol*. 44, 578–591. doi: 10.1097/00005072- 198511000-00004


77, 677–682. doi: 10.1016/S0031- 9384(02)00929-0


relationships. *Hum. Brain Mapp.* 34, 384–395. doi: 10.1002/hbm.21437


study. *J. Physiol.* 587, 5393–5410. doi: 10.1113/jphysiol.2009.175885


contralateral effects of unimanual training. *J. Appl. Physiol.* 4, 136–144.


versus unimanual coordination: what makes the difference? *Neuroimage* 22, 1336–1350. doi: 10.1016/j.neuroimage.2004.03.012


motor activation in patients with multiple sclerosis: a functional and anatomical study. *Hum. Brain Mapp.* 28, 636–644. doi: 10.1002/ hbm.20305


(2008). Intracortical inhibition and facilitation with unilateral dominant, unilateral nondominant and bilateral movement tasks in left- and right-handed adults. *J. Neurol. Sci*. 269, 96–104. doi: 10.1016/j.jns.2007.12.033


learning: evidence from performance measures. *Cogn. Psychol.* 19, 1–32. doi: 10.1016/0010-0285(87) 90002-8


section of the anterior part of the body of the corpus callosum: evidence from an interhemispheric transcallosal approach. *J. Neurol*. 259, 1860–1867. doi:


Ruddy and Carson Cross education of motor function

the cortical motor system: new concepts. *Electroencephalogr. Clin. Neurophysiol.* 106, 283–296. doi: 10.1016/S0013-4694(98)00022-4


*Exp. Brain Res.* 98, 523–534. doi: 10.1007/BF00233989


ahead. *Nature* 371, 413–416. doi: 10.1038/371413a0


*Neurosci. Lett.* 445, 209–213. doi: 10.1016/j.neulet.2008.09.013


somatotopy, and link between microstructure and function. *J. Neurosci.* 27, 12132–12138. doi: 10.1523/JNEUROSCI.2320-07.2007


on motorcortical inhibition induced by blockade of GABA uptake in humans. *J. Physiol.* 517(Pt 2), 591–597. doi: 10.1111/j.1469-7793.1999.0591t.x


mechanisms of cross education. *Exerc. Sport Sci. Rev.* 28, 177–184.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 05 April 2013; accepted: 07 July 2013; published online: 29 July 2013.*

*Citation: Ruddy KL and Carson RG (2013) Neural pathways mediating cross education of motor function. Front. Hum. Neurosci. 7:397. doi: 10.3389/ fnhum.2013.00397*

*Copyright © 2013 Ruddy and Carson. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

## Control of automated behavior: insights from the discrete sequence production task

#### *Elger L. Abrahamse1 \*, Marit F. L. Ruitenberg2, Elian de Kleine2 and Willem B. Verwey2*

*<sup>1</sup> Department of Experimental Psychology, University of Ghent, Ghent, Belgium*

*<sup>2</sup> Department of Cognitive Psychology and Ergonomics, University of Twente, Enschede, Netherlands*

#### *Edited by:*

*Rachael D. Seidler, University of Michigan, USA*

#### *Reviewed by:*

*Lara Boyd, University of British Columbia, Canada Carol Seger, Colorado State University, USA David Rosenbaum, Pennsylvania State University, USA*

#### *\*Correspondence:*

*Elger L. Abrahamse, Department of Experimental Psychology, Faculty of Psychology and Educational Sciences, University of Ghent, Henri Dunantlaan 2, Ghent 9000, Belgium. e-mail: elger.abrahamse@ugent.be*

Work with the discrete sequence production (DSP) task has provided a substantial literature on discrete sequencing skill over the last decades. The purpose of the current article is to provide a comprehensive overview of this literature and of the theoretical progress that it has prompted. We start with a description of the DSP task and the phenomena that are typically observed with it. Then we propose a cognitive model, the dual processor model (DPM), which explains performance of (skilled) discrete key-press sequences. Key features of this model are the distinction between a cognitive processor and a motor system (i.e., motor buffer and motor processor), the interplay between these two processing systems, and the possibility to execute familiar sequences in two different execution modes. We further discuss how this model relates to several related sequence skill research paradigms and models, and we outline outstanding questions for future research throughout the paper. We conclude by sketching a tentative neural implementation of the DPM.

**Keywords: motor skill, sequence learning, automated behavior**

## **INTRODUCTION**

Many of our daily activities are testimony to the possession of motor skill. One may think of riding a bike, lacing a shoe, or writing one's signature. Accordingly, within the fields of cognitive psychology and cognitive neuroscience ample research has been devoted to understanding how the brain represents and controls motor events. This venture is hindered, among other things, by a lack of direct conscious access to motor processes, and by the considerable time that the acquisition of motor skill typically takes. Nevertheless, various experimental tools have been developed over the last decades from which the workings of motor control—and its constant interaction with higher-order cognition—can be inferred with surprising detail. These experimental tools may be classified within two major experimental paradigms, motor adaptation1 , and motor sequence learning (e.g., Doyon et al., 2003). The focus of the current paper is on motor sequence learning.

Motor sequence learning refers to the acquisition of the skill to rapidly and accurately produce a sequence of movements with limited effort and/or attentional monitoring. Such learning is typically based on repeated practice and (a mixture of) explicit instruction, explicit trial-and-error discovery and more elaborated hypothesis testing, or implicit detection of regularity. As most, if not all, of our goal-directed actions involve some kind of sequential structure, the human capacity to acquire sequential motor skill has been a topic of extensive research over the last decades. This research has led to a large variety of laboratory sequence acquisition tasks that typically involve finger-to-thumb opposition movements, finger presses on response boxes or key boards, movements of the whole arm, isometric forces, or oculomotor movements. The purpose of the current article is to provide a comprehensive overview on the contribution of one of these tasks, the *discrete sequence production* (DSP) task (Verwey, 2001), to our understanding of the execution of well-learned, discrete movement patterns.

The current review, then, is narrow in focus in the sense that it centers on work with the DSP task. Other sequence learning tasks and their major findings will not be discussed in detail (they have been reviewed elsewhere before: e.g., Rhodes et al., 2004; Perruchet and Pacton, 2006; Doyon et al., 2009; Abrahamse et al., 2010; Rosenbaum, 2010). However, the current review ultimately aims to outline from the DSP research a framework for sequence skill that aspires to a much broader application. This framework builds on the notion that sequential control occurs at both the cognitive level and at an autonomous motor level, and that it is the interplay between these levels that optimizes performance in sequential movement tasks.

In the next section we will (a) provide a description of the DSP task, (b) situate the DSP task within the larger domain of motor sequence learning in order to identify both its strengths and limitations, and (c) provide an overview of the typical phenomena associated with the DSP task. Overall, this section thus constitutes a sort of user's manual of the DSP task. In the third section, we will present the framework. This so-called *dual processor model*

<sup>1</sup>Motor adaptation, a form of (re-)learning characterized by gradual improvement in performance in response to altered task conditions, can be studied both with arm and eye movements, and either by using visuomotor adaptation (i.e., distortion of the visual but not the proprioceptive consequences of the motor commands) or by using force-field adaptation (i.e., distortion of both the visual and proprioceptive consequences of motor commands). For reviews see Krakauer and Mazzoni (2011) and Shadmehr et al. (2010).

(DPM) was proposed already by Verwey (2001). However, based on more recent work with the DSP task, we here extent and specify the model. Finally, in the fourth section we will describe a tentative neuropsychological architecture that may underlie the DPM.

## **THE DSP TASK: A USER'S MANUAL**

## **EXPERIMENTAL SETTING**

The DSP task involves participants resting four to eight fingers on the designated keys of the keyboard (**Figure 1** and **Table 1**) <sup>2</sup> . A similar number of placeholders (usually small squares) is displayed on the screen, and each placeholder corresponds to one of the keys of the keyboard in a spatially compatible manner. Whenever a placeholder is lights up, the participant is instructed to rapidly press the spatially compatible key. Then the next stimulus is displayed. A typical DSP sequence involves two fixed series of 3–7 stimuli which results in the execution of two equally long key-press sequences. Usually, these sequences are carried out in a random order. This implies that a DSP task with, for example, two alternative 6-key sequences turns with practice from two series of 6-choice RT tasks into a single 2-choice RT task in which an entire 6-key sequence constitutes a single response. We use *Sn* to denote the *n*-th stimulus of



*See the main text for elaboration.*

2See this link for a downloadable EPrime version of the DSP task: http://www*.* utwente*.*nl/gw/cpe/en/Employees%20CPE/Verwey/Research/research*.*doc/

a sequence, R*<sup>n</sup>* to denote the *n*-th response in the sequence, and T*<sup>n</sup>* to denote the RT associated with S*n*. Sometimes these RTs are referred to as inter-key-intervals (IKIs) but this only holds in the typical case when response-to-stimulus-intervals are 0 ms.

Two methodological features of the DSP task are worth noting. First, the DSP task starts off with a practice phase (including 500–1000 repetitions per sequence) to develop the building blocks; These so-called *motor chunks* are assumed to represent a limited number of responses that can be selected and executed as if they are a single response in a control hierarchy (Book, 1908; Miller et al., 1960; Pew, 1966; Newell and Rosenbloom, 1981; Verwey, 1996). Following practice, the properties of these motor chunks are studied in a test phase in which a novel ("unfamiliar") sequence is usually taken as control condition.

Second, by counterbalancing the fingers of individual participants across the sequential positions of the sequence, fingerspecific effects at individual sequential positions are ruled out because each of the fingers contributes equally to the RTs at each sequential position. For example, when participants are using the D, F, G, J, K, and L keys on a keyboard, one participant may practice the 6-key sequence KFGDJL, the next participant the 6-key sequence LGJFKD (each key is shifted rightward relative to the first participant), and so on. This counterbalancing procedure also implies that the same sequences can be used as familiar and as unfamiliar, control, sequences so that RT differences between familiar and unfamiliar sequences are not related to inconspicuous differences in keying order, but rather are clean indicators of the underlying control processes.

## **SITUATING THE DSP TASK**

We consider research with the DSP task as a way to study the building blocks of more complex behavioral patterns that make up everyday behavior (Paillard, 1960; Eysenck and Frith, 1977; Gallistel, 1980). For example, driving a car builds on movement sequences that underlie switching gears, steering through corners, looking in your mirror and back, etc. As such, the DSP task is representative for the way in which more complex real-world actions are acquired and controlled.

The DSP task was inspired by earlier studies that employed discrete keying sequences (e.g., Povel and Collard, 1982; Rosenbaum et al., 1983; Kornbrot, 1989). The use of key-press sequences to study the development and application of sequential skills has the benefit that they allow exploring sequential control *per se* because executing a single sequence element takes very little time (e.g., MacKay, 1982; Rhodes et al., 2004). This makes RTs in a keying sequence a more sensitive indicator for the underlying control processes as compared to when, for example, series of arm movements are studied and control processes may occur during execution of individual sequence elements (which will take relatively long).

Various other tasks have been used to study the acquisition and control of sequential movement skills, such as the pursuit rotor task (e.g., Grafton et al., 1992), the tracing of cut-out mazes (e.g., Van Mier et al., 1998), the *m* × *n* task (Hikosaka et al., 1995), a sequential elbow flexion and extension task (Park et al., 2004) and the serial reaction time (SRT) task (e.g., Nissen and Bullemer, 1987). Two of these tasks are especially interesting to elaborate upon here because their experimental designs overlap substantially with the DSP task; that is, they also aim at studying sequential representation on the basis of repeatedly performing key-press sequences. First, the *m* × *n* task involves trial-and-error based responding to sets of stimuli that eventually end up in fluent sequential skill. Like with the DSP task, the *m* × *n* task allows for exploring motor chunking; however, because practice involves trial-and-error search followed by relatively few repetitions once the sequence is fully discovered (i.e., with virtually error-free performance), the task differs from the DSP task that focuses on fast and effortless skill acquisition. Still, as will be elaborated on below, the model that Hikosaka et al. (1999) derived from mainly the *m* × *n* task has substantial conceptual overlap with the model that we propose below on the basis of DSP studies.

Second, in the SRT task participants cycle through a fixed and continuously repeating series of stimulus-response (S–R) events. The regularity between events is not explicitly conveyed to participants beforehand, and participants are often picking up on the regularity (as shown by performance measures) without being aware of it. Hence, in contrast to the DSP task, the SRT task mainly involves an implicit learning paradigm and does not employ discrete sequences. More importantly even, the SRT task does not typically involve motor chunking (Jiménez et al., 2011), Again, despite these differences, below we claim that various aspects of SRT skill overlap with DSP skill.

The DSP task as defined here (cf. Verwey, 2001) can also be distinguished from various earlier discrete sequence learning studies in three respects. First, the typical practice phase in DSP studies involves the execution of two sequences for around 500–1000 repetitions each. This results in performance that is characterized by substantial preparation before execution starts, which is indicated by the very fast RTs after T1 (sometimes reaching averages below 100 ms), and the alleged use of motor chunks. Earlier research employed much less practice. For example, Restle (1970), Simon (1972), Jones (1974), and Rosenbaum et al. (1983) employed only a few dozen repetitions per sequence. As it is known that the amount of practice has both quantitative and qualitative (e.g., differential sensitivity to interference from secondary tasks; e.g., Poldrack et al., 2005) effects on sequence skill, this might limit the generalizability of results from DSP studies to less practiced movement sequences. However, as we outline below, we believe that the framework we propose still has ramifications for situations with substantially less or more practice.

Second, the DSP task as defined here employs spatially defined key-specific stimuli that are presented throughout practice. These are mapped in a spatially compatible way to the response keys in order to minimize effects of (new) S-R learning. This differs from many earlier discrete sequence learning studies, in which participants were asked to explicitly learn the sequences after which their execution was triggered by either a simple go-signal (Rosenbaum et al., 1983, 1986) or by a pre-learned indicator (e.g., "O" for sequence 1 and "X" for sequence 2; Rosenbaum et al., 1984), or they were presented with word (or letter) series that were then to be spoken or typed in response to a go-signal (Sternberg et al., 1978).

Finally, the aim of DSP research is to explore the creation and exploitation of newly acquired sequence representations that ultimately lead to the development of motor chunks. It does not typically employ sequences that are described by pre-stored chunks or rule knowledge (like 12344321 and 12123434, Restle, 1970; Jones, 1981; Rosenbaum et al., 1983). In that situation, sequence learning is a matter of recognizing and reproducing the underlying rules rather than learning an arbitrary series of movements (cf. Coynel et al., 2010).

Hence, the DSP task as first specified in Verwey (2001) can be distinguished from earlier work on discrete sequence learning in terms of the overall amount of practice, the sequential structure, and the learning procedure. Later in this paper we return to these distinctions and elaborate on how we believe that they relate to the theoretical framework we propose. We will now first describe some of the major phenomena that are systematically observed across DSP studies.

## **TYPICAL PHENOMENA**

The literature on the DSP task reports a number of robust findings. These include (a) distinct phases of discrete sequence skill, and the spontaneous segmentation of longer sequences, (b) distinct coding systems that underlie sequence representations, and (c) the development of explicit sequence knowledge.

## *Processing phases of sequence skill: Initiation, concatenation and execution*

The overall execution of a well-learned keying sequence can be related to three distinct processing phases that we believe are reflected in the respective RTs. The first phase is here referred to as sequence *initiation* and is reflected in T1. In case of a choice RT paradigm such as the typical DSP task, T1 is assumed to involve the selection and preparation of the sequence. As **Figure 2**

**FIGURE 2 | Executing a 6-key sequence and its typical reaction time pattern.** It involves the processing phases initiation, concatenation, and (mere) execution. Please note that with smaller sequence lengths (*<*5 key-presses) the relatively slow T half way through (concatenation) is not typically observed.

illustrates, this first key-press is typically much slower than subsequent key-presses (e.g., Verwey, 1999). This slow start is caused, in part, by suboptimal anticipation to the presentation of S1, as the slow first response can be observed even when a short, random series of key-presses is carried out (Verwey, 2003b). However, when there is a fixed keying order the difference between the first and later Ts increases considerably with practice because of the increasing possibility to prepare the later key-presses (Verwey et al., 2010). Possibly, the tendency to prepare an increasing number of elements also affects T1 itself: decreases of T1 with practice may be counteracted by the increasing time to prepare more responses in advance as the sequence becomes more familiar.

In line with the notion that T1 involves selection and preparation of forthcoming key-presses, T1 has been found to increase with the number of elements (i.e., key-presses) in the sequence (e.g., Verwey, 1999). This sequence length effect is commonly explained by the notion that individual responses are loaded immediately before sequence initiation into a short term *motor buffer* (Henry and Rogers, 1960; Sternberg et al., 1978; Hulstijn and Van Galen, 1983; Van Galen, 1991; Thomassen and Van Galen, 1992).

The sequence length effect appears to level off as sequences get longer (Sternberg et al., 1978, 1988; Rosenbaum, 1987). This is attributed to the notion that only a limited number of responses can be prepared in the motor buffer, and that preparation of later responses is postponed until after sequence initiation. This is referred to as concurrent, or on-line, programming. A related phenomenon is that the sequence length effect on T1 reduces with practice. This has been observed for, among others, the DSP task (Verwey, 1999). As the reduction of the sequence length effect with practice is associated with sequence-specific improvement (Verwey, 1999), it is assumed that this reduction indexes the development of a motor chunk that allows an entire sequence—or at least the first part of it—to be initiated like a single response.

The key-presses following sequence initiation are typically very fast—sometimes with RTs below 100 ms. This is possible because these involve just execution processes; selection and preparation processes of these keys have already occurred during the initiation phase. Together, these key-presses are referred to as the (mere) *execution* key-presses (see **Figure 2**). Key-presses reflecting initiation and execution can be dissociated through experimental manipulations. For example, Verwey (1999) showed that reversing the mapping between a sequence-specific stimulus and the sequence slowed initiation but not execution (see also Verwey et al., 2009).

Usually, longer sequences (*>*4 key-presses) show a relatively slow response half way through the sequence (Brown and Carr, 1989; Verwey et al., 2002; Kennerley et al., 2004; Bo and Seidler, 2009). Based on this observation, and the aforementioned finding that the sequence length effect levels off as sequence length increases, Verwey and Eikelboom (2003) argued that longer, fixed sequences are divided into multiple motor chunks due to assumed limitations in the length of a single motor chunk—in strong analogy to the well-known chunk-based capacity limitations of working memory (Miller, 1956; Cowan, 2000). Detailed examination of the effects of extensive practice and regularities in key-pressing order suggested that indeed most participants executed a 6-key sequence as 2 or more successive segments. Such segmentation is complemented by what is referred to as *concatenation*: the processes that allow distinct motor chunks within a sequence to be executed in rapid succession as smoothly as possible. The relatively slow response halfway through, then, is assumed to index the transition from one motor chunk to the next, and can be referred to as the concatenation point (see **Figure 2**). The slowing may be indicative of the involvement of higher cognitive processes such as preparation processes for the upcoming motor chunk (e.g., Verwey et al., 2010), or strategic parsing (Wymbs et al., 2012), and may eventually disappear with extensive practice when the initially separated motor chunks become rearranged and behave as a single larger motor chunk.

The idea that concatenation involves other processes than mere execution of key-presses is supported by a double dissociation between execution and concatenation key-presses; they have been shown to be affected by different manipulations. Specifically, the RTs reflecting the concatenation point increased less than RTs from execution key-presses after changing the location of the hand relative to the body (De Kleine and Verwey, 2009a), when using fingers adjacent to the ones used during practice (Verwey et al., 2009), and when discrete sequences were executed by dyslexics (De Kleine and Verwey, 2009b). Conversely, the concatenation point was lengthened more than the execution key-presses after applying transcranial magnetic stimulation (TMS) to the pre-supplementary motor area (pre-SMA; Kennerley et al., 2004). Initiation and concatenation are assumed to both involve loading and initiating the upcoming motor chunk, but the initiation phase will most likely include more general preparatory processes too (Verwey, 2003b).

Various studies have explored the notion that higher cognitive processes are mainly involved in the concatenation of successive motor chunks. If so, a cognitively demanding secondary task should especially slow concatenation as compared to execution key-presses. After some initial contradicting findings (Brown and Carr, 1989; Verwey, 2003b), we recently explored this prediction with a secondary task that required participants to count tones that were presented at a random moment during sequence execution (Verwey et al., 2010, 2013). This secondary task indeed slowed responses, but slowing was not larger for the alleged concatenation response than for the other responses. This finding was explained by the notion that concatenating motor chunks in a fixed sequence does not necessarily require cognitive processing after substantial practice. Apparently, motor chunks can become associated within a single sequence representation, so that executing one motor chunk primes the commonly ensuing next chunk (just like individual responses can become associated in an SRT task, Abrahamse et al., 2010). This can explain why concatenation has been found to get faster with practice (e.g., De Kleine and Verwey, 2009a).

Overall, we thus propose that initiating, concatenating and executing key-presses involve distinct processes of sequence skill that are reflected in their respective RTs. This suggests that these distinct phases are differentially affected by various experimental manipulations.

*Imposing segmentation.* For experimental purposes, it is a challenge that (depending on the structure of the sequence) the relatively long RT that is assumed to index the concatenation phase has been found to occur at different sequential locations for different persons. Consequently, across a group of participants a single long RT cannot always be easily observed (Sakai et al., 2003; Verwey, 2003b; Verwey and Eikelboom, 2003; Kennerley et al., 2004; Bo and Seidler, 2009). Instead, the second and the last responses are often faster than the responses in between (Verwey, 2003b; Verwey and Eikelboom, 2003). This could be interpreted as concatenation processes being distributed across these in-between responses for a group of individuals.

In the literature, several methods have been proposed for artificially imposing segmentation at the same location within the sequence across participants. A first procedure is to introduce regularities in response order. Such regularities appear to induce the same segmentation across participants (e.g., Restle, 1970; Povel and Collard, 1982; Koch and Hoffmann, 2000; Sakai et al., 2004). For example, De Kleine and Verwey (2009a) observed a highly similar segmentation across participants with their sequences, which was attributed to the occurrence of a reversal (A-B-A) halfway through the sequence. This particular regularity may have initially affected the parsing into subsets of responses, which eventually consolidated into motor chunks.

Second, when during practice a pause is inserted between two successive stimuli (yielding a so-called *prestructured* sequence), participants are typically observed to segment the sequence at the location of the pause when subsequently the pause is removed. This suggests that the position where concatenation occurs is determined by the pause position during practice (e.g., Stadler, 1993; Verwey and Dronkert, 1996; Verwey et al., 2009, 2010). The possibility that this segmentation involves learning of a particular temporal pattern, a rhythm, has been refuted because (a) the various intervals did not adhere to the expected integer ratios (Verwey, 1996; Verwey and Dronkert, 1996), (b) the temporal pattern did not transfer to another sequence (Verwey et al., 2009), and (c) segmentation patterns did not correlate with the individual's temporal control abilities (Bo et al., 2009; Bo and Seidler, 2009; also see, Sakai et al., 2004).

Finally, Jiménez et al. (2011) used differently colored keyspecific stimuli to distinguish different segments in an SRT task (i.e., stimuli signaling the responses that were to be segmented together were presented in the same color). This successfully induced consistent segmentation/concatenation across participants, but has yet to be tested and validated for discrete movement sequences.

*Assessing segmentation and concatenation.* Several methods have been reported to identify spontaneous chunking behavior in a *post-hoc* fashion. First, some studies have compared the slowest T after the T1 (assumed to be the concatenation point) against the others (e.g., Verwey et al., 2010). This procedure can be refined by first testing all T's (after T1) against its directly surrounding neighbors, and look for a significantly longer T that can subsequently be labeled as the concatenation point. However, this method relies on assumptions that during training chunk boundaries are relatively static and that, eventually, short chunks are not combined into larger chunks. This method is relatively insensitive to measuring how the chunking structures develop with practice.

Second, Jiménez et al. (2011) proposed a different manner of studying motor chunking. Instead of identifying the precise concatenation point, these authors developed a method to index chunk formation that was inspired by the logic of the analysis of variance. In brief, segmentation and concatenation of motor chunks are assumed to be indexed by an increase of the ratio between the variance between elements of the sequence and the variance within sequence elements. Hence, it relies on the variance concerned with differences in responding to distinct parts of the sequence (between-element variance), while controlling for variance caused by general factor such as practice or fatigue (within-element variance). It needs to be said, though, that this method was validated within the context of an SRT task, and has yet to be tested for a DSP task.

Third, Wymbs et al. (2012) modeled chunking behavior by using so-called modularity-optimization algorithms to seek for groups of T's (i.e., IKIs) that are more tightly connected to each other relative to their connections to T's in other groups. Such modeling allowed calculating a measure for the ease with which the network could be divided into smaller communities, and the inverse of this measure was used to index chunk magnitude. This procedure allows tracing chunk development over practice.

## *Coding movement sequences*

Several studies have investigated the type of representation that forms with practice in discrete movement sequences. The general notion is that initial sequence execution relies on effector-unspecific sequence knowledge (also referred to as effector-independent coding) and that with practice execution becomes increasingly dependent on effector-specific knowledge (also referred to as effector-dependent coding; Hikosaka et al., 1999; Bapi et al., 2000; Verwey, 2001; Verwey and Wright, 2004; Verwey et al., 2009).

Verwey and Wright (2004) examined the contribution of effector-dependent and -independent representations with respect to sequence learning in the DSP task. In their study, participants practiced two 5-key sequences, using three fingers of either a single hand or across both hands. When performing these sequences with the unpracticed hand configuration in a subsequent test phase, execution was slower than with the practiced hand configuration. Still, it was faster than the execution of unfamiliar sequences. This finding suggested that with extensive practice in the DSP task the sequence representation includes an effector-dependent and an effector-independent component.

In a subsequent DSP study, Verwey et al. (2009) found that the execution rate of 6-key sequences was slowed also when participants used the adjacent fingers of the same, practiced hands. However, this slowing was clearly less than in Verwey and Wright's (2004) study in which transfer to fingers of the other hand was assessed. The authors suggested that effector-specificity in the DSP task may result from hand-based visuo-spatial coding: using adjacent fingers could well allow involvement of the same handbased reference frame for coding locations as during practice (e.g., Cho and Proctor, 2002). That hand-based spatial coding is probably not the whole story, however, is suggested by indications that effector-specific sequence learning involves adjustment to the biomechanical properties of the effector used (Park and Shea, 2003), and that one effector may start moving before the previous movement has been executed (i.e., co-articulation; Daniloff and Moll, 1968; Jordan, 1995; Sosnik et al., 2004; Berner and Hoffmann, 2009).

Finally, the extent to which sequence coding involves effectordependent and -independent information may be related to the experimental design too, as indicated by the following discrete sequence studies: (a) Bapi et al. (2000) showed that with practice reliance on an effector-independent representation decreases, and control becomes more effector-specific (i.e., motor based; cf. Hikosaka et al., 1999; Park and Shea, 2003). (b) Gruetzmacher et al. (2011) showed that only physical but not observational practice results in coding in motor coordinates. (c) Several studies showed that with extensive practice, representations for key-pressing sequences include an effector-dependent component (e.g., Bapi et al., 2000; Verwey and Wright, 2004; Verwey et al., 2009), while for elbow flexion and extensions sequences effector-independent representations seem to remain dominant with extended practice (Kovacs et al., 2009b). (d) The complexity of a movement sequence influences the use of motor as opposed to visuo-spatial representations (Kovacs et al., 2009a; Panzer et al., 2009). Finally, (e) Panzer et al. (2011) suggested that the coding of movement sequences depends on individual characteristics in that with a relatively complex flexion/extension sequence older participants (over 60) appeared to rely more on motor coding while young adults (23–31 years) used visuo-spatial coding.

In sum, there is now substantial reason to believe that sequential movement skill involves several types of representation. Some involve a slowly developing motor code (e.g., in terms of joint angles and forces), while other representations probably code movement patterns in terms of more rapidly developing spatial reference systems (Hikosaka et al., 1999; Panzer et al., 2009). Finally, even abstract symbolic codes, like verbal codes, may be used. Which codes are dominant in a particular task seems to depend on the amount and type of practice, the number and type of responses in the sequence, individual capacities, and the strategies used during practice.

## *Explicit sequence knowledge*

It is usually accepted that sequence learning can be both implicit and explicit. Implicit learning refers to a learning process that proceeds in the absence of conscious awareness of both the learning itself and the end product of learning. As mentioned above, implicit learning is the main object of study in the SRT literature. Explicit knowledge may be based on explicit sequence descriptions in the instructions, but can also develop online by testing hypotheses about the regularity of events (e.g., Haider and Frensch, 2005; Rünger and Frensch, 2010).

Participants in DSP studies are commonly informed that they are performing fixed keying sequences. In combination with the saliency of DSP sequences this has led to the notion that the DSP task is an explicit sequence learning paradigm (Bo and Seidler, 2009). However, it has been demonstrated that participants in DSP studies do not always possess explicit, in-depth and verbalizable knowledge of the order in which the elements were carried out (e.g., Verwey et al., 2010). That is, they have no *structural knowledge* even though they know that there is a fixed regularity in the sequences (i.e., *judgment knowledge*, Dienes and Scott, 2005). Furthermore, even when participants were able after the experiment to report on the structure of their sequences, a substantial number of them indicated to have reconstructed this knowledge in the recall task after the experiment by tapping the sequences in their mind or on the table top (e.g., Verwey et al., 2010; Verwey and Abrahamse, 2012). Two potential explanations may underlie this lack of explicit, structural knowledge of the DSP sequences. It may be that participants obtain substantial (or full) explicit knowledge of the sequential structure early on in training, but later gradually lose out on it as performance becomes more and more automatized. Alternatively, some participants may never develop structural sequence knowledge. Interestingly, participants with substantial structural knowledge are often only a little faster than less aware participants—if any. This indicates that skill in this task does not depend much on explicit (structural) knowledge (Verwey et al., 2009, 2010; Verwey, 2010), in line with the notion that in the DSP task motor coding is dominant.

Here we finish the user's manual of the DSP task. In the next sections we will first describe a framework on discrete sequence skill referred to as the DPMDPM that we have derived from our work with the DSP task, and then provide a tentative neuropsychological architecture that may underlie the DPM.

## **COGNITIVE UNDERPINNINGS OF DISCRETE SEQUENCE EXECUTION**

Over the last decades, various cognitive models have been proposed to account for our capacity to develop sequential skill. Here we present an updated version of the DPM, which has resulted from work with the DSP task. Additionally, we speculate about its relationship with sequencing models that have been developed in different research paradigms.

## **DUAL PROCESSOR MODEL**

The DPM claims that a cognitive processor and a motor processor are responsible for skill in executing discrete movement sequences. During early practice, the cognitive processor translates each externally presented stimulus into the associated response, and prompts the motor processor to execute this response. In case of relatively novel but explicitly known sequences (e.g., through instructions), it may also load, one by one and before execution, a limited number of individual responses into the motor buffer. This motor buffer is assumed to be a part of working memory (Smyth and Pendleton, 1989; Tattersall and Broadbent, 1991; Verwey, 1999). However, as short series of movements are repeatedly executed in close temporal proximity, these series are assumed to gradually integrate into a single representation, the motor chunk. The availability of motor chunks allows the cognitive processor to eventually select and load this motor chunk from long term memory in a single processing step into the motor buffer, as if each motor chunk constitutes a single response (Verwey, 1999).

After loading the motor buffer, the cognitive processor triggers the motor processor to start reading the codes for the individual movements from the motor buffer and to execute the movement series in a relatively autonomous fashion. The rapidity with which familiar sequences can be selected and executed through this buffer-mediated process, is what makes up the sequence skill. According to the DPM sequential movement skills can be considered automatic to the extent that (a) little cognitive processor involvement is required when motor chunks are executed by the relatively autonomous motor processor, and that (b) with practice the contribution of the cognitive processor may even be further reduced as entire motor chunks may become triggered by external stimuli as if they involve prepared reflexes (cf. Hommel, 2000).

The model has two additional features. First, when the task, participant strategy and the available processing resources allow it, the cognitive and the motor processor may "race" each other to initiate each response in a familiar sequence; the motor processor triggers the individual responses stored in the motor buffer, while the cognitive processor selects each response on basis of key-specific stimuli (Verwey, 2001) or by using explicit sequence knowledge (Ruitenberg et al., 2012). This race will be elaborated upon below.

Second, whereas the cognitive processor initially is responsible for selecting each motor chunk and loading it into the motor buffer, with practice this may automatize for the later motor chunks of a sequence. That is, associations between successive motor chunks—in strong analogy with associative learning between individual responses in, for example, the SRT task—may facilitate or even take over the selection and loading (i.e., the concatenation) processes from the cognitive processor. Empirical support for this notion was provided by Verwey et al. (2010, 2013), who showed that the concatenation interval is not slowed any more by a secondary task than other key-presses. This suggests that, after substantial practice, the cognitive processor is no longer required for concatenating motor chunks when they are repeatedly executed in a fixed order.

## *Dual processors*

Two major issues for the DPM concern the justification for the assumptions of (a) two distinct processors instead of a single graded processing resource, and (b) a race between the two processors. We believe that justification for the two processor assumption comes from several findings. The first relates to the notion that action slips have been found to mainly occur at the decision points in an action sequence, where highercognitive involvement is required for adequate action selection (e.g., Reason, 1992; Botvinick and Bylsma, 2005), and not the moments where behavior is guided more automatically. This is in line with two qualitatively distinct processors; one controlling and the other executing behavior. Similarly, two such processors can also explain why action sequences sometimes continue even though the situation requires sudden termination. In that case the cognitive processor is temporarily unavailable (e.g., by distraction) or disengaged (e.g., in case of absent-mindedness), and the motor processor simply continues the habitual course of action. Second, we believe that two distinct processors fit well with the notion that both the qualitative features and underlying neural substrate differ greatly between early and late practice stages. Below this is discussed in more detail.

Third, and most importantly, there is also empirical support for two processors from DSP studies. One source of support is that selecting a forthcoming action (a single key-press, or a motor chunk) slows ongoing sequence execution, but this slowing is unaffected by the load of the selection process itself (when manipulated in terms of S-R compatibility and reversing a learned stimulus-sequence association, Verwey, 1995, 2001). This cannot be easily explained by a single resource or single processor model. Another type of behavioral support comes from a dual task study by Verwey et al. (2010). This study involved a tone counting task as secondary task to force participants to allocate their cognitive processor away from executing the sequence (for an earlier version, see Verwey, 1993). It appeared that in familiar sequences each tone was followed by slowing of the three ensuing responses by maximally 30 ms. In a follow-up study, Verwey et al. (2013) further showed that slowing was larger for identifying and counting a tone than for merely identifying a tone. These dual task findings are in line with two processors: while the secondary task allocated the cognitive processor away from executing the sequence, the motor processor enabled the continuation of sequence execution—with the moderate slowing being caused by the cognitive processor no longer racing with the motor processor. Additionally, taking away the key-specific stimuli (after the first) in a familiar keying sequence has been found to also slightly slow sequence execution (Verwey, 1999, 2010). This is entirely consistent with the notion that this largely eliminated the contribution of the cognitive processor to triggering individual responses in the familiar keying sequence—with performance based merely on efforts of the motor processor.

We would like to close this section by outlining how the DPM rests on assumptions similar to models developed for various other types of tasks. First, the notion of separate cognitive and motor processors is found across (models derived from) various research paradigms. For example, Sternberg (1998) suggested that sensory and motor processing stages might be carried out by processors independent from a central processor that is responsible for cognitive processing stages (like stimulus identification, and response selection). Moreover, results obtained with the Psychological Refractory Period (PRP) paradigm (e.g., Welford, 1952; Pashler, 1994) showed that the processing stages that are affected by a central bottleneck include response selection, response initiation, decision, and certain perceptual judgments (e.g., Pashler, 1992, 1994; De Jong, 1993). While the central bottleneck may be caused by a cognitive processor dealing with one process at the time, the initial perceptual processes and the final motor execution stages are assumed to be carried out by dedicated processors (Pashler, 1994). Indeed, the overall notion that a cognitive processor performs a prepared series of processing operations has been proposed many times before in more general information processing architectures (e.g., Norman and Shallice, 1986; Detweiler and Schneider, 1991; Meyer and Kieras, 1997; Anderson et al., 2004; Salvucci and Taatgen, 2008). The order of these processing stages, and whether sensory and motor processors are to be used, would be set during task preparation by creating a superordinate control structure (e.g., Norman

and Shallice, 1986; De Jong, 1995; Klapp, 1995; Salvucci and Taatgen, 2008). Such a schema-based processing procedure is in line with our notion of a cognitive processor setting in advance the processing operations and autonomous processors to be used.

Second, the notion that different processors are racing to trigger a response in a familiar keying sequence fits well with the many indications that the execution of a movement sequence involves the simultaneous use of different codings (motor, egocentric, and allocentric spatial, verbal; see e.g., Hikosaka et al., 1999; Bapi et al., 2000; De Kleine and Verwey, 2009a; Verwey et al., 2010; Panzer et al., 2011; Shea et al., 2011; Verwey and Abrahamse, 2012). Moreover, it relates strongly to other models that assume a race between different processing routes (e.g., Logan, 1988; Kornblum et al., 1990).

### *Modes of sequence execution*

Verwey (2003a) noted that sequencing performance in the DSP task can be based on at least two execution modes. The first is a *reaction mode* in which participants use each key-specific stimulus to select a response. This mode is especially used when encountering new sequences, and involves closed-loop control. As a discrete sequence is repeatedly executed, participants learn the order of stimuli and responses, and switch to performing the sequence (or short parts of it; i.e., motor chunks) in response to just the first stimulus. Subsequent stimuli can be ignored and participants are said to be performing in the *chunking mode*. This mode can be envisaged as open-loop control in the sense that key-specific stimuli after the first are no longer needed (though, as said, they may still be used when the cognitive processor races with the motor processor).

Recently, indications have been found that discrete keying sequences can be carried out in a third execution mode too. Earlier studies had demonstrated that when participants switch from slow to fast execution of a familiar sequence they briefly produce the sequence at some intermediate rate (Verwey, 2003a), and that elderly do not use motor chunks in discrete keying sequences but still benefit from practice (Verwey, 2010; Verwey et al., 2011). Inspired by these findings, Verwey and Abrahamse (2012) tested the notion that an SRT-like *associative mode* develops with DSP practice. In this mode successive reactions are primed by the preceding responses but still require stimulus processing for actual execution—as would occur in SRT learning (see Abrahamse et al., 2010). Verwey and Abrahamse (2012) argued and confirmed that in the DSP task the effect of the associative mode would emerge only when the much faster chunking mode is not used. Skilled participants performed a condition in which familiar, discrete keying sequences were carried out while most of them included 2 deviants (i.e., key-specific stimuli at unpredictable positions) that effectively disabled the chunking mode. As expected, the few sequences in this condition without deviants were executed much slower than the familiar sequences in a non-manipulated condition. Importantly, however, they were executed faster than unfamiliar sequences. Analysis of the RT distributions showed that this effect could not be attributed to sequences occasionally being performed in the chunking mode. The authors interpreted the intermediate execution rate as resulting from reactions to stimuli being primed by the preceding responses, just as observed by Verwey (2003a). That this associative mode develops seems reasonable given that responding to successive stimuli in early DSP practice mimics the SRT task.

These findings led to the proposal that familiar movement sequences can be executed in two different modes, the associative mode which continues to require external guidance by movement-specific stimuli and does not involve no use of motor chunks, and the chunking mode which is based on advance preparation of motor chunks and which does not require guidance by movement-specific stimuli. In the next section we attempt to integrate these execution modes with the DPM.

## *A general architecture*

The reaction and chunking modes can be easily accounted for by the DPM (see below). The theoretical challenges concern the implementation of the associative mode, especially with respect to the representational level. It is generally accepted that representing sequential information may involve coding across the perceptual, cognitive, and response-based/motor levels (e.g., Hikosaka et al., 1999; Keele et al., 2003; Abrahamse et al., 2010; Goschke and Bolte, 2012). The chunking mode would mostly depend on associations at the motor level from which motor chunks can develop. Conversely, the associative mode could be tentatively linked to various types of visuo-spatial associations—in line with the SRT literature (Abrahamse et al., 2010). However, the possibility cannot be excluded that the associative mode derives directly from the same associations that underlie the chunking mode: rather than being just static propositions waiting to be used for the chunking mode, motor chunks may continuously influence ongoing processing (Cleeremans, 2008). They may, for example, prime the selection of individual responses. To comply with the notion of distributed coding (cf. Hikosaka et al., 1999; Abrahamse et al., 2010), we assume an event-based sequence representation—where *event* refers to a specific S-R episode—that potentially involves associations at both the visuospatial (e.g., between successive stimuli or response locations) and motor level. Its precise features will probably depend on the task requirements, the context, and the amount of practice.

**Figure 3** depicts a cognitive architecture for the skilled production of movement sequences. It shows how a response (R*n*) is generated on the basis of stimulus input (S*n*) by the concerted action of the cognitive and motor processors. These processors may use a motor buffer that can temporarily hold representations that concern a limited number of responses. In the reaction mode, which is dominant with unfamiliar or random sequences, the cognitive processor processes sensory input and selects the appropriate response separately for each particular stimulus. Next, it puts the motor processor to work for the actual execution of the response. With repeated execution of the same sequence of events, associations develop between successive events. The resulting representation allows for response selection processes to be primed when they are executed in a familiar order on the basis of preceding events (associative mode). Moreover, when the representation becomes sufficiently strong at the motor level, it allows for the temporary activation of a short series of movements (i.e., motor chunks) as if

**motor chunking).** S*<sup>n</sup>* and R*<sup>n</sup>* denote the current stimulus and corresponding response within the sequence, respectively. Black arrows and boxes denote the relevant processing routes. **(A)** In the reaction mode, responses are selected by the cognitive processor (CP) on the basis of S-R translation. **(B)** Ongoing response selection by the CP is facilitated by the first, still weak, sequence knowledge that develops. **(C)** Motor chunks have developed, and the codes from the motor buffer, and response selection by the cognitive processor on basis of continued S-R translation or explicit sequence knowledge (dark gray arrow with black lining). Also note (II) that a fourth theoretical possibility is not depicted here, namely that the CP can load the motor buffer *not* by selecting motor chunks, but *rather* by (the slower process of) selecting and loading individual response elements of a (relatively) unfamiliar sequence.

they are loaded in a single step into a motor buffer. Next, the motor buffer content is read and executed by the motor processor. Because the motor buffer capacity is limited, the number of simultaneously prepared and executed responses is limited. Finally, the independence of the cognitive processor and motor processor allows a race between them in that the cognitive processor selects responses at the cognitive ("response selection") level, and the motor processor triggers responses from the motor buffer.

The DPM forwards a number of testable predictions on the dynamic interplay between the different modes and the types of sequence knowledge acquired. For example, the model predicts that for participants without explicit sequence knowledge, the effect of a secondary task on executing a DSP sequence will vanish if key-specific stimuli after the first are no longer presented (i.e., single-stimulus condition). The reason is that without explicit knowledge and external stimuli, the cognitive processor is no longer able to race with the motor processor, and thus never enhances skilled (i.e., motor processor based) performance. Additionally, if after extensive practice the chunking mode is prevented through, for example, introducing (auditory) stop-signals during a specific proportion of sequences within a block (requiring to terminate sequence execution), it can be expected that executing a familiar sequence in a single-stimulus condition is only better than executing an unfamiliar sequence for aware (and not for unaware) participants because their explicit knowledge still allows the cognitive processor to enhance performance beyond pure S-R translation. Furthermore, artificially slowing execution rate by using more complex responses will increase the presence, and contribution, of explicit sequence knowledge and/or the associative mode because there is more time to contribute. These and other (types of) predictions need to be addressed in future research.

## **GENERALIZING THE DUAL PROCESSOR MODEL**

In our efforts above to situate the DSP task within the larger domain of sequence learning, we already anticipated a discussion about how the DPM relates to other work on sequence skill. Here we outline such a link, first, with respect to discrete sequence skill, and second, with respect to the models that stem from related sequence learning paradigms. This results in various issues for future research.

## *Practice levels and sequence complexity*

The end-product of motor learning is typically related to automaticity in the sense that control over behavior becomes fully encapsulated and cognitively impenetrable. For example, it is difficult to verbalize the procedure of how one laces one's shoes. Without disclaiming this notion of automaticity in discrete sequence skill, the DPM features both cognitive and motor control as continuously interacting components of even welltrained movement sequences. This model is based on research with the DSP task, which typically employs sufficient practice to reach substantial performance improvements as compared to unfamiliar movement sequences but it does *not* account for overlearned sequences (such as when a single sequence is practiced for many sessions across multiple days or even weeks; e.g., Lehéricy et al., 2005; Coynel et al., 2010). Hence, the DPM may not generalize to overlearned movement sequences. However, we believe that overlearned sequence skill can still be explained by the DPM by assuming that with more extensive practice with the same movement sequence, the contribution of the cognitive processor is increasingly reduced as processing becomes automatized (i.e., stimulus-based selection of entire motor chunks; successive motor chunks becoming either fully represented into a larger motor chunk, or concatenated in a largely automatic manner).

As mentioned above, there are numerous earlier discrete sequence learning studies that employed relatively little practice, mostly in combination with a learning procedure that did not involve key-specific stimuli (Restle, 1970; Simon, 1972; Jones, 1974; Rosenbaum et al., 1983). We believe that these studies did not involve sequence execution in the chunking mode. Rather, performance in those studies seems to have been based on a dominant cognitive processor using simple rules that describe the entire sequence. As such, the phenomena observed in those studies seem to inform us primarily on the cognitive constraints of the cognitive processor.

One such major phenomenon that has been shown across multiple sequential motor tasks is referred to as the parameter remapping effect (Rosenbaum et al., 1986). This implies that a sequence is more difficult to execute when the number-oftaps carried out by a particular finger changes throughout the sequence than when each finger always taps a fixed number of times. One could say that the sequential structure provides limitations on the ease with which movement sequences are prepared. It is, however, not clear whether this effect can be found also after more substantial practice. The DPM suggests that the development of motor chunks could shield against interference by parameter remapping, but this requires explicit examination.

Finally, as noted above, the various discrete sequence studies that employed little practice *also* employed sequences of limited length and/or salient rule-based structure, which can be easily transferred to long-term memory with even little practice. This leaves two possibilities. First, it may be that the fast development of long-term memory representations for these short and/or rule-based sequences actually allows for motor chunking even with little practice. This is tentatively supported by the observation that practice on 3-key sequences quickly reaches a performance asymptote (e.g., Rosenbaum et al., 1983). Alternatively, motor chunking may be highly dependent on substantial practice, and involve different processing mechanisms (and neural correlates) than the execution of short and/or salient sequences with little practice. We here argue for the latter case, which is supported by the general notion that coding in motor coordinates requires ample physical practice, and the finding that the relatively high execution rate of simple 2-key sequences disappeared with increasing cognitive load (Verwey, 2001). As such, we believe that discrete sequence learning studies with short and/or rulebased sequences, too, are strongly based on a dominant cognitive processor that controls performance by the one-by-one loading of individual response elements with no motor chunks involved. Future studies are required to further explore this issue.

In short, even though the DPM is built on DSP studies that are characterized by substantial practice with relatively short, complex sequences, other discrete sequence learning studies can be tentatively related to this model, and—more importantly—can inform us about the characteristics of the two processors and their interplay.

#### *Relationship with other sequence skill models*

As mentioned above, the production of movement sequences has been studied with several tasks. The cognitive models that are proposed to account for the results in those studies share several features with the DPM. First and foremost, it should be noted that these models generally agree with the DPM that cognitive and motor processing involve independent systems (e.g., Pew, 1966; Allport, 1980; MacKay, 1982; Schmidt, 1988). One particularly interesting model has been proposed by Klapp (1995, 2003). He developed it for series of timed (Morse code) key-presses and speech sequences. It assumes, like the DPM, that longer sequences involve several chunks, each of which may initially consist of a single element (key-press or syllable) and later, of short series of these elements. The so-called INT process programs the internal structure of each chunk, which may in simple RT conditions occur before sequence initiation. After loading the motor buffer, and after the go-signal has been detected, the SEQ process then places these chunks in the correct order so that the sequence of chunks can be executed correctly. In longer sequences, the INT processes dedicated to later chunks occur during sequence execution (Klapp, 2003). One could argue that these INT and SEQ processes are a specification of two roles carried out by the cognitive processor proposed in the DPM when timing is crucial. Indeed, this model leaves actual execution to some unspecified motor process.

The Hikosaka et al. (1999) model suggests that, in what they called the pre-learning stage, each stimulus triggers a movement without any effect of preceding or subsequent stimuli (like the DPM's reaction mode). With practice, visuo-spatial and motor learning develop, with the former developing at faster rate. The visuo-spatial learning may be tentatively related to the associative mode of the DPM: successive events prime each other on the basis of visuo-spatial sequential representations, either at the perceptual (e.g., stimulus location learning) or the response (e.g., response location learning) level. The motor learning system becomes dominant during later stages of sequence learning, and can be tentatively linked to the chunking mode of the DPM.

Keele et al. (2003) proposed a dual system framework for sequence learning in the SRT task. This model is designed to explain results from a continuous sequence learning task that does not include preparation and chunk development. Instead, the main focus is on the implicit-explicit divide. The framework assumes a unidimensional system that is composed of multiple modules that each associate information within a single informational dimension. There also is a more overarching multidimensional system that enables associations both within and across informational dimensions. Together, these two systems can account for a number of dual-task studies on SRT learning. The DPM's cognitive processor is clearly reminiscent of Keele et al.'s (2003) multidimensional processor, but the unidimensional modules do not seem to correspond well to the motor processor of the DPM. Though the latter two share features in terms of their relatively autonomous functioning, there are some essential differences. Most importantly, whereas the motor processor is assumed to be executive in nature and fully dependent on input from the cognitive processor, the unidimensional modules from Keele et al. are primarily representational systems. Both the multidimensional system and the unidimensional modules are related to what we referred to as the associative mode: they are both responsible for the relatively automatic priming of responses on the basis of inter-trial contingencies and do not involve the possibility of preparing series of responses and using motor chunks. This is entirely reasonable given that the Keele et al. model was developed in the SRT research domain where motor chunks do not develop (e.g., Jiménez et al., 2011).

Finally, based on a number of discrete sequence learning studies with relatively little practice, Rosenbaum et al. (1984) and Rosenbaum (1987) proposed the hierarchical editor (HED) model. The HED model builds on the notion that a hierarchically organized motor program is first "edited" to specify open parameters, only after which the sequence can be executed. We believe, in line with notions from above, that the HED model mainly describes the cognitive constraints that are related to the workings of the cognitive processor in preparing and/or controlling sequence execution after limited practice. With substantial practice and the resulting development of strong motor chunks it may be questioned if similar hierarchical structures work on series of whole motor chunks.

Overall, we believe that there is a clear overlap between the DPM and these other models. This overlap supports the merit of the DPM as a general model of sequence performance. The most important features of the DPM are that (a) it distinguishes the associative and chunking modes of sequence execution (and thereby their respective literatures), (b) it is able to explicitly account for automaticity of skill by the relative autonomous execution processes of a motor system (motor processor and motor buffer), and (c) it allows for explaining the overall dynamic interplay between cognitive and automatic processes in daily life.

## **NEURAL UNDERPINNINGS OF THE DUAL PROCESSOR MODEL**

In this section we discuss on the basis of cognitive-neuroscientific findings (e.g., Hikosaka et al., 1999; Ashby et al., 2010; Stocco et al., 2010; Penhune and Steele, 2012) how the cognitive architecture proposed above may be implemented in the human brain. Specifically, we develop a mapping of the DPM on specific cortico-striatal loops (Seger, 2006; Doyon et al., 2009; Ashby et al., 2010). The nature of this mapping is admittedly speculative as very little of the work discussed here strictly builds from the DSP task itself, but we feel that this effort nevertheless will inspire progress in the understanding of discrete sequence skill from a combined cognitive and neuroscientific approach.

We explicitly distinguish the three modes in which sequences can be executed, and thus focus mostly on implementation and less on representation of sequence skill. Though this endeavor probably results in an oversimplification and a somewhat artificial separation of massively interacting networks (e.g., corticostriatal loops cannot be strictly separated; Seger and Spiering, 2011), we believe that this effort will guide future research. In brief, we propose that S-R based performance in the reaction mode is related to the associative cortico-striatal loop (AL) in concert with prefrontal cortex (ALPFC). With practice, sensorimotor cortico-striatal loops (SLs) gradually take over and enable both more automatic S-R translation and sequence based performance in close interaction with premotor and primary motor

cortices<sup>3</sup> . For the associative mode we propose the sensorimotor loop to involve the premotor cortex (SLPMC), while for the chunking mode the SMA is involved instead (SLSMA). In the chunking mode, an ALPRE−SMA loop may remain involved for the actual loading of motor chunks. Hence, besides building from the accepted distinction between the AL and the SL, we also propose functional divisions of both the AL and the SL.

## **REACTION MODE**

The execution of an individual movement on the basis of an external stimulus (like when a random or unfamiliar sequence is being executed) probably involves areas that are consistently related to spatial response selection, such as the premotor cortex (PMC), the parietal cortex and the prefrontal cortex (PFC) (Iacoboni et al., 1996; Dassonville et al., 2001; Merriam et al., 2001; Schumacher and D'Esposito, 2002; Jiang and Kanwisher, 2003; Schumacher et al., 2003, 2005, 2007). The associative striatum enables a functional network between prefrontal and posterior areas (i.e., AL; Seger, 2008) to support the initial S-R translation processes that underlie the reaction mode (i.e., performance is driven by goaldirected control based on the S-R mappings that are held in working memory). Indeed, activity in the associative striatum has been linked to the early stages of training in sequence learning and habit formation tasks (Jankowski et al., 2009; Ashby et al., 2010). Moreover, it has been shown that activity in the associative striatum (i.e., anterior caudate) is closely correlated with (the rate of) learning the associations between visual cues and specific motor responses (Williams and Eskandar, 2006). However, the involvement of PFC may soon decrease as the highly compatible spatial S-R mapping of the DSP task allows for less controlled response selection that involves PMC in concert with the sensorimotor striatum—in line with the special role that is assumed for PMC in translating spatial information into motor output (Hikosaka et al., 1999) and with PMC involvement in habit formation (i.e., automatic S-R translation; Ashby et al., 2010).

## **SEQUENCE SKILL**

With more practice and the development of a sequence representation, activity will further shift from the AL toward SLs. The SLs are networks that involve the sensorimotor striatum, premotor (PMC, supplementary motor area or SMA) and motor cortices. Various findings support this notion of activity shifts. First, Miyachi et al. (2002) found that the sensorimotor striatum is home to most of the striatal neurons that show their strongest response to highly practiced motor sequences. Furthermore, whereas the temporary inactivation of the sensorimotor striatum impairs performance on already acquired motor sequences, it hardly affects the learning of new motor sequences (Miyachi et al., 1997). Second, practice-based transition in activity can also be observed at the cortical level. Specifically, whereas PMC

<sup>3</sup>It should be noted that, with an amount of practice that exceeds the level typical for the DSP task, the sensorimotor cortico-striatal loops may even enable direct cortical-cortical representations to form on the base of slow Hebbian learning (Ashby et al., 2010; Karni et al., 1998). Yet, this will not be covered in the present review.

is typically activated relatively early in learning, later in training this activation decreases while SMA activity gradually increases (Jenkins et al., 1994; Toni et al., 1998; Wymbs and Grafton, 2013). It is assumed that SMA is strongly related to memory-based sequence performance (Mushiake et al., 1991; Haaland et al., 2004), thus independent of external action cuing, while PMC underlies skill that is stimulus-based. Below we specify this for both the associative and chunking modes that we defined above, starting with the latter because it better relates to the existing neuro-imaging work with discrete movement sequences.

## *Chunking mode*

The crucial role of the BG for motor chunking has become evident over the last decades. Studies on stroke (Boyd et al., 2009) and Parkinson's disease (e.g., Hayes et al., 1998; Tremblay et al., 2010) led to the conclusion that the ability to form motor chunks is impaired in patients with BG damage. Additionally, rodent research has shown that activity in the striatum is strongly related to, and essential for, motor chunking (Yin and Knowlton, 2006; Graybiel, 2008; Jin and Costa, 2010). Performance in the chunking mode is dominated by the cognitive processor selecting and loading a motor chunk that is subsequently executed by the motor processor. While the overall involvement of BG is evident, we here speculate about the chunking mode in some more detail, subsequently considering (1) the segmentation of sequences, (2) the motor buffer, (3) the loading of the motor buffer, and (4) chunk-based performance.

First, as noted before, discrete movement sequences exceeding about four or five responses are usually spontaneously segmented into two parts. Recent studies suggest that such segmentation of longer sequences into multiple smaller chunks is based on fronto-parietal networks. Pammi et al. (2012) observed selective activation of a fronto-parietal network in the early learning stage with increasing sequence length (in the *m* × *n* task). This notion also fits well with two studies by Verwey and colleagues who showed that the ability to segment long sequences into chunks is impaired in elderly (Verwey, 2010; Verwey et al., 2011), which could be related to reduced cortical capacity (Resnick et al., 2003; Raz et al., 2005). The segments that are created can be assumed to gradually transform into relatively rigid motor chunks, with concatenation processes required for the fluid transitions between motor chunks. In a recent fMRI study on human subjects, Wymbs et al. (2012) related these latter processes to the bilateral putamen of the BG.

Second, the chunking mode involves reading responses from a motor buffer. As noted above, we conceive of the motor buffer as a part of working memory. Over the last decades, an increasing number of researchers understand working memory as the activated part of long term memory (e.g., Cowan, 1995; Postle, 2006). The long term representations for sequence skill (i.e., motor chunks) are highly distributed, and may even shift between areas with practice. However, there is no overall consensus. For example, storage has been proposed to relate to premotor areas (Jacobsen, 1934; Fulton, 1935), to the sensorimotor parts of the BG (Lehéricy et al., 2005), to the cerebellum (e.g., Hikosaka et al., 2002; Doyon et al., 2009), and, with long term practice, to the primary motor cortex itself (e.g., Matsuzaka et al., 2007). Additionally, equally strong arguments have been proposed against some candidate regions. For example, PMC activation may not reflect the representation of motor commands *per se* but rather their associations with specific sensory cues (e.g., Halsband and Lange, 2006), while the BG may contribute to skill by training cortical-cortical and thalamo-cortical representations rather than by storing procedural knowledge (e.g., Ashby et al., 2010; Desmurget and Turner, 2010). Overall, then, it is difficult to pinpoint the representation that develops with short, discrete keying sequences in the DSP task. Sequence representations are probably highly task- and context-dependent, and relevant neuro-imaging work with the DSP task is currently lacking.

Third, on the basis of a study by Kennerley et al. (2004) we propose that loading the motor buffer (in the chunking mode) is related to pre-SMA. In this TMS study the authors showed for extensively practiced sequences (a) that the pre-SMA is involved in the initiation of a motor chunk, but (b) that this only holds when the motor chunk needs to be retrieved from memory as a "superordinate set of movements without the aid of a visuomotor association" (p. 978). Conversely, the pre-SMA was shown to not be involved in general execution processes. Pre-SMA, then, through its dense connections with PFC, is assumed here to selectively activate the relevant long-term memory representations (i.e., load the motor buffer) that are stored elsewhere. This initiating role of the pre-SMA fits well with findings from monkey research that pre-SMA neurons are mostly active during pre-movement and not during actual movement (Halsband and Lange, 2006). Because pre-SMA is typically related to the AL with the basal ganglia, the loading of the motor buffer may require a stable involvement of the ALpre-SMA in even more advanced sequence skill, although, as mentioned above, the ALPFC gradually reduces its impact.

Fourth, the true chunking based performance is proposed to rely on the SLSMA. This fits well with the notion that SMA is typically involved in memory-based performance: though stimuli are still presented in the DSP task even after substantial practice, these are assumed to be no longer dominant in the response selection process—as evidenced, among others, by average RTs of sometimes below 100 ms. It is also consistent with various other findings. For example, a study with mice by Jin and Costa (2010) indicates that initiating (and also aborting) action sequences is related to nigro-striatal circuits—as if start (and stop) signals are represented within these circuits. In sum, from the notion that action sequences are generally goal-directed, we propose that initiation of well-learned action sequences is based on sequence (or motor chunk) selection and loading through PFC (Averbeck et al., 2006) and pre-SMA, after which a sequence-specific SLSMA is involved in prompting sequence execution.

Finally, we could speculate on a different (or possibly just complementary) function for the BG in sequence skill. Specifically, as discrete sequence skill involves the activation by PFC/pre-SMA of particular sequence (motor chunk) representations laid out somewhere else in the brain (i.e., loading the motor buffer; see above), the effectiveness of this advance preparation can be assumed to require the temporary inhibition of execution processes. The BG are well-suited to moderate this process as they are involved in go- (cf. direct pathway) and no-go-signals (cf. indirect and hyperdirect pathways; Nambu et al., 2002) that determine thalamico-cortical output. Various observations are in line with such a moderator role. For example, the BG have been shown to be heavily involved in tasks that require inhibiting a planned action program such as in the stop-signal task (Aron and Poldrack, 2006), and there is at least tentative support for BG involvement in motor imagery (Guillot et al., 2012), which probably also relates to the inhibition of motor commands. Moreover, Elsinger et al. (2006) observed enhanced activity in the anterior putamen when sequences were held in memory for delayed execution, which could be related to inhibitory processes as well. As such, loading of the motor buffer during the preparation of skilled DSP may require inhibitory processes within BG.

## *Associative mode*

We propose that the major difference between the chunking and the associative mode relates to the sensorimotor loop that is involved. Whereas the SLSMA loop underlies the chunking mode, the associative mode builds from a SLPMC because performance in the associative mode is still partly under stimulus-based control. The latter loop will be engaged either when practice has not yet developed strong enough representations for memory-based performance (i.e., the chunking mode driven by the SLSMA), or when the chunking mode has been disengaged through experimental manipulations. This fits well with studies that relate both the SL and the PMC to implicit sequence learning in the SRT task (e.g., Grafton et al., 2002; Bischoff-Grethe et al., 2004; Seger, 2006), which is typically seen as a form of associative learning (e.g., Abrahamse et al., 2010) that remains at least partly stimulusdriven and does not include motor chunking (Jiménez et al., 2011). Also inspired by the SRT literature, the storage in the brain of knowledge that underlies the associative mode is highly taskand/or context-dependent, but probably involves at least areas across parietal cortex (e.g., Jenkins et al., 1994; Grafton et al., 1998) that are related to visuo-spatial coding.

## **CONCLUSIONS AND QUESTIONS FOR FUTURE RESEARCH**

In the current paper we have described the DSP task, the major behavioral phenomena that can be typically observed with it, and

## **REFERENCES**


an update of the DPM. The DPM holds that discrete sequence skill builds from the continuous and dynamic interplay between a cognitive processor and a motor system comprising a motor processor and a motor buffer, with the former being dominant early on in practice, and the latter taking over execution as practice evolves. The notion that movement skill is characterized by automaticity is explained by the relative autonomy of the motor system from the cognitive processor. As we have outlined, this model generates various predictions of the model at the behavioral level that await further exploration. We have emphasized that the DSP literature that underlies the DPM is limited in scope in terms of practice amount and sequence structure, and future studies should aim to clarify how the DPM relates to these features; from there is should also be explored if the general notions of DPM hold across other sequence learning paradigms.

As to the neural underpinnings of the DPM, we suggest (a) that striatum and PMC (possibly in concert with more posterior areas) define a functional loop that underlies the reaction mode from the moment that S-R translation becomes relatively automatic (cf. habit formation). In the case of the DSP task this would develop quite rapidly because of the high spatial compatibility of stimuli and responses. We further suggest (b) that a sensorimotor-PMC loop underlies the associative mode, and (c) that a sensorimotor-SMA loop underlies the chunking mode. The main distinction between the associative and the chunking modes may lie in the efforts of the BG to inhibit execution during the activation of (cortical or subcortical) areas that contain relevant sequence representations. Besides generating predictions for future research, we believe that this tentative mapping of the DPM's execution modes on specific cortico-striatal loops will contribute to explorations on the biological plausibility of DPM.

## **ACKNOWLEDGMENTS**

Elger L. Abrahamse was supported by the Netherlands Organization for Scientific Research (NWO) under contract number 446-10-025 and by the Research Foundation—Flanders (FWO) under contract number 12C4712N. Marit F. L. Ruitenberg was supported by the Netherlands Organization for Scientific Research (NWO) under contract number 400-07-097.

effector-independent components of sequencing skill. *J. Mot. Behav.* 41, 30–44.


sequences. *J. Neurophysiol.* 101, 3116–3125.


mechanisms for reducing interference in concurrent performance. *J. Exp. Psychol. Hum. Percept. Perform.* 15, 686–700.


processing information. *Psychol. Rev.* 63, 81–97.


practice on coding and learning of movement sequences. *Q. J. Exp. Psychol.* 62, 1265–1276.


brain activation. *Hum. Brain Mapp.* 17, 193–201.


S. Sternberg (Cambridge: MIT), 703–863.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 30 October 2012; accepted: 01 March 2013; published online: 19 March 2013.*

*Citation: Abrahamse EL, Ruitenberg MFL, de Kleine E and Verwey WB (2013) Control of automated behavior: insights from the discrete sequence production task. Front. Hum. Neurosci. 7:82. doi: 10.3389/fnhum.2013.00082*

*Copyright © 2013 Abrahamse, Ruitenberg, de Kleine and Verwey. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

**MINI REVIEW ARTICLE** published: 14 March 2013 doi: 10.3389/fnhum.2013.00081

## Basic principles of sensorimotor adaptation to different distortions with different effectors and movement types: a review and synthesis of behavioral findings

## *Otmar Bock\**

*Institute of Physiology and Anatomy, German Sport University, Köln, Germany*

#### *Edited by:*

*Rachael D. Seidler, University of Michigan, USA*

#### *Reviewed by:*

*Herbert Heuer, Leibniz Research Centre for Working Environment and Human Factors, Germany Lauren E. Sergio, York University, Canada*

#### *\*Correspondence:*

*Otmar Bock, Institute of Physiology and Anatomy, German Sport University, Am Sportpark Müngersdorf 6, 50933 Köln, Germany. e-mail: bock@dshs-koeln.de*

This article reviews seemingly conflicting behavioral data about sensorimotor adaptation. Some earlier studies assert that one common mechanism exists for multiple distortions, and others that multiple mechanisms exist for one given distortion. Some but not others report that adaptation is direction-selective. Some submit that adaptation transfers across effectors, and others that a single effector can adapt to multiple distortions. A model is proposed to account for all these findings. It stipulates that adaptive mechanisms respond to multiple distortions, consist of directionally tuned special-purpose modules, can be switched in dependence on contextual cues, and are connected to practiced movement types with a higher weight than to unpracticed ones.

**Keywords: motor learning, plasticity, context-dependence, transfer, multiple adaptation**

Human sensorimotor adaptation has been evaluated with a baffling number of experimental paradigms. Subjects were exposed to distortions of visual (Stratton, 1897), acoustic (Mikaelian, 1974) and proprioceptive inputs (Lackner and DiZio, 1994), to topographical (Kohler, 1955; Cunningham and Welch, 1994) and to dynamical distortions (Shadmehr and Mussa-Ivaldi, 1994; Bock, 2003), to distortions experienced while tracking (Cunningham and Welch, 1994), pointing (Mikaelian, 1974) or grasping with the hand (Gentilucci et al., 1995; Weigelt and Bock, 2007), while executing pursuit eye movements (Carl and Gellman, 1986), reflexive (McLaughlin, 1967) or volitional saccades (Deubel, 1995). Given this wealth of paradigms, it seems reasonable to question whether all authors dealt with the same phenomenon: is all adaptation achieved by one common mechanism, or rather by multiple mechanisms, each specific for a given paradigm?

This question has been addressed in behavioral studies by testing for the transfer of adaptation from one visual rotation to another, or from one lateral shift to another. This work invariably found that subjects started under the second distortion with the behavior they acquired under the first, and then gradually modified it until it became adequate for the second distortion; as a consequence, they performed better than novices when the second distortion was *larger* than the first, but worse than novices when the second distortion was *opposite* to the first (Lazar and van Laer, 1968; Wigmore et al., 2002; Bock et al., 2003). Thus transfer was compulsory, occurring even where it degraded performance. Other work found compulsory transfer even between distortions of a different type, i.e., between a visual rotation and a visual velocity-dependent lateral shift (Thomas and Bock, 2010), between a visual rotation and a force field (Bock and Thomas, 1999), and between a visual and an acoustic rotation (Kagerer and Contreras-Vidal, 2009). In those studies, performance benefits again emerged when both distortions were of equal sign, and costs when they were of opposite sign. Taken together, these findings suggest that adaptation to a wide range of distortions might be based on a common mechanism; this is illustrated in **Figure 1A**, where a universal adaptive mechanism receives sensory inputs *I*<sup>j</sup> from different sensory modalities distorted in different ways, and sends motor outputs *O*<sup>k</sup> to different effectors executing different types of movement.

Other findings have refined this view by indicating that the proposed universal mechanism can be subdivided into several functionally specialized modules. Thus, subjects exposed to different visual rotations perform less and less well as the magnitude of rotation increases toward 90◦, but improve again as rotation continues to increase from 90◦ toward 180◦; in fact, performance under a 180◦ rotation is not dramatically poorer than under no rotation (Cunningham, 1989; Abeele and Bock, 2001). Furthermore, subjects exposed to a rotation of more than 90◦ quickly change their response direction by 180◦ and then gradually change it "back" toward the required angle (Bock et al., 2003). These findings call for the existence of two functional modules, one that gradually changes spatial coordinates by up to 90◦, and a second one that quickly changes them by 180◦; the latter module possibly exploits the mathematical equivalence between a 180◦ rotation and an inversion of the horizontal and vertical axis.

Further work suggests that the presumed gradual-change modules are selective to only a limited range of movement directions around the practiced direction (Krakauer et al., 2000; Wang and Sainburg, 2005). This range can be estimated from published data as 45◦ (Tanaka et al., 2009) to 80◦ (Roby-Brami and Burnod, 1995), which fits well with the finding that adaptation shows only modest signs of interference when eight targets, located 45◦ apart, are associated with different rotational transformations (Werner and Bock, 2010). We posit that the axis-inversion modules are

"fnhum-07-00081" — 2013/3/13 — 9:34 — page 1 — #1

distorted in different ways, and sends outputs *O*<sup>k</sup> to different effectors executing different types of movement. **(B)** More elaborate model that includes functional modules *G*<sup>i</sup> for gradual changes, *X* <sup>i</sup> for axis inversions, mechanisms linked to the motor output by a context-dependent switch, and weighting factors that are higher for practiced than for unpracticed effectors and movement types.

direction-selective as well, i.e., they operate only for movement directions similar to the trained ones; however, this issue has not been addressed experimentally yet. In contrast, adaptation to a new scaling factor seems not to be directionally tuned: adaptation of one movement direction transfers obligatorily to the full 360◦ range of possible directions (Bock, 1992; Krakauer et al., 2000). **Figure 1B** therefore depicts an adaptive mechanism that responds to multiple distortions with a number of special-purpose modules: several directionally tuned ones for gradual changes of direction (*G*i), several directionally tuned ones for axis inversions (*X*j), and a single one for scaling (S). This layout correctly predicts the obligatory transfer between distortions, the concurrence

"fnhum-07-00081" — 2013/3/13 — 9:34 — page 2 — #2

of quick and gradual changes under one given distortion, and the distinct adaptation characteristics with rotations and scalings.

The interplay of special-purpose modules such as those in **Figure 1B** can be readily illustrated with available data on the adaptation to mirror-reversed vision. This distortion initiates quick 180◦ changes of response directions for targets presented at the left and right, quick 180◦ changes followed by gradual 90◦ clockwise changes for targets along the right diagonal, quick 180◦ changes followed by gradual 90◦ counter-clockwise changes for targets along the left diagonal, and only a transient increase of response variability for targets at the top and bottom (Werner and Bock, 2010). This pattern of findings can be easily explained by the model in **Figure 1B**: targets at the left, right, and along either diagonal activate the corresponding directionally tuned axis-inversion modules, and targets along the diagonals additionally activate the corresponding gradual-change modules. Note that such an interpretation puts the minimum number of gradual-change modules to eight: the distortion activates four modules tuned to the diagonal directions, and has no effect on four modules tuned to the interleaved orthogonal directions. As noted above, this number of modules fits well with their reported tuning width of 40–80◦, since 360/8 = 45. Similarly, the minimum number of axis-inversion modules seems to be 4: the distortion activates modules at the right and left, but not those at the top and bottom. For reasons of parsimony, one might therefore postulate eight gradual-change and four axis-inversion modules, but for reasons of symmetry, one might postulate eight modules of either type. Further research is needed to resolve this issue.

Adaptation to a given distortion does not transfer well to unpracticed movement types. A moderate transfer was observed between manual tracking and pointing (Abeele and Bock, 2003; Bock, 2005), grasping and pointing (Weigelt and Bock, 2010), as well as volitional saccades and pointing (Cotti et al., 2007), but no transfer was found between reactive and volitional saccades (Deubel, 1995), nor between reactive saccades and pointing (Cotti et al., 2007). Transfer between the two arms varied widely between studies and seems not to be obligatory, since both arms can concurrently adapt to opposite visual rotations with no sign of interference (Prablanc et al., 1975; Wang and Sainburg, 2003; Bock et al., 2005). Similarly, manual pointing and reactive saccades can concurrently adapt to two opposite distortions with only moderate interference (Grigorova et al., 2013). It even has been shown that one single arm, pointing at a single set of targets, can concurrently adapt to two opposite distortions if they are coded by contextual cues such as hemi-workspace (Ghahramani and Wolpert, 1997; Woolley et al., 2007), head position (Seidler et al., 2001), or screen color (Wada et al., 2003). In fact, subjects can adapt with no noticeable interference to as many as *four* distortions, each coded by a unique combination of arm and hemiworkspace (Thomas and Bock, 2012). Even when contextual cues are not available, subjects can use a "probing" movement to find out whether a previously established adaptive change should be preserved or rather abandoned (Wang and Sainburg, 2003). To account for these findings, **Figure 1C** shows four distinct multidistortion mechanisms that can be alternately connected to the motor output via a context-dependent switch; the signal is then weighted, with the trained effector and movement type receiving the highest weight.

A model of sensorimotor adaptation, consisting of multiple mechanisms that are selectable by context, has been proposed before (Ghahramani and Wolpert, 1997; Wolpert and Kawato, 1998). The present article refines this model by adding multidistortion sensitivity, special-purpose modules, directional tuning, and output weighting. The available database provides robust evidence for the existence of these key characteristics of adaptation, but future experimental findings may require an increase in the number of adaptive mechanisms and/or special-purpose modules. Additional research is also desirable to find out whether adaptive mechanisms are truly universal, i.e., respond to any conceivable type of distortion, and to determine the actual tuning widths of modules and weights of outputs. This would allow a quantitative rather than qualitative comparison of experimental data with model predictions.

The model in **Figure 1C** was designed to illustrate the known functional characteristics of adaptation; it was not meant to show the actual anatomical layout of the underlying neuronal circuitry. In fact, given the preponderance of parallel distributed processing in the brain, it is quite likely that the depicted modules and mechanisms are implemented within a highly interconnected neural network with only a limited topographical segregation. In a way, the model in **Figure 1C** could be interpreted as a specific version of schema theory, which posits that movements are executed by tailoring a generalized motor program to the needs of a specific movement (Schmidt, 1975).

As complex as it is, the model proposed in **Figure 1C** still disregards two crucial aspects of sensorimotor adaptation. One of them is the existence of multiple time scales. Gradual rotation proceeds with a time constant τ <sup>1</sup> in the order of several movements, and a second one with a time constant τ <sup>2</sup> in the order of several tens of movements (Snoddy, 1926; Smith et al., 2006); additional time scales in the order of days to months have been reported by classical accounts (Stratton, 1897; Kohler, 1955) and by recent spaceflight studies (Bock et al., 2010; Gaveau et al., 2011; Mulavara et al., 2012). Since the model in **Figure 1C** is mainly based on findings about long-term adaptation, it most likely represents the τ <sup>2</sup> component. Little is known about the characteristics of the τ <sup>1</sup> component, except that it acts in parallel rather than in series to τ <sup>2</sup> (Lee and Schweighofer, 2009), requires working-memory resources (Anguera et al., 2010), is context-independent (Lee and Schweighofer, 2009) and exhibits its own distinctive directional tuning (Bock and Schmitz, 2011). It still is unknown whether axis inversion and scaling also proceeds along multiple time scales.

The second neglected aspect is the contribution of strategies. Exposure to a distortion initiates not only the adaptive recalibration of sensorimotor pathways, but also the use of workaround strategies such as cognitive reinterpretations of sensory signals, anticipations, associative stimulus–response learning, postural changes, and error-based corrections (Redding and Wallace, 1996; McNay and Willingham, 1998; Clower and Boussaoud, 2000). These strategies are thought to be situation-specific and shortlived, and thus to modify performance during exposure to a distortion, but not after removal of the distortion or after transfer to a new movement type. Evidence for the role of strategies is

"fnhum-07-00081" — 2013/3/13 — 9:34 — page 3 — #3

therefore largely based on the dissociated effects of higher-order mental functions on subjects' performance *during* but not *after* exposure, e.g., the effects of aging (McNay and Willingham, 1998; Bock, 2005), emotional state (Bock, 2010), and explicit knowledge (Werner and Bock, 2007).

Summing up, **Figure 1C** presents a model for the slow component of adaptive recalibration that accounts for a wide range of

## **REFERENCES**


seemingly contradictory behavioral phenomena: compulsory versus partial versus null transfer, common mechanism for multiple distortions versus multiple mechanisms for one distortion, presence versus absence of direction-selectivity, and eye–arm transfer versus multiple adaptation of a single arm. Additional experiments are needed to verify the model, determine its parameter values, and possibly add further functional details.

feedback in reaching movements. *Exp. Brain Res.* 193, 315–321.


"fnhum-07-00081" — 2013/3/13 — 9:34 — page 4 — #4

distorted object size and orientation. *Exp. Brain Res.* 181, 139–146.


with different regions of workspace. *Exp. Brain Res.* 179, 155–165.

**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 02 November 2012; accepted: 28 February 2013; published online: 14 March 2013.*

*Citation: Bock O (2013) Basic principles of sensorimotor adaptation to different distortions with different effectors and movement types: a review and synthesis of behavioral findings. Front. Hum. Neurosci. 7:81. doi: 10.3389/fnhum.2013. 00081*

*Copyright © 2013 Bock. This is an openaccess article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

"fnhum-07-00081" — 2013/3/13 — 9:34 — page 5 — #5

## Context-dependent generalization

#### *Jordan A. Taylor1 \* and Richard B. Ivry2,3*

*<sup>1</sup> Department of Psychology, Princeton University, Princeton, NJ, USA*

*<sup>2</sup> Department of Psychology, University of California, Berkeley, CA, USA*

*<sup>3</sup> Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA*

#### *Edited by:*

*Rachael D. Seidler, University of Michigan, USA*

#### *Reviewed by:*

*Joseph M. Galea, University College London, UK Pietro Mazzoni, Columbia University, USA Rajiv Ranganathan, Northwestern University, USA*

*\*Correspondence: Jordan A. Taylor, Department of Psychology, Princeton University, 3-S-13 Green Hall, Princeton, NJ 08544, USA. e-mail: jordanat@princeton.edu*

The pattern of generalization following motor learning can provide a probe on the neural mechanisms underlying learning. For example, the breadth of generalization to untrained regions of space after visuomotor adaptation to targets in a restricted region of space has been attributed to the directional tuning properties of neurons in the motor system. Building on this idea, the effect of different types of perturbations on generalization (e.g., rotation vs. visual translation) have been attributed to the selection of differentially tuned populations. Overlooked in this discussion is consideration of how the context of the training environment may constrain generalization. Here, we explore the role of context by having participants learn a visuomotor rotation or a translational shift in two different contexts, one in which the array of targets were presented in a circular arrangement and the other in which they were presented in a rectilinear arrangement. The perturbation and environments were either consistent (e.g., rotation with circular arrangement) or inconsistent (e.g., rotation with rectilinear arrangement). The pattern of generalization across the workspace was much more dependent on the context of the environment than on the perturbation, with broad generalization for the rectilinear arrangement for both types of perturbations. Moreover, the generalization pattern for this context was evident, even when the perturbation was introduced in a gradual manner, precluding the use of an explicit strategy. We describe how current models of generalization might be modified to incorporate these results, building on the idea that context provides a strong bias for how the motor system infers the nature of the visuomotor perturbation and, in turn, how this information influences the pattern of generalization.

**Keywords: motor control, motor learning, motor adaptation, models, theoretical, generalization (psychology)**

## **INTRODUCTION**

Generalization following practice of a new motor task has provided an important tool for evaluating the specificity of learning. By examining whether or not the effects of training extend to untrained movements and novel contexts, we gain insight into the representational changes that have occurred during learning (Poggio and Bizzi, 2004). Generalization designs have been widely used in studies of sensorimotor adaptation with the pattern of generalization providing clues as to how movement is computed and updated through learning (Ghahramani et al., 1996; Thoroughman and Shadmehr, 2000; Donchin et al., 2003; Thoroughman and Taylor, 2005). These studies have revealed that the motor system does not learn by a simple look-up table (Atkeson, 1989; Conditt et al., 1997; Mussa-Ivaldi, 1999), but rather builds an internal model to approximate the sensorimotor mapping required for controlling reaches in a particular environment.

One common method for studying adaptation is to perturb the visual feedback, either by imposing a lateral translation (e.g., prism glasses) or an angular deviation (e.g., visuomotor rotation). These perturbations introduce an error between the expected and actual visual feedback, a signal that is used to modify an internal model. In generalization studies of visuomotor adaptation, training is restricted to movements in a particular direction or some subregion of the workspace, followed by testing with movements in novel directions or regions of the workspace (Pine et al., 1996; Krakauer et al., 2000).

The form and extent of generalization show distinct characteristics for these two types of perturbations. Following a translation, generalization is broad, spanning the entire workspace (Ghahramani et al., 1996). In contrast, generalization following visuomotor rotation has been found to be relatively narrow, with strong generalization for movements similar to the training direction and falling off rapidly as the probe directions deviate from this direction. When considered in polar coordinates, the degree of generalization falls to approximately 25% for movements 45◦ away from the training location (Krakauer et al., 2000). Nonetheless, most studies have found a small degree of generalization throughout the entire workspace (Pine et al., 1996; Krakauer et al., 2000; Tanaka et al., 2009), although the magnitude at distant locations is generally less than what is observed with translational shifts (Ghahramani et al., 1996).

Studies in which participants adapt to a visuomotor rotation have, for the most part, reported generalization patterns that are consistent with the direction of the rotation. However, the results of two recent studies suggest that generalization may entail another component, one that indicates that participants may be inferring a translational shift, either in addition to a rotation (Brayanov et al., 2012), or in lieu of a rotation (Taylor et al., 2013). Taylor et al. (2013) trained participants to reach to a target to the right of the starting position, imposing a counterclockwise (CCW) rotation that was counteracted by movements in the clockwise, or downward direction. When the participant was provided with full, online visual feedback, reaches to novel targets positioned in the opposite side of the workspace showed trajectories that were also deviated in the clockwise (CW) direction (now upward), consistent with what would be expected if a rotational had been learned (**Figure 1**). In contrast, when only endpoint feedback was provided during training, the trajectories to the novel locations were deviated in the counterclockwise direction, or downward, consistent with a translational perturbation (**Figure 1**). Thus, training under different forms of feedback led to very different patterns of generalization. Importantly, modeling of these different patterns of generalization revealed an alternative account of the broad, albeit modest, generalization observed in the online feedback condition. This generalization could be due to incidental training for movements in the direction of the generalization targets that occurred as the participants either made corrective movements to the training target or moved back to the starting location during the training phase of the experiment.

The behavioral and theoretical work on generalization have focused on how an internal model is modified, based on the tuning properties of the motor system and the form of the error signal. Ignored in this discussion is how the environmental context may also influence learning and, as such, constrain generalization. This is surprising given that contextual effects have been shown to provide a powerful source of constraint in a wide range of motor tasks (Hommel, 1993; McNevin et al., 2000; Mechsner et al., 2001; Ivry et al., 2004). In sensorimotor adaptation studies, the context can be defined by the layout of the target locations. For example, the targets might be limited to a single location, constrained to fall within a limited part of space (e.g., fixed radial distance from a start location), or broadly distributed across the workspace. Interestingly, previous studies of generalization have always confounded the arrangement of the target locations and the type of visual perturbation in that experimenters have employed a reaching environment consistent with the perturbation. In studies where the perturbation was a rotation, the targets were arranged in a circular manner. In contrast, in studies where the perturbation involved a translation, the targets were arranged in a rectilinear manner (Ghahramani et al., 1996). Thus, there has always been a confound between the form of the visual errors (rotation or translation) and the arrangement of the targets (circular or rectilinear). This confound makes it impossible to evaluate the relative contribution and interaction of these factors with respect to generalization, and in particular, to understand why the extent of generalization varies for different perturbations (e.g., narrow for rotation, broad for translation).

The present study was designed to untangle this confound. We first conducted an experiment in which the context and perturbation were consistent, similar to what has been implicit in previous studies of generalization. One group of participants learned to overcome a rotation with targets that were arranged on a circle while a second group of participants learned to overcome a translational shift with targets that were arranged on a set of lines. Our goal here was to replicate previous work, but in a single experiment in which all other factors were identical for the two

groups. In a second experiment, we swapped the context for the two types of perturbations, creating conditions in which these two factors were inconsistent with one another. One group of participants learned to overcome a rotation with targets that were arranged on a line while a second group of participants learned to overcome a translational shift with targets that were arranged on a circle. Comparing the results, both within and between these experiments, should allow us to assess the relative contribution of context and visual error signals to generalization.

## **MATERIALS AND METHODS**

## **PARTICIPANTS**

Forty participants (24 females/16 males, ages 18–24) were recruited from the Department of Psychology research participation pool at the University of California, Berkeley. Participants received class credit for participation. All participants were right handed, measured by the Edinburgh handedness inventory (Oldfield, 1971). Sixteen participants participated in experiment one, sixteen in experiment two, and eight in experiment three. The experimental protocol was approved by the institutional review board of the University of California, Berkeley.

## **EXPERIMENTAL APPARATUS**

Participants held onto a digitizing pen and made center-out reaching movements to visually displayed targets (7 mm diameter) by sliding the pen across a digitizing tablet (Intuous 3, Wacom, Vancouver, WA, USA). The targets and other task stimuli were displayed on a 15-in., 1280 × 1024 pixel resolution LCD monitor, mounted 25.4 cm above the tablet. The monitor was oriented horizontally to match the plane of the tablet. This configuration occluded vision of the hand and feedback of hand position was limited to a small cursor (3.5 mm diameter); when veridical, the feedback cursor was directly above the hand. The experimental task was implemented using custom software written in Python (open source) and run on a laptop computer.

## **EXPERIMENT 1**

The 16 participants were assigned to one of two experimental groups (**Figure 2**). For the CircleRotation group, a circular ring (7 cm radius) was always visible on the screen. The visual target could appear at one of eight locations on the ring, with polar angles of 0◦, 45◦, 90◦, 135◦, 180◦, −135◦, −90◦, and −45◦. For the LineTranslation group, two vertical lines were always visible, displaced 7 cm to the left and right of the starting position. A single target could appear at one of eight possible locations on the lines, four per line. The targets were equally spaced along the lines, and when defined in polar coordinates, were at 0◦, 35.2◦, 54.7◦, 125.3◦, 144.7◦,180◦, −144.7◦, and −35.2◦. Since the targets were not arranged in a circular manner, the distances to targets varied across the four target pairs (leftward or rightward amplitudes, from bottom to top of 8.57, 7.0, 8.57, and 12.12 cm). Note that the locations were chosen such that the second target location from the bottom on each side was co-linear with the starting position (**Figure 2**), and the neighboring targets were within 10◦ of corresponding target positions for the CircleRotation group.

On each trial, a single target was presented and the participant was instructed to make a fast reaching movement, "slicing" through the target. Although the movements terminated beyond the targets, endpoint feedback was limited in the main part of the experiment to the appearance of a red cursor that appeared along the contextual landmark (circular ring or vertical lines). The feedback cursor remained visible for 2 s. To motivate the participants to move quickly, a pleasant "ding" sound was played whenever the target amplitude was reached within 500 ms. If the movement time exceeded this criterion, an aversive "buzz" sound was presented. At the end of the feedback period, the feedback cursor was replaced by a ring with a diameter corresponding to the distance between the hand and starting position. By moving toward the starting circle, this ring became progressively smaller. When the hand was within 1 cm of the starting circle, the ring was transformed into the white feedback cursor, allowing the participant to position the hand within the starting circle. This form of feedback provided a way to guide the participant back to the starting position without providing information about the visuomotor perturbation (e.g., rotation or translation, see below). The participant was required to keep the cursor within the starting position for 1 s, at which time the next target appeared. Feedback of the cursor was removed when the position of the hand exceeded 1 cm from the starting position. While we emphasized movement speed, we did not put any constraint on reaction time. Movements were generally initiated within 500 ms.

The experimental session consisted of 266 reaches, divided into six blocks (**Figure 2**—bottom row). The first 24 trials (Base1) were designed to familiarize participants with the experimental task and the guidance method for returning to the starting position. Veridical online feedback was presented during the outbound portion of the movement until the hand passed through the ring or line, for the circular and rectilinear contexts, respectively. Each target was presented three times. For the next 40 trials (Base2), the online feedback was replaced by endpoint feedback. This was followed by a final baseline block of 32 trials (Base3) during which the endpoint feedback was only presented on 50% of the trials. A pseudorandom procedure was employed such that, for each target location, endpoint feedback was presented on two trials and withheld on two trials. Participants were informed that feedback would be withheld on half of the trials. On these trials, the auditory feedback concerning movement time also served as a cue that the movement had reached the required target amplitude. We included these trials because we wanted to familiarize the participants with the no-feedback procedure that would be critical for the assessment of generalization.

Participants then completed a 40-trial visuomotor perturbation training block (Training). In this block, all reaches were to the 0◦ location (target located directly to the right of the starting position) and the visual feedback was perturbed. For the CircleRotation group, the perturbation consisted of a 30◦ counterclockwise (CCW) rotation of the feedback cursor relative to true hand position. For the LineTranslation group, the perturbation consisted of an upward 3.5 cm vertical shift relative to true hand position. The magnitude of this vertical shift was chosen to equal the angular distortion induced by a 30◦ rotation for a 7 cm movement to the 0◦ target location. Endpoint feedback was provided on all trials. We chose to use endpoint feedback because a translational perturbation involving a constant shift relative to

hand position would introduce a discontinuity with online feedback (e.g., the cursor would jump the distance of the perturbation at movement onset). Participants were not informed of the perturbation, nor that the target would always appear at the same location.

**(C)** and the visual perturbation for the training target was a vertical shift of

Generalization was tested in the last two blocks. In the Test block, reaches to the training location (0◦) were interleaved with reaches to three of the target locations (those corresponding to 135◦, 180◦, and −135◦ in the CircleRotation group and the targets approximating these positions in the LineTranslation group,

feedback was only provided on trials to the training target location.

see **Figure 2**). These three probe locations were chosen because they would be the most informative for determining the extent and form of generalization. Endpoint feedback was only provided for reaches to the training location; all reaches to the three probe locations were performed without any visual feedback. The Test block consisted of 90 movements, 45 to the training location and 15 to each of the three probe locations. Thus, feedback was provided on 50% of the trials. The trial sequence was pseudorandomly distributed such that, for every four movements, two were to the training location and two were to probe locations.

The second generalization block (No Feedback) consisted of 40 trials, five to each of the eight target locations. No visual feedback was presented on any of these trials, including those in which the target appeared at the training location. This block provided a full assessment of the generalization function.

Participants completed the series of six blocks in approximately 45 min.

## **EXPERIMENT 2**

To unconfound context and perturbation, we repeated the procedure of Experiment 1, but now employed an inconsistent mapping (**Figure 2**) by assigning the two perturbations to the opposite context. Thus, a rotation was employed in the rectilinear context, while a translation was employed in the circular context. Sixteen naive participants were assigned to one of two experimental groups. For the CircleTranslation group, targets appeared on a circular ring, but the perturbation, when present, was a 3.5-cm upward, vertical translation. In this condition, the feedback cursor at the onset of the training block was usually displaced outside of the circular ring. In contrast, in the LineRotation group, targets appeared on the vertical lines, but the visual perturbation was a 30◦ CCW rotation. Here, the feedback cursor at the onset of the training block was usually displaced inside of the vertical line. The organization of the 266 trials was identical to that of Experiment 1.

## **EXPERIMENT 3**

Various lines of evidence indicate that sensorimotor learning entails the operation of multiple learning processes. These processes can vary in terms of the weight they give to the error signal, how they decay over time, and their accessibility to awareness. We focus on the awareness issue in a third experiment, given that the broad generalization observed with a visuomotor translation might be taken to reflect the operation of a process not specific to adaptation *per se*, but one that might result from the generic application of a strategy. We imposed the visuomotor perturbation in a gradual manner since this method has been shown to constrain learning to processes associated with adaptation of a visuomotor mapping (Kagerer et al., 1997; Saijo and Gomi, 2010; Taylor et al., 2011). We limited testing to the translation condition to ask if the broad generalization observed with this kind of perturbation was eliminated when strategic processes were excluded.

Eight naive participants were trained with the rectilinear context. The translational shift was introduced in small increments, increased linearly from 0 cm to 3.5 cm over the course a 160 trial training block (a shift of 0.023 cm or 0.188◦ per trial for the first 152 trials, then held constant over the last 8 trials). The structure of the baseline blocks and generalization blocks was the same as in Experiments 1 and 2.

## **DATA ANALYSIS**

Kinematic and statistical analyses were performed with Matlab (MathWorks, Natick, MA). To assess adaptation and generalization, we focused on the angular difference between the target location and the hand position when the hand intersected the circular ring or the vertical line. Each movement trajectory, regardless of the actual target location, was rotated to a common axis such that the target location was at 0◦. A straight line was connected between the starting position and the actual hand position, and we computed the angle between this line and the 0◦ reference line. With this convention, positive angles indicate a positive deviation (CCW) along the *y*-axis and negative angles indicate a negative deviation (CW) along this axis.

To assess performance prior to the introduction of the perturbation, we performed two separate analyses on reaches made during the Base3 block. First, the endpoint hand angles, averaged over all target locations, were calculated for each participant. For Experiments 1 and 2, these values were submitted to a two-sample *t*-test to determine if there were significant differences between groups. In addition, the movement time and reaction time data were analyzed to see if these variables were influenced by the two contexts.

Second, we performed a more restricted analysis on the reaches in Base3 to the three probe locations since these will be of greatest interest in our assay of generalization. The average endpoint hand angle was calculated separately at each probe location. These values were submitted to a mixed-model repeated measures ANOVA with the within-participant factor, Probe Location, and the between-participant factor, Context. As shown below, this analysis revealed that there were systematic differences in endpoint hand angle between the three probe locations independent of the training environment, an effect that is most likely due to biomechanical biases. To compensate for these biases, we subtracted out the Base3 endpoint hand angles from the comparable values in the generalization blocks (see below).

To quantify learning of the visual perturbations, the endpoint hand angles of the last five trials during the Training block were averaged. We performed a two-step analysis with these data. First, we compared these values to 30◦ to determine if participants were fully adapted to the perturbation. Second, we conducted a twosample *t*-test to determine if there were significant differences between the groups. In addition, we fit each participants' time series of hand angles in the training block with an exponential function using the Levenberg-Marquardt method for nonlinear least squares. To determine if there were differences in learning between the groups in Experiments 1 and 2, these values were also submitted to a two-sample *t*-test. The alpha value was set to 0.05 when only one test was performed and set to 0.025 when we performed a two-step analysis.

To assess generalization, we focused on the three probe locations in the Test block. The endpoint hand angle at each probe was calculated for each participant and the Base3 endpoint hand angles on trials without feedback were subtracted from these values. We then performed a two-step analysis. The first analysis was to determine if there was significant generalization at the probe locations within each group (differences greater than zero). The second analysis compared the endpoint hand angle data for the probe locations between the two groups. We also analyzed the Training and Test block data in a between-experiment supplemental analysis to directly compare performance in Experiments 1 and 2, using a two-way ANOVA with the factors Context and Perturbation.

The data from the No Feedback block provides a picture of the full generalization function. While we present these data qualitatively, our statistical analysis was restricted to two subregions. One subregion was composed of target locations near the training location. This included the target locations at 45◦ and −45◦ with the circular context and the target locations at 35.2◦ and −35.2◦ with a rectilinear context. The second subregion was composed of the three probe locations (circular context: 135◦, 180◦, and −135◦; rectilinear context: 144.7◦, 180◦, and −144.7◦). For each subregion, we performed the two-step analysis described above, again subtracting out the endpoint hand angles from the Base3 block. We did not statistically evaluate performance for the other targets (circular context: 90◦ and −90◦; rectilinear context: 54.7◦ and 125.3◦) because these locations did not have corresponding target locations within the other context.

## **RESULTS**

#### **EXPERIMENT 1**

Prior to the introduction of the visual perturbation, participants generally reached straight toward the target locations, terminating their movements just past the targets. The two exceptions were the 54.7◦ and 125.3◦ target locations in the rectilinear context, where the participants tended to not pass entirely through the targets because this was near the extent of a comfortable reach distance for these locations. Endpoint hand angle during the Base3 block did not differ between the groups [*t(*14*)* = 1*.*65, *p* = 0*.*12]. The added movement distance required to reach the targets in the rectilinear context did not lead to a significant increase in movement time [*t(*14*)* = 1*.*11, *p* = 0*.*28]. In fact, the trend was in the opposite direction, with average movement times of 280 ± 79 ms and 228 ± 28 ms for the circular and rectilinear contexts, respectively. The mean reaction time was 368 ± 31 ms in the circular context and 452 ± 59 ms in the rectilinear context, values that were significantly different [*t(*14*)* = 2*.*47, *p* = 0*.*03].

As described in the Methods, we performed a restricted analysis on the three probe locations for the Base3 data. A mixedmodel, repeated measures ANOVA revealed a significant effect of Probe Location [*F(*2*,* <sup>28</sup>*)* = 2*.*47, *p* = 0*.*03]. Reach trajectories toward the −135◦ and −144.7◦ target locations terminated slightly CCW relative to the target, while reaches toward the 135◦, 144.7◦, and 180◦ locations terminated slightly CW relative to the target. The effect of Context was not significant [*F(*1*,* <sup>7</sup>*)* = 0*.*02, *p* = 0*.*89], nor was the interaction of these two factors [*F(*1*,* <sup>7</sup>*)* = 1*.*33, *p* = 0*.*27]. Given that the probe location differences were independent of context, we assume that the effect reflects biomechanical biases associated with different limb configurations required for the different target locations. To minimize the effect of this bias in the subsequent analyses, we subtracted the Base3 endpoint hand angles to these three locations from the endpoint hand angles in the generalization blocks (see below).

During the Training block, participants in both groups altered their movement trajectories to compensate for the perturbation (**Figure 3**). The average endpoint hand angle for the last five movements in the Training block was −25*.*9 ± 4*.*4◦ for the CircleRotation group and −22*.*9 ± 3*.*7◦ for the LineTranslation group. These values indicate that adaptation was not complete [CR: *t(*7*)* = 2*.*66, *p* = 0*.*03; LT: *t(*7*)* = 5*.*50, *p <* 0*.*001], with the average position of the feedback cursor falling below the target in both conditions. Nonetheless, this level of learning was similar between the groups [*t(*14*)* = 1*.*47, *p* = 0*.*16]. To assess the overall learning functions, the time series of endpoint hand angles was fit with an exponential function. No differences were found between the CircleRotation and LineTranslation groups in the rate of learning [*t(*14*)* = 1*.*10, *p* = 0*.*29], the final asymptotic level of learning [*t(*14*)* = 1*.*99, *p* = 0*.*07], or the magnitude of learning [*t(*14*)* = 1*.*02, *p* = 0*.*33].

During the Test block, reaches to the training target location were interspersed with reaches to the three probe locations. Visual feedback was presented on reaches to the training target and was withheld on reaches to the probe targets. For the training target location, participants continued to compensate for the visual perturbation. For the CircleRotation group, the average reach angle

**FIGURE 3 | Group averaged endpoint hand angle across trials in Experiment 1.** The visuomotor mapping was veridical for the first 96 trials (Base1, Base2, Base3). Dashed vertical lines mark when the visual perturbation was present during the Training block (movements 97–136) and during the Test block (movements 137–226). Filled circles represent movements to the training target location and open circles represent movements to other target locations (blue: CircleRotation group; red: LineTranslation group). Endpoint position for the LineTranslation group was converted from Cartesian to polar coordinates since the visual perturbation was identical in polar space for the two groups.

was −19*.*0 ± 3*.*5◦ over the first five movements of the Test block, which was significantly less than at the end of the Training block [*t(*7*)* = 3*.*01, *p* = 0*.*02]. Participants in the LineTranslation group also showed a significant reduction in adaptation with an average reach angle of −19*.*5 ± 3*.*4◦ over the first five movements of the Test block compared to the end of the Training block [*t(*7*)* = 3*.*28, *p* = 0*.*01]. We attribute the reduced adaptation to the fact that there was a short set break (approximately 30 s) between the end of the Training block and start of the Test block. Importantly, however, there was no significant difference between the two groups over these first five movements toward the training target [*t(*14*)* = 1*.*15, *p* = 0*.*26] nor over the last five movements [*t(*14*)* = 1*.*36, *p* = 0*.*19].

Of greatest interest in terms of generalization is the participants' performance when reaching to the three probe locations. These data are presented in **Figure 4** as colored lines (blue = CircleRotation; red = LineTranslation), alongside the trajectories to the same targets in the Base3 block (black). We subtracted the Base3 endpoint hand angle data from each value and performed a two-step analysis, first asking if the trajectories within each group deviated from a straight path to the targets (after correcting for the biases observed in Base3), and then comparing the two groups. For the CircleRotation group, generalization at the probe target locations was not significant [*t(*7*)* = 0*.*65, *p* = 0*.*53]. In contrast, generalization was significant for the LineTranslation group [*t(*7*)* = 4*.*89, *p* = 0*.*002]. When the data for the two groups were directly compared, generalization for the LineTranslation group was significantly greater than for the CircleRotation group [*t(*14*)* = 3*.*38, *p* = 0*.*005].

The No Feedback block provided a picture of the full generalization function (**Figure 5**). The statistical analysis, however, was restricted to two subregions, one selected to be far from the training target location (the three probe locations), and one selected to be near the training target location (the two adjacent locations). Again, the Base endpoint hand angles were subtracted out to remove systematic biases at each target location. Generalization was significant at locations near the training location for both the CircleRotation group [*t(*7*)* = 3*.*13, *p* = 0*.*02] and the LineTranslation group [*t(*7*)* = 4*.*90, *p* = 0*.*001], and the degree of generalization was similar between the two groups at these near locations [*t(*14*)* = 0*.*50, *p* = 0*.*62]. For the far locations, the pattern of generalization was similar to what had been observed in the Test block. Generalization was significant for the LineTranslation group [*t(*7*)* = 3*.*05, *p* = 0*.*02], but not for the CircleRotation group [*t(*7*)* = 0*.*94, *p* = 0*.*38]. When the two groups were directly compared, the difference was only marginally significant [*t(*7*)* = 1*.*91, *p* = 0*.*08]. Note that, while we did not observe generalization at the probe locations for the CircleRotation group, the small shifts were actually in the opposite direction from what would be expected if participants had learned a rotation.

As can be seen in a comparison of **Figures 4** and **5**, the magnitude of generalization at the probe locations is weaker in the No Feedback block compared to the Test block. This result is expected given that adaptation decays over time in the absence of visual feedback (Hatada et al., 2006; Criscimagna-Hemminger and Shadmehr, 2008; Huang and Shadmehr, 2009).

## **EXPERIMENT 2**

The results of Experiment 1 suggest that there is broader generalization for a Cartesian translation compared to a polar rotation when each perturbation is presented in a context consistent with its respective perturbation. However, it is unclear if the broader generalization in the former condition is due to the type of perturbation (translation vs. rotation), the context (rectilinear vs. circular), or a combination of these factors. In Experiment 2, we examined these hypotheses by swapping the contexts for the two types of perturbations. One group (CircleTranslation) was presented with a translational perturbation when reaching to targets arranged in a circular context while a second group (LineRotation) was presented with a rotation when reaching to targets arranged in a rectilinear context (**Figure 2**).

**LineTranslation group (red) compared with the averaged trajectories**

Black circles represent the values for each participant.

During the Base3 block, there were no significant differences between the groups: participants showed similar target errors [*t(*14*)* = 1*.*48, *p* = 0*.*16], reaction times [*t(*14*)* = 0*.*12, *p* = 0*.*27], and movement times [*t(*14*)* = 0*.*75, *p* = 0*.*46], indicating that the contexts did not affect reaching behavior in the absence of a visuomotor perturbation. When the heading analysis was restricted to the three probe locations, we again observed a significant effect of Probe Location [*F(*2*,* <sup>28</sup>*)* = 9*.*48, *p* = 0*.*001], but no effect of Group [*F(*1*,* <sup>7</sup>*)* = 5*.*65, *p* = 0*.*76] or interaction of these factors [*F(*2*,* <sup>14</sup>*)* = 0*.*43, *p* = 0*.*52]. Participants exhibited a similar bias pattern to that observed in Experiment 1.

Participants showed rapid learning of both visual perturbations (**Figure 6**). The average endpoint hand angles over the last five trials during the Training block were −24*.*7 ± 2*.*47◦ for the CircleTranslation group and −28*.*9 ± 3*.*00◦ for the LineRotation group, values that were significantly different [*t(*14*)* = 3*.*02, *p* = 0*.*01]. Compared to the value corresponding to full adaptation (30◦), the CircleTranslation group showed incomplete learning [*t(*7*)* = 6*.*05, *p <* 0*.*001]. This comparison was not reliable for the LineRotation group [*t(*7*)* = 1*.*07, *p* = 0*.*32], a null result consistent with complete learning. An exponential fit of the time series of endpoint hand angles also revealed a difference in the asymptotic level of learning between the groups [*t(*14*)* = 2*.*30, *p* = 0*.*04], consistent with greater learning in the LineRotation group. However, the groups did not differ in terms of learning rate [*t(*14*)* = 1*.*27, *p* = 0*.*23] or magnitude of learning [*t(*14*)* = 1*.*00, *p* = 0*.*34].

Despite the subtle performance differences during the training block, we observed dramatic differences in generalization at the probe locations in the Test block (**Figure 7**). After correcting for the Base3 biases, significant generalization was observed for both the CircleTranslation group [*t(*7*)* = 4*.*96, *p* = 0*.*002]

**Experiment 2.** Block structure was the same as in **Figure 2**. Filled circles represent movements to the training target location and open circles represent movements to other target locations (cyan: CircleTranslation group; purple: LineRotation group).

and the LineRotation group [*t(*14*)* = 6*.*88, *p <* 0*.*001]. However, the magnitude of generalization was considerably larger in the LineRotation group [*t(*14*)* = 4*.*82, *p <* 0*.*001]. Note that this increase in generalization was observed despite the fact that this group had shown less adaptation during the training phase.

Differences in the amount of generalization between the two groups was also evident in the No Feedback block, with larger changes in hand angle for the LineRotation group at both near and far locations (**Figure 8**). For the CircleTranslation group, small but significant generalization was observed for both the near target locations [*t(*7*)* = 2*.*71, *p* = 0*.*03] and probe target locations [*t(*7*)* = 3*.*88, *p* = 0*.*006]. Generalization was also reliable at both subregions for the LineRotation group [near: *t(*7*)* = 9*.*01, *p <* 0*.*001; far: *t(*14*)* = 7*.*78, *p <* 0*.*001]. When the two groups were compared, the LineRotation group exhibited larger generalization for the near [*t(*14*)* = 5*.*65, *p <* 0*.*001] and far [*t(*14*)* = 5*.*85, *p <* 0*.*001] subregions.

It is important to note that the direction of generalization was similar for both groups, with the trajectories shifted in the downward direction. This would be expected if participants were learning an internal model of a translation shift since compensation for the upward shift requires a downward displacement of the trajectory. However, it is opposite of what would be expected if participants were learning an internal model of a rotation. Generalization of a rotation to the probe locations would require an upward displacement of the trajectories to these targets.

Black circles represent the values for each participant.

To directly compare the data from Test blocks in the two experiments, we employed a two-way ANOVA with the factors Context (circular vs. rectilinear) and Perturbation (rotation vs. translation). There were no significant effects in the degree of adaptation at the training target location during the training block, although both main effects and the interaction approach significance [Context: *F(*1*,* <sup>14</sup>*)* = 2*.*69, *p* = 0*.*11; Perturbation: *F(*1*,* <sup>14</sup>*)* = 1*.*17, *p* = 0*.*29; interaction: *F(*1*,* <sup>7</sup>*)* = 1*.*44, *p* = 0*.*24]. In terms of generalization, only the effect of Context was significant [*F(*1*,* <sup>14</sup>*)* = 33*.*6, *p <* 0*.*001]. The type of perturbation was not significant [*F(*1*,* <sup>14</sup>*)* = 0*.*18, *p* = 068], nor was the interaction of these factors [*F(*1*,* <sup>14</sup>*)* = 2*.*34, *p* = 0*.*14]. When averaged across the three probe locations and between experiments, the mean shifts in hand angle were 15.4 ± 7.5◦ and 2.66 ± 4.7◦ for the rectilinear and circular contexts, respectively. Thus, when the two experiments are considered together, the results clearly demonstrate that generalization, at least to targets far from the training location, is constrained more by the context rather than the error information.

#### **EXPERIMENT 3**

In Experiments 1 and 2, the visual perturbations were introduced abruptly and participants were likely cognizant at the beginning of the training block that their performance was no longer accurate. It is possible that learning with rectilinear context induced the adoption of a generic strategy rather than the adaptation of an internal model (Taylor and Ivry, 2012). For example, the participants may have noticed that the reaches were terminating above the target location and decided to aim to a location below the target. Generalization would appear broad if this strategy was applied to all of the targets. By this hypothesis, the results of Experiments 1 and 2 would indicate that the rectilinear arrangement leads participants to adopt a strategy, whereas the circular arrangement does not. Alternatively, it may be that the rectilinear arrangement produces greater sensorimotor adaptation than the circular arrangement.

To assess these two hypotheses, we employed a procedure that has been used to prevent strategy use in previous studies of visuomotor adaptation. Instead of introducing the perturbation abruptly, a small, incremental perturbation was introduced over the course of an extended, 160-trial Training block. Under such conditions, participants exhibit minimal, if any, awareness of the perturbation (Malfait and Ostry, 2004; Saijo and Gomi, 2010; Taylor et al., 2011). Given that our focus here is to understand why the rectilinear context produces broad generalization, we only tested one group of participants (*n* = 8), using the LineTranslation condition in the rectilinear context. The translational shift was introduced in small increments, increased linearly from 0 cm to 3.5 cm over the course a 160 trial training block (a shift of 0.023 cm or 0.188◦ per trial for the first 152 trials, then held constant over the last 8 trials). The remaining structure of the baseline blocks and generalization blocks was the same as in Experiments 1 and 2, resulting in a total of 386 movements.

Participants learned to offset the gradual perturbation during the Training block (**Figure 9**). Over the last five trials, the average endpoint hand angle was −25*.*1 ± 1*.*96◦, which fell short of complete learning of the 30◦ perturbation [*t(*14*)* = 7*.*08,

**Experiment 3.** A vertical shift (black solid line) was introduced in an incremental manner during the Training Block, reaching a final value of 4 cm (movements 97–256). Feedback was only provided for reaches to the training target location during the Test block (movements 256–226). Visual feedback was never provided in the final, No Feedback block. Filled circles represent movements to the training target location and open circles represent movements to other target locations.

*p <* 0*.*001]. Since the perturbation was introduced linearly during the Training block, we used a linear function to fit the time course data. The observed slope of 0.17 ± 0.02◦ per trial was slightly less than the 0.188◦ slope of the perturbation function [*t(*7*)* = 52, *p <* 0*.*001].

While participants continued to compensate for the translational shift when reaching to the training target location during the Test block, there was a initial decrease in hand angle (**Figure 9**), likely due to decay during the transition between the Training and Test blocks. Over the first five movements to the training target, the average hand angle was −19*.*5 ± 1*.*79◦, which was less than that observed in the last five trials of the Training block [*t(*7*)* = 5*.*92, *p <* 0*.*001]. Generalization at the probe locations was observed in the Test block [*t(*7*)* = 2*.*96, *p* = 0*.*02; **Figures 10A,B**] and was also evident across the workspace in the No Feedback block [near targets: *t(*14*)* = 5*.*49, *p <* 0*.*001; far, probe targets: *t(*14*)* = 3*.*18, *p* = 0*.*02; **Figure 10C**]. Indeed, the magnitude of generalization at the probe locations was similar to that observed for the LineTranslation groups in Experiment 1 in a between-experiment comparison [*t(*14*)* = 1*.*35, *p* = 0*.*20].

It is possible that the increased error at the beginning of the Test block may have led to some awareness of the perturbation. However, there are a few reasons why we do not think that this led to the observed pattern of generalization. First, generalization at the probe locations was apparent at the start of the Test block, measured over the average of the first five reaches [*t(*7*)* = 5*.*92, *p <* 0*.*001]. Second, while post-experiment questionnaires

revealed that most participants had a general sense that there was some sort of experimental manipulation of the feedback, they were not able to able to articulate the manipulation and reported reaching straight toward the probe target location. Third, if the generalization was due to the generic application of a strategy, we would expect it to be of similar magnitude as that observed at the training location; however, the results show that is was considerably attenuated. In sum, the results of Experiment 3 indicate that broad generalization for the rectilinear context was not dependent on conditions that might induce the use of a strategy.

## **DISCUSSION**

## **SUMMARY**

The set of experiments presented here highlight an important constraint on generalization following sensorimotor adaptation. Experiment 1 replicated previous work, showing that generalization was much broader when a visuomotor perturbation involved a translational shift compared to a rotation. Previous accounts of this difference have attributed it to how the type of perturbation, and its resultant error, is used to update an internal model (Thoroughman and Taylor, 2005; Hinder et al., 2008; Shabbott and Sainburg, 2010; Taylor et al., 2013). However, the results of Experiment 2 provide compelling evidence that the training context is the primary factor underlying this difference. Broad generalization was observed with a rotation when the targets were in a rectilinear arrangement, and became much smaller for a translation when the targets were arranged in a circular arrangement. The broad generalization pattern for the line context also held when participants were largely unaware of the visual perturbation (Experiment 3). Taken together, these results show that the pattern and breadth of generalization is strongly constrained by the training environment.

## **THE ROLE OF ERROR SIGNALS IN MODELS OF GENERALIZATION**

In examining generalization, researchers have focused on the breadth of the generalization function and the reference frame in which generalization is expressed. Across a number of studies, a picture has emerged in which the pattern of generalization varies for different visual perturbations. Generalization of rotations has been shown to be quite narrow, with the modest generalization at distant locations showing trajectory deviations that are consistent with the rotation. For example, with a clockwise rotation, the small amount of generalization for probe locations 180◦ from the training location are also in the clockwise direction (Pine et al., 1996; Krakauer et al., 2000). In contrast, generalization following gain changes (Krakauer et al., 2000; Pearson et al., 2010) and translational perturbations is quite broad, with the latter evident following prism adaptation (Bedford, 1993) or cursor shifts in one Cartesian dimension (Ghahramani et al., 1996). Thus, for translational perturbations, the trajectory deviations are consistent with what would be expected if participants had learned to compensate for a translation (Ghahramani et al., 1996). However, recent studies have suggested that generalization may entail multiple components, and that these may be expressed in multiple reference frames (Brayanov et al., 2012; Taylor et al., 2013).

Differences in generalization have been attributed to the error signal caused by the perturbation (Krakauer et al., 2000; Taylor et al., 2013). In a previous study involving a rotation, we found that systematically increasing the quality of visual error information led to different patterns of generalization (Taylor et al., 2013). When online feedback was provided throughout the entire movement, generalization was manifest as trajectory deviations that would suggest the participants had learned a rotational perturbation. In contrast, when feedback was limited to knowledge of results (endpoint feedback), the trajectory deviations were in the opposite direction, consistent with what would be expected if the participants had inferred a translational perturbation. Intermediate levels of feedback led to reference frame effects that fell between that observed with full online and endpoint only feedback.

Computational models of adaptation have employed radial basis function networks to explain how error signals are used to update a sensorimotor mapping and to explore the constraints on generalization (Thoroughman and Shadmehr, 2000; Thoroughman and Taylor, 2005; Tanaka et al., 2009; Pearson et al., 2010). The basis function network provides a representation of movement direction, through the weighted sum of individual units that are tuned to a particular movement direction. As such, the network activity results in a population vector, with each unit voting for a preferred direction of movement (Georgopoulos et al., 1986). Gradient descent is used to update the weight of each unit, with the change a function of the degree of an unit's activity level and the size of the visual error signal. Narrow generalization arises in this model because the tuning function serves as a weight on the error signal; that is, the effect of adaptation is greatest for units that are active at the time that the error is experienced. A critical feature of our model for generalization is that the different feedback conditions afforded different opportunities for error-driven learning. Endpoint feedback, with its discrete feedback, provided only a single opportunity for updating the internal model. In contrast, we proposed that online feedback provided additional opportunities for updating. Thus, we modeled conditions with online feedback corrections or movements that returned to the target with a second update during each trial. Here the rotational errors were experienced when units were active with a directional tuning quite different from those active when initially reaching to the training target (e.g., if during a return movement, the active units would be in the opposite direction). In this manner, adaptation could occur across a broad set of the basis functions, providing a mechanistic account of generalization.

A key insight from this work is that differences in the pattern of generalization between the feedback conditions were not necessarily inherent to differences in tuning functions, but rather an incidental by-product of the state of the network at the time of error updates. For example, a counterclockwise rotation during the outbound portion of a movement (see **Figure 1**) would adjust units tuned toward 0◦ in the clockwise direction. The same rotation during the return movement, would also produce a shift in the clockwise direction, but here the effect is on units tuned toward 180◦. Thus, when generalization is tested for movements around 180◦, the trajectories would exhibit a clockwise shift, suggesting that the participants had learned a rotation. However, the model suggests that this is not generalization *per se*, but rather the incidental effect of local adaptation for movements in this direction. That is, generalization with online feedback is the composite effect of multiple local adaptation effects.

An alternative perspective on the difference between translational and rotational generalization focuses on the reference frame within which learning occurs. A translation can be viewed as a perturbation defined in an extrinsic reference frame; for example, the displacement of a soccer kick from any point on the field will be affected in a similar manner by a strong wind. The reference frame for a rotation is more ambiguous. It could be in extrinsic space, defined by polar coordinates. Or it could be defined intrinsically as has been shown in force field adaptation where learning generalizes in joint space (Shadmehr and Mussa-Ivaldi, 1994).

It is important to note that in the current study, as well as several other studies of visuomotor adaptation, the reference frame of learning cannot actually be inferred from the pattern of generalization. Generalization always appeared to be translational, consistent with the idea that it operates in an extrinsic reference frame. However, it remains unclear if this pattern reflects the reference frame of learning, or if it reflects an inference about the nature of the perturbation. With endpoint feedback, the motor system may be unable to infer the precise nature of the perturbation and a translation may be the default inference, even with a rotational perturbation. With online feedback, a rotational perturbation would result in curved trajectories. This may bolster an inference that the perturbation is, in fact, rotational. Insight into the reference frame of learning can be gained by having participants reach to the same set of target locations, but with an altered limb configuration during generalization trials. Using this approach, Brayanov et al. (2012) found that generalization of a rotation entailed a mixture of multiple reference frames. It remains an open question if a rotation induces adaptation in multiple reference frames, or if performance reflects a mixture of multiple inferences about the nature of the perturbation.

## **THE INFLUENCE OF CONTEXT**

Independent of the reference frame debate, the present results pose a problem for current models of generalization. It does not seem likely that context would affect low-level representations of movement, such as the tuning function of the units in the basis function model. The current results indicate that a full model of generalization must go beyond consideration of tuning functions and error signals, incorporating the influence of context in how participants make inferences about the nature of the error signal. There are two related issues to keep in mind here. First, and most compelling, generalization was much more substantial at the probe locations with the rectilinear context compared to the circular context. Second, for a given context, there was little difference between the two types of perturbations: Generalization was broad and substantial for the rotational and translational perturbations with a rectilinear context, and minimal for both types of perturbations for the circular context.

There are various ways in which to consider this contextual effect. One idea is that both the error and context define the reference frame for learning. For example, the rectilinear context may promote a conceptualization that is extrinsic or world based, whereas the circular context may promote a conceptualization that is intrinsic or body based. If the error signal is always in extrinsic coordinates, then the context and error both converge on a common, extrinsic reference frame. In contrast, the context and error would be in opposition for a circular context. By this view, the minimal generalization seen at distant locations with the circular context is due to the canceling effects of the two factors, whereas the broad generalization at these locations with the rectilinear context is due to their complementary effects.

A second idea relates back to the idea that generalization may be captured by a mixture of experts model (Ghahramani and Wolpert, 1997; Krakauer et al., 2000; Pearson et al., 2010), one form of which is reflected in models in which generalization involves a combination of local and global components. In these models, the perturbation may be learned by modular decomposition by expert modules at a very local level (as with direction tuned units), and then combined with a weighting, or gating function to account for generalization that is manifest at the global level (Ghahramani and Wolpert, 1997; Pearson et al., 2010). In current versions of this model, adaptation of the local modules is based on the size of the error signal, regardless of context. Indeed, this weighting idea has previously been considered in terms of the how translational and rotation error signals might produce different patterns of generalization (Ghahramani et al., 1996; Ghahramani and Wolpert, 1997; Krakauer et al., 2000; Pearson et al., 2010). However, context may change how these local units are combined at the global level. By this hypothesis, the effect of context could be viewed, not in terms of how it influences the reference frame of generalization, but rather in terms of how it constrains the weighting function. One potential problem for a simple weighting function model is that learning at the training location and generalization to the near probes was similar for the two contexts. As such, it cannot be that the rectilinear context simply produced an overall increase in the weighting function. Rather, a two-process model would be required, one in which local adaptation is based on the error signal independent of context, and a second in which context constrains how that information is broadcast globally.

Why might the motor system give less weight to a circular context (and rotational perturbation)? One hypothesis is that the weighting function is modulated by perturbation uncertainty. A rotational perturbation is inherently nonlinear and more complex than a translational perturbation. Because of this complexity, the participant is more uncertain about the perturbation. Increased uncertainty may attenuate the weighting function, resulting in weaker generalization at distant locations. While the uncertainty idea could be considered with respect to the error signal, the current results make clear that the weighting hypothesis must be

### **REFERENCES**


and experiments in human motor control. *J. Neurosci.* 23, 9032–9045. Georgopoulos, A. P., Schwartz, A. B., and Kettner, R. E. (1986). Neuronal population coding of movement direction. *Science* 233, 1416–1419.


modified to consider context as a key constraint. Specifically, the arrangement of the targets may provide clues to the motor system as to the nature of the perturbation. A linear arrangement of the environment could bias the system to infer a linear solution to offset the perturbation. A circular arrangement of the environment could bias the system to infer a more complex, non-linear solution. As with the error-based models, this more complex (or ambiguous) environment results in an attenuated weighting function due to uncertainty.

We recognize that we are only offering speculative ideas about the mechanisms through which context influences generalization. We do believe the ideas outlined here provide a framework for considering constraints on motor learning, with the key insight that the context must be part of the equation. Future experiments could better manipulate how the combination of information in the error signal and the training environment guide learning and generalization. Ultimately, the motor system is faced with an inductive inference problem, especially when sensory information is limited, to make predictions about the underlying state of the world. The error signal and the context within which that information is presented are exploited to best resolve an ambiguous inference problem.

### **ACKNOWLEDGMENTS**

We thank Laura Hieber and Christina Gee for help with data collection. We thank Maurice Smith for helpful comments on the manuscript. Jordan Taylor was supported by an NRSA fellowship (F32NS064749) from the National Institute of Neurological Disorders and Stroke, and Richard Ivry was supported by 5R01HD060306 from the National Institute of Child Health and Human Development.


adaptation a cognitive response to the sudden introduction of load? *J. Neurosci.* 24, 8084–8089.


scaling factors and reference axes for reaching movements. *Neuroreport* 7, 2357–2361.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 18 January 2013; accepted: 16 April 2013; published online: 06 May 2013.*

*Citation: Taylor JA and Ivry RB (2013) Context-dependent generalization. Front. Hum. Neurosci. 7:171. doi: 10.3389/fnhum.2013.00171*

*Copyright © 2013 Taylor and Ivry. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

## Unlearning versus savings in visuomotor adaptation: comparing effects of washout, passage of time, and removal of errors on motor memory

#### *Tomoko Kitago1, Sophia L. Ryan1, Pietro Mazzoni 1, John W. Krakauer 1,2,3 and Adrian M. Haith2 \**

*<sup>1</sup> Motor Performance Laboratory, Department of Neurology, Columbia University College of Physicians and Surgeons, New York, NY, USA*

*<sup>2</sup> Department of Neurology, Johns Hopkins University, Baltimore, MD, USA*

*<sup>3</sup> Department of Neuroscience, Johns Hopkins University, Baltimore, MD, USA*

#### *Edited by:*

*Rachael D. Seidler, University of Michigan, USA*

#### *Reviewed by:*

*K. Thoroughman, Washington University, USA Dezso Nemeth, University of Szeged, Hungary*

#### *\*Correspondence:*

*Adrian M. Haith, Department of Neurology, Johns Hopkins University, 600 N. Wolfe Street, Baltimore, MD 21287, USA e-mail: adrian.haith@jhu.edu*

Humans are able to rapidly adapt their movements when a visuomotor or other systematic perturbation is imposed. However, the adaptation is forgotten or unlearned equally rapidly once the perturbation is removed. The ultimate cause of this unlearning remains poorly understood. Unlearning is often considered to be a passive process due to inability to retain an internal model. However, we have recently suggested that it may instead be a process of reversion to habit, without necessarily any forgetting *per se*. We compared the timecourse and nature of unlearning across a variety of protocols where unlearning is known to occur: error-clamp trials, removal of visual feedback, removal of the perturbation, or simply a period of inactivity. We found that, in agreement with mathematical models, there was no significant difference in the rate of decay between subject who experienced zero-error clamp trials, and subjects who made movements with no visual feedback. Time alone did lead to partial unlearning (over the duration we tested), but the amount of unlearning was inconsistent across subjects. Upon re-exposure to the same perturbation, subjects who unlearned through time or by reverting to veridical feedback exhibited savings. By contrast, no savings was observed in subjects who unlearned by having visual feedback removed or by being placed in a series of error-clamp trials. Thus although these various forms of unlearning can all revert subjects back to baseline behavior, they have markedly different effects on whether long-term memory for the adaptation is spared or is also unlearned. On the basis of these and previous findings, we suggest that unlearning is not due to passive forgetting of an internal model, but is instead an active process whereby adapted behavior gradually reverts to baseline habits.

#### **Keywords: adaptation, visuomotor rotation, unlearning, decay, savings**

## **INTRODUCTION**

Human subjects adapt rapidly to systematic perturbations to their movements through an error-driven, model-based learning mechanism (Huang et al., 2011; Haith and Krakauer, 2013). However, behavior rapidly reverts to baseline when the errors that drive adaptation are removed. Although behavior in adaptation paradigms has been studied in tremendous detail, this process whereby recent adaptation is apparently forgotten remains poorly understood. We will adopt the term *unlearning* for the reversion to baseline. We do so because it allows us to remain agnostic as to whether reversion to baseline reflects decay (forgetting), or competition between intact memories.

Unlearning of a perturbation can occur in at least four distinct ways. Switching off the perturbation leads to errors in the opposite direction to those which drove the initial adaptation, leading to rapid adaptation back to baseline. However, unlearning can also occur in more spontaneous fashion if movement errors are artificially eliminated through error-clamp paradigms that create the illusion of perfect performance (Scheidt et al., 2000; Criscimagna-Hemminger and Shadmehr, 2008; Huang et al.,

2011; Shmuelof et al., 2012). For purely visual perturbations, errors can be removed entirely by removing visual feedback, which also leads to a steady return toward baseline (Galea et al., 2011). Finally, unlearning can simply occur with the passage of time; sitting idle for a period of minutes to hours leads to a reduction in the extent of compensation for a perturbation (Criscimagna-Hemminger and Shadmehr, 2008). All of these manipulations lead to ostensibly the same outcome: that subjects make movements that are the same as those made at baseline. However, just because all four conditions lead to a reversion to the same baseline phenotype does not mean that they are in the same state in terms of retained motor memories (Smith et al., 2006).

Adaptation is commonly described mathematically with the state space model framework (Thoroughman and Shadmehr, 2000; Donchin et al., 2003; Cheng and Sabes, 2006; Zarahn et al., 2008). This framework essentially assumes that subjects adapt their behavior in proportion to the size of performance errors. The same set of equations describing learning can be derived based on assumptions of gradient descent on the squared movement error, or based on Bayesian estimation of the imposed perturbation. Unlearning can be conveniently accommodated in such models through a trial-to-trial forgetting rate. This forgetting rate also has the benefit of being able to capture the fact that adaptation is never able to quite reach an asymptote of zero error; learning from residual error in each trial is eventually balanced by unlearning between trials. Although all four varieties of unlearning described above can be modeled within the state space model framework, simply describing the data mathematically overlooks the deeper question of why unlearning should occur at all.

Adaptation according to state space dynamics is generally thought to occur through updating of an internal model that predicts the outcomes of a motor command. However, we and others have recently shown that an additional success-based, model-free learning mechanism (Huang et al., 2011; Izawa and Shadmehr, 2011) also plays a role in adaptation. In particular, the phenomenon of savings, i.e., faster re-learning upon re-exposure to a previously encountered perturbation, depends on this modelfree learning mechanism (Huang et al., 2011). The fact that both model-based and model-free learning processes participate during adaptation raises the question as to which of these processes actually gives rise to the unlearning. In this study, we compared four different methods of eliciting unlearning: error clamps, removal of visual feedback, washout by removal of the perturbation, and the passage of time. We hypothesized that these four different manipulations would result in qualitatively different kinds of unlearning that would be revealed both by the timecourse of the unlearning itself and by the presence or absence of savings on subsequent re-exposure to the original perturbation.

## **MATERIALS AND METHODS EXPERIMENTAL METHODS**

Forty healthy, right-handed individuals (age 28*.*3 ± 7*.*4 years, 18 women) were recruited from the local community. All participants were naïve to the purpose of the study and signed a written consent form that was approved by the Columbia University institutional human research review board.

Subjects were seated at a glass surface table and moved a cursor by making planar reaching movements (**Figure 1A**). Hand position, calibrated to the position of the fingertip, was monitored using a Flock of Birds (Ascension Technology, Burlington, Vermont, USA) magnetic movement recording system at a frequency of 120 Hz. Real-time hand position was used to control the visual display and to provide on-line visual feedback. The hand itself was not visible to subjects. One condition of the experiment made use of error-clamp trials, in which the angular position of the cursor relative to the start location was clamped to a straight line between the start location and the target. Subjects still maintained direct control of the radial distance of the cursor from the start location during these error-clamp trials.

Subjects were instructed to make out-and-back movements from a center start circle to a single target (radius 1 cm, at the 135◦ position, 8 cm from the start circle), reversing within the target. The experimental paradigm consisted of 4 epochs (**Figure 1B**). The first epoch (*Baseline*) consisted of 40 trials with unperturbed feedback. The second epoch (*Initial learning*) consisted of 80 trials in which visual feedback was rotated 30◦ counterclockwise (CCW). In the third epoch (*Unlearning*), subjects were placed in one of four unlearning conditions: (1) 200 error-clamp trials (*Clamp*), (2) 200 trials with no visual feedback (*No Feedback*), (3) 200 trials with veridical visual feedback (*Washout*), or (4) sitting idle for 740 s (∼12 min) (*Time*), which was the average amount of time taken by subjects in the other groups to complete 200 trials. In the final epoch (*Re-learning*), subjects were re-exposed to the perturbation for a further 80 trials to test whether any memory of the prior adaptation would be present in the form of savings.

## **DATA ANALYSIS**

Trajectory data were smoothed using a 2nd-order Savitzky–Golay filter. Movement initiation was determined based on the first time that movement speed exceeded 2.4 cms<sup>−</sup>1. Initial reach direction was determined based on the angle between lines connecting the hand position at movement initiation with position of the hand at

(30◦ counterclockwise rotation), *Unlearning*, and *Re-learning* (30◦

(Washout), or inactivity for 740 s (Time).

peak velocity and the center of the target. We subtracted from this reach angle a baseline reach direction for each subject, estimated from the last 20 trials of the *Baseline* epoch.

We determined the rate of unlearning for each subject through the slope of a linear regression between the initial reach directions on consecutive trials over the course of the *Unlearning* epoch. We quantified the overall extent of decay in the *Unlearning* epoch by taking the ratio between the reach direction immediately preceding (last 20 trials of the first adaptation block) and following (first trial of the second adaptation block) the unlearning block.

To assess savings, we assumed that subject behavior followed a linear state-space model given by:

$$\varkappa\_{i+1} = A\varkappa\_i + Be\_i + \eta\_i \tag{1}$$

$$\mathbf{y}\_{i} = \mathbf{C}\mathbf{x}\_{i} + \mathbf{e}\_{i} \tag{2}$$

In this model, *xi* corresponds to the state of the subject's internal model of the perturbation on trial *i*, *yi* reflects the hand position on trial *i*, and *ei* represents the directional error on trial *i*. *A* ≤ 1 is the trial-to-trial retention rate, *B* is the adaptation rate, *C* = 1, and η*<sup>i</sup>* and ε*<sup>i</sup>* are independent noise terms, with η*<sup>i</sup>* : *N(*0*, Q)* and ε*<sup>i</sup>* : *N(*0*, R)*, and *x*<sup>0</sup> : *N(*μ*,V*0*)*. We estimated the remaining parameters (*A, B,*μ*,V*0*, Q, R*) separately for each individual subject using maximum likelihood estimation (Ghahramani and Hinton, 1996; Cheng and Sabes, 2006). Trials that were excluded were treated as unobserved variables by setting *C* = 0 on these trials. In order to minimize the risk of overfitting the model by allowing too many free parameters, all parameters were assumed to be constant throughout the experiment except for the learning rate *B*, which we allowed. to take different values in each epoch. We considered savings to have occurred if the estimated value of *B* during re-learning was greater than the corresponding value during initial learning. A power analysis based on data from Zarahn et al. (2008) suggested that 9 subjects would be an appropriate minimum sample size using a power of 0.9 with two-tailed alpha of 0.05.

Note that we could, in principle, have allowed the forgetting rate *A* to also have varied across trials, since a change in *A* would also have influenced the learning rate. In practice, changing the forgetting rate A tends to have a far larger effect on the asymptote of learning than on the initial rate. Varying B has a strong effect on initial adaptation rate and a weaker effect on the asymptote. Although this means that these parameters can in principle be dissociated in the kind of data we consider here, in practice jointly estimating these two parameters from small datasets yields correlated estimates that are highly prone to overfitting (Cheng and Sabes, 2006). We therefore considered it best to compare estimated learning rates across epochs assuming all other things to be equal and therefore allowed only the learning rate *B* to vary across epochs.

## **RESULTS**

Four groups of 10 subjects each participated in an experiment to test the effect of different types of feedback on prior visuomotor adaptation: *Clamp*, *No Feedback* (*NoFB*), *Washout* (*WO*), and *Time*. All groups exhibited a comparable amount of adaptation during the *Initial Learning* epoch. Across all subjects, the asymptotic error (last 20 trials of initial learning) was 7*.*4 ± 3*.*6◦, and did not differ significantly across groups (*p* = 0*.*94).

#### **INFLUENCE OF FEEDBACK TYPE ON UNLEARNING**

First, we compared the trial-by-trial rate of unlearning in the *Clamp*, *NoFB*, and *Washout* groups. **Figure 2A** illustrates the average behavior for each group during the *Unlearning* epoch. Standard models of adaptation suggest that a constant proportion of prior adaptation is forgotten on each trial, leading to an exponential timecourse of decay. Assuming that this is the case, we estimated the time-constant of this unlearning by performing a linear regression between reach directions on consecutive trials. We quantified the decay rate as 1 minus the slope of the regression. For the *Clamp* and *NoFB* groups, this is equivalent to estimating *A* in Equation 1, for the *Washout* group this is equivalent to estimating *A* + *B* (although note that for the purpose of this analysis is was not necessary to fit a full state space model to the data).The estimated unlearning rates are shown in **Figure 2B**. The decay rate varied significantly across the three groups [*F(*2*,* <sup>27</sup>*)* = 37*.*8, *p <* 10<sup>−</sup>7]. As expected, there was a significant difference in the rate of unlearning between the *Washout* group and both the *Clamp* and *NoFB* groups [*t(*9*)* = 7*.*86, *p <* 10<sup>−</sup>4; *t(*9*)* = 5*.*84, *p* = 0*.*0012; Bonferroni-corrected). Although the rate of unlearning appeared slower in the *Clamp* group compared to the *NoFB* group, this difference was not statistically significant (*p* = 0*.*82 after Bonferroni-correction).

Next, we compared the total amount of retention of the initial adaptation by the end of the unlearning epoch. We determined the amount of retained adaptation for each subject through a retention factor that quantified the proportion of the total amount of initial adaptation that remained following the *Unlearning* block. **Figure 2C** shows the average retention factor across subjects for each group. Only the *Time* group exhibited retention that was significantly different from zero [*t(*9*)* = −3*.*845, *p*Time *<* 0*.*05 Bonferroni-corrected; *p*Clamp = 0*.*69; *p*NoFB = 0*.*31; *p*WO = 0*.*54), i.e., all other groups had returned to baseline. Although the *Time* group did not fully return to baseline, they did exhibit partial unlearning, evidenced by the fact that they had a retention factor that was significantly smaller than 1 [*t(*9*)* = 3*.*414, *p <* 0*.*01).

## **PRESENCE OF SAVINGS FOLLOWING THE DIFFERENT UNLEARNING PROTOCOLS**

Following the unlearning manipulation, we re-exposed subjects to the 30◦ CCW rotation perturbation to assess whether or not a memory of the prior adaptation was present in the form of savings. **Figures 3A**–**D** compares the initial learning and re-learning.

Following (Cheng and Sabes, 2006; Zarahn et al., 2008), we fitted state space models to each subject's data (see Materials and Methods).The critical parameter of interest with regard to savings is the sensitivity to error, *B* in Equation 1. We allowed this parameter of the model to take different values in each epoch, in order to capture the difference in adaptation rates between the first and second exposures (Zarahn et al., 2008). All other parameters were assumed to be fixed throughout the experiment. Savings would therefore be evident as a change in the learning rate *B* during the *Re-learning* epoch relative to the *Initial Learning* epoch. **Figure 3E**

shows the average estimated learning rate for each group during the *Initial Learning* and *Re-learning* epochs. We found that the change in learning rate was significantly different across groups (Mixed-effects ANOVA, Group × Epoch interaction, *F(*3*,* <sup>36</sup>*)* = 4*.*493; *p <* 0*.*01). *Post-hoc* comparisons showed a marginally significant change for the *Washout* group (*t* = 3*.*00, *p* = 0*.*0598, Bonferroni-corrected), and a strongly significant effect following time (*t* = 6*.*23, *p <* 0*.*001). Thus, we observed highly robust savings following unlearning due to the passage of time with no movements, less reliable savings following 200 trials of washout and no savings following either *Clamp* or *NoFB* blocks.

## **DISCUSSION**

Trial-by-trial learning during adaptation paradigms is believed to depend on sensory prediction errors (Mazzoni and Krakauer, 2006) driving updates to an internal forward model in the cerebellum (Bastian, 2006; Tseng et al., 2007; Shadmehr and Krakauer, 2008; Taylor et al., 2010). This model of learning is expressed mathematically through state-space models (Thoroughman and Shadmehr, 2000; Donchin et al., 2003; Cheng and Sabes, 2006). Unlearning has typically been accommodated within such models through a trial-to-trial retention factor, with the general supposition that this unlearning reflects a forgetting of the recentlylearned internal model.

The basic state-space framework may be extended to include multiple components that learn and decay at different rates (Smith et al., 2006; Körding et al., 2007). Enriching the model in this way enables it to account for the characteristic twotimescale learning curves, accounts for spontaneous recovery of recently washed out learning during clamp trials, and suggests

a mechanism for savings: that the faster relearning is supported by a latent slow-process memory. However, such a model is unable to account for our results. The fact that learning passively returned to baseline during the *Clamp* and *NoFB* implies that even the slower-decaying process must have decayed back to baseline values. Although we saw no savings following this unlearning, consistent with the predictions of such models, we did observe savings following a comparable number of trials of washout by a null perturbation. This would not be predicted by a multi-rate state-space model, since the decay of the memory of the initial learning is governed purely by the number of trials since exposure and should therefore be the same as for the *Clamp* and *No Feedback* conditions. One way in which the state-space model may be extended in order to account for our findings would be to include a capacity to contextually switch between multiple learned states (Lee and Schweighofer, 2009; Berniker and Körding, 2011; Pekny et al., 2011). The difference between *Clamp*/*NoFB* and *Washout* could then be explained by the fact that washout trials did engage such a contextual switch, but *Clamp* and *NoFB* did not.

Our results demonstrate that removing errors altogether (*No Feedback*) has a qualitatively similar effect to artificially clamping errors to zero (*Clamp*), both in terms of the time-course of unlearning and the abolition of subsequent savings. The unlearning part of the result is predicted by the state-space framework since, in both cases, the only change in internal state between trials stems from the retention coefficient (*A* in Equation 1). The similarity between *Clamp* and *NoFB* is interesting because it suggests that in the absence of feedback, subjects may implicitly presume success based on their forward model predictions about the outcome of their movements. Sitting idle for a comparable passage of time had a far weaker effect of unlearning, implying that it is necessary to actively make movements in order for unlearning to occur.

There is a potential mechanism that may support a model-based interpretation of movement-dependent unlearning. Cerebellar learning depends critically on plasticity at the parallel fiber-Purkinje Cell (PF/PC) synapse (Coesmans et al., 2004; Jörntell and Hansel, 2006). Long-term depression (LTD) at this synapse occurs when simple spikes, movement related activity carried by mossy-fiber inputs to cerebellar cortex, co-occur with complex spikes, which are driven by climbing fiber inputs. This LTD must be balanced by long-term potentiation (LTP) in order for the cerebellum to be able to maintain flexibility in what it can learn. LTP occurs when simple spike activity occurs in the absence of complex spikes. Popular models of cerebellar learning posit that the climbing fiber signal reflects a prediction-error signal. The absence of a complex spike therefore should signal perfect performance. However, if no sensory feedback is available to validate the prediction made by the cerebellum, then presumably this must also be encoded by the absence of a complex spike. Thus, this mechanism can potentially explain both why making movements leads to strong forgetting (increased simple spike activity), while also explaining why unobservable errors should lead to similar amount of forgetting as observed zero error.

Although it may be possible to interpret our findings here in terms of multiple internal cerebellar-based internal models, our recent work has proposed a fundamentally different view of motor learning. We have argued that behavior in adaptation paradigms is in fact governed by a combination of two qualitatively distinct learning processes (Huang et al., 2011; Shmuelof et al., 2012; Haith and Krakauer, 2013). Although initial learning may proceed through updates to a forward model in a cerebellar-dependent, model-based manner consistent with statespace model dynamics, savings upon re-learning appears to be due instead to a distinct, model-free learning mechanism that depends on the basal ganglia rather than the cerebellum. Actions that prove to be successful during initial learning are remembered and recalled during subsequent exposures, leading to accelerated adaptation during re-learning (Huang et al., 2011). The slow process invoked by multi-rate state space models of learning may in fact inadvertantly provide a means to approximate model-free components of learning.

The presence of multiple, qualitatively different learning systems raises the question of which learning system the unlearning is truly occurring in. Specifically, unlearning might be a modelfree phenomenon, reflecting a gradual reversion to old (baseline) habits, rather than forgetting of a forward model. It is quite possible that spontaneous unlearning in clamp trials and following removal of feedback is due to a combination of forgetting of a forward model and reversion to baseline habits. Indeed, unlearning behavior in clamp trials shows two distinct timescales (Smith et al., 2006), suggesting that two distinct processes are implicated.

We recently showed that the point which subjects decay to in clamp trials can be shifted to a new action by inserting a period of binary reinforcement of an adapted action (Shmuelof et al., 2012). Vector error feedback about task performance was removed following initial adaptation, forcing subjects to rely on binary feedback alone and precluding them from maintaining and using an accurate internal model. Thus, subjects had to rely on an alternative learning strategy, which we hypothesize uses the same model-free mechanism that is responsible for savings (Huang et al., 2011). This result can be explained quite naturally within a multiple learning systems framework in terms of a shift in the balance between learning systems caused by the removal of vector error. However, it is problematic to explain this result within a state-space model framework in which learning of all components is driven by vector error.

The fact that changing the nature of feedback can alter patterns of unlearning suggests that unlearning may usually occur because of a reversion to a baseline, model-free habit, rather than as a consequence of passive unlearning of an internal model. A partial reduction in the amount of decay is also seen following transcranial direct current stimulation of the motor cortex (Galea et al., 2011). We similarly interpret this result as being due to the promotion of model-free learning in motor cortex and not to halting decay of an internal model. Interestingly, transcranial direct current stimulation of the cerebellum accelerates initial adaptation but has no effect on the timecourse of unlearning (Galea et al., 2011), further calling into question the notion that unlearning is a cerebellar-based phenomenon. Although these previous experiments suggest that unlearning is due to an active return to a habitual baseline rather than passive decay of a recently-learned internal model, this does not necessarily mean that the forward model is not also forgotten. It is difficult to establish the state of the internal model when overt behavior may be dictated by additional overlying processes.

Unlearning during washout was faster than in the *Clamp* and *NoFB* conditions. This result is unsurprising since it reflects an active re-adaptation toward baseline, rather than more spontaneous unlearning. More interestingly, however, we found that savings was stronger following washout than following *Clamp* and *NoFB* trials. Interestingly, the magnitude of the savings we observed following washout was weaker than we have observed previously in paradigms that used a smaller number of washout trials (Zarahn et al., 2008; Huang et al., 2011). Savings is likely dependent on the number of trials of washout (Krakauer et al., 2005). Here we used a relatively long washout block of 200 trials, compared to previous studies that employed only 80 trials (Zarahn et al., 2008; Huang et al., 2011). We suggest that this may have affected savings by increasing the value associated with baseline movements, rather than directly diminishing the value of the previously reinforced action at the end of adaptation.

We interpret the lack of savings in the *Clamp* and *No Feedback* groups as reflecting the fact the reinforced action has been completely erased. However, an alternative explanation is that the memory is indeed retained but subjects are unable to retrieve it due to interference caused by the multitude of movements made during the unlearning block that may have been equally reinforced. Indeed we have argued previously that interference is attributable to competition for retrieval rather than over-writing one memory by another (Krakauer et al., 2005). Although there was no direct reinforcement in the *No Feedback* group, subjects may have presumed that their movements would be successful, therefore receiving a comparable reinforcement and therefore giving rise to the same kind of interference.

We observed the greatest extent of savings in the *Time* condition. The Bayesian explanation for the faster re-learning following a period of inactivity is that uncertainty about the plant and perturbation increased during the idle period, so that new prediction errors had a relatively stronger influence on updating subjects' estimate of the perturbation (Körding et al., 2007; Wei and Körding, 2010). This logic should, however, apply equally to the condition in which visual feedback was removed. We found no evidence to support this theory in our data, however, since the learning rate during re-learning was identical when visual feedback was removed, compared with when feedback was clamped at zero error. We therefore favor the idea that savings was maximal after a period of inactivity because there were fewer (zero) intervening washout trials to reinforce baseline.

In summary, our findings, in conjunction with our previous work and that by others, lead us to conclude that spontaneous unlearning reflects reversion to baseline actions (which have presumably been strongly reinforced throughout life) from a new action that has been more weakly reinforced during adaptation. The presence of savings implies that the adapted action is not entirely forgotten. Thus, a weakly reinforced action can either

## **REFERENCES**


be out-competed but not forgotten (*Time* and *Washout*) or outcompeted and forgotten (*Clamp* and *No Feedback*). Future work will need to establish the degree to which our findings generalize to other motor learning paradigms, such as force field adaptation during reaching (Pekny et al., 2011) or split-belt adaptation of locomotion (Reisman et al., 2005), and to further clarify the interaction between internal models, presumably in the cerebellum, with a reinforced controller, presumably in motor cortex.

## **ACKNOWLEDGMENTS**

We would like to thank Eric Zarahn for very helpful advice on the experimental design and analysis. This work was supported by grant R01NS052804 (to John W. Krakauer).

*J. Neurosci.* 26, 3642–3645. doi: 10.1523/JNEUROSCI.5317-05.2006 Pekny, S. E., Criscimagna-Hemminger,


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 01 January 2013; accepted: 08 June 2013; published online: 28 June 2013.*

*Citation: Kitago T, Ryan SL, Mazzoni P, Krakauer JW and Haith AM (2013) Unlearning versus savings in visuomotor adaptation: comparing effects of washout, passage of time, and removal of errors on motor memory. Front. Hum. Neurosci. 7:307. doi: 10.3389/fnhum. 2013.00307*

*Copyright © 2013 Kitago, Ryan, Mazzoni, Krakauer and Haith. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

## Functional near-infrared spectroscopy-based correlates of prefrontal cortical dynamics during a cognitive-motor executive adaptation task

## *Rodolphe J. Gentili 1,2,3\*, Patricia A. Shewokis 4,5,6, Hasan Ayaz 4,5 and José L. Contreras-Vidal <sup>7</sup>*


#### *Edited by:*

*Rachael D. Seidler, University of Michigan, USA*

#### *Reviewed by:*

*Julien Doyon, Institut universitaire de gériatrie de Montréal, Canada Sean K. Meehan, University of Michigan, USA*

#### *\*Correspondence:*

*Rodolphe J. Gentili, Department of Kinesiology, School of Public Health, University of Maryland, School of Public School Building (#255), College Park, MD 20742, USA e-mail: rodolphe@umd.edu*

This study investigated changes in brain hemodynamics, as measured by functional near infrared spectroscopy, during performance of a cognitive-motor adaptation task. The adaptation task involved the learning of a novel visuomotor transformation (a 60◦ counterclockwise screen-cursor rotation), which required inhibition of a prepotent visuomotor response. A control group experienced a familiar transformation and thus, did not face any executive challenge. Analysis of the experimental group hemodynamic responses revealed that the performance enhancement was associated with a monotonic reduction in the oxygenation level in the prefrontal cortex. This finding confirms and extends functional magnetic resonance imaging and electroencephalography studies of visuomotor adaptation and learning. The changes in prefrontal brain activation suggest an initial recruitment of frontal executive functioning to inhibit prepotent visuomotor mappings followed by a progressive de-recruitment of the same prefrontal regions. The prefrontal hemodynamic changes observed in the experimental group translated into enhanced motor performance revealed by a reduction in movement time, movement extent, root mean square error and the directional error. These kinematic adaptations are consistent with the acquisition of an internal model of the novel visuomotor transformation. No comparable change was observed in the control group for either the hemodynamics or for the kinematics. This study (1) extends our understanding of the frontal executive processes from the cognitive to the cognitive-motor domain and (2) suggests that optical brain imaging can be employed to provide hemodynamic based-biomarkers to assess and monitor the level of adaptive cognitive-motor performance.

**Keywords: visuomotor adaptation-learning, frontal executive, functional near infrared spectroscopy, internal models, arm reaching movement**

## **INTRODUCTION**

Humans have the ability to adapt their movements to various environments and/or perturbations through practice or experience. A possible approach to investigate human adaptation capabilities is to simultaneously examine the brain dynamics and behavioral changes during arm movements in the presence of a visual distortion of the movement trajectory (e.g., Contreras-Vidal and Kerick, 2004; Anguera et al., 2007; Seidler and Noll, 2008; Gentili et al., 2009, 2011). Under such conditions, individuals are required to learn the internal representation of the novel visuomotor transformation (i.e., a hand-screen cursor rotation) to perform accurate movements (e.g., Kluzik et al., 2008; Kagerer and Contreras-Vidal, 2009; Gentili et al., 2011). Visuomotor adaptation paradigms require inhibiting prepotent visuomotor mappings that are no longer task-relevant and consequently may interfere with the ongoing adaptation process.

Brain dynamics during visuomotor task adaptations have been investigated by employing various neuroimaging techniques. For instance, numerous studies combined an adaptation task with functional Magnetic Resonance Imaging (fMRI) (e.g., Seidler et al., 2006; Seidler and Noll, 2008). However, the constrained movement amplitudes and the unnatural placement of the subject's body in a supine position while performing the task in a magnet provided limited ecological validity; since daily physical motor activities are usually performed in a seated or standing position. To address the issues of limited ecological validity and task performance in natural settings, functional near infrared spectroscopy (fNIR) enables monitoring of cortical activity in natural settings was used (e.g., Hatakenaka et al., 2007; Ikegami and Taga, 2008; Leff et al., 2008a,b; Ayaz et al., 2009, 2011, 2012a,b,c; Ohuchida et al., 2009; James et al., 2010, 2012; Gentili et al., 2010a). In addition, there are few magnetoencephalography (MEG) and electroencephalography (EEG) investigations of brain dynamics during performance of visuomotor adaptation tasks (Contreras-Vidal and Kerick, 2004; Anguera et al., 2009; Bradberry et al., 2009; Gentili et al., 2009, 2011; Perfetti et al., 2011). These recent EEG studies evidenced a refinement of the cortical dynamics throughout adaptation for individuals facing the distortion whereas no changes in brain dynamics or behavior were observed in individuals who did not face the distortion challenge. However, for individuals who faced the distortion, there was an increase in alpha power in the prefrontal regions that reflect a progressive derecruitment of the prefrontal inhibitory functions. Thus, these prefrontal inhibitory functions are highly engaged during early learning to inhibit the prepotent motor responses whereas they become irrelevant to the task demand during late adaptation (Gentili et al., 2010a, 2011).

Beyond these studies, there is a critical need to investigate hemodynamic changes in ecological situations (e.g., seated positions) as the brain adapts by considering alternative neuroimaging approaches such as fNIR. While EEG provides a measure of neural electrical activity, by contrast fNIR measures blood oxygenation levels via infrared light (e.g., Izzetoglu et al., 2007; Ayaz et al., 2009, 2011, 2012a,b,c). In essence, fNIR can provide different and complementary biological markers for brain dynamics with increased robustness to artifacts during cognitive and motor performance under everyday conditions and in real life environments (e.g., Coyle et al., 2007; Hatakenaka et al., 2007; Leff et al., 2008a,b; Abdelnour and Huppert, 2009; Ayaz et al., 2009, 2011, 2012a,b,c; Gentili et al., 2010a; James et al., 2010, 2012; Power et al., 2012; Sweeney et al., 2012). Comparatively, it was demonstrated that fNIR could indicate various levels of cognitive workload (Izzetoglu et al., 2005; Ayaz et al., 2009, 2012a,b,c; James et al., 2012; Power et al., 2012) as well as changes in motor performance (Hatakenaka et al., 2007; Ikegami and Taga, 2008; Leff et al., 2008a,b; Morihiro et al., 2009; Gentili et al., 2010a).

Among the rare fNIR investigations that focused on motor learning (e.g., Leff et al., 2011; Ayaz et al., 2012a,b,c; James et al., 2012), none investigated adaptive brain dynamics capabilities along with the concomitant changes in performance during a visuomotor adaptation task where individuals faced a cognitive-motor challenge such as the inhibition of prepotent motor responses that are no longer task-relevant.

Therefore, the present study examined functional brain activation by employing fNIR with a particular emphasis on the prefrontal regions since the visuomotor task we employed solicited these specific cortical regions that inhibit prepotent motor responses to facilitate adaptation processes. We predicted that as adaptation happened, there would be a progressive reduction of the cortical activity (i.e., a reduction of oxygenation level) for individuals who experienced the visual distortion since during early adaptation, frontal executive (inhibitory, updating) functions are necessary to adapt to the task demands whereas these same executive functions would become much less relevant by late adaptation. In addition, no cortical or behavioral changes were expected to be observed for individuals who were not exposed to the visual distortions since the engagement of these executive functions was unnecessary.

## **MATERIALS AND METHODS**

## **PARTICIPANTS AND APPARATUS**

Twenty-six right-handed and healthy adults (12 males and 14 females ranged from 20 to 35 years old) with normal or correctedto-normal vision volunteered to participate in this study that was approved by the Institutional Review Board at the University of Maryland-College Park. Participants were seated at a table while facing a computer screen that was placed in front of them at a distance of ∼60 cm; while they were required to draw a line by moving a pen on a digitizing tablet (12 WACOM, InTuos). Pen trajectories were displayed in real time as solid black lines on the computer screen by means of custom software (Oasis v.8.29 Kikosoft, Neijmegen); however, a horizontal board prevented vision of the arm/hand moving on the digitizing tablet. Participants had to execute with their right arm/hand "center-out" movements to draw lines from a home target circle (ø = 5 mm) placed in the center of the screen to one of four peripheral target circles (ø = 5 mm). As such, the home target circle represented the origin of a polar frame of reference in which the pointing target circles were positioned at 10 cm from the origin and located at 45◦, 135◦, 225◦, and 315◦, respectively (**Figure 1**). Concurrently, optical brain imaging signals were recorded by the continuous-wave dual-wavelength fNIR system first described by Chance et al. (1993) and developed at Drexel University (Ayaz et al., 2011, 2012a). Accurate and repeatable positioning of the sensor pad was ensured by using the International 10–20 system for electrode placement and by matching the center of the sensor with the vertical axis of symmetry that passes through the nasion. This fNIR system included three components: (1) a flexible headpiece (sensor pad) which incorporates both light sources and detectors enabling therefore a fast placement of all 16 optodes (channels), (2) a control box for hardware processing, and (3) a computer for data acquisition with triggers to synchronize events with the fNIR signal. The sensor had a temporal resolution of 500 milliseconds per scan with 2.5 cm source-detector separation allowing for ∼1.25 cm penetration depth. There are four light emitting diodes (LED) that can shine in 730 and 850 nm wavelengths and 10 photo detectors on the flexible headband sensor. The configuration of light source and detectors yielded to a total of 16 active channels that composed the probe covering a space of 14.1 cm (width) by 3.5 cm (height). Such a system was designed and previously employed to monitor dorsal and inferior frontal cortical regions underlying the forehead (e.g., Izzetoglu et al., 2005; Bunce et al., 2006; Ayaz et al., 2010, 2012a,b,c) (see **Figure 1A**). Cognitive Optical Brain Imaging (COBI) Studio software (Drexel University) was used for data acquisition and visualization (Ayaz et al., 2011). During the task, a serial cable between the fNIR data acquisition computer and stimulus presentation computer was used to synchronize the fNIR and kinematic signals.

**FIGURE 1 | Experimental setup and devices. (A)** fNIR probe (Drexel University™) including the 16 channels (represented by a filled white square) and its placement on the prefrontal region of the forehead subjects (Ayaz et al., 2012a). **(B)** Sequence composing a reaching trial and set-up allowing for continuously recording of both fNIR and kinematic signals via the digitizing tablet during arm reaching movement. MO, Movement onset.

#### **EXPERIMENTAL PROCEDURE**

To become familiarized with the experimental device, the participants performed 20 practice trials as an orientation/familiarization phase (these trials were not included in the subsequent analyses). Before starting the experiment, a 10 s baseline was recorded while the participants were inactive and relaxed. This baseline was then employed to compute the changes in concentration of fNIR markers (e.g., oxygenated hemoglobin) used in the subsequent standardization of the fNIR data. Then, participants performed 20 trials (i.e., 1 block) under normal (i.e., without visual distortion) visual feedback of cursor movement (i.e., pre-exposure). The 26 participants were randomly assigned to the learning and control groups with each group including 13 participants. The experimental (learning) group participants performed 180 (i.e., 9 blocks × 20 trials) drawing movements during which the screen cursor was suddenly rotated 60◦ counterclockwise (i.e., exposure) whereas no visual distortion was imposed on the control group. Lastly, all participants executed a block of 20 trials under normal visual feedback (i.e., no visual distortion) to assess for after-effects (i.e., post-exposure) and to determine if the internal model of the novel visuomotor map was effectively encoded by participants. Targets were self-selected and movements were self-initiated (i.e., no forced paced), and all targets were displayed throughout each trial. At the beginning of each trial, an 8 s rest period was considered during which the subject fixated on the home target (Sitaram et al., 2007). Then a beep sounded, indicating to the participants that they were allowed to enter the screen cursor inside the home target circle (without any time constraints). Next, participants had to select one of the four peripheral targets without moving the pen and prepare their movement. Once the peripheral target was selected, they had to draw a line as straight and as fast as possible linking the home and pointing target. Movements that started earlier than 2 s after target presentation were terminated, and the trial was restarted. Thus, participants had enough time to both select a target and plan their movement (at least 2 s) and could start whenever they felt ready after this 2 s period. Once a successful trial was performed, all visual stimuli were erased from the screen in preparation for the next trial. To minimize fatigue and maintain attention, brief relaxation periods were allowed as needed (**Figure 1B**).

## **DATA ANALYSIS** *Movement kinematics*

The kinematics of the hand were low-pass filtered using a 5-Hz, dual-pass eighth-order Butterworth filter. Then, to quantify the motor performance, three kinematic parameters were computed. Movement time (MT) defined as the time elapsed between leaving the home target and acquiring the reaching target and that reflected regulations during movement performance. Movement length (ML) represented the distance traveled in each trial between the home and the reaching target. Finally, root mean squared error (RMSE) was computed to quantify any discrepancy between the movement trajectory and the "ideal" straight line linking the home and the reaching target. After resampling of the trajectories to reach an equal amount of data points between the actual and "ideal" straight trajectory, the RMSE was computed according to the following formula:

$$\text{RMSE(in cm)} = \sqrt{\frac{\sum\_{i=1}^{N} \left[ (x\_a - x\_i)^2 + (y\_a - y\_i)^2 \right]}{N}} \tag{1}$$

where *xa*, *ya*, and *xi*, *yi* are corresponding points of the actual (index *a*), resampled trajectory and the ideal (index *i*) trajectory, respectively. *N* is the number of points in the path.

In addition, a measure of directional error labeled initial directional error (IDE) was computed as the difference between the angle formed by the vector from the home position to the current hand position 80 ms after movement onset and the vector extending from the home position to the goal target (target to reach). Since this directional error is measured before visual feedback is available, measurement error information can inform about planning processes and the current state of the internal model of the perturbation. The kinematic parameters (MT, ML, and RMSE) and the directional error (IDE) were standardized with respect to the pre-exposure stage for each participant to account for any differences in a participants' performance during the pre-exposure stage (i.e., without perturbation) as well as to focus on changes that are solely due to adaptation. The values were standardized according to the following equation:

$$\text{SP}\_{i}(\text{SD}) = \frac{P\_{i} - \overline{P\_{\text{Pr\\_Exp}}}}{\text{SDP}\_{\text{Pr\\_Exp}}} \tag{2}$$

where *Pi* (*P*: Parameter) is the value of a kinematic parameter computed for the *i*th single trial performed during exposure, and *P*Pr \_*Exp* and *SDP*Pr \_*Exp* represent the mean and standard deviation across trials of the same parameter computed during the pre-exposure block, respectively. The *SPi* (*SP*: Standardized Parameter) values were then averaged within blocks and participants. As such, a standardization process was applied to the kinematic data, which were expressed in standard deviation units (i.e., *SD* units) relative to the pre-exposure stage for each participant.

## *fNIR signal processing*

For each participant, raw fNIR data (16 optodes × 2 wavelengths) were low-pass filtered with a finite impulse response, linear phase filter with an order 20 and cut-off frequency of 0.1 Hz to attenuate the high frequency noise, respiration, and cardiac cycle effects (Izzetoglu et al., 2005; Ayaz et al., 2011). To check for any saturation, in which light intensity at the detector was higher than the analog-to-digital converter limit or motion artifact, both visual inspection and sliding window motion artifact rejection technique was used (Ayaz et al., 2010). fNIR data epochs for the baseline and task periods were extracted from the continuous data using time synchronization markers. Blood oxygenation and volume changes within each of the 16 optodes were calculated using the modified Beer-Lambert Law (Chance et al., 1993; Villringer and Chance, 1997) for task periods with respect to the baseline at beginning of the experiment with fnirSoft (Ayaz, 2010). For each task period, concentration changes of four parameters were calculated: oxygenated-hemoglobin (HBO2), deoxygenated hemoglobin (HB), total hemoglobin (HBT), and oxygenation (OXY—defined as the difference between HBO2 and HB). In order to ensure consistency in our data processing, the approach used for fNIR values was similar to that employed for kinematic parameters and error measurement. The fNIR values were also standardized by employing Equation 2. Then, the first eight optodes (1–8) were averaged to represent the left hemisphere while the last eight optodes (9–16) were average to represent the right hemisphere within the prefrontal cortex.

## *Statistical procedures and data fitting*

Given that the purpose of this study was to investigate the relationship between cortical hemodynamics and performance by replicating a study using EEG power values and a visuomotor adaptation task (Gentili et al., 2011), the statistical plan was similar.

### *Statistical procedure for kinematic parameters*

The average standardized values of the kinematic parameters (MT, ML, and RMSE) were assessed regarding meeting the parametric assumptions of normality using a *Kolmogorov-Smirnov test* *using a Lilliefors correction* as well as histograms. To assess the behavioral efficacy of the adaptation 2 × 2 Group (Learning and Control) by Period (Early and Late adaptation periods) mixed model ANOVAs with repeated measures on the last factor were computed separately for the kinematic parameters (MT, ML, RMSE) and the directional error (IDE). Adaptation periods were defined as early- (the two first blocks) and late- (the last two blocks) of task performance. The choice of the definition of the period for the early and late period was guided by previous studies that defined these periods in a similar way (e.g., Anguera et al., 2009; Gentili et al., 2011). Significance criterion for all tests was 0.05, percent change and 95% confidence intervals of the mean differences were calculated and presented in **Table 1**. For significant effects, partial omega-squared (ω2) is the effect size index presented for the data interpretation. Number Cruncher Statistical System (NCSS) 8 (www.ncss.com) software was used for the statistical analyses.

## *Statistical procedure for the fNIR parameters*

A 2 × 2 × 2 [Group (learning; control) × Hemisphere (right; left) × Period (early; late)] mixed model ANOVA with repeated measures on the last two factors was applied separately to the four fNIR markers. The between subjects factor, Group, and the within subjects factors of Hemisphere and Period were fixed factors while the subject factor was a random effect. A Huynh– Feldt correction was applied (Huynh and Feldt, 1976) when the assumption of sphericity was violated. Any significant interaction effects were assessed by Tukey HSD tests for interactions. Cohen's d effect sizes were also computed and used to aid in data interpretation.

### *Data fitting of fNIR parameters*

To characterize the cortical dynamics throughout the adaptation stage, the fNIR parameters were fitted throughout the nine practice blocks. A visual inspection of the fNIR data clearly suggested considering a linear fit. Thus, for the left and the right hemisphere and across the adaptation task, the changes in standardized fNIR parameters were fitted using a linear model for each participant. For each linear-fitted model, the coefficient of determination (*r*2) and its slope were obtained. Then, for each of the standardized fNIR parameters, the slopes of the linear models were statistically tested, as for the kinematic parameters, by employing a *Kolmogorov–Smirnov test using a Lilliefors correction* and histograms. A non-parametric *Wilcoxon tes*t was used to compare if the model values were statistically different (1) from 0 (i.e., if these models revealed a significant decrease or increase) and (2) between the participants of the learning and control group (i.e., if the dynamics of these model revealed a significant difference between the learning and the control group).

## *Relationship between cortical dynamics and kinematics parameters*

Finally, to more directly examine the relationships between performance and the cortical hemodynamics, the values for the significant fNIR (HBO2, OXY) parameters were plotted as a function of the kinematics (MT, ML, and RMSE) for the left and


**Table 1 | Descriptive statistics of the standardized kinematic parameters and the directional error across the groups including 95% confidence intervals.**

*MT, movement time; ML, movement length; RMSE, root mean squared error; IDE, initial directional error; EXP, experimental; CON, control; SE, standard error.*

the right hemispheres. A visual inspection of the data suggested four main possible fitting curves: a linear [*f(x)* = *ax* + *b* {*a, b*} ∈ *<sup>R</sup>*], a logarithmic [*f(x)* <sup>=</sup> *<sup>a</sup>* log*(x)* {*a*} ∈ *<sup>R</sup>*], a rational [*f(x)* <sup>=</sup> *<sup>a</sup> <sup>x</sup>* {*a*} ∈ *R*], and a composite function that combined a rational function and a linear component [*f(x)* = *<sup>a</sup> <sup>x</sup>* + *bx* + *c* {*a, b,c*} ∈ *R*]. The best fit was selected by considering *r*<sup>2</sup> of the fit.

## **RESULTS**

## **MOVEMENT KINEMATICS**

During early exposure to the visuomotor perturbation the participants of the learning group revealed movement similar to counter-clockwise spirals trajectories that included sudden reversals and slow progression toward the targets whereas during the late exposure stage movement trajectories were faster, straighter, and with a reduction of the RMSE as noted in **Table 1**. Movement kinematics resulted in significant differences for the Group × Period interaction, Period main effect and Group main effects which are reported in **Table 2**. The significant Group × Period interactions and Period main effects revealed that, compared to early adaptation, MT, ML, RMSE, and initial directional error were reduced during the late-exposure stage (see **Figures 2A,B**; left column). In addition, a very large effect ωpartial<sup>2</sup> = 0*.*37 − 0*.*64 was detected for the movement kinematics as a function of the interactions or Period main effects. During the post-exposure stage, (i.e., once the distortion was removed), movement trajectories showed distortions (after-effects) with movements in the opposite direction compared to the early stages of adaptation revealing the participants had encoded the internal model of the new visuomotor transformation. Conversely, participants of the control group did not reveal any changes in performance throughout the entire task as suggested by the absence of changes in the hand paths (see **Figures 2A,B**; right column).

## **fNIR VALUE: EARLY (FIRST TWO) vs. LATE (LAST TWO) BLOCKS OF TRIALS**

A 2 × 2 × 2 ANOVA (Group × Hemisphere × Period) was applied separately to the four fNIR markers (i.e., HBO2; OXY; HBT; and HB). Significance criterion for all tests was 0.05 and 95% confidence intervals of the mean differences and percent change were calculated and presented in **Table 3**.

**Table 2 | Results of the mixed model ANOVAs (2 Group × 2 Period) for the kinematic parameters and the directional error obtained for the learning and control groups during the early and late adaptation periods.**


*MT, movement Time; ML, movement length; RMSE, root mean squared error; IDE, initial directional error.* ω*2, effect size index (proportion of variance explained) for between subjects main effect;* ω*partial2, effect size index for interactions and within subjects main effect.*

## **HBO2**

The results of the ANOVA revealed a two-way interaction between Group and Period [*F(*1*,* <sup>24</sup>*)* = 4*.*67, *p <* 0*.*05] for the HBO2 marker. This analysis showed that HBO2 in the prefrontal region was significantly lower for the late (*M* = −4*.*31 sd units, *SE* = 1*.*19; *d* = 1*.*26) compared to the early (*M* = −0*.*15 sd units, *SE* = 0*.*59) adaptation phase in the learning group (*p <* 0*.*012) whereas no change was detected (*p >* 0*.*98) in the control group (**Figures 2C**, **3A**).

## **OXY**

ANOVA revealed a Period main effect [*F(*1*,* <sup>24</sup>*)* = 6*.*66, *p <* 0*.*05] for the OXY marker suggesting that, compared to the late adaptation (*M* = −2*.*51 sd units, *SE* = 1*.*76*, d* = 0*.*66), the OXY was

significantly higher during early adaptation stage (*M* = 0.35 sd units, *SE* = 0*.*65) for both groups. Interestingly, ANOVA revealed a tendency regarding a two-way interaction between Group and Period (*p* = 0*.*06) which suggest that the Period main effect was likely mainly driven by changes in the learning group. Although not significant, the OXY in the prefrontal region tended to be lower for the late (*M* = −3*.*95 sd units, *SE* = 1*.*52, *d* = 1*.*21) compared to the early (*M* = 1.03 sd units, *SE* = 0*.*76) adaptation phase in the learning group whereas the same comparison for

adaptation for early, middle, and late exposure (the gray area illustrates the

the control group revealed a smaller difference (early: *M* = −0*.*33 sd units, *SE* = 0*.*46 vs. late: *M* = −1*.*03 sd units, *SE* = 1*.*91, *d* = 0*.*17; see **Figures 2C**, **3B**). The high standard errors may have contributed to reducing the power of this test.

## **HB**

Right; L, Left.

ANOVA did not reveal any effect (*p >* 0*.*05) for the HB marker suggesting that, both the learning and control group had a comparable level of HB throughout practice (see **Figures 2C**, **3C**).


**Table 3 | Descriptive statistics of the standardized hemodynamic parameters across the learning and control groups and prefrontal sides including 95% confidence intervals.**

*HBO2, oxygenated hemoglobin; OXY, oxygenation; HB, deoxygenated hemoglobin; HBT, total hemoglobin; EXP, experimental; CON, control; SE, standard error.*

## **HBT**

ANOVA revealed a Group main effect [*F(*1*,* <sup>24</sup>*)* = 5*.*07, *p <* 0*.*05] for the HBT marker suggesting that during practice, in comparison to the control group (*M* = 0.12 sd units, *SE* = 4*.*21, *d* = 0*.*61), the HBT was lower in the learning (*M* = −2*.*48 sd units, *SE* = 0*.*98) group. In addition, the Period main effect exhibited a tendency in the same direction for the early and late phases which was comparable to the OXY and HBO2 biomarkers (*p* = 0*.*08). The high variability during the late phase contributed to the reduced statistical power and higher Type II error for this effect (see **Figures 2C**, **3D**).

## **LINEAR MODEL OF fNIR MARKERS ACROSS ALL BLOCKS OF TRIALS**

The data fitting approach (see **Figure 4**, top row) revealed that the linear fitting [*f(x)* = *ax* + *b* {*a, b*} ∈ *R*] model captured accurately the changes in hemodynamics indicated by the HBO2 marker and revealed a significant linear decrease in both the left (*r*<sup>2</sup> = 0*.*92; *pslope <* 0*.*002; *a* = −0*.*59; *b* = 1*.*07) and right (*r*<sup>2</sup> = 0*.*93; *pslope <* 0*.*001; *a* = −0*.*65; *b* = 1*.*41) hemisphere throughout the nine blocks of trials in the learning group whereas such change was not significant for the control group (*r*<sup>2</sup> *<* 0*.*51; *pslope >* 0*.*50; *a >* −0*.*11; *b >* 0*.*26). Also, compared to the control group, the linear decrease for the right hemisphere was significantly higher (*p <* 0*.*013) for the learning group whereas a tendency in a similar direction was observed for the left hemisphere (*p* = 0*.*06). The oxygenation level indicated by the OXY marker revealed a very large effects and significant linear decrease in both the left (*r*<sup>2</sup> = 0*.*95; *pslope <* 0*.*007; *a* = −0*.*71; *b* = 2*.*10) and right (*r*<sup>2</sup> = 0*.*96; *pslope <* 0*.*004; *a* = −0*.*78; *b* = 2*.*71) hemisphere throughout the nine blocks of trials in the learning group whereas the control group did not exhibit these changes (*r*<sup>2</sup> *<* 0*.*63; *pslope >* 0*.*57; *a >* −0*.*14; *b >* −0*.*14). Also, compared to the control group, the linear decrease for the right hemisphere revealed a tendency to be higher (*p* = 0*.*07) for the learning group (see **Figure 4**, second rows). The same linear modeling applied to the HB did not reveal any significant decrease or increase for both

the learning and the control groups (0*.*20 *< r*<sup>2</sup> *<* 0*.*79; *pslope >* 0*.*20; *a >* −0*.*12; *b >* −1*.*67). This linear fit revealed a significant decrease of HBT in the right hemisphere for the learning group (*r*<sup>2</sup> *>* 0*.*78; *pslope <* 0*.*013; *a* = −0*.*41; *b* = −0*.*16) whereas no significant change was observed in the control groups (*r*<sup>2</sup> *<* 0*.*43; *pslope >* 0*.*54; *a >* −0*.*11; *b >* 0*.*60; see **Figure 4**, third and fourth rows).

## **RELATIONSHIP BETWEEN fNIR PARAMETERS LEVEL AND TASK PERFORMANCE**

To establish a more direct relationship between the observed changes in fNIR markers and kinematic performance throughout the visuomotor adaptation performance, correlational, and

data fitting analyses were conducted. Generally, the relationship between the changes in oxygenation levels (i.e., HBO2, OXY) in both hemispheres and the kinematic parameters (MT, ML, RMSE) observed in the learning group was best modeled by means of the composite function (**Figure 5**). Specifically, this fitting model accurately captured the relationship between the HBO2 and the three kinematic parameters (*r*<sup>2</sup> *>* 0*.*72; see **Figure 5**, left column) as well as between the OXY measurement and the three same kinematics parameters (*r*<sup>2</sup> *>* 0*.*70; see **Figure 5**, right column) for the participants of the learning group. The coefficient of determination was very large and accounted for a substantial amount of the explained variance in the hemodynamic variables (HBO2 and OXY) as a function of the kinematic measures.

The model could not capture the same relationship considering HB (which actually presented an opposite directionality to

learning and control groups, respectively.

the logarithmic function) whereas it could moderately capture the relationship between the changes in HBT and the three kinematics parameters for the learning group (0*.*38 ≤ *r*<sup>2</sup> ≤ 0*.*48; not shown in **Figure 5**). Finally, the same analyses could not fit the relationship between the hemodynamic markers and the kinematics performance for the control group since no particular patterns was observed in the data. Rather the data represented a simple clustering effect [see **Figure 5**; the gray empty circles and filled squares for the HBO2 (right column) and OXY (left column)].

## **DISCUSSION**

The main results of this investigation were the reduction of the hemodynamics and oxygenation level as indicated by the changes in HBO2 and to a lesser extent in OXY in the prefrontal regions as participants of the learning group progressively encoded the internal model of the new visuomotor transformation and thus improved their performance during the cognitive-motor adaptation task. Importantly, these reductions of oxygenation in the prefrontal regions were accompanied by a simultaneous decrease in variability and by a reduction in MT, ML, RMSE and directional error that resulted in performance enhancement. No changes either in brain hemodynamics or behavioral performance were observed in the participants of the control group who performed the same task as the learning group without any visuomotor distortion. Therefore, it appears reasonable to consider that the changes in oxygenation (HBO2 and OXY) observed in the prefrontal regions of the participants of the learning group were associated with adaptation of the prefrontal cortical dynamics that translated into enhanced quality of motor performance. Such changes in the learning group provide hemodynamic-based brain biomarkers (Georgopoulos et al., 2007; Gentili et al., 2009, 2010a,b, 2011) that can be employed to track the state of motor adaptation and more generally the changes in quality of performance.

## **FRONTAL EXECUTIVE FUNCTIONING FOR ADAPTIVE COGNITIVE-MOTOR CHALLENGE**

As expected, a reduction of the oxygenation in the prefrontal regions was observed throughout adaptation. A decrease in oxygenation reflects a progressive reduction of activation of these prefrontal regions suggesting that there is a reduction of the role of the prefrontal cortex as adaptation progresses. Specifically, the highest magnitude of oxygenation was observed during early adaptation suggesting an initial pronounced engagement of these prefrontal regions that was progressively reduced as participants adapted to the task demands. This evidence was also supported by the non-standardized values of HBO2 and OXY (not reported here) which, although similar during the pre-exposure session, suddenly increased when the perturbation was introduced and then decreased as adaptation progressed. Specifically, compared to the control group a 65% and a 82% larger increases for the HBO2 and OXY markers were observed for the learning group, respectively. Furthermore, the effect size for the early to late oxygenation transition for the learning group was large (*d* = 1*.*21) while the control group had a small to negligible effect (*d* = 0*.*17). These findings support our hypothesis about the differential involvement of the frontal executive functioning during early and late adaptation to a new visuomotor transformation. More precisely, while the prefrontal regions play a critical role in multiple neural processes, a possible explanation for this gradual prefrontal derecruitment would be related to the executive functioning processes and, specifically, to the inhibitory and the updating functions. Thus, during early exposure the introduction of the sudden kinematic perturbation challenged the individuals of the learning group since they are now facing a new visuomotor map: (1) for which their prepotent visuomotor transformation (i.e., their usual motor response in absence of perturbation) becomes suddenly irrelevant and (2) that creates a mismatch between the visual feedback related to the cursor trajectory and the kinesthetic signals from the arm/hand movement, resulting in sensorimotor conflicts and poor performance. Thus, during early exposure, the inhibitory function would play a critical role by inhibiting the unsuitable prepotent visuomotor transformation. This inhibition would allow for an efficient adaptation (e.g., Miller and Cohen, 2001; Basso et al., 2006) by facilitating the selection process of a new and well-suited visuomotor plan to solve sensorimotor conflicts and meet the task requirements (i.e., move the cursor as straight and fast as possible). Concurrently, the frontal updating function would constantly update working memory by evaluating newer incoming external (visual input) and internal (kinesthetic) information (Miyake et al., 2000; Shimamura, 2000). It was previously suggested that both functions could be implemented in the prefrontal regions (Collette and Van der Linden, 2002), and specifically, in the dorsolateral prefrontal and frontopolar cortices which are used to evaluate externally (visual, kineasthetics) and internally (motor command) generated information during movement preparation (Christoff and Gabrieli, 2000; Braver and Bongiolatti, 2002), respectively. Over time, as the participant adapts, the role of these inhibitory and updating functions progressively decreases, resulting in a gradual deactivation (reflected by a reduction of the oxygenation level) of the prefrontal regions by late exposure.

Importantly, by employing an alternative neuroimaging methodology, the present findings confirm and extend those from previous EEG and fMRI studies that revealed an increased role of frontal and prefrontal (dorsolateral, ventral) regions during early compared to late visuomotor adaptation and particularly underscore the role of the frontal executive (inhibitory, updating functions) when a new visuomotor transformation is being encoded (e.g., Shadmehr and Holcomb, 1997, 1999; Ghilardi et al., 2000; Graydon et al., 2005; Lacourse et al., 2005; Anguera et al., 2007; Gentili et al., 2010a, 2011). In a previous study, Gentili et al. (2011) analyzed EEG and kinematics using exactly the same visuomotor task including a learning group where participants had to adapt to a new visuomotor transformation and a control group who performed the same visuomotor task without any perturbation. The results of the Gentili et al. (2011) EEG study reported a bilateral increase in alpha power in the prefrontal regions that translated into the improvement of the quality of performance as individuals of the learning group adapted to the task demands. By considering that the alpha power is inversely related to brain activation, thus, an increase in EEG alpha power reflects a progressive cortical idling or in other words a progressive refinement of the cortical activity (Pfurtscheller et al., 1996; Hatfield et al., 2004). Therefore, in the present investigation, the reduction of oxygenation level observed bilaterally in the prefrontal regions is consistent with the reduction of the cortical activity previously reported by an increase in EEG alpha power. In addition, the findings of the Gentili et al. (2011) EEG study also revealed that no change either in performance or alpha power (i.e., no change in cortical activation) was observed in participants of the control group. This finding is also consistent with the present results for the control group where no substantial change either in performance or in the oxygenation level (and thus in cortical activation) was observed. Although the limited spatial resolution of EEG and fNIR do not allow to accurately identify which brain regions would implement such inhibitory function, our results are in accordance with a previous fMRI study that employed a similar adaptation task and suggested the involvement of the ventral prefrontal cortex during inhibition of competing internal models of visuomotor transformations (Shadmehr and Holcomb, 1999).

In addition to these EEG and fMRI studies, and although still relatively rare, the few fNIR learning studies provide a developing body of evidence that cortical hemodynamics change as a function of learning new motor skills and practice (Hatakenaka et al., 2007; Ikegami and Taga, 2008; Leff et al., 2008a,b; Morihiro et al., 2009; Gentili et al., 2010a,b). Particularly, in agreement with our results, a reduction of cortical hemodynamics was revealed in the prefrontal cortex throughout practice while performance was enhanced (Leff et al., 2008a,b; Ohuchida et al., 2009; Ayaz et al., 2012a). The authors suggested that such attenuation would reflect a refinement of the prefrontal regions activity involved in executive functioning and particularly those related to attentional processes as well as to working memory supervised by the updating function in order to encode new spatiotemporal arrangements. The observed changes in cortical hemodynamics during this adaptation task could also reflect changes in attentional processes that were previously identified as common practice effects in various studies examining procedural skills learning (e.g., Leff et al., 2008a,b, 2011; Ohuchida et al., 2009). Importantly, this change in cortical hemodynamics could provide a complementary explanation to the procedural skills attention processes that are related to inhibitory control. During early adaptation, the attentional resources are largely depleted by the task. This depletion of resources occurs in conjunction with the need to inhibit familiar responses, however when performance becomes more automatic during late adaptation the attentional resources are regained.

Also, it must be noted that the refinement of the cortical hemodynamics (HBO2, OXY) and kinematics have different time scales since they follow linear and non-linear (exponential) dynamics, respectively. Such time-scale discrepancies between cortical hemodynamic and kinematics were also observed in previous EEG studies that used the same task (Gentili et al., 2009, 2011; Rietschel, 2011). A possible explanation for the timescale discrepancies would be that, although performance strongly improves over a certain number of trials, the brain is still engaged in a substantial effort to perform the task successfully and thus a high degree of cortical activity is observed. Although, at some point, additional practice does not necessarily result in a substantial improvement of the behavioral performance, the additional practice definitely contributes to enhancing the automaticity of performance that translates into continuous refinement of the prefrontal activity and thus into a continuous reduction of the oxygenation level (Rietschel, 2011). With additional practice, we contend that the prefrontal hemodynamics would also reach an asymptotic level.

Nevertheless, it must be noted that a prefrontal asymptotic leveling response was not systematically observed in experiments that involved motor practice. For instance, during a rotor pursuit practice task, while a reduction in the activation of the presupplementary motor area was observed, no particular hemodynamic change was revealed in the prefrontal cortex (Hatakenaka et al., 2007). Such discrepancy may be due to differences in experimental paradigms and to the nature and/or the demands of the task. Contrary to our task, practice of the pursuit rotor task (Hatakenaka et al., 2007) required mainly refining existing motor commands without the need to inhibit prepotent motor plans that could interfere with task performance.

Therefore, the high magnitude of activation in the prefrontal regions during early learning would be primarily related to executive functioning and particularly to the updating function to appraise working memory and inform changes in attentional resources along with the inhibitory function to suppress prepotent motor responses of inappropriate actions. The role of such frontally mediated functions is reduced during late learning and thus leads to a smaller activation of these prefrontal regions (Ghahremani et al., 2010). Overall, our results confirm and extend those from previous studies employing various neuroimaging techniques (e.g., fMRI, PET, EEG, fNIR) as well as reinforces that idea that the frontal executive has not only a critical role for cognitive control involved in purely cognitive tasks (e.g., Stroop task, Collette et al., 2006), but also for cognitive-motor/sensorimotor learning challenges, contributing to bridging the gap between the cognitive and cognitivemotor/sensorimotor control fields. It must be noted that the current study employed a fNIR probe that only covered the forehead. Thus, although the use of this technique was guided by our hypothesis, additional research is needed by employing a whole head fNIR system in order to further examine the hemodynamic responses of other cortical regions during adaptation.

## **HEMODYNAMICS-BASED BRAIN BIOMARKERS FOR ECOLOGICALLY VALID COGNITIVE-MOTOR PERFORMANCE**

To our knowledge, the present study is the first fNIR investigation of a visuomotor adaptation task where the participants need to inhibit prepotent motor plans while performing multi-joint arm reaching movements from a seated position allowing certain latitude in term of mobility. The present study extends and confirms the notion that fNIR technology allows for recording and analyzing the neural activity during cognitive-motor performance and learning task in ecologically valid situations where individuals can be seated rather than constrained to a supine position as is the case when employing fMRI (Hatakenaka et al., 2007; Ikegami and Taga, 2008; Ayaz et al., 2009, 2011, 2012a,b,c; Morihiro et al., 2009; Gentili et al., 2010a,b). In addition, this type of signal may be more resilient to noise compared to EEG such as less susceptibility for artifacts from eye-movements, muscular activity, and surrounding electrical interferences (Coyle et al., 2004; Sweeney et al., 2012) and therefore also contribute to reinforce its suitability for applications in the field. From an applied perspective, cortical hemodynamics may play a significant role in a broad range of applications in the field of operational neurosciences. For instance, these hemodynamic-based biomarkers may be useful for monitoring brain activity during ecologically valid adaptive movements where upper limbs are involved in learning/re-learning a motor task or adapting to a new tool/environment in rehabilitation and/or a human factors context. Besides the critical advantage of fNIR to perform a task in an ecologically valid environment, another important advantage of fNIR over fMRI is that additional biomarkers can be derived from the information related to hemodynamic responses which include HBO2, OXY, and HBT and not just HB (Leff et al., 2011; Ayaz et al., 2012a). Although multiple hemodynamic markers can be derived from fNIR, Leff et al. (2011) noted that many fNIR studies only considered one hemodynamic marker to examine brain activation (Miyai et al., 2003; Suzuki et al., 2004; Takeda et al., 2007; Ohuchida et al., 2009). We are in agreement with reporting multiple hemodynamic markers in fNIR studies and thus, the current investigation assessed HBO2, HB, and HBT, as well as the OXY marker which reflected the level of total oxygenation.

However, it is also important to note that despite its numerous advantages, fNIR also has important limitations such as a limited temporal resolution compared to other techniques (e.g., EEG). This is an important limitation since such reduced temporal resolution does not allow investigators to examine separately the planning and execution phase as previously done with studies using EEG (Gentili et al., 2009, 2011). Another limitation of fNIR is its sensitivity to head orientation since this can change the blood flow and thus impact the fNIR signals irrespective of the task effects and of the multiple existing approaches developed to eliminate such artifacts (Boas et al., 2001; Zhang et al., 2009; Ayaz et al., 2010; Izzetoglu et al., 2010; Sweeney et al., 2012; Umeyama and Yamada, 2013). Also, the spatial resolution of spectroscopybased systems is limited in the optode screening depth to half of the distance between the light source and detector (Strangman et al., 2002). Therefore, both fNIR and EEG techniques appear to be complementary techniques that could be combined in order to provide multi-modal hemodynamic and electrical-based brain markers.

Interestingly, as described earlier, the results obtained in this fNIR study are consistent with those previously obtained employing EEG with exactly the same task (Gentili et al., 2011). Although no co-registration of fNIR and EEG was performed, the strict parallel of this experimental protocol and that employed by Gentili et al. (2011) contributes to reinforce the idea to combine these two neuroimaging approaches. One important advantage of combining these two neuroimaging technologies would be to have access to multiple markers derived from both electrical activity and hemodynamic responses that act at different temporal and spatial scales. This would be particularly helpful for investigating the brain dynamics and more generally for brain monitoring applications to accurately assess the level of cognitive-motor performance (Gentili et al., 2010a,b). Because the underlying physics principles of these multimodal technologies, which may be attributed to the fact that the light signal does not interfere with electrical or magnetic fields (Coyle et al., 2007), we posit that a combination of fNIR and EEG or fNIR and fMRI seem plausible. Thus, a system combining both EEG and fNIR technologies could be deployed in the field with the possible capabilities to provide different and complementary brain biomarkers that can be used to robustly investigate brain functioning in ecologically valid, naturalistic situations (Coyle et al., 2007; Roche-Labarbe et al., 2007; Gentili et al., 2010b).

## **CONCLUSION**

This was the first study to investigate adaptive arm reaching movement employing fNIR technology. Specifically, the findings revealed that throughout adaptation to a new visuomotor transformation, it was possible to derive fNIR-based hemodynamic markers in terms of oxygenation levels (HBO2 and OXY) in the prefrontal regions to assess the ongoing progression of the adaptation processes. Our findings are supported by previous EEG results obtained by employing exactly the same reaching task adaptation paradigm. The study confirms the previously proposed principle that the refinement of the cortical dynamics during adaptation translate into the quality of the motor performance. More precisely, the gradual reduction of oxygenation in the prefrontal regions observed throughout adaptation suggest a pronounced initial involvement of frontal executive processes such as inhibitory and updating functions that is progressively derecruited as the internal model of the new visuomotor transformation is gradually encoded and the task is mastered. The present findings contribute to expand our understanding of the role of frontal executive functioning beyond the purely cognitive field to the cognitive-motor/sensorimotor control field. Overall the observed changes in the cortical hemodynamics represent potential brain biomarkers (Georgopoulos et al., 2007; Gentili et al., 2010a,b, 2011) that could be combined with different and complementary EEG markers (Gentili et al., 2011) to assess the level of adaptive cognitive-motor performance.

## **ACKNOWLEDGMENTS**

Rodolphe J. Gentili was supported by La Fondation Motrice (Paris, France). Patricia A. Shewokis and Hasan Ayaz were partially supported by NSF HCC/IIS-1064871. Jose Contreras-Vidal was supported in part by NSF HCC/IIS-1064703. The authors would like to acknowledge the contributions of Dr. Bradley D. Hatfield for his extremely helpful discussion on the role of the prefrontal cortex in cognitive-motor behavior as well as Mr. Hyuk Oh and Mr. Cyrus Hadavi for their help regarding the data fitting and data collection.

## **REFERENCES**


near infrared-based brain computer interface system. *Found. Augment. Cogn. Neuroergonom. Oper. Neurosci.* 5638, 699–708. doi: 10.1007/978-3-642-02812-0\_79


*Neuroimage* 21, 936–945. doi: 10.1016/j.neuroimage.2003.10.037


brain activation associated with different forms of motor learning. *Brain Res.* 871, 127–145. doi: 10.1016/S0006-8993(00)02365-9


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 30 December 2012; accepted: 27 May 2013; published online: 04 July 2013.*

*Citation: Gentili RJ, Shewokis PA, Ayaz H and Contreras-Vidal JL (2013) Functional near-infrared spectroscopybased correlates of prefrontal cortical dynamics during a cognitive-motor executive adaptation task. Front. Hum. Neurosci. 7:277. doi: 10.3389/fnhum. 2013.00277*

*Copyright © 2013 Gentili, Shewokis, Ayaz and Contreras-Vidal. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

## Waiting for a hand: saccadic reaction time increases in proportion to hand reaction time when reaching under a visuomotor reversal

## *Irene T. Armstrong1, Melissa Judson2, Douglas P. Munoz 1,2,3, Roland S. Johansson4 and J. Randall Flanagan1,2,3\**

*<sup>1</sup> Centre for Neuroscience Studies, Queen's University, Kingston, ON, Canada*

*<sup>2</sup> Department of Psychology, Queen's University, Kingston, ON, Canada*

*<sup>3</sup> Department of Biomedical and Molecular Sciences, Queen's University, Kingston, ON, Canada*

*<sup>4</sup> Section for Physiology, Department of Integrative Medical Biology, Umeå University, Umeå, Sweden*

#### *Edited by:*

*Rachael D. Seidler, University of Michigan, USA*

#### *Reviewed by:*

*Shanqing Cai, Boston University, USA Matthew Heath, The University of Western Ontario, Canada*

#### *\*Correspondence:*

*J. Randall Flanagan, Department of Psychology, Queen's University, 62 Arch Street, Humphrey Hall Room 232, Kingston, ON, K7L 3N6, Canada e-mail: flanagan@post.queensu.ca*

Although eye movement onset typically precedes hand movement onset when reaching to targets presented in peripheral vision, arm motor commands appear to be issued at around the same time, and possibly in advance, of eye motor commands. A fundamental question, therefore, is whether eye movement initiation is linked or yoked to hand movement. We addressed this issue by having participants reach to targets after adapting to a visuomotor reversal (or 180◦ rotation) between the position of the unseen hand and the position of a cursor controlled by the hand. We asked whether this reversal, which we expected to increase hand reaction time (HRT), would also increase saccadic reaction time (SRT). As predicted, when moving the cursor to targets under the reversal, HRT increased in all participants. SRT also increased in all but one participant, even though the task for the eyes—shifting gaze to the target—was unaltered by the reversal of hand position feedback. Moreover, the effects of the reversal on SRT and HRT were positively correlated across participants; those who exhibited the greatest increases in HRT also showed the greatest increases in SRT. These results indicate that the mechanisms underlying the initiation of eye and hand movements are linked. In particular, the results suggest that the initiation of an eye movement to a manual target depends, at least in part, on the specification of hand movement.

#### **Keywords: eye-hand coordination, saccadic reaction time, reaching movements, motor adaptation, humans**

## **INTRODUCTION**

Hand movements to visual targets are typically accompanied by saccadic eye movements that bring gaze to the target ahead of the hand. When reaching to targets presented in peripheral vision, the eyes generally begin moving before the hand (Prablanc et al., 1979; Biguer et al., 1982; Jeannerod, 1988; Land et al., 1999; Johansson et al., 2001; but see Bekkering et al., 1995). However, much of the delay in hand movement onset, relative to eye movement onset, can be attributed to the greater inertia of the arm. Indeed, a recent study demonstrates that the motor commands underlying coordinated eye and hand movements appear to be issued in close temporal proximity and that commands for hand movement may even precede those for eye movement (Biguer et al., 1982; Gribble et al., 2002; Sailer et al., 2005). Given this timing, an important question is whether the mechanism underlying eye movement initiation is dependent on processes responsible for the planning and control of hand movement.

Several lines of evidence indicate that hand movement can influence saccadic initiation. For example, saccadic reaction time (SRT) is greater when eye movement is accompanied by hand movement compared to when the eyes move alone (Mather and Fisk, 1985; Bekkering et al., 1995) and SRT and hand reaction time (HRT) both increase when reaching to targets in contralateral versus ipsilateral space (Fisk and Goodale, 1985). In addition, people appear to be unable to shift their gaze away from the current reach target (toward a new gaze target), until the hand completes the reach (Neggers and Bekkering, 2000, 2001).

We investigated hand-eye coupling using a task in which participants moved a cursor, controlled by the unseen hand, to reach targets located at varying distances to the left or right of a central start position that also served as the initial fixation point. The targets and cursor were presented in the plane of hand motion. We sought to manipulate HRT by adapting participants to a visuomotor reversal (180◦ rotation) between the hand and the position of the cursor. Under the reversal, a hand movement to the right resulted in a cursor movement to the left and vice versa. We expected HRT to increase under this visuomotor reversal (Fitts and Deininger, 1954; Fernandez-Ruiz et al., 2011). Under the hypothesis that processes involved in the programming of hand movement influences saccade initiation, we predicted that the reversal would also result in an increase in SRT despite that fact that the task for the eyes—shifting gaze to the target—is ostensibly unchanged.

Hand movement may not only influence the initiation but also the execution of saccades. Recent studies have shown that saccadic velocity increases when saccades are accompanied by coordinated hand movements to the same target (Epelboim et al., 1995; Snyder et al., 2002) but not when the hand is directed to a second target located in the opposite direction of the saccadic target (Snyder et al., 2002). The latter result could arise because eye and hand movements are aimed in different directions or because they have different spatial goals. By examining eye only and eye plus hand movements under the visuomotor reversal, we can address this question. If hand movement increases the velocity of accompanying saccades under the visuomotor reversal, we could conclude that it does so because of a shared spatial goal independent of hand movement direction *per se*.

## **METHOD**

All procedures were approved by the Queen's University human research ethics board and were in compliance with the Helsinki declaration.

## **PARTICIPANTS**

Six university undergraduates (5 women and 1 man) participated in this study after giving informed consent. All were right handed and all had normal or corrected-to-normal vision. All participants completed two experiments.

## **APPARATUS**

Participants grasped the handle of a light-weight manipulandum (Phantom Haptic Interface 3.0, Sensable Technologies, Inc., Cambridge, MA) that measured the position of their dominant hand in three dimensions at 1000 Hz. The handle was constrained to move in a horizontal plane (see **Figure 1**). A visual projection system that prevented vision of the hand and arm was used to display a start marker, visual targets, and a cursor controlled by the hand (all 1 cm diameter circles) in a horizontal plane located at the top of the handle. The start marker was located 32 cm below and 33 cm in front of the left eye and was thus about 46 cm from the eye.

An infrared video-based eye-tracking device (RK-726PCI pupil/corneal tracking system, ISCAN Inc., Burlington, MA) recorded the gaze position of the left eye in the horizontal plane of the targets at 240 Hz. To obtain accurate recordings, the head was stabilized using a forehead rest and a small bite bar. To calibrate gaze, we first used ISCAN's 5 point calibration procedure and then performed an additional 25 point calibration (Johansson et al., 2001). In both cases, the calibration points (5 or 25) were projected onto the horizontal plane of the targets and distributed such that the outer rectangle formed by the points enclosed the locations of the targets used in the experiment. We calibrated a plane rather than just a line (alone which the targets were presented) so that we could measure any gaze errors in any direction in the plane. The spatial resolution of gaze in the horizontal plane, defined as the average standard deviation of all calibrations, was 0.31 cm. This translated into an error of 0.36◦ visual angle when gaze is located at the start position. Gaze was calibrated before the experiment began and the calibration was checked throughout the experiment. Additional calibrations were

**FIGURE 1 | Apparatus used to measure gaze and hand movements and to present visual feedback about targets and cursor position.** While seated, participants used their right hand to grasp the handle of a light-weight manipulandum that measured the position of the hand in three-dimensions. The handle of the manipulandum was supported by an "air-sled" that rode across a horizontal glass surface on a cushion of air. With this support, the hand was effectively constrained to move in a horizontal plane. A video projector (not shown), above and to the right of the participant, projected targets, and cursor onto a projection screen via a 45◦ mirror. Participants viewed the cursor targets in a semi-silvered mirror located halfway between the projection screen and the plane of hand movement and could not see their hand. Thus, the targets and cursor appeared in the same plane as the hand.

performed if necessary; however, a single calibration was usually adequate.

## **DESIGN AND PROCEDURE**

All participants completed two experiments. All completed Experiment 1 first and completed Experiment 2, on average, 2 weeks later.

## *Experiment 1*

A trial began when the centrally located start marker appeared on the display. Participants were required to fixate and, in trials involving hand movements, move the cursor to this marker. An eccentric target appeared once the participant's gaze and cursor were within 2 and 0.5 cm, respectively, of the start marker position for half a second. Participants were asked to move their gaze or, in hand movement trials, the cursor to the target as quickly and accurately as possible and then maintain gaze or the cursor at the target until it disappeared 2 s after target presentation. In trials in which participants were required to move the cursor to the target, no explicit instruction was given regarding eye movement; however, participants always shifted their gaze to the target and held their gaze at the target while the cursor remained at the target. The target appeared at one of three eccentricities (5, 10, and 15 cm; 6.2, 12.3, and 18.1◦ of visual angle) to the left and right of the start marker.

**Figure 2** shows the sequence of experimental conditions. The experiment began with a training period in which participants moved the cursor, and therefore their gaze as well, to the presented target on each trial. Each target location was presented six times in a randomized order yielding 36 training trials.

After training, participants performed test trials presented in blocks based on movement type. There were three movement types. In *eye only* trials participants were instructed to look at the target without a hand movement. In *eye* + *hand* trials participants were instructed to move the cursor (and hence the hand) to the target. These cursor movements were always accompanied by an eye movement that shifted gaze to the target. In *eye* + *anti*-*hand* trials participants were instructed to look at the target but move the cursor (and hence the hand) away from the target in the opposing direction. Four trials for each of the six target locations were randomly ordered within each block and four such blocks of 24 trials were completed for each of the movement types for a total of 12 blocks. This yielded 288 trials in total with 16 trials for each combination of target location and movement type.

Following the first test period, participants were adapted to a visuomotor reversal where the direction of cursor movement was rotated 180◦ from the direction of hand movement. Thus, to move the cursor to the right, participants had to move their hand to the left and vice versa. During this adaptation period, participants completed 40 *eye* + *hand* trials per target location in randomized order (240 trials in total). Previous studies have shown that most participants can adapt to visuomotor rotations within 240 trials (e.g., Krakauer et al., 1999; Wigmore et al., 2002; Caithness et al., 2004).

After the adaptation trials, and with the visuomotor reversal in effect, participants completed a second test phase. Only two movement types were included in this phase of the experiment: *eye only* trials and *eye* + *hand* trials where participants were instructed to move the cursor to the target (requiring a hand movement in the direction opposite the target). With this

**Frontiers in Human Neuroscience www.frontiersin.org** July 2013 | Volume 7 | Article 319 |

and the cursor controlled by the hand was veridical. In contrast, in the last two phases the cursor position was rotated 180◦ about the start position such that a rightward hand movement produced a leftward cursor movement. During training and adaptation, participants made only

gaze to the target (dashed box).

(Experiment 1) or randomly interleaved (Experiment 2). In the first test phase in Experiment 1, participants also completed a block of trials in which they had to move the cursor away from the target while shifting instruction, participants always shifted their gaze to the target. There were four blocks of 24 trials for each movement type and the eight blocks, in total, were performed in a randomized order. Within each block, there were four trials for each target location randomly interleaved within each block. This yielded 192 trials in total with 16 trials for each combination of target location and movement type.

## *Experiment 2*

The second experiment was similar in format to the first with the following major exception: Movement types were randomly interleaved over trials rather than blocked. The type of movement required on a trial was indicated by the color of the start position (red or blue) at each trial onset. For half the participants, blue indicated an *eye only* trial and red indicated and *eye* + *hand* trial; for the remaining participants, the color instructions were reversed.

Participants in Experiment 2 performed the same sequence of training, test, adaptation, and re-test trials used in Experiment 1 except *eye* + *anti*-*hand* trials were not included (see **Figure 2**). In the training period, they completed 12 *eye only* trials and 12 *eye* + *hand* trials (two replicates per target) with the 24 trials randomly interleaved. The test trials (both before and after adaptation) included 36 trials (six trials per target) for each movement type: *eye only* and *eye* + *hand*. The 72 trials were randomly interleaved. Participants completed 180 *eye* + *hand trials* during the adaptation period in which they moved by the cursor 30 times to each of the six targets presented in randomized order.

## **DATA ANALYSIS**

Hand and gaze positions in the horizontal plane of the targets were sampled at 1000 Hz. This involved over-sampling the gaze data (recorded by the ISCAN system at 240 Hz). The x (left–right) and y (anterior–posterior) hand and gaze positions were digitally smoothed using a low-pass, dual-pass, fourth-order Butterworth filter with cut-off frequencies of 14 and 25 Hz, respectively. The ISCAN system smoothed the gaze data on-line with a 10-point moving average. To compensate for the temporal delay produced by this averaging, we time advanced the gaze signal by 19 ms, equal to one over the sampling rate (240) multiplied by (10-1)/2. To determine the start and end of eye and hand movements, the x and y gaze and hand positions were differentiated with respect to time to obtain velocities in the horizontal plane. The slope of the resultant of these velocity signals was then computed. A saccade began when the gaze velocity slope exceeded 15 m/s/s and ended when the slope during the deceleration phase exceeded −15 m/s/s. Hand movement start and end times were determined in the same way using thresholds of 0.5 m/s/s and −0*.*5 m/s/s, respectively. For each gaze and hand movement, we determined movement amplitude and the peak resultant velocity, which for simplicity, we will refer to as peak velocity. We visually inspected all data scoring to ensure that this algorithm worked successfully.

For analysis, we removed trials in which the first saccade undershot or overshot the center of the target by more than 2 cm (2.5◦ visual angle). This resulted in the exclusion of less than 5% of all trials. The great majority of hand movements also landed within 2 cm of the target center, even in the *eye* + *anti*-*hand* condition in Experiment 1 in which both the hand and cursor moved away from the target. We also excluded trials in which participants made hand direction errors (i.e., when the hand started to movement in the incorrect direction for a given condition). In Experiment 1, hand direction errors occurred in 14% of the trials and were primarily observed in the *eye* + *anti*-*hand* trials and the *eye* + *hand* trials under the reversal. In Experiment 2, errors occurred in only 3.6% of the trials despite the increased uncertainty due to the random interleaving of movement conditions. The absence of *eye* + *anti*-*hand* trials in Experiment 2 presumably contributed to the lower error rate but practice effects (from Experiment 1) may also have contributed.

In order to compare saccadic velocities across conditions, it is important to control for possible differences in saccadic amplitude because saccadic velocity increases with amplitude (Bahill et al., 1975; Fuchs et al., 1985). Therefore, for each participant and for each combination of target direction, amplitude, movement type, reversal, and experiment, we determined the predicted peak saccadic velocity that would be expected if the participant made a perfectly accurate eye movement to the target. This involved fitting a linear regression line relating peak saccadic velocity to saccadic amplitude to the individual trial data. The slope and intercept were then used to find the predicted peak saccadic velocity corresponding to the amplitude of the target. Snyder et al. (2002) used a similar approach to assess differences in saccadic velocity across conditions. Because hand velocity also scales with movement amplitude, we computed predicted peak hand velocities using the same method.

Repeated measures ANOVAs, based on participant averages across trials, were used to assess the effects of movement conditions, target amplitude, and target direction on various parameters of the eye and hand movements including maximum velocity and reaction time. The alpha level was set at 0.05.

## **RESULTS**

**Figure 3** shows means and standard errors (based on participant averages) of SRT **(A,C)** and HRT **(B,D)** as a function of movement type, visuomotor mapping (normal versus reversed), and target distance for Experiment 1 **(A,B)** and Experiment 2 **(C,D)**. The figure only includes data from the two test phases, and not from the adaptation phase. Because the results for leftward and rightward targets were very similar (as revealed by preliminary analyses of SRT, HRT, and saccadic velocity), the data were collapsed across target direction. In the following analysis, we excluded *eye* + *anti*-*hand* trials because they were only included in the Experiment 1 and only in the pre-adaptation phase. The results for this movement type are described at the end of the Results. As expected, One-Way repeated measures ANOVA revealed that HRT (Mean ± SE: 322 ± 7 ms) was greater (*P <* 0*.*001) than SRT (209 ± 8 ms). We used separate repeated measures ANOVAs to examine the effects of our experimental manipulations on HRT and SRT.

## **HAND REACTION TIME**

A 2 × 2 × 3 repeated measures ANOVA was used to assess the effects of trial structure (i.e., blocked versus randomly interleaved *eye only* and *eye* + *hand* trials), visuomotor mapping, and target

distance on HRT. As predicted, HRT was longer (*P <* 0*.*001) under the visuomotor reversal (350 ± 9 ms) than under normal visual feedback conditions (293 ± 7 ms). The trial structure also strongly affected HRT (*P <* 0*.*001). HRT was about 57 ms longer when movement types were randomly interleaved (350 ± 8 ms) compared to when they were delivered in blocks (292 ± 8 ms). HRT did not differ across target distances (*P* = 0*.*10) and target distance did not interact with other factors.

## **SACCADIC REACTION TIME**

A 2 × 2 × 2 × 3 (movement type, trial structure, visuomotor mapping, target distance) repeated measures ANOVA was used to examine experimental effects on SRT. Note that this analysis of SRT includes a factor (i.e., movement type) not included in the analysis of HRT, because HRT could not be computed for *eye only* trials. Unlike HRT, SRT increased with target distance (189 ± 13, 215 ± 8, and 222 ± 5 ms for the 5, 10, and 15 cm targets, respectively; *P* = 0*.*011). The increase in SRT with distance was larger for *eye* + *hand* trials compared to *eye only* trials resulting in an interaction between target distance and movement type (*P* = 0*.*001). SRT was longer (*P* = 0*.*007) for *eye* + *hand* trials (220 ± 9 ms) than for *eye only* trials (198 ± 8 ms). There was no reliable main effect of trial structure on SRT.

Our main research question concerned the effect of the visuomotor reversal on SRT. We were primarily interested in *eye* + *hand* trials but also asked whether SRT in *eye only* trials would be affected by adaptation to the visuomotor reversal. Overall, SRT was longer (*P* = 0*.*017) when visual feedback was reversed (225 ± 10 ms) compared when visual feedback was veridical (192 ± 8 ms). However, there was an interaction between movement type and visuomotor mapping (*P* = 0*.*031) with the effect of the visuomotor reversal being stronger for *eye* + *hand* trials than *eye only* trials. Therefore, we carried out separate 2 × 2 (trial structure, visuomotor mapping) repeated measures ANOVAs for each movement type. For *eye* + *hand* trials, there was a main effect of visuomotor mapping where SRT was clearly delayed (*P* = 0*.*024) under reversed (241 ± 15 ms) compared to veridical (194 ± 8 ms) visual feedback. However, there was no effect of trial structure and no interaction between trial structure and visuomotor mapping. For *eye only* trials, a main effect of visuomotor mapping was also observed where SRT was longer (*P* = 0*.*039) under reversed (206 ± 9 ms) compared to veridical (190 ± 7 ms) visual feedback. There was no main effect of trial structure. However, the interaction between trial structure and visuomotor mapping was marginally significant (*P* = 0*.*064). Further analysis revealed a reliable simple main effect of visuomotor mapping when trials were interleaved (*p <* 0*.*05) but not when trials were blocked.

#### **RELATIONSHIP BETWEEN HAND AND SACCADIC REACTION TIMES**

As described above, in *eye* + *hand* trials the visuomotor reversal produced increases in both HRT, as expected, and SRT, as hypothesized. Moreover, the increases in HRT and SRT were roughly similar in magnitude. On average, HRT was 57 ms longer under the reversal whereas SRT, in the same *eye* + *hand* trials, was 50 ms longer under the reversal. To test whether the effects of the reversal on HRT and SRT were different, we carried out a 2 × 2 × 2 (effector, trial structure, visuomotor mapping) repeated measures ANOVA using *eye* + *hand* trials. No interaction between effector and visuomotor mapping was observed (*P* = 0.406) indicating that the reversal produced similar increases in HRT and SRT. No other two-way interactions were observed and there was no three-way interaction.

If the initiation of saccadic eye movements depends, in some way, on hand movement planning and control, then there should be a correlation, across participants, between the effects of the reversal on HRT and the effects of the reversal on SRT in the same *eye* + *hand* trials. **Figure 4** shows the relationship between *-* SRT and *-* HRT where  refers to the difference between preand post-adaptation reaction time and positive values indicate longer reaction times following adaptation. Although the effects of the visuomotor reversal on reaction time could, for some participants, be quite different for the two experiments, in both experiments the relationship between *-* SRT and *-* HRT was linear and strongly positive (*r*<sup>2</sup> = 0*.*88; *P* = 0*.*02 for Experiment 1 and *r*<sup>2</sup> = 0*.*97; *P* = 0.001 for Experiment 2). For Experiment 1, the intercept and slope were −46.9 ms and 1.7 and for Experiment 2 the intercept and slope −47.9 ms and 1.7. The fact that the slope, in both cases, is greater than one suggests that the relative effects

**FIGURE 4 | The relationship between** *-* **SRT and** *-* **HRT for each participant in Experiment 1 (filled symbols) and Experiment 2 (open symbols).**  represents the effect of reversal adaptation, i.e., the difference between pre- and post-adaptation reaction time when the eyes and hand moved concurrently. A positive value indicates the reaction time was longer after adaptation. Different symbols are used for each of the six participants. Separate regression lines are shown for each experiment. However, these two lines have similar slopes and intercepts.

of the reversal on HRT and SRT varied with the effect on HRT. That is, the increases in HRT and SRT, due to the reversal, were most similar when the reversal produced larger increases in HRT.

#### **PEAK SACCADIC AND HAND VELOCITIES**

**Figure 5** shows means and standard errors (based on participant averages) of saccadic **(A,C)** and hand **(B,D)** velocities (corrected for saccadic and hand movement amplitude, respectively, see Methods) as a function of target distance, movement type, and reversal for both Experiment 1 **(A,B)** and Experiment 2 **(C,D)**. As with reaction times, we collapsed across movement directions because the results for leftward and rightward targets were very similar. A 2 × 2 × 2 × 3 (trial structure, movement type, visuomotor mapping, target distance) repeated measures ANOVA revealed that saccadic velocity increased with target distance (*P <* 0*.*001) and was lower (*P* = 0.029) with reversed (265 ± 3◦/s) compared to veridical (270 ± 3◦/s) visual feedback. There was no main effect of trial structure or movement type but the two factors interacted (*P* = 0.014). Specifically, for blocked trials (Experiment 2), saccadic velocity was faster for *eye only* trials (269 ± 3◦/s) than *eye* + *handtrials* (267 ± 4◦/s) whereas, for randomly interleaved trials (Experiment 1), saccadic velocity was faster for *eye* + *hand* trials under blocked conditions but faster for the *eye* + *hand* trials (269 ± 4◦/s) than eye only trials (266 ± 3◦/s). The latter finding is consistent with the effect reported by Snyder et al. (2002) who randomly interleaved trials with and without hand movement.

A 2 × 2 × 3 (trial structure, movement type, target distance) repeated measures ANOVA confirmed that hand velocity increased with target distance (*P <* 0*.*001). Hand velocity was greater (*P <* 0*.*05) before adaptation to the visuomotor reversal (570 ± 52 cm/s) than after adaptation (493 ± 52 cm/s). Thus, although participants learned to make quite rapid hand movement under the reversal, hand speeds did not match those observed prior to the adaptation period. No effect of trial blocking was observed on hand velocity.

### **RELATIONSHIP BETWEEN SACCADIC REACTION TIME AND SACCADIC VELOCITY**

Because SRT and saccadic velocity varied across movement conditions, the question arises whether there is a link between them. To examine this question we computed the correlation between SRT and saccadic velocity for each participant and each combination target distance, movement type, reversal, and trial structure. For each of these the 24 combinations (3 × 2 × 2 × 2), we computed the mean correlation coefficient averaged across participants. Correlations significantly different than zero were found for only 2 of the 24 cases and both of these were small (*r*<sup>2</sup> *<* 0*.*198). Thus, we did not find evidence for a robust relationship between saccadic velocity and SRT. This finding is consistent with the results of Snyder et al. (2002) who found that the increase in saccadic velocity with a coordinated hand movement was independent of SRT.

#### **HAND AND SACCADIC REACTION TIMES DURING ADAPTATION**

**Figure 6** shows HRT and SRT (means and standard errors based on participant averages) as a function of successive 10 trial

blocks during the adaptation periods of Experiment 1 (filled circles) and Experiment 2 (open circles). The figure also shows, for each experiment, HRT and SRT for eye + hand trials during the pre-adaptation and the post-adaptation phases (means and standard errors based on participant averages). During the adaptation period of Experiment 1, experienced first by all participants, HRT decreased over the first 150 trials or so and then leveled off. At the start of the adaptation period of Experiment 2, HRT was reduced compared to the start of the same period in Experiment 1. HRT then decreased slightly before leveling off. Thus, as judged by HRT, participants appeared to retain learning of the visuomotor reversal across the successive experimental sessions. Because all participants completed Experiment 1 first, we cannot logically rule out the possibility that the reduced HRT at the start of the adaptation period in Experiment 2 (compared to Experiment 1) is due to some other factor other than learning across experiments. However, we would emphasize that participants only performed *eye* + *hand* movements during the adaptation phases of both Experiments and can think of no reason why adaptation of HRT would be different—apart from the learning or order effect. In any event, HRT at the end of the adaptation period was similar in the two experiments. This is consistent with the observation (see above) that the increase in HRT due to the reversal was similar in Experiments 1 and 2. Interestingly, the changes in HRT, within and across successive adaptation periods, are qualitatively similar to changes in direction error observed when participants adapt to a visuomotor rotation over successive sessions (e.g., Krakauer et al., 1999; Wigmore et al., 2002). Finally, in contrast to HRT, SRT did not appear to change appreciably during the adaptation period (**Figure 6**). We observed that early in the adaptation period, some participants occasionally kept their gaze at the start marker, presumably to watch which way the cursor would go, and only then shifted their gaze to the target. Although these trials were excluded from our analysis (due to excessive SRTs; see Methods), this gaze strategy resembles the gaze behavior observed when participants first learning a complex visuomotor transformation (Sailer et al., 2005).

## **HAND AND SACCADIC REACTION TIMES IN ANTI-HAND TRIALS**

Although we have not focused on *eye* + *anti*-*hand* trials, the results shown in **Figures 3A,B** indicate that HRT and SRT were prolonged in these trials even more than in *eye* + *hand* trials under the reversal. A One-Way repeated measures ANOVA, performed on the data from Experiment 1, confirmed that both HRT (*P* = 0.039) and SRT (*P* = 0.016) were greater in *eye* + *anti*-*hand* trials (HRT: 338 ± 10 ms; SRT: 274 ± 16 ms) than in *eye* + *hand* trials under the reversal (HRT: 318 ± 7 ms; SRT 231 ± 23 ms). This result provides additional evidence of a coupling between eye and hand movement initiation. The result also indicates that moving the hand away from the target is less disadvantageous when the cursor moves to the target than when it moves with the hand.

## **DISCUSSION**

As expected, we found that the visuomotor reversal produced a marked increase on HRT. This effect on HRT, which was consistent across experimental sessions, permits us to scrutinize our main research question—whether an increase in HRT under the reversal would be accompanied by an increase in SRT. Our results, clearly answering this question in the affirmative, provide strong support for the hypothesis that processes involved in the programming of hand movement in visually guided reaching influence the initiation of eye movements directed to the reach targets.

In trials involving coordinated eye and hand movement, the reversal increased SRT by an average of 50 ms, an effect comparable to the 57 ms increase in HRT caused by the reversal. Critically, the increase in SRT occurred despite the fact that the required eye movement was unaffected by the visuomotor reversal. That is, participants always directed their gaze to the target. The rough equivalence between increases in HRT and SRT is consistent with the results of Fisk and Goodale (1985) showing that both hand and eye movements are delayed by some 405–0 ms when reaching to targets in contralateral versus ipsilateral space. In other words, Fisk and Goodale found that SRT for a given target varied depending on whether the reach was performed by the ipsilateral or contralateral hand. Importantly, for coordinated eye and hand movements, we found a strong coupling, across participants, between the effects of the reversal on HRT and SRT. In general, the greater the increase in HRT caused by the reversal, the greater the increase in SRT. This finding is consistent with previous work showing a correlation between HRT and SRT (Gielen et al., 1984; Mather and Fisk, 1985; Frens and Erkelens, 1991; Bekkering et al., 1995; Neggers and Bekkering, 1999, 2002).

Our results suggest that in rapid visually guided reaching to targets presented in peripheral vision, saccade initiation depends on, or is yoked to, hand movement. The question remains as to why saccade initiation would be delayed when hand movement initiation is delayed. Several studies of eye-hand coordination have suggested that hand movement commands are specified in gaze-centered coordinates (Andersen and Buneo, 2002; Engel et al., 2002; Crawford et al., 2004) and that both visual targets and the hand are represented in gaze-centered frames of reference in the posterior parietal cortex (Batista et al., 1999; Buneo et al., 2002; Medendorp et al., 2003; Khan et al., 2005), premotor cortex (Mushiake et al., 1997) and the superior colliculus (Stuphorn et al., 2000). One reason why saccade initiation may be delayed when additional time is required to initiate hand movement is because a saccade lunched too early may disrupt or distort the internal representation of the target before the hand movement is specified (Henriques and Crawford, 2000). Although this representation could be updated quickly during or even prior to the saccade (Duhamel et al., 1992; Jordan and Hershberger, 1994; Colby et al., 1995), even a brief disruption may be undesirable in a manual reaction time task such as the one we examined. This notion may also explain why SRTs increase when eye movements are accompanied by hand movement compared to when they are made in isolation (Mather and Fisk, 1985; Bekkering et al., 1994, 1995). Assuming that hand movement planning takes longer than eye movement planning (a reasonable assumption given that the geometry and dynamics of the arm are more complex than those of the eye), a delay in saccadic initiation would be expected. Interestingly, Bekkering et al. (1994) have shown that whereas SRT is delayed when eye movement is accompanied by a hand movement to the same target, SRT is not delayed when the hand is required to make a button press response. Because button pressing does not require specification of target location, there would be no need for the saccade to "wait for the hand".

The delayed saccadic initiation observed under the visuomotor reversal may be compared to the gaze-locking mechanism reported by Neggers and Bekkering (2000, 2001) whereby participants, during rapid target pointing movements, failed to comply with the instruction to generate a saccade to a new visual target (presented during the reach) until the hand reaches the vicinity of the hand target. Neggers and Bekkering (2001) suggested that gaze is anchored on the reach target so that retinal and extraretinal gaze-related signals can be used to ensure pointing accuracy. Keeping gaze on target until the hand arrives may also maintain correlations between different sensory signals (e.g., visual, proprioceptive, tactile) and predicted sensory signals linked to goal achievement; correlations that would be used to uphold the sensorimotor mappings required for visually guided actions (Johansson et al., 2001). In contrast, we are suggesting that, in our task, gaze is "anchored" at the fixation point (until hand movement is specified) so as to ensure a stable reach target representation. Although the function of gaze may differ in the two situations, it is possible that similar pathways are involved in inhibiting gaze shifts based on hand movement signals.

The anchoring of gaze on the target during hand movement (Neggers and Bekkering, 2000, 2001) and the apparent anchoring of gaze prior to hand movement, observed in the present study, suggests that there must be a signal from brain circuits involved in hand movement planning and control to the circuits underlying saccade initiation. Neggers and Bekkering (2001) suggested that this linkage may involve the interaction between saccadic neurons in the superior colliculus (SC) and putative reach-related neurons in the same structure (Stuphorn et al., 2000) that are thought to receive projections from premotor and motor cortices (Fries, 1984, 1985; Werner et al., 1997a,b). However, there are also extensive interactions between gaze and hand movement related signals in parietal cortex (Colby et al., 1995; Crawford et al., 2004) and frontal cortex (Fujii et al., 2002).

The effect of the visuomotor reversal on SRT was not confined to coordinated eye and hand movements. When *eye* + *hand* and *eye only* trials were randomly interleaved (Experiment 2), the reversal led to an increase in SRT in *eye only* trials. In contrast, no such effect was observed when the different trial types were blocked (Experiment 1). The increase in SRT in randomly interleaved *eye only* trials can be explained if one assumes that hand movement are programmed in all trials, that the execution of the hand movement is actively inhibited in the *eye only* trials, and that the inhibition of the hand delays saccadic initiation. In contrast, when the two types of trials are delivered in separate blocks, participants presumably do not prepare hand movements in *eye only* trials and inhibition of hand execution is not required.

Snyder et al. (2002) showed that, in non-human primates, there is a small but reliable increase in saccadic velocity when simultaneously executed eye and hand movements are directed to the same target and not when the eye and hand movements is directed to different target located in opposite directions. We replicated the basic finding of Snyder and co-workers by showing that saccadic velocity increased for eye movement accompanied by a hand movement to the same target for trials with veridical visual feedback of hand movement in the experiment with randomly mixed *eye only* and *eye* + *hand* trials (Experiment 2). However, hand movement did not facilitate saccadic velocity when *eye only* and *eye* + *hand* trials were performed in blocks (Experiment 1). As suggested above, when these trial types are randomly mixed, hand movements may be actively inhibited during *eye only* trials. A spilling over of this putative hand movement inhibition to eye movement in the randomly mixed condition may account for the lower saccadic velocity in *eye only* trials compared to *eye* + *hand* trials. When the different trial types are delivered in blocks, hand movements need not be inhibited in *eye only* trials and hence no decrease in saccadic velocity is observed.

In previous studies showing facilitation of saccadic velocity by hand movement, the effect has been demonstrated under conditions in which there was spatial congruency of eye and hand movement directions as well as eye and hand targets (Epelboim et al., 1995; Snyder et al., 2002). We sought to determine whether spatial congruency of spatial targets alone could produce this phenomenon. To this end, we compared *eye only* and *eye* + *hand* trials performed under the visuomotor reversal in these trial types were randomly mixed. We found that saccadic velocity still tended to be greater in *eye* + *hand* trials but that the effect was marginally non-significant. Thus, whereas we can speculate that target congruency contributes to the hand effect on saccadic velocity, we cannot rule out a contribution of movement direction congruency.

Although it is well established that the coordination of eye and hand movements involves parietofrontal circuits, little is know about how these circuits would handle visuomotor transformations that alter the mapping between visual inputs and motor outputs. Barash (2003) suggested that "paradoxical" neurons, found in both the prefrontal (e.g., Funahashi et al., 1993) and parietal (e.g., Zhang and Barash, 2000, 2004) cortices, may play a key role in this regard. These neurons, which have been identified in the context of the memory-guided anti-saccade task, exhibit both visual and motor responses. However, what distinguishes them from other visual-motor neurons is that the visual and motor responses can be differentially classified based on temporal and spatial criteria. Thus, the motor response may be linked to the direction of the target whereas its visual response may be linked to the timing of target presentation. It is tempting to speculate that similar populations of neurons may play a part in the control of hand movement under altered visuomotor conditions such as those employed here.

In summary, we have provided strong evidence supporting the hypothesis that, in visually guided reaching, processes involved in hand movement programming affect the initiation of saccadic eye movements that are naturally directed to the reach targets. Our results also suggest that processes related to hand movement control can influence saccades in trials requiring eye movements alone (*eye only* trials) when such trials are randomly interleaved with trials requiring hand movement (*eye* + *hand* trials) but not when these trials types are performed in blocks. Thus, our findings point to a strong influence of task set on the control of eye movements.

## **ACKNOWLEDGMENTS**

This study was supported by the Canadian Institutes for Health Research (CIHR), the Canadian Foundation for Innovation, the Ontario Innovation Trust, the Swedish Research Council Project 08667, and the Strategic Research Program in Neuroscience at the Karolinska Institute.

## **REFERENCES**


updating of the representation of visual space in parietal cortex by intended eye movements. *Science* 255, 909–902. doi: 10.1126/science.1553535


anti-saccade task. *Nature* 365, 7537–7556. doi: 10.1038/365753a0


modulated by direction of gaze. *J. Neurophysiol.* 78, 5675–5671.


saccadic cells. *Exp. Brain Res.* 115, 2062–2016.


*Nature* 408, 9719–9775. doi: 10.1038/35048530

Zhang, M., and Barash, S. (2004). Persistent LIP activity in memory antisaccades: working memory for a sensorimotor transformation. *J. Neurophysiol.* 91, 14241–14441. doi: 10.1152/jn.00504.2003

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 15 March 2013; accepted: 10 June 2013; published online: 01 July 2013.*

*Citation: Armstrong IT, Judson M, Munoz DP, Johansson RS and Flanagan JR (2013) Waiting for a hand: saccadic reaction time increases in proportion to hand reaction time when reaching under a visuomotor reversal. Front.*

*Hum. Neurosci. 7:319. doi: 10.3389/ fnhum.2013.00319*

*Copyright © 2013 Armstrong, Judson, Munoz, Johansson and Flanagan. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

## Different mechanisms contributing to savings and anterograde interference are impaired in Parkinson's disease

## *Li-Ann Leow\*, Aymar de Rugy , Andrea M. Loftus and Geoff Hammond*

*School of Psychology, University of Western Australia, Crawley, WA, Australia*

#### *Edited by:*

*Rachael D. Seidler, University of Michigan, USA*

#### *Reviewed by:*

*Edwin M. Robertson, Harvard Medical School, USA Burak Erdeniz, University of Michigan, Ann Arbor, USA*

#### *\*Correspondence:*

*Li-Ann Leow, Department of Psychology, The Natural Sciences Centre, The Brain and Mind Institute, Western University, London, ON N6A 5B7, Canada. e-mail: liann.leow@gmail.com*

Reinforcement and use-dependent plasticity mechanisms have been proposed to be involved in both savings and anterograde interference in adaptation to a visuomotor rotation (cf. Huang et al., 2011). In Parkinson's disease (PD), dopamine dysfunction is known to impair reinforcement mechanisms, and could also affect use-dependent plasticity. Here, we assessed savings and anterograde interference in PD with an A1-B-A2 paradigm in which movement repetition was (1) favored by the use of a single-target, and (2) manipulated through the amount of initial training. PD patients and controls completed either limited or extended training in A1 where they adapted movement to a 30◦ counter-clockwise rotation of visual feedback of the movement trajectory, and then adapted to a 30◦ clockwise rotation in B. After subsequent washout, participants readapted to the first 30◦ counter-clockwise rotation in A2. Controls showed significant anterograde interference from A1 to B only after extended training, and significant A1-B-A2 savings after both limited and extended training. However, despite similar A1 adaptation to controls, PD patients showed neither anterograde interference nor savings. That extended training was necessary in controls to elicit anterograde interference but not savings suggests that savings and anterograde interference do not result from equal contributions of the same underlying mechanism(s). It is suggested that use-dependent plasticity mechanisms contributes to anterograde interference but not to savings, while reinforcement mechanisms contribute to both. As both savings and anterograde interference were impaired in PD, dopamine dysfunction in PD might impair both reinforcement and use-dependent plasticity mechanisms during adaptation to a visuomotor rotation.

#### **Keywords: visuomotor rotation, motor learning, motor adaptation, anterograde interference, savings, Parkinson's disease**

## **INTRODUCTION**

Motor adaptation is the process through which the motor system alters movements to maintain performance in response to perturbations or changes in the state of the effector and/or of the environment. These perturbations evoke discrepancies between the predicted motor outcome and the actual motor outcome, which are thought to drive the iterative updating of an internal model that predicts the consequences of motor commands (i.e., a forward model; Thoroughman and Shadmehr, 1999). However, this internal-model based account of motor adaptation cannot fully explain persistent effects of initial learning on subsequent performance after the motor output is returned to the original, unadapted state (Huang et al., 2011). Persistent effects of initial learning can be evident in savings, when initial learning enhances subsequent adaptation to a similar perturbation, and in anterograde interference, when initial learning impairs subsequent adaptation to an opposing perturbation. These effects could be explained by a two-process model which posits a fastlearning, fast-forgetting process that occurs by updating an internal model, as well as a slow-learning, slow-forgetting process that does not occur by updating an internal model (Huang et al., 2011). Two mechanisms have been suggested for this "model-free" slow process: reinforcement learning, where repeatedly pairing the adapted movement with a rewarding outcome (e.g., hitting the target) reinforces that movement such that there would be a subsequent bias toward reselecting that movement, and use-dependent plasticity, where repetition alone of a particular movement (i.e., independently of a reward associated with the adaptation) would bias subsequent movements toward the repeated movement (Huang et al., 2011).

Savings is thought to occur through reinforcement mechanisms (Huang et al., 2011). Consistent with this proposal, savings is impaired in Parkinson's disease (PD) (Marinelli et al., 2009; Bedard and Sanes, 2011; Leow et al., 2012), a neurological disorder characterized by dysfunctional dopamine neurotransmission and consequent reinforcement learning deficits (Frank et al., 2004; Shohamy et al., 2006; Rutledge et al., 2009). Despite unimpaired initial learning where the rate and extent of error reduction is indistinguishable from that of controls, substantial deficits in savings have been repeatedly observed on PD patients using various protocols (Marinelli et al., 2009; Bedard and Sanes, 2011; Leow et al., 2012). Deficient savings is evident within the same test session (Bedard and Sanes, 2011; Leow et al., 2012), between test sessions separated by a 24-h delay (Marinelli et al., 2009; Bedard and Sanes, 2011; Leow et al., 2012), and during single-target (Leow et al., 2012) and multiple-target adaptation (Marinelli et al., 2009; Bedard and Sanes, 2011). In healthy adults, A1-B-A2 savings (i.e., savings in A2 after adapting to a first perturbation in A1 followed by an opposing perturbation in B) is also evident after extended training in A1, but not after limited training in A1 (Krakauer et al., 2005). A reinforcement interpretation suggests that with limited training, reinforcing the adapted movement for the A1 perturbation and subsequently reinforcing the adapted movement for the opposing B perturbation forms two competing movement-reward associations, which inhibits reselection of the A1-adapted movement in A2, thus preventing savings (Krakauer, 2009; Huang et al., 2011). In contrast, extended training in A1 strengthens the association of the A1 adapted movement with reward, increasing the bias to reselect it in A2, thus evoking savings. Anterograde interference may similarly be interpreted in terms of reinforcement: reinforcing a first adapted movement might bias the selection of that particular movement in subsequent learning of an opposing perturbation and cause interference (Huang et al., 2011). If reinforcement mechanisms contribute to A1-B-A2 savings and anterograde interference, then reinforcement learning deficits in PD should impair both A1-B-A2 savings and anterograde interference.

The role of use-dependent plasticity in savings and anterograde interference is unclear. Although previous studies suggest that use-dependent plasticity is neither necessary nor sufficient for savings (Huang et al., 2011), it might contribute to anterograde interference, which is typically measured in B without washing out movement biases induced by movement repetition in A1 (Sing and Smith, 2010). Like reinforcement learning, usedependent plasticity is dopamine sensitive: the formation of use-dependent movement biases is accelerated by the dopamine precursor Levodopa in healthy adults (Floel, 2005; Floel et al., 2008), and is slowed by dopamine antagonists in schizophrenia patients (Daskalakis et al., 2008). While there is still no direct evidence that use-dependent plasticity is impaired in PD, it is likely to be affected by dysfunctional dopamine neurotransmission, and might thus impair anterograde interference in PD.

The present study examined A1-B-A2 savings and anterograde interference in PD patients and older adult controls. In A1, participants first adapted to a 30◦ counter-clockwise rotation of the visual feedback of the movement trajectory, with either limited (25 trials) or extended (80 trials) training. Subsequently in B, all participants completed a block of adaptation trials with a 30◦ clockwise rotation. After subsequent washout with veridical feedback trials, all participants re-adapted to the first 30◦ counterclockwise rotation in A2. As dopamine dysfunction in PD could affect both reinforcement and use-dependent mechanisms, it was hypothesized that PD patients would show both impaired savings and impaired anterograde interference.

## **METHODS**

## **PARTICIPANTS**

A total of 16 mild-to-moderate PD patients and 18 neurologically intact older adult controls who were naive to the experimental design were recruited from the Parkinson's Western Australia newsletter and local newspapers. This study was approved by the Human Research Ethics Committee at The University of Western Australia. All participants provided written informed consent. All participants were tested on their dominant hand, had normal or corrected-to-normal vision, and scored within the normal range (*>*24) on the Montreal Cognitive Assessment (Nasreddine, 2005). All PD patients were tested on-peak of their medication schedule.

The limited training condition was completed by seven PD patients (aged 59–78 years, 4 female) and nine older adult controls (aged 54–75 years, 5 female). All of these PD patients were on Levodopa (mean daily Levodopa dose: 408 ± 102 mg). Four of these PD patients were also on the dopamine agonist Pramipexole (mean daily dose 2.55 ± 0.67 mg). Disease duration ranged from 7 months to 8 years. PD patient severity rated according to the motor subscale of the Movement Disorders Society Sponsored Revised Unified PD Rating Scale (MDS-UPDRS) (Goetz et al., 2007) ranged from 7 to 30.

The extended training condition was completed by nine PD patients (aged 52–79 years, 3 female) and nine older adult controls (aged 59–77 years, 6 female). Eight of these PD patients were on Levodopa (mean daily Levodopa dose 472 ± 257), and four of these PD patients were also on the dopamine agonist Pramipexole (mean daily dose: 2.2 ± 0.9 mg). Disease duration ranged from 6 months to 9 years, and MDS-UPDRS motor subscale scores ranged from 10 to 44.

## **APPARATUS**

Participants were seated on a height-adjustable chair in front of a laptop computer placed ∼50 cm away from the participant along their midline. Participants held a digitizing pen (15.95 cm long, 1.4 cm wide, 17 g) on a WACOM Intuos 2 digitizing tablet (size: 30.48 cm × 30.48 cm, resolution ±0.025 mm). The pen's position on the tablet (XY coordinates) was sampled at 100 Hz and displayed on the computer monitor as a circular cursor with a 5 pixel radius (1.25 mm). Direct vision of the hand was prevented by placing the tablet and the hand directly beneath a stand, with the laptop placed atop the stand.

## **GENERAL EXPERIMENTAL PROCEDURE**

The experimental task required participants to move the onscreen cursor from a start circle to a target circle by moving the digitizing pen on the digitizing tablet. Participants were first instructed to move a cursor representing the pen's position into the start circle. After the cursor came within the start circle for 2 s, a single-target circle of radius 23 pixels (6.08 mm) appeared 75 mm at 45◦ from the target. This single-target was used throughout the task. A tone sounded immediately after the target circle appeared, signaling participants to move the cursor to the target. Participants were instructed to move the cursor from the start circle to the target circle as accurately and as quickly as possible, in a single, uncorrected movement. Visual feedback of the movement trajectory was shown on-screen in real-time, and remained on-screen for 1 s after movement completion.

## **EXPERIMENTAL DESIGN**

Prior to adaptation, all participants completed a minimum of 30 baseline trials with veridical feedback, until three out of four consecutive movements were made with directional error of less than or equal to 3◦ and movement time was less than 1000 ms. Once this criteria were met, the test phase commenced. At the beginning of the test phase, participants completed a first block (A1) of either 25 (limited training condition) or 80 adaptation trials (extended training condition) in which visual feedback was rotated 30◦ counter-clockwise relative to the start circle. To compensate for the rotation, participants had to move in the 30◦ clockwise direction relative to the original movement direction. Previous work shows that 66 trials (per target) in A1 was sufficient to result in A1-B-A2 savings (Krakauer et al., 2005), and thus 80 trials with a single-target in A1was thought to constitute sufficient overlearning to evoke A1-B-A2 savings in controls. Participants then completed a second block of 25 adaptation trials with an opposing 30◦ clockwise rotation of visual feedback (B), such that to completely compensate for the rotation, participants had to move in the 30◦ counter-clockwise direction. Participants subsequently deadapted with 15 washout trials with veridical feedback. Previous work indicates that 15 washout trials were sufficient for directional error to reduce to pre-perturbation levels (Leow et al., 2012). In the ensuing third adaptation block A2, participants completed another 25 adaptation trials with the 30◦ counter-clockwise rotation previously experienced in A1. Finally, participants completed a further 15 washout trials with veridical feedback.

## **DATA ANALYSIS**

Cartesian XY coordinates were recorded and used to plot movement trajectory. Directional error was scored at either (1) 100 ms into the movement after moving at least 5 mm (Bedard and Sanes, 2011) or (2) at 25% of movement trajectory, whichever came earlier. Directional error was calculated as the angular difference between this movement direction and an idealized movement direction starting from the start circle to the target circle. A negative value in directional error indicates that the onscreen movement trajectory was counter-clockwise to an ideal movement trajectory plotted from the start to the target, while a positive value denotes the opposite. To examine anterograde interference, it was necessary to compare negatively signed directional error in A to positively signed directional error in B. Thus, positively signed directional errors in B were converted to corresponding negatively signed values. A single-exponential function was fit to the group mean trial-by-trial directional error for each adaptation block for graphical depiction.

Savings and anterograde interference were quantified by examining block-to-block changes in percent adaptation calculated from the rapid error reduction phase (taken as Trials 2–15) of each block (Leow et al., 2012). The first trial of each block was not considered as there is no opportunity to correct error on the first trial. The method of evaluating block-to-block changes using percent adaptation in the rapid error reduction phase has been previously validated (Krakauer et al., 2005). Percent adaptation was computed with the formula: Percent adaptation = 100% × [1 − (Mean directional error/30◦)]. Mean directional error was calculated from the mean of directional error in Trials 2–15, as rapid error reduction occurred in Trials 2–15 in the current study. Mixed ANOVAs and paired *t*-tests were used to evaluate block-to-block changes in percent adaptation within each participant group. Where applicable, Bonferroni corrections were used to correct for violations of sphericity. Effect sizes were quantified using η<sup>2</sup> and Cohen's *d*. By convention, η<sup>2</sup> values were categorized as: 0.01∼ small, 0.06∼ medium, 0.14∼ large, and Cohen's *d*-values were categorized as: 0.20∼ small, 0.50∼ medium, 0.80∼ large. Block-to-block changes in percent adaptation were reported as means ± standard errors of the mean. A1-B-A2 savings was quantified by increased percent adaptation from A1 to A2. Anterograde interference was quantified by decreased percent adaptation from A1 to B.

It is noted that other studies quantify anterograde interference by comparing performance in B in a group that has completed A1 to performance in B of a control group that did not previously complete A1 (Cothros et al., 2006). However, the current method of quantifying anterograde interference by comparing adaptation performance in B with that in A1 has been shown to be a sensitive measure of anterograde interference (Sing and Smith, 2010).

## **RESULTS**

## **PD PATIENTS SHOW SIMILAR RATE AND EXTENT OF A1 ERROR REDUCTION**

**Figure 1** shows group mean trial-by-trial directional error in all adaptation phases in PD patients (red lines) and controls (black lines) for the limited (left panel) and the extended training condition (right panel). In A1, PD patients and controls appeared to reduce directional error at a similar rate in both the limited and extended training conditions. Mixed ANOVAs with betweensubjects factor Group (PD, controls) and within-subjects factor Trial (Trials 1–25) were run separately for the limited and the extended training condition. In both analyses, there was no significant main effect of Group, and no significant Group by Trial interaction. To evaluate if PD patients and controls differed in the extent of error reduction in A1, mixed ANOVAs with between-subjects factor Group (PD, controls) and withinsubjects factor (Trials 16–25) were run separately for the limited and extended training conditions. These trials were selected to estimate asymptotic directional error as little further error reduction occurred beyond Trial 16. In the limited training condition, there was no significant main effect of Group [*F(*1*,* <sup>14</sup>*)* = 1*.*33, *p* = 0*.*3, η<sup>2</sup> = 0*.*09], and no significant Group by Trial interaction [*F(*5*.*0*,* <sup>69</sup>*.*8*)* = 0*.*93, *p* = 0*.*5, η<sup>2</sup> = 0*.*07]. Similarly, in the extended training condition, there was no significant main effect of Group [*F(*1*,* <sup>16</sup>*)* = 0*.*24, *p* = 0*.*6, η<sup>2</sup> = 0*.*02], and no significant Group by Trial interaction [*F(*4*.*8*,* <sup>76</sup>*.*5*)* = 1*.*00, *p* = 0*.*4, η<sup>2</sup> = 0*.*06]. Hence PD patients and controls did not differ in the extent of adaptation in A1 in either the limited or the extended training condition. To evaluate if PD patients differed from controls in variability of directional error at asymptote in A1, trial-by-trial variability of directional error at asymptote in A1 was estimated using standard deviations calculated from Trials 16–25 of A1. Variability of directional error at asymptote did not differ significantly between PD patients and controls in either

the limited [*F(*1*,* <sup>14</sup>*)* = 2*.*06, *p* = 0*.*2, η<sup>2</sup> = 0*.*13] or the extended training condition [*F(*1*,* <sup>17</sup>*)* = 0*.*66, *p* = 0*.*4, η<sup>2</sup> = 0*.*04].

#### **IMPAIRED A1-B-A2 SAVINGS IN PD**

**Figure 1** also shows that in A2, mean directional error in PD patients appeared to decrease more slowly than in controls after both limited and extended training in A1. To facilitate comparison of savings, data from A1 and A2 are replotted in **Figure 2**. Controls reduced directional error more rapidly in A2 (open circles) than in A1 (closed circles) in both the limited (**Figure 2** top left panel) and the extended training condition (**Figure 2** top right panel), indicating A1-B-A2 savings. PD patients showed similar rates of error reduction in A1 and A2 in both the limited (**Figure 2** bottom left panel) and the extended training condition (**Figure 2** bottom right panel) indicating a lack of A1-B-A2 savings.

Percent adaptation averaged from Trials 2–15 of A1 (filled bars) and A2 (clear bars) are shown in **Figure 2** insets. To evaluate the effect of participant group and training on block-to-block changes in percent adaptation, a mixed-ANOVA with betweensubjects factors Group (PD, controls) and Training (Limited, Extended) and within-subjects factors Block (Block A1 and A2) was run on percent adaptation data. The Group by Block interaction [*F(*1*,* <sup>30</sup>*)* = 3*.*78, *p* = 0*.*06, η<sup>2</sup> = 0*.*09] suggests that controls and PD patients might have differed in the way percent adaptation changed from Block A1 to A2. *T*-tests showed that in the limited training condition, controls significantly increased percent adaptation from A1 to A2 [*t(*8*)* = 2*.*78, *p* = 0*.*02, *d* = 0*.*71, mean increase: 12.73 ± 4.58%], but PD patients did not [*t(*6*)* = 0*.*19, *p* = 0*.*8, *d* = 0*.*08, mean increase: 1.26 ± 6.60%]. Similarly in the extended training condition, controls significantly increased percent adaptation from A1 to A2 [*t(*8*)* = 2*.*43, *p* = 0*.*034, *d* = 0*.*40, mean increase: 6.98 ± 2.87%], but PD patients did not [*t(*8*)* = 1*.*11, *p* = 0*.*3, *d* = 0*.*21, mean increase: 3.94 ± 3.56%]. Hence while controls showed significant A1-B-A2 savings after both limited and extended training, PD patients did not show significant A1-B-A2 savings after either limited or extended training.

#### **IMPAIRED ANTEROGRADE INTERFERENCE IN PD**

**Figure 1** shows that PD patients reduced directional error more quickly than controls in B after extended training in A1, suggesting that PD patients showed less anterograde interference from A1 to B than controls. Mean trial-by-trial directional error of A1 and B are replotted in **Figure 3** to facilitate comparison of anterograde interference. Both PD patients (bottom panels) and controls (top panels) showed large directional error in the first trial of B of approximately twice the magnitude of directional error in the first trial of A1, thus reflecting the change in rotation from 30◦ counter-clockwise in A1 to 30◦ clockwise in B. After limited training in A1 (**Figure 3**, left panels), both controls (top panel), and PD patients (bottom panel) showed similar rates of error reduction in A1 and B, indicating little anterograde interference from A1 to B. After extended training in A1, however, controls showed greater error in B than in A1 (**Figure 3**, top right panel), indicating anterograde interference, whereas PD

**training condition (right panels).** A single-exponential function was fit to

**panels) for the limited training condition (left panels) and the extended**

patients did not (**Figure 3**, bottom right panel), indicating little anterograde interference. Anterograde interference was quantified as a reduction in percent adaptation averaged from Trials 2–15 of Block A1

and B, nd is shown in insets in **Figure 3**. These scores were subjected to mixed ANOVAs with between-subjects factors Group (Controls, PD) and Training (Limited, Extended) and within-subjects factors Block (A1, B). A significant Group by Block by Training interaction [*F(*1*,* <sup>30</sup>*)* = 4*.*67, *p* = 0*.*04, η<sup>2</sup> = 0*.*11] suggests that groups differed in block to block changes in percent adaptation depending on training condition. This Group by Block by Training interaction was followed up with mixed-ANOVAs with a between-subjects factor Group (PD, Controls) and a within-subjects factor Block (A1, B) run separately for the limited and the extended training conditions.

In the limited training condition, neither the main effect of Block [*F(*1*,* <sup>14</sup>*)* = 3*.*12, *p* = 0*.*09, η<sup>2</sup> = 0*.*15] nor the main effect of Group [*F(*1*,* <sup>14</sup>*)* = 0*.*53, *p* = 0*.*5, η<sup>2</sup> = 0*.*04] or their

directional error of Trials 2–15 for each adaptation block. Error bars represent standard errors of the mean.

interaction [*F(*1*,* <sup>14</sup>*)* = 0*.*05, *p* = 0*.*8, η<sup>2</sup> = 0*.*00] were significant. Both controls (**Figure 3** top left panel inset) and PD patients (**Figure 3** bottom left panel inset) showed little reduction in percent adaptation from A1 to B, suggesting a lack of anterograde interference. Percent adaptation did not decrease significantly from A1 to B in either the control group [*t(*8*)* = 0*.*09, *p* = 0*.*9, *d* = 0*.*02, mean reduction: 7.21 ± 5.89%], or the PD group [*t(*6*)* = 1*.*25, *p* = 0*.*3, *d* = 0*.*61, mean reduction: 9.25 ± 7.38%].

In the extended training condition, there was a significant Group by Block interaction [*F(*1*,* <sup>16</sup>*)* = 6*.*74, *p* = 0*.*01, η<sup>2</sup> = 0*.*24] which resulted from a decrease in percent adaptation from A1 to B in the control group (**Figure 3**, top right panel inset) but not in the PD group (**Figure 3**, bottom right panel inset). The decrease in percent adaptation from A1 to B was significant in the controls [*t(*8*)* = 2*.*93, *p* = 0*.*02, *d* = 1*.*36, mean decrease: 26.43 ± 9.13%], showing anterograde interference. The decrease in percent adaptation was not significant in the PD group [*t(*8*)* = 0*.*67, *p* = 0*.*5, *d* = 0*.*23, mean decrease: 7.35 ± 8.80%], showing a

lack of anterograde interference. Hence extended training in A1 evoked significant anterograde interference in controls but not in PD patients. The top right panel in **Figure 3** shows that, for controls after extended in A1, directional error was greater in B than A1 not only in Trials 2–15, but also in Trials 16–25 where little further error reduction occurred. This suggests anterograde interference was not limited to the rapid error reduction phase, but persisted through the asymptotic phase. To evaluate this possibility, asymptotic directional error was estimated by averaging Trials 16–25 of adaptation block A1 and B for each dataset. Mean asymptotic error was larger in B (−11*.*65 ± 10*.*08◦) than in A1 (−5.33 ± −4.28◦) in these trials and this difference approached significance [*t(*8*)* = 1*.*55, *p* = 0*.*07, one-tailed], with a moderate effect size (*d* = 0*.*72).

## **DISCUSSION**

The current study yielded two main findings. First, controls showed savings after both limited and extended training in A1, but showed anterograde interference after extended but not limited training in A1. Second, PD patients did not show anterograde interference or savings after either limited or extended training in A1. These results indicate that different mechanisms contribute to savings and anterograde interference, and that these mechanisms are both impaired in PD.

## **DIFFERENT MECHANISMS CONTRIBUTE TO ANTEROGRADE INTERFERENCE AND SAVINGS**

The current data show that savings and anterograde interference require different amounts of training. A limited training regime of 25 trials was sufficient to elicit savings, but not anterograde interference. That extended training was necessary to elicit anterograde interference but not savings shows that a two-state model comprising a fast and a slow process (Smith et al., 2006) cannot account for both savings and anterograde interference. If the same mechanism(s) in this model contributes to both savings and anterograde interference, the same amount of training should produce both savings and anterograde interference. We suggest that the model-free mechanisms of reinforcement learning and use-dependent plasticity have different training requirements and show different contributions to anterograde interference and savings: while limited training might be sufficient to engage reinforcement mechanisms responsible for savings, extended training might be necessary to additionally engage other mechanisms to elicit anterograde interference. Use-dependent plasticity is a plausible candidate, as it requires extended movement repetition (Classen et al., 1998). Extended training with a single-target in A1 entailed extended repetition of a single adapted movement, likely generating a use-dependent bias in the same direction, thus slowing error reduction in B.

Savings has been attributed to reinforcement mechanisms which associate the adapted movement with reward at initial learning such that the adapted movement is preferentially selected when relearning the same perturbation, speeding up adaptation (Huang et al., 2011). Limited training of 25 trials thus appears sufficient to engage this reinforcement mechanism. At first glance, this finding seems inconsistent with Krakauer et al. (2005) who found that 33 cycles in A1 (33 visits to each of 8 different targets) was insufficient to elicit A1-B-A2 savings. This could be due to the different number of targets used: in the current single-target design, a single adapted movement was reinforced, whereas in the multiple-target design of Krakauer et al. (2005), multiple movements to spatially separated targets were reinforced. Reinforcement mechanisms may be more effective in a single-target design where the adapted movement is repeated and reinforced in consecutive trials than in a multiple-target design.

Our results also indicate that savings is unlikely to result from use-dependent plasticity mechanisms, because repetitioninduced movement biases should have been eliminated by the washout trials prior to A2. This is consistent with previous findings showing that use-dependent plasticity alone is insufficient for savings. For instance, repeating a movement in the direction of an ideally adapted movement in the absence of a perturbation failed to elicit savings in subsequent adaptation (Huang et al., 2011). Furthermore, use-dependent plasticity might not be crucial to savings, as savings is not decreased when repetition of the fully adapted movement is reduced via a gradual adaptation schedule (Klassen et al., 2005), or even when repetition of the adapted movement is prevented completely (Huang et al., 2011).

It is not thought that use-dependent plasticity alone is sufficient to elicit anterograde interference. Findings of anterograde interference even with a 24-h delay between A1 and B (Cothros et al., 2006) appear inconsistent with the suggestion that usedependent plasticity alone is responsible for anterograde interference, as use-dependent movement biases typically decay after 60 min (Classen et al., 1998). Reinforcement mechanisms likely to contributes to anterograde interference: a rewarding outcome resulting from execution of the adapted movement reinforces that movement such that it is preferentially selected even when the perturbation in subsequent learning opposes that in initial learning, slowing the rate of subsequent learning (Huang et al., 2011).

It is noteworthy that anterograde interference in controls was not only evident in the error reduction phase, but also in the asymptotic phase, where directional error remained larger in B than in A1. This phenomenon has previously been observed (Tong and Flanagan, 2003; Cothros et al., 2006; Sing and Smith, 2010; Zach et al., 2012), but has received little attention. Larger asymptotic error in B cannot be completely attributed to use-dependent plasticity as it was also evident when repetition of movement to a single direction was prevented by a multiple-target design (Tong and Flanagan, 2003; Cothros et al., 2006; Zach et al., 2012). The persistence of the previously reinforced movement in A1 could additionally contribute to larger asymptotic error in B. This proposal is consistent with a recent finding that reinforcing an adapted movement without error feedback during asymptote increases persistence of that movement in subsequent error clamp trials (Shmuelof et al., 2012). We therefore suggest that both usedependent and reinforcement mechanisms elicited from extended training contribute to anterograde interference.

## **SAVINGS AND ANTEROGRADE INTERFERENCE ARE BOTH IMPAIRED IN PD**

Unlike controls, who showed savings after both limited and extended training, PD patients did not show A1-B-A2 savings after either limited or extended training. This is the first time that impaired A1-B-A2 savings in PD has been demonstrated, and this extends previous findings of impaired savings in PD with an A1 washout-A2 paradigm (Marinelli et al., 2009; Bedard and Sanes, 2011; Leow et al., 2012). Dopamine dysfunction and consequently deficient reinforcement mechanisms in PD may result in difficulty associating the adapted movement for A as well as the adapted movement for B with reward, such that in A2, the adapted movement for A is not preferentially selected, attenuating savings. On the other hand, the finding of impaired anterograde interference in PD is novel, and suggests that intact dopaminergic function is important to the use-dependent plasticity mechanisms thought to contribute to anterograde interference.

Dopaminergic treatment in PD patients often overdoses the relatively unaffected ventral striatum while treating the more affected dorsal striatum (for a review, see Cools, 2006). While impaired savings has been shown even in drug-naïve PD patients who are unaffected by dopamine medication overdose effects (Marinelli et al., 2009), the current findings of impaired anterograde interference in medicated PD patients could result from overdosing the less affected ventral striatum. Future studies examining anterograde interference in drug-naïve PD patients should clarify if dopamine denervation alone can impair anterograde interference.

It is important to bear in mind that reinforcement and usedependent mechanisms were not directly manipulated in this study. Instead, the dopamine dysfunction in PD that impairs reinforcement and use-dependent plasticity mechanisms was used to explore the role of these mechanisms in savings and interference. Our interpretation was built upon current knowledge of the role of reinforcement in adaptation learning (Diedrichsen et al., 2010; Huang et al., 2011; Izawa and Shadmehr, 2011; Pekny et al., 2011; Shmuelof et al., 2012), the role of dopamine in reinforcement (Frank, 2005) and use-dependent plasticity (Floel, 2005; Floel et al., 2008; Rösser et al., 2008). However, we cannot rule out the possibility that other mechanisms might additionally contribute to savings and interference.

## **POTENTIAL NEURAL MECHANISMS OF SAVINGS AND ANTEROGRADE INTERFERENCE**

The primary motor cortex (M1) has been shown to play an important role in savings and anterograde interference. While altering M1 activity during adaptation does not affect initial rate of adaptation learning, decreasing M1 excitability using repetitive transcranial magnetic stimulation selectively impaired both anterograde interference (Cothros et al., 2006) and savings (Riek et al., 2012), while increasing M1 excitability using transcranial direct current stimulation of M1 markedly increased retention of the learned visuomotor rotation (Galea et al., 2011). M1 is thought to encode a longer-term representation of motor adaptation, as repeating the adapted movement after attaining asymptote changes the preferred direction of a subgroup of M1 neurons to the adapted movement direction (Gandolfo et al., 2000; Li et al., 2001; Paz et al., 2003), and this change persists across test sessions spanning several days (Paz et al., 2003; Richardson et al., 2012), despite washout (Li et al., 2001; Paz et al., 2003) and subsequent adaptation to an opposing perturbation (Zach et al., 2012). However, it is unclear whether use-dependent and/or reinforcement mechanisms contribute to this phenomenon since there is at present no direct evidence supporting this suggestion. Future studies could elucidate if and how use-dependent plasticity and reinforcement mechanisms influence the longer-term representation of motor adaptation in M1 by systematically varying movement repetition and reward during adaptation while recording or disrupting M1 activity.

Midbrain dopaminergic signals to M1 may be important to both model-free slow mechanisms of reinforcement and usedependent plasticity and might thus contribute to the formation of a longer-term representation of adaptation learning in M1. M1 is connected to the midbrain through indirect and direct projections (for a review, see Luft and Schwarz, 2009).

## **REFERENCES**


Diedrichsen, J., White, O., Newman, D., and Lally, N. (2010). Usedependent and error-based learning of motor behaviors. *J. Neurosci.* 30, 5159–5166.


Dopamine reward signals influence M1 activity, as M1 excitability is modulated by the probability of reward in neurologically intact adults but not in unmedicated PD patients (Kapogiannis et al., 2008, 2011). Midbrain dopaminergic signals influence the LTPlike processes thought responsible for use-dependent plasticity in M1 (Floel et al., 2008), and dopamine denervation in PD impairs M1 LTP-like plasticity in PD (Morgante et al., 2006; Suppa et al., 2011; Kishore et al., 2012). Hence blunted midbrain dopaminergic signals in PD resulting in attenuated modulation of M1 activity, might impair both reinforcement and use-dependent mechanisms.

## **SUMMARY**

This study shows that in neurologically intact controls, extended training of 80 trials in A1 was necessary to elicit anterograde interference but not necessary to elicit A1-B-A2 savings, which was evident even after limited training of 25 trials in A1. We suggest that while reinforcement mechanisms evoked by limited training are sufficient to elicit A1-B-A2 savings, additional use-dependent plasticity mechanisms evoked by extended training is necessary to elicit anterograde interference. Furthermore, this study also shows that dopamine dysfunction in PD impairs both anterograde interference and A1-B-A2 savings, which suggests that dopamine is important to both reinforcement and use-dependent mechanisms activated during motor adaptation.

## **ACKNOWLEDGMENTS**

The authors would like to acknowledge the support of Parkinson's Western Australia, Parkinson's Centre (ParkC), and all study participants who generously donated their time.

of the cerebellum and motor cortex during adaptive learning: the motor cortex retains what the cerebellum learns. *Cereb. Cortex* 21, 1761–1770.


reward prediction errors during motor adaptation. *PLoS Comput. Biol.* 7:e1002012. doi: 10.1371/journal.pcbi.1002012


visuomotor rotation," in *Progress in Motor Control: A Multidisciplinary Perspective*, ed D. Sternad (New York, NY: Springer), 405–421.


foraging task. *J. Neurosci.* 29, 15104–15114.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 01 November 2012; accepted: 11 February 2013; published online: 27 February 2013.*

*Citation: Leow L-A, de Rugy A, Loftus AM and Hammond G (2013) Different mechanisms contributing to savings and anterograde interference are impaired in Parkinson's disease. Front. Hum. Neurosci. 7:55. doi: 10.3389/fnhum. 2013.00055*

*Copyright © 2013 Leow, de Rugy, Loftus and Hammond. This is an openaccess article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

## Structural correlates of skilled performance on a motor sequence task

## *Christopher J. Steele1,2\*, Jan Scholz 3, Gwenaëlle Douaud4, Heidi Johansen-Berg4 and Virginia B. Penhune1*

*<sup>1</sup> Department of Psychology, Concordia University, Montréal, QC, Canada*

*<sup>2</sup> Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany*

*<sup>3</sup> Mouse Imaging Centre, Hospital for Sick Children, Toronto, ON, Canada*

*<sup>4</sup> FMRIB Centre, University of Oxford, John Radcliffe Hospital, Headington, Oxford, UK*

### *Edited by:*

*Rachael D. Seidler, University of Michigan, USA*

#### *Reviewed by:*

*William C. Gaetz, The Children's Hospital of Philadelphia, USA David Vaillancourt, University of Florida, USA*

#### *\*Correspondence:*

*Christopher J. Steele, Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, Stephanstraße 1a, 04103 Leipzig, Germany. e-mail: steele@cbs.mpg.de*

The brain regions functionally engaged in motor sequence performance are well-established, but the structural characteristics of these regions and the fiber pathways involved have been less well studied. In addition, relatively few studies have combined multiple magnetic resonance imaging (MRI) and behavioral performance measures in the same sample. Therefore, the current study used diffusion tensor imaging (DTI), probabilistic tractography, and voxel-based morphometry (VBM) to determine the structural correlates of skilled motor performance. Further, we compared these findings with fMRI results in the same sample. We correlated final performance and rate of improvement measures on a temporal motor sequence task (TMST) with skeletonized fractional anisotropy (FA) and whole brain gray matter (GM) volume. Final synchronization performance was negatively correlated with FA in white matter (WM) underlying bilateral sensorimotor cortex—an effect that was mediated by a positive correlation with radial diffusivity. Multi-fiber tractography indicated that this region contained crossing fibers from the corticospinal tract (CST) and superior longitudinal fasciculus (SLF). The identified SLF pathway linked parietal and auditory cortical regions that have been shown to be functionally engaged in this task. Thus, we hypothesize that enhanced synchronization performance on this task may be related to greater fiber integrity of the SLF. Rate of improvement on synchronization was positively correlated with GM volume in cerebellar lobules HVI and V—regions that showed training-related decreases in activity in the same sample. Taken together, our results link individual differences in brain structure and function to motor sequence performance on the same task. Further, our study illustrates the utility of using multiple MR measures and analysis techniques to specify the interpretation of structural findings.

**Keywords: superior longitudinal fasciculus, individual differences, motor sequence performance, fractional anisotropy, diffusion tensor imaging, gray matter volume**

## **INTRODUCTION**

Even with identical practice, no two individuals are able to reach the same level of performance on a motor skill—nor do they follow the same trajectory of improvement as they learn. As neuroscientists, we assume that such individual differences are related to brain structure and function, but relatively few studies have linked performance variability to variability in the brain. Over the last 20 years, work with animals and functional neuroimaging studies in humans have identified the major brain regions involved in learning and performing motor skills (Hikosaka et al., 2002; Doyon and Benali, 2005; Ashe et al., 2006; Doyon et al., 2009; Penhune and Steele, 2012). Work from our lab and that of others has examined the relationship between individual differences in motor performance and brain function (Seidler et al., 2002; Penhune and Doyon, 2005; Grafton et al., 2008; Seidler and Noll, 2008; Orban et al., 2010; Steele and Penhune, 2010), but individual differences in structure have rarely been explored (Van Horn et al., 2008). However, recent studies have shown that individual differences in white matter (WM) supporting visuospatial attention (Tuch et al., 2005), motor cortical connectivity through the corpus callosum (Johansen-Berg et al., 2007), and connectivity between the motor regions of the cerebellum and motor cortex (Della-Maggiore et al., 2009; Tomassini et al., 2011) can be related to motor performance. Only one of those studies combined measures of WM and gray matter (GM) structure with functional MRI (Tomassini et al., 2011). Crucially, the authors found adjacent functional- and WM-performance correlations in the dorsal premotor region, and WM-performance correlations that were adjacent to co-localized functional- and GMcorrelations in cerebellar crus I–II and lobule HVI (Tomassini et al., 2011). These results underscore the importance of combining data from multiple methodologies to provide a more nuanced view of how brain structure and function are related to behavior. Following this model, the current study combines fMRI data from a study of motor sequence learning (Steele and Penhune, 2010) with diffusion tensor imaging (DTI—to assess WM integrity and perform probabilistic tractography) and voxelbased morphometry (VBM—to assess GM volume). The goal is to examine the relationship between individual differences in performance, brain function, and underlying structure at the end of training. Results from the fMRI experiment revealed learningand performance-related functional changes in motor, cerebellar, and parietal cortex. Based on this, we hypothesized that individual differences in WM and GM structure in these regions would be related to individual differences in motor sequence performance.

The majority of structural studies of individual differences find that better performance is associated with higher fractional anisotropy (FA) or greater GM volume (Golestani and Pallier, 2007; Golestani et al., 2007; Bermudez et al., 2009; Della-Maggiore et al., 2009; Jäncke et al., 2009; Scholz et al., 2009; Foster and Zatorre, 2010a; Tomassini et al., 2011). Individual differences in structural measures reflect differences in the microstructural organization of tissue related to task performance. Greater FA, an index of fiber integrity, may represent a greater ability for neurons in connected regions to communicate (Fields, 2005, 2008); greater GM volume may indicate greater cell density and synaptic connections that could support enhanced information processing. However, some studies have found that better performance is associated with lower FA values (Tuch et al., 2005; Taubert et al., 2010). These somewhat counter-intuitive findings have been interpreted as potentially resulting from fibers that cross the identified tract. Analyses that could assess the contribution of crossing fibers to FA values have typically not been conducted. FA values in WM are affected by factors such as axon myelination, diameter, and packing density (Beaulieu, 2002; Alexander et al., 2007), but may also be influenced by the presence of crossing fibers (Douaud et al., 2009, 2011; Jbabdi et al., 2010). FA values in one fiber population can be affected by the relative strength of a second crossing fiber population in the same region. One way to assess the contribution of crossing fibers to FA is by assessing the differential contributions axial and radial diffusivity. However, because axial and radial diffusivity are defined relative to the axis of greatest diffusivity, rather than to particular tracts, their interpretation is non-trivial in a complex human brain with multiple fiber crossings (Jbabdi et al., 2010; Douaud et al., 2011). Therefore, fiber tractography should also be used to determine the underlying tract orientation in addition to clarifying FA correlations and/or differences by analysing axial and radial diffusivity.

While most neuroimaging studies examine task performance after a single day of training, the current study examined skilled performance and improvement after 5 days of practice. By combining behavioral data with cross-sectional DTI and T1 structural images obtained on the final day, we can identify the structural correlates of skilled motor performance and compare them with the brain regions functionally responsible for learning and performance on the same task. The results of our previous fMRI experiment showed that though most motor-related regions decreased in activity with learning, there were performancerelated increases in specific regions including primary motor cortex, cerebellar lobule HVIIIa/VIIb, and superior parietal lobule (PLs) (Steele and Penhune, 2010). Therefore, in the current study we hypothesized that motor performance would be positively correlated with FA and GM volume in the regions functionally implicated in this task: motor cortex, cerebellum, and PLs. The secondary goal of this study was to more fully describe the contributions of axial and radial diffusivity to our FA findings and discuss them within the context of underlying tract organization defined by DTI tractography.

## **MATERIALS AND METHODS**

## **PARTICIPANTS**

The participants in this study were those tested in a previously published fMRI study (Steele and Penhune, 2010). The sample consisted of 13 participants (five female) between the ages of 18 and 27 (*M* = 22*.*4; *SD* = 2*.*9 years) who gave written informed consent. All were right handed [assessed using a handedness questionnaire adapted from Crovitz and Zener (1962)], neurologically normal, and had less than 3 years of musical experience [assessed using the Index of Musical Training and Experience; (Penhune et al., 1999)]. The experimental protocol was approved by the McGill University MNH/I Research Ethics Board and the Concordia University Human Research Ethics Committee.

## **TASK, STIMULI, AND PROCEDURE**

The temporal motor sequence task (TMST) used in this experiment requires participants to reproduce a temporally complex sequence of finger taps in synchrony with a visual stimulus. This task can be used to detect both long- and short-term changes in performance and brain activity (Penhune and Doyon, 2005; Savion-Lemieux and Penhune, 2005; Steele and Penhune, 2010). Performance on this task can be separated into two components: (1) accuracy—the order of short and long key-presses in the sequence; and (2) synchronization—the precise timing of movements. A detailed description of the task, stimuli, and procedure is presented in a previously published functional imaging study (Steele and Penhune, 2010). In brief, participants learned to press and release a mouse button in synchrony with the onset and offset of a visually-presented sequence of 10 elements [5 (S)hort— 300 ms; 5 (L)ong—600 ms; interstimulus interval—300 ms]. Each element was presented on screen for the specified duration as a large colored block—participants were instructed to press the mouse button when the block appeared and release when it disappeared. Five long and five short elements were arranged to create a sequence corresponding to a non-standard musical rhythm that is difficult to learn (the learning sequence—LRN: S L L S L S S L S L), a simple control sequence of five long followed by five short (L L L L L S S S S S), and a control sequence that was only observed. Four sequences of each condition were combined to create 40 s blocks. Four blocks of each condition were pseudorandomly arranged and interleaved with three 40 s blocks of rest to create a single training run. Participants were trained on the stimuli and taught LRN on the beginning of the first day and practised it for three runs of four blocks (16 trials) per day over 5 consecutive days, for a total of 240 trials. The current study focused on the relationship between the slope of improvement and final performance on LRN and cross-sectional structural imaging data acquired on the final day of training. T1 and diffusion-weighted images (DWI) were acquired with an eight-channel head coil in a Siemens Trio 3T MRI scanner on the final day of practice (T1–TR = 23 ms, TE = 7.4 ms, FOV = 256 mm, flip angle = 30◦, 1 × 1 × 2 mm; DWI–3 runs of 32 directions, TR = 5000 ms, TE = 104 ms, FOV = 220 mm, *b* = 1000 s/mm2, 1*.*7 × 1*.*7 × 5 mm, five *b* = 0 images per run).

## **DATA ANALYSIS**

### *Behavioral*

Motor sequence skill was assessed with two measures of performance for each practice run: percent correct (PCOR)—the percentage of correctly produced long and short key-presses within the sequence, a measure of the accurate production of elements within the sequence, and percent synchronization (PSYN)—a measure of the synchronization of key-press responses with visual stimuli. Means (M) and standard deviations (SD) for short and long elements were calculated based on individuals' performance on training trials at the beginning of each day. PCOR was then defined as the percentage of key-press responses that were initiated between 300 ms before the stimulus and the end of the stimulus and had key-press duration of less than M + 2SD (for short elements) or greater than M - 2 SD (for long elements). PSYN was defined as the sum of the absolute lag between the onset and offset of the stimulus and the onset and offset of the response, divided by the actual stimulus element duration (Steele and Penhune, 2010). As this calculation results in values that are smaller for better performance, scores were subtracted from 100 to obtain a score that increased with performance. A score of 100% on PCOR represents perfect knowledge of the ordering of elements within the sequence. A score of 100% on PSYN indicates that the key-press and release response exactly matched the onset and offset of the visual stimuli.

For the purposes of this study two measures were used: final performance—PCOR and PSYN for the last run of training on Day 5; and slope of improvement—*r*-value of the best fit linear regression line passing through participants' PCOR and PSYN run averages for the 15 runs of the experiment (PCORslp, PSYNslp). Both measures index performance potential (how proficient you can become and how quickly that level can be attained) that we reasoned may be represented within the structure of the brain (Tomassini et al., 2011). Final PCOR, PSYN, PCORslp, and PSYNslp were then correlated with imaging measures as described below.

#### *Diffusion imaging*

All imaging data were analysed using the FMRIB Software Library (FSL 4.1.5) (Smith et al., 2004). Diffusion images from three diffusion runs were concatenated, corrected for eddy current, and averaged. The FMRIB's Diffusion Toolbox (FDT) was used to create voxelwise maps of diffusion parameters including FA and the eigenvalues of the diffusion tensor. Images were then analysed using FSL's tract-based spatial statistics (TBSS) (Smith et al., 2006) which first requires images to be non-linearly aligned to the FMRIB58\_FA standard space template. The mean FA image was calculated and thinned to produce the study-specific FA skeleton—which represents the centers of all tracts common to all participants. FA data were then projected onto individual FA skeletons that were subsequently used in group permutationbased non-parametric statistical analyses. The mean FA skeleton was thresholded at FA *>* 0.25 to limit analyses to regions where major tracts existed in all individuals.

To determine the fiber regions that are important for skilled performance on this task, FA was correlated with final performance and slope measures for each participant with age as a covariate of no interest. Regions where FA was found to correlate significantly with performance were further investigated by assessing axial and radial diffusivity values. Whole-brain axial and radial diffusivity images were registered to the standard space using each individual's non-linear warp field (obtained from the FA image registration) and projected onto the mean FA skeleton. Regions identified in the FA correlational analysis were used to extract axial and radial diffusivity values from the same skeleton regions in all individuals. Partial correlation analyses, with age as a covariate of no interest, were then used to identify relationships between variables.

Probabilistic tractography was used to better characterize the directions of fiber tracts in regions of interest. This allows the interpretation of diffusion measures within the context of the underlying fiber tract organization. Significant voxels from the FA analysis were converted into a binary mask in each individual's 1 mm isotropic transformed diffusion space and then used to seed probabilistic tractography. Two different tractography analyses were conducted: one with target masks placed superiorly and inferiorly along the putative corticospinal tract (probable CST; inclusion planar regions at *z* = 54, 6, −11; exclusion at *x* = ±42 *y* = 43), the other with target masks placed laterally, anteriorly, and posteriorly to capture the association fibers/probable superior longitudinal fasciculus (probable SLF; inclusion planar regions at *x* = ±35, ±47 *y* = 42, −50; exclusion regions identical to CST inclusion). An additional exclusion plane was placed at *x* = 0 for both fiber populations. Both fiber directions were randomly sampled 10,000 times for each voxel in the seed mask. Each fiber population was averaged across participants and thresholded at 10% of the maximum particle number to obtain anatomically plausible tracts. This analysis produces delineations of the fiber tracts passing through the mask region, and can be used to visually differentiate the different fiber populations.

## *Voxel-based morphometry*

To assess individual differences in GM volume that were related to task performance, T1 images were analysed with the VBM tools in FSL (Douaud et al., 2007). Images were brain extracted (Smith, 2002), then segmented by tissue type to produce 3D GM partial volume images (Zhang et al., 2001). Each image was first aligned to the MNI152 template brain with an affine transform (Jenkinson et al., 2002). A study-specific GM template was generated by averaging all linearly aligned GM images. The group mean GM image was used as the target for non-linear registration of the original native space GM images using a b-spline representation of the registration warp field (Rueckert et al., 1999). The resulting non-linearly aligned GM images were smoothed with a Gaussian kernel of σ = 4 (∼9.4 mm) prior to statistical analyses. Whole-brain GM volume values were correlated with behavioral measures to identify cortical regions responsible for skilled performance and rate of improvement on this task.

Statistical analyses of FA and VBM data were conducted using FSL's randomize with 5000 permutations and thresholdfree cluster enhancement (Smith and Nichols, 2009). All analyses were controlled for the effects of age (entered as a covariate of no interest) and results were considered significant at *p <* 0*.*05, corrected for multiple comparisons. Analyses resulting in significant correlations were rerun while controlling for both age and gender to confirm that the unequal number of males and females in this sample did not bias the results.

## **RESULTS**

#### **BEHAVIORAL**

Subjects were trained to produce accurate and synchronized button presses in response to a 10 element visually presented motor sequence across 5 days of practice. Within-subjects ANOVAs revealed that PCOR, the more explicit sequence ordering measure, improved significantly over the course of the experiment [PCOR: *F(*4*,* <sup>48</sup>*)* = 9*.*80, *p <* 0*.*01, η<sup>2</sup> = 0*.*45] while PSYN, the more procedural sensorimotor integration measure, showed a statistical trend toward improvement on LRN (*p* = 0*.*10) (see Steele and Penhune, 2010 for further details). Final performance scores indicated that participants were able to perform well by the end of the experiment (PCOR: *M* = 93*.*13, *SD* = 7*.*06; PSYN: *M* = 70*.*16, *SD* = 10*.*84). Final PCOR performance was very high while final PSYN performance was lower and had greater variability across participants. **Figure 1** plots each individual's PCOR and PSYN scores from the final run of practice—illustrating the inter-individual variability in behavioral performance. PCORslp (*M* = 0*.*80, *SD* = 0*.*82) and PSYNslp (*M* = 0*.*5, *SD* = 0*.*96) measures indicated that participants, on average, improved over the course of the experiment. Individual final PCOR and PSYN, PCORslp, and PSYNslp values were used in subsequent correlational analyses to explore the relationship between motor performance and brain structure.

## **DIFFUSION MEASURES**

#### *FA: Behavioral correlation*

FA was correlated with behavior to identify individual differences in WM integrity related to performance. Final PSYN was found to correlate negatively with FA within bilateral CST, such that participants with greater final synchronization performance had lower FA in these clusters. In the left hemisphere, one cluster was located directly below the hand area of the primary motor cortex (M1) (Yousry et al., 1997) and the other was located more inferiorly in the CST and extended into the temporal/parietal junction. The significant clusters in the right hemisphere were located in approximately the same regions as those in the left, though they were smaller. **Figure 2** shows the regions of the FA skeleton negatively correlated with final PSYN performance overlaid with the regions where functional activity was positively correlated with PSYN performance (see **Table 1** for a list of peak voxels and their locations). Final PCOR showed a similar relationship with FA in the same region of the left hemisphere as final PSYN, though this relationship was not significant after correcting for multiple comparisons. These findings indicate that the relationship between FA in this region and the task is a general one, rather than specific to a particular hemisphere or component of performance. A subsequent analysis including gender as an additional covariate of no interest found the same pattern of results as reported above: the cluster with peak voxel at −27, −30, 16 remained significantly correlated with Final PSYN and the remaining clusters dropped below significance to *p* = 0*.*07, fully corrected for multiple comparisons. There were no statistically significant correlations between PCORslp/PSYNslp and FA.

## *Axial and radial diffusivity*

Axial and radial diffusivity values were extracted from the identified region to investigate their contributions to the negative correlation between FA and final PSYN. Axial diffusivity is the diffusivity along the axis of greatest diffusion and radial diffusivity is the mean of diffusivity in the two perpendicular axes. Radial diffusivity was found to positively correlate with final PSYN (*r* = 0*.*79, *p <* 0*.*005) while axial diffusivity did not (*p* = 0*.*18). In addition, FA and radial diffusivity were negatively correlated (*r* = −0*.*91, *p <* 0*.*001). **Figure 3** shows the partial correlation between radial diffusivity and final PSYN performance. The positive correlation between radial diffusivity and performance combined with the strong negative correlation between FA and radial diffusivity indicates that the observed negative relationship between FA and performance is driven by the positive relationship between radial diffusivity and performance.

## *Tractography*

Probabilistic tractography was used to identify the tracts crossing the region identified in the FA-behavioral analysis to more precisely interpret the results of the FA, axial, and radial diffusivity analyses. Based on location, we expected that the clusters identified in the behavioral regression analyses could contain fibers not only from the CST but also from the SLF. To test this possibility, we performed probabilistic tractography on two

combinations of target and exclusion masks designed to delineate ascending and descending (probable CST) from association fibers (probable SLF). Using the region where FA was found to significantly correlate with final PSYN as the seed (inclusion planar regions at *z* = 54, 6, −11; exclusion at *x* = ±42 *y* = 43), the ascending and descending tract extends superiorly to the sensorimotor cortex and inferiorly to the brainstem; this tract location is consistent with the CST (colored Red-Yellow in **Figure 4**) (Wakana et al., 2004) and the cortical target of its trajectory corresponds well with the motor cortical regions found to increase with improvements in PSYN (green in **Figure 4**) (Steele and Penhune, 2010). The tract identified with same seed and lateral, anterior, and posterior target masks (inclusion planar regions at *x* = ±35, ±47 *y* = 42, −50; exclusion regions identical to CST inclusion) is consistent with the course of the SLF: extending anteriorly to the frontal lobe along the external capsule, posteriorly across the superior part of the CST to the parietal lobe, and laterally to the auditory cortical regions of the temporal lobes (colored Blue-Lightblue in **Figure 4**) (Mori et al., 2002; Wakana et al., 2004; Makris et al., 2005). The tract termination points show remarkable agreement with the parietal and auditory cortical regions previously found to be involved in optimizing this component of the task (green in **Figure 5**) (Steele and Penhune, 2010). The excellent correspondence between the functionallydefined motor, parietal, and auditory cortical regions important for PSYN optimization and the tracts identified in this analysis underscore the importance of these regions and their connections in the optimization and performance of this task.

#### **VOXEL-BASED MORPHOMETRY AND PERFORMANCE**

To compliment the WM findings, we used VBM to examine regions of the GM that may contribute to the acquisition and performance of the TMST. PSYNslp was positively correlated with GM volume in right cerebellar lobules HVI and V (Schmahmann et al., 2000), two regions known to be specifically connected to the motor cortex (**Figure 5**, depicted in red to yellow) (Kelly and Strick, 2003; O'Reilly et al., 2010; Stoodley and Schmahmann, 2009). Refer to **Table 1** for a list of peak voxels and their locations. Importantly, these regions showed significant learning-related decreases in blood-oxygenation-level-dependent (BOLD) signal between Day 2 and Day 5 in the functional study with the same participants (**Figure 5**, depicted in green) (Steele and Penhune, 2010, supplementary materials). An additional analysis including age and gender as covariates of no interest found the same two regions to be significantly correlated with PSYNslp, though at a reduced spatial extent. There were no significant correlations with any of the other measures.

## **DISCUSSION**

The current study examined the relationship between individual differences in the ability to perform a motor task and structural brain measures collected on the final day of practice. Importantly, we compared these findings with the results of a previous functional brain imaging study in the same sample. Behavioral regression analyses found that better final synchronization performance was negatively correlated with FA bilaterally in fiber tracts underlying sensorimotor cortex, such that participants with lower FA

**Table 1 | Coordinates and peak** *t***-statistics for significant correlations.**


**FIGURE 2 | Relationship between final synchronization performance and FA in WM underlying motor cortex in the context of regions functionally involved in this task.** This negative correlation suggests that those with greater performance have lower FA in these regions that correspond well to areas functionally implicated in the task. Red-Yellow: correlation between FA

and final synchronization (*p <* 0*.*05, corrected for multiple comparisons); Green: functional correlation between synchronization performance and BOLD across the 5 days of the experiment (*p <* 0*.*001, cluster corrected); Yellow: mean FA skeleton (FA *>* 0.25). Significant voxels in the FA skeleton were thickened and overlaid on the ICBM 152 T1 for display.

showed better final performance. The direction of this relationship may appear counter-intuitive. However, radial diffusivity in this region was positively correlated with performance and multifiber tractography revealed that this region is an area of CST and SLF crossing fibers—meaning that the interpretation of FA in which bigger is better may not always apply. Functional imaging results with the same sample found positive relationships with

**FIGURE 3 | Relationship between radial diffusivity and final performance on PSYN.** This plot depicts the partial correlation between final PSYN and radial diffusivity extracted from the significant correlation with FA in bilateral sensorimotor cortex (**Figure 2**) after the effects of age have been removed. Each value is a residual converted to a *z*-score, and represents individual scores. The dotted line represents the best fit linear regression line through the data (*r* = 0*.*79, *p <* 0*.*005).

synchronization performance in motor, parietal, and auditory cortical regions that correspond well with both identified tracts (Steele and Penhune, 2010). These findings raise the possibility that skilled performance on this task is associated with enhanced fiber integrity in the SLF. Enhanced fiber integrity in the SLF could result in reduced FA in regions where it crosses the CST. Additional VBM analyses revealed a positive relationship between rate of improvement and GM volume in right cerebellum that were co-localized with functional decreases observed in the fMRI data (Steele and Penhune, 2010), thus providing further evidence for the cerebellum's role in skilled motor performance.

where BOLD activity decreased across learning on this task. Red-Yellow: significant correlation between GM volume and rate of improvement (*p <* 0*.*05, corrected for multiple comparisons); Semi-Transparent Green: task-specific decrease in BOLD activity between Day 2 and Day 5 (*p <* 0*.*001, cluster corrected). Significant regions are overlaid on the ICBM 152 T1 average for display.

**FIGURE 4 | Mean probabilistic tractography results for each target region, originating from the performance-FA seed mask.** Red-Yellow: tract resulting from the ascending/descending waypoint masks; Blue-Lightblue: tracts resulting from the lateral waypoint masks; Green: functional correlation between synchronization performance and BOLD across the 5 days of the experiment (*p <* 0*.*001, cluster corrected). The delineation between the ascending/descending fibers of the CST and

the laterally projecting fibers can be clearly seen. Tractography was conducted on each individual, averaged, and thresholded at 10% of maximum for display. Lighter colors indicate higher particle count. The tractography seed mask contained all voxels in the skeleton that showed a significant negative correlation between FA and final PSYN performance in both hemispheres. Tracts have been overlaid on the ICBM 152 T1 average for display.

Better final performance on the TMST was related to lower FA in the CST and SLF inferior to bilateral sensorimotor cortex, and this effect was mediated by a positive correlation with radial diffusivity (**Figures 2**, **3**). Our tractography result confirmed that this region contained fibers from both the CST and SLF (Makris et al., 2005) (**Figure 4**). Although we were unable to detect tract-specific relationships—likely as a result of the non-isotropic voxel sizes used in the current study—we speculated that greater diffusivity along the course of the SLF in this region may be responsible for the observed positive correlation of performance with radial, rather than axial, diffusivity. Though increases/greater radial diffusivity has been linked to dysmyelination in uniformly oriented fiber populations (Pierpaoli et al., 2001; Sun et al., 2008), the presence of crossing fibers in this region makes interpretation more difficult (Jbabdi et al., 2010; Douaud et al., 2011). Therefore, we have hypothesized that radial diffusivity in part reflects the fiber integrity of the SLF, where the principle diffusion direction is typically oriented anterior-posterior. The possibility that the negative correlation between FA and skilled performance could be driven by variation in the SLF is an attractive one. The SLF connects parietal and auditory cortical regions functionally implicated in performance of this task and in an fMRI study with the same participants (Penhune and Doyon, 2002, 2005; Steele and Penhune, 2010). In support of our hypothesis, a previous study found a *positive* relationship between FA in the SLF and motor sequence learning (Tomassini et al., 2011). This finding is in a more anterior region of the SLF (*y* = −10) that would be unlikely to be influenced by crossing fibers from the CST. Also possibly consistent with our findings, a recent study showed that *non-musicians* had *greater* FA than musicians in bilateral CST regions similar to those observed in our results (Imfeld et al., 2009). The authors speculate that their counter-intuitive findings are due to increased axonal permeability due to long-term sensorimotor training in musicians, but do not consider the potential effect of crossing fibers. The overlap between the regions functionally implicated in improvement on the TMST and the tractography results presented here provides further evidence for the importance of the SLF in skilled motor sequence performance.

The results of behavioral regression analyses with VBM GM values showed that individual differences in cerebellar lobules HVI and V were related to the rate of improvement of synchronization on the TMST. These regions overlap with those that showed learning-related decreases in activity in the fMRI data from the same subjects (**Figure 5**). Co-localization of behaviorally-relevant structural differences and functional changes identified with independent analyses provides further evidence for the role of the cerebellum in motor tasks—a finding that is compatible with a proposed role for the cerebellum in processing error-related feedback (Ohyama et al., 2003). Crucially, lobules HVI and V are structurally and functionally connected to motor cortex (Kelly and Strick, 2003; O'Reilly et al., 2010; Stoodley and Schmahmann, 2009), show performance-related changes in functional activity during motor tasks (Penhune and Steele, 2012), and form part of a network of regions responsible for the optimization of motor behavior (Ramnani, 2006).

Studies identifying relationships between cerebellar GM volume and performance are rare, with only two that use non-expert populations (Tomassini et al., 2011; Kühn et al., 2012). Our results are in agreement with those of Kühn et al. (2012), who found that GM volume in lobule VI was related to fine motor control, and directly support those of Tomassini et al. (2011) who also identified a relationship between motor sequence performance and GM volume in lobule VI. Though the design of the current study did not allow us to address learning-related changes in GM volume, previous work has identified increases in cerebellar synapse number and glial cell volume as a result of practice and learning (Kleim et al., 2002, 2007). We hypothesise that the observed performance-related individual differences in GM are in part due to differences arising from previous training and experience. Thus, greater cell or synaptic density in the cerebellum may support enhanced information processing ability (and thus a faster rate of behavioral improvement) that is related to decreasing functional activity as performance improves.

The causes of inter-individual variability in brain structure are not fully understood, but likely include pre-existing genetic contributions and contributions from learning and the environment. The design of our study was not able to disentangle these affects. FA is affected by WM properties including axon myelination, diameter, and packing density. Differences in these properties could lead to the individual differences in performance observed in our study through pre-existing differences or training-induced changes in axon conduction velocity and synaptic synchronization (Fields, 2005, 2008), or density of innervation. Greater fiber integrity along the SLF would be consistent with the idea, proposed by Fields, that greater myelination observed in relation to performance may underlie enhancements in synchronized activity between task-relevant regions (Fields, 2005, 2011). Similar to WM measures, individual differences in GM volume could be influenced by multiple factors such as neuronal and glial cell density, synaptic density, vascular architecture, and cortical thickness. Though the physiological basis for GM volume differences in humans has not been fully explained, previous work has established the feasibility of identifying individual differences in brain structure that are related to: timed finger tapping (Ullén et al., 2008), performance on musical tasks (Foster and Zatorre, 2010b), bimanual coordination (Johansen-Berg et al., 2007), and learning of foreign language sounds (Golestani and Pallier, 2007; Golestani et al., 2007). This study identified regions where performance is related to brain structure but its design does not allow us to conclude whether the observed effects are due to previous experience, training, or a combination of the two. Our study comprised a brief training regime (5 days) followed by structural data acquisition on the final day. A number of studies have identified structural changes after multiple weeks of training (Draganski et al., 2004; Boyke et al., 2008; Scholz et al., 2009; Taubert et al., 2010), but others have also reported changes with short-term training (Landi et al., 2011), TMS (May et al., 2007), and drug intervention (Tost et al., 2010). The current study provides a link between skilled performance and brain structure in regions known to be functionally involved with task performance. With only a single timepoint we cannot comment on how the regions that we have identified may change as a result of practice; however, given the overlap with previous fMRI results, structural changes in the SLF and cerebellar lobules HVI and V may occur with training on similar motor sequence tasks. Future studies employing longitudinal methods and longer periods of training could be used to address these questions.

We have identified individual differences in performance that are related to brain structures important for motor sequence performance. There was a negative relationship between FA and performance in a region of the CST-SLF fiber crossing that may reflect greater fiber integrity in the SLF of skilled performers—and is consistent with the idea of enhanced communication/synchronization between regions functionally important for this task. Two regions of the cerebellum (lobules HVI and V) where GM volume is important for the speed at which

## **REFERENCES**


Anatomically related grey and white matter abnormalities in adolescentonset schizophrenia. *Brain* 130, 2375–2386.


sequence skill is acquired were also identified. Our multimodal cross-sectional individual differences design also illustrates the importance of considering multiple structural measures (GM volume, FA, diffusivities, tractography) within the context of functional results to help provide a more global interpretation of the processes involved in skilled motor sequence performance.

## **ACKNOWLEDGMENTS**

We are grateful for financial support from the Natural Sciences Engineering and Research Council of Canada (Christopher J. Steele), Canadian Institutes of Health Research (Virginia B. Penhune), and Wellcome Trust (Heidi Johansen-Berg).


learning induces astrocytic hypertrophy in the cerebellar cortex. *Behav. Brain Res.* 178, 244–249.


nature of the relationship between performance and brain activity in motor sequence learning. *Neuroimage* 49, 694–702.


sequence learning. *J. Neurosci.* 30, 8332–8341.


variability in brain activity: a nuisance or an opportunity? *Brain Imaging Behav.* 2, 327–334.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 23 July 2012; accepted: 01 October 2012; published online: 19 October 2012.*

*Citation: Steele CJ, Scholz J, Douaud G, Johansen-Berg H and Penhune VB (2012) Structural correlates of skilled performance on a motor sequence task. Front. Hum. Neurosci. 6:289. doi: 10.3389/fnhum.2012.00289*

*Copyright © 2012 Steele, Scholz, Douaud, Johansen-Berg and Penhune. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

## *S. Sami\* and R. C. Miall*

*Behavioural Brain Sciences Centre, School of Psychology, University of Birmingham, Birmingham, UK*

#### *Edited by:*

*Rachael D. Seidler, University of Michigan, USA*

#### *Reviewed by:*

*Christopher J. Steele, Max Planck Institute for Human Cognitive and Brain Sciences, Germany David Vaillancourt, University of Florida, USA Shahabeddin Vahdat, Centre de Recherche de l'Institut Universitaire de Gériatrie de Montréal, Canada*

#### *\*Correspondence:*

*S. Sami, Behavioural Brain Sciences Centre, School of Psychology, University of Birmingham, Birmingham B15 2TT, UK. e-mail: s.sami@bham.ac.uk*

Recent studies have demonstrated that following learning tasks, changes in the resting state activity of the brain shape regional connections in functionally specific circuits. Here we expand on these findings by comparing changes induced in the resting state immediately following four motor tasks. Two groups of participants performed a visuo-motor joystick task with one group adapting to a transformed relationship between joystick and cursor. Two other groups were trained in either explicit or implicit procedural sequence learning. Resting state BOLD data were collected immediately before and after the tasks. We then used graph theory-based approaches that include statistical measures of functional integration and segregation to characterize changes in biologically plausible brain connectivity networks within each group. Our results demonstrate that motor learning reorganizes resting brain networks with an increase in local information transfer, as indicated by local efficiency measures that affect the brain's small world network architecture. This was particularly apparent when comparing two distinct forms of explicit motor learning: procedural learning and the joystick learning task. Both groups showed notable increases in local efficiency. However, a change in local efficiency in the inferior frontal and cerebellar regions also distinguishes between the two learning tasks. Additional graph analytic measures on the "non-learning" visuo-motor performance task revealed reversed topological patterns in comparison with the three learning tasks. These findings underscore the utility of graph-based network analysis as a novel means to compare both regional and global changes in functional brain connectivity in the resting state following motor learning tasks.

**Keywords: fMRI, resting state, graph analysis, complex networks, motor learning**

## **INTRODUCTION**

The combination of resting state neuroimaging methods with motor learning paradigms has ushered in a new era to the investigations of adult brain plasticity. Until recently neuroimaging paradigms examining motor learning were almost exclusively investigated during the execution of a learning task. This has generated a wealth of data showing rapid neural changes occurring during the execution of the learning task. Although the vast majority of these studies were investigated with fMRI, other techniques such as diffusion weighted imaging have shown that long term motor practice can induce structural changes in both gray (Maguire et al., 2000) and white matter (Scholz et al., 2009; Johansen-Berg, 2010; Tomassini et al., 2011). So, given that learning a new skill alters both functional and structural brain networks, one key unanswered question is how the rapid functional changes seen in task related activity contribute to sustain longer term changes in structure or function i.e., in essence the relationship between short-term and long term motor memory. While it has been previously speculated that resting state functional networks may hold at least a partial answer to this question (Miall and Robertson, 2006; Albert et al., 2009; Ma et al., 2010), it was not until recently that such a link has been provided (Taubert et al., 2011; Vahdat et al., 2011).

However, many questions about the very nature of functional resting states remain unanswered (Deco et al., 2011). Ever since Biswal and colleagues measured spontaneous activity over the motor cortex there has been a great interest in resting state networks (Biswal et al., 1995). "Resting state activity" usually measures endogenous and spontaneous rhythms and can be considered low frequency fluctuations in the BOLD signal. It has now been established that resting state-brain networks (RSNs) are highly reliable, showing reproducible traits over time, over subjects and across testing sessions, as well as having a strong association to task-related activation patterns (Smith et al., 2009). Recent studies have investigated the functional relevance of resting state networks by striving to link changes in RSNs with known functionally active task-related networks. One of the first studies was by Albert et al. (2009) investigating the effect of a visuomotor learning task on resting state BOLD. They found that the fronto-parietal and cerebellar networks are particularly engaged following learning, highlighting that functional changes seen in resting state immediately following motor training are representative of changes generally seen during motor learning task performance. Moreover, this comparative approach has given us an additional insight into RSNs, highlighting common characteristics between brains regions that share a common function (Smith et al., 2009). The comparisons between RSN and task-based network modulations has been largely achieved through the use of novel neuroimaging techniques like seed-based correlations and ICA, and have allowed the categorization of further functional sub-networks (Van den Heuvel and Hulshoff Pol, 2010).

Even though a number of key networks have been identified through ICA little is known about their network properties. More recently, graph theoretical network analysis has provided a novel approach to identify biologically plausible network architectures and this could provide insight into organizational rules as well as the processing properties of these networks following learning (Heitger et al., 2012).

Graph analysis of neuroimaging data is still a very new technique. Until now the most common use of graph-based analysis of resting fMRI data, has been to characterize normal functional connectivity at rest, and to examine differences in brain networks in healthy individuals compared to patients with neurological disorders (Liu et al., 2008; Lynall et al., 2010). Most recently a few studies have utilized graph analysis of neuroimaging data related to motor learning. Bassett et al. (2011) looked at dynamic changes following a simple motor learning task focusing on modular network changes only, while Heitger et al. (2012) looked at a more complete set of graph analytic measures investigating motor learning in a task based experiment. Given the paucity of work on resting state graph based analysis following motor learning, we were interested in whether these techniques can usefully complement more common ICA-based approaches. Here we utilize this graph analytic approach to compare immediate changes induced in the resting state following four motor tasks. Two groups of participants performed a visuo-motor target-tracking task with one group adapting to a transformed relationship between joystick and cursor. Two further groups were trained in either explicit or implicit procedural sequence learning. Based on our previous ICA results (Albert et al., 2009) and on a recent meta-analysis of the motor learning literature (Hardwick et al., 2012), we hypothesize that the visuo-motor tasks will show significantly stronger cerebellar activity while the procedural sequence-learning tasks will show more widespread cortical activity.

## **MATERIALS AND METHODS**

We used resting state BOLD signal data from four motor tasks, two variants of a sequence-learning task requiring rapid finger button presses in a learned sequence, and two variants of a targetreaching task using a joystick. Both sequence-tasks were designed to induce learning, one explicit and one implicit; one of the visuomotor tasks was a learning task, the other a non-learning control task. In each we compared resting activity before and after the learning period.

## **TASKS 1 AND 2: SEQUENCE LEARNING** *Participants*

Two groups of twelve healthy individuals participated in either an explicit (task 1) or implicit (task 2) version of the serial reaction time task (SRTT; Robertson, 2007). All participants were right handed, as confirmed by the Edinburgh handedness questionnaire. All participants (mean age 23*.*6 ± 5*.*2 years) gave written informed consent, and received either cash or credit for their participation. Participants recalling more than four items of the sequence were excluded from the implicit condition. All the participants were instructed to respond as quickly and as accurately as possible to the target location by a button press. Moreover, instructions to participants differed depending on which group they participated in. The implicit participants were unaware of the underlying sequence; while the explicit participants were aware of the existence of a sequence that was highlighted by a different color than the embedded random sequences. The two tasks were equalized in terms of the testing block size to avoid durational performance effects. The task lasted approximately 10 min for the explicit group while it was only slightly longer (∼by 2 min) for the implicit group. The local ethics committee at the University of Birmingham approved the experiment.

## *Procedure*

Participants were scanned with a 3T Philips Achieva MRI scanner as they completed a fixed set of tasks. First they viewed a dynamic point light display of human body movements, as a dummy task (Albert et al., 2009). They were then instructed to lie still with their eyes open while fixating on cross displayed in the middle of the screen during the initial rest scan which lasted for 10 min. An explicit or implicit procedural learning SRTT task was then issued for approximately 15 min, dependent upon individual reaction times. Participants responded with their right hand using a 4-button response box. The dummy task was then repeated for 5 min. Finally participants remained for a second 10 min rest scan conducted ∼5 min after the end of the SRTT task.

## **TASKS 3 AND 4: VISUO-MOTOR LEARNING**

Data from a previously reported study have been reanalyzed here. Details of the procedures are found in the original report (Albert et al., 2009). In summary: two groups of twelve individuals participated in one of two visuo-motor tracking tasks. Participants used an MR compatible joystick to control a cursor with their non-preferred left hand. For the test group (task 3) there was an angular displacement between target and cursor accumulating by 10◦ every min for 10 min, reaching a maximum of 90◦, while for the control group (task 4) there was no such displacement, and the movements of joystick and cursor were congruent. As in tasks 1 and 2 each of the visuo-motor tasks were interleaved between the two rest sessions and had the same dummy task preceding every rest period acquisition.

## *Imaging parameters*

For all 4 experiments, scans were conducted at the Birmingham University Imaging Centre (BUIC), University of Birmingham, Birmingham, UK; the experiments were approved by the University's local ethical panel, and all participants gave written informed consent. The MRI unit was a 3 Tesla Philips Achieva scanner (Koninklijke Philips Electronics N.V., Eindhoven, Netherlands). Each participants had a highresolution T1-weighted structural scan where the *TR* = 8*.*4 ms, *TE* = 3*.*8 ms, flip angle = 8◦ and FOV = 232 × 288 × 175 mm). In all functional scans the *TR* = 2800 ms, *TE* = 35 ms, and flip angle = 85◦. An 8 channel (SENSE factor 2) head coil was used. EPI volumes consisted of forty-nine 96 × 96 axial slices of 2*.*5 × 2*.*5 × 3 mm voxels. Using an FOV of 240 × 147 × 240 mm, the entire cerebral cortex and cerebellum were covered.

## *Image pre-processing*

All data were motion-corrected and normalized to a standard template using the statistical parametric mapping software (SPM8; Friston et al., 2006). Pre-processing included regression of motion parameters, nuisance signals, and global signal, followed by band-pass filtering at 0.01–0.1 Hz to isolate the low-frequency fluctuations characteristic of resting connectivity. Data was then parcellated into 116 regions using the Automatic Anatomical Labeling (AAL) atlas as implemented by IBASPM (Tzourio-Mazoyer et al., 2002; Alemán-Gómez et al., 2006). This resulted in an averaged fMRI time series for 116 regions (nodes) for each subject, which were used for subsequent graph network analysis (see **Figure 2**).

## *Network graph construction*

To create network graphs for every participant (see **Figure 1**), we used the Matlab-based Connectivity Decoding Toolkit (Richiardi et al., 2011). This software applies the outcome of the widely adopted IBASPM structural atlas to form a functional atlas by averaging the time series data for each region. It then performs a discrete wavelet transformation on the averaged time series data, filtering it into four separate frequency sub-bands. Here we adopted the use of a standard sub-band (0.06–0.1 Hz), which has been widely used for resting state analysis. In practice this sub-band has been shown to effectively filter out physiological noise in upper frequencies, and avoids measurement errors connected with lower frequencies (Fornito et al., 2010; Richiardi et al., 2011). For functional connectivity between the 116 parcellated regions, the Pearson correlation was computed between all pairs of node time series to generate a 116 × 116 correlation matrix (i.e., the adjacency matrix, *Aij*) for each subject (see **Figure 2**). The adjacency matrix represents a very densely connected network that makes it difficult to test the reliability of the connections. For simplicity the adjacency matrix is thresholded and further binarized to maintain only the most reliable connections (Rubinov and Sporns, 2011). In this study we adopted five thresholds of *r* = 0*.*3, 0.4, 0.5, 0.6, and 0.7.

**(B)** Application of parcellation scheme anatomical template image to each individual fMRI dataset creating a regional mean fMRI time series. **(C)** Wavelet analysis was used to bandpass filter the regional time series

matrices **(E)** The adjacency matrices were thresholded and binarized then these undirected graphs were used for the creation of **(F)** network topological metrics were further evaluated by statistical testing.

### *Network measures*

Once the binary graphs were constructed, the Brain Connectivity Toolbox (Rubinov and Sporns, 2010) was used to calculate the network measures. All network measures used thresholded and binarized graphs with the exception of the *strength* measure, which can only be applied to complete graphs. Although a large number of measures can be used (Rubinov and Sporns, 2010), in this study we used 10 selected measures of the thresholded binary graphs, choosing those most consistent across the literature, yet allowing us to capture the important features of the complex graphs. Network based measures used in this study include *degrees, hubs, characteristic path length, clustering coefficients, local and global efficiency, small worldness, betweenness centrality*, *modularity, and participation coefficients*.

## *Topological properties of the network*

*General measures of connectivity.* One of the fundamental measures widely used in graph analysis is that of connectivity degree. The degree *Di* of a node/region *i* is characterized as the total number of edges connecting that node/region to its neighbors (see **Figure 1**). An increase in level of global network interaction for a given region is signified by increase in degrees. Nodes with the highest degrees can also be signified as hubs. The degree *D* of a graph B is the mean of the degrees for the total number of nodes in the graph (Heitger et al., 2012).

$$D = \frac{1}{N} \sum\_{i \in B} D\_i$$

Another measure of global connectivity is *strength* (*Si*). For a given region this is defined and computed as the sum of weights *wij* (connection density) of all the connections of a region/node *i*, providing information on the total level of weighted pair-wise correlations of the region/node. In mathematical terms:

$$S\_i = \sum\_{j \in N} w\_{ij}$$

In turn, the total connection *strength* S of the graph was computed as the mean of *Si* for all nodes (Sporns, 2011; Heitger et al., 2012).

*Path length* provides information on global information transfer efficiency, as a shorter path would allow for the more rapid distribution of information between brain regions, with shorter paths entailing a greater prospect for integration (see **Figure 1**).

The mean shortest path length *Li* of a node *i* is:

$$L\_i = \frac{1}{N - 1} \sum\_{i \neq j \in B} L\_{i,j}$$

The characteristic path length *L* of a network is the mean of the shortest path length between the nodes (Sporns, 2011; Heitger et al., 2012).

Furthermore, *global efficiency* of a network is also associated with path length and generally defined as the mean of the inverse shortest path length (Latora and Marchiori, 2001).

*Region based measures of functional connectivity.* Densely interconnected groups of nodes are known as clusters within the network. These clusters can be defined on either a regional or network level (see **Figure 1**). The *clustering coefficient* of a node or region *Ci* is a ratio between the numbers of existing edges among the node's neighbors divided by the total number of all the regions possible edges:

$$C\_i = \frac{R\_i}{D\_i(D\_i - 1)/2}$$

*Ri* is the total number of connected pairs between all neighbors of node *i*. On a network level the clustering coefficient *C* is defined as the mean of the clustering coefficient of all nodes (Sporns et al., 2004; Sporns, 2011; Heitger et al., 2012).

*Local efficiency Ei*\_*loc* of a node *i* is linked to the clustering coefficient and is defined as:

$$E\_{i\\_loc} = \frac{1}{\nu\_i(\nu\_i - 1)} \sum\_{j,\ \nu \neq H\_i} \frac{1}{L\_{j,\nu}}$$

where the sub-graph*Hi* represents nodes that are connected to the node *i* and in which *Lj,<sup>v</sup>* is the minimal number of edges connecting node *j* and node *v* (similar to shortest path description) and *vi* (similar to *N*). *Ei*\_*loc* discloses how efficient the communication is between node *i* and its neighbors. The mean local efficiency of a graph, is merely the mean of the local efficiency of all the nodes in the graph (Sporns, 2011; Heitger et al., 2012).

*Small-world brain connectivity.* Small-world networks can be described as networks that have approximately the same characteristic path length as random networks, yet are notably more clustered than random networks, (Watts and Strogatz, 1998), Formally:

$$\gamma = C^{\text{real}} / C^{\text{rand}} > 1$$

$$\lambda = L^{\text{real}} / L^{\text{rand}} \approx 1$$

where the *L*real and *C*real are the characteristic path length and clustering coefficient of the real network, the *L*rand and *C*rand are the mean characteristic path length and clustering coefficient of an comparable random network, i.e., a random network that has similar graph characteristics in terms of size and edges as the real network (Maslow and Sneppen, 2002; Sporns et al., 2004). The *small worldness* coefficient is defined as a ratio σ = γ/λ, where values of sigma greater than 1 can be considered small world (Sanz-Arigita et al., 2010).

Measures founded on the notion of *centrality* are described as the most important nodes that contribute to the shortest paths inside a network and as a result act as central controls of information flow (Rubinov and Sporns, 2010). A commonly adopted *centrality* measure is *betweenness centrality Xi* of a node *i*, is defined as:

$$X\_i = \frac{1}{(N-1)(N-2)} \sum\_{\substack{f,\,j\in G\\f\neq i,\,f\neq i,\,j\neq i}} \frac{Y\_{\vec{f}\vec{\jmath}(i)}}{Y\_{\vec{f}\vec{\jmath}}}$$

in which *Yfj* is the total sum of shortest paths connecting nodes *f* and *j* and *Yfj(i)* is the total sum of shortest paths linking nodes *f* and *j* that go through node *i*.

The principle nodes often referred to as hubs can also be described as those nodes with the greatest *betweenness centrality* in a complex network (He et al., 2007; Shu et al., 2009; Rubinov and Sporns, 2010).

*Modularity.* A module is defined as a sub-network of highly inter-connected nodes that are comparatively sparsely linked to nodes in other modules (see **Figure 1**). Modularity in brain networks is associated with densely connected neighboring functional or anatomical cortical areas or communities, while connections between modules tend to be comparatively long distance (Meunier et al., 2010). The modularity detection algorithm we used was based on the Louvain method (Blondel et al., 2008) and visualized with a circular diagram. This is an efficient method for identifying modular structures. This is based on an algorithm that maximizes modular detection by iterative searching over the possible divisions of a network until modularity for a given module cannot be further improved.

The *modularity* measure: *Q* is originally defined as an unweighted and undirected network that is partitioned into subnetworks (Newman, 2004; Meunier et al., 2010)

$$Q = \frac{1}{2\alpha} \sum\_{Z \in P} \times \sum\_{j,\ D \in B\_l} \left( A\_{ij} - \frac{k\_i k\_j}{2\alpha} \right)^2$$

where *A* is the adjacency matrix of the network; α is the total number of edges; *ki* and *kj* are the degrees of node *i* and *j*. The index *Z* runs over the modules of the community or partition *P*. *Modularity* compares the number of links between the numbers of possible connections for all pairs of nodes within a subnetwork, against the number of such edges for a corresponding random graph.

Following the optimal partitioning of a network into modules, individual nodes can be ascribed to characterize their impact for within and between -modular transfer of information (Guimerà and Amaral, 2005; Meunier et al., 2010). The *participation coefficient* of a given node is the proportion of edges linking it to nodes in other modules.

$$\Omega\_{\hat{\jmath}} = 1 - \sum\_{W=1} \left( \frac{\beta\_{\hat{f}\hat{c}}}{\beta\_{\hat{\jmath}}} \right)^2$$

where β*jc* is the number of links of node *i* to nodes in module *W* and β*<sup>j</sup>* is the degree of node *i*. If all the edges of node *i* are distributed within their module, then β*jc* = β*<sup>j</sup>* and the participation coefficient *<sup>j</sup>* is 0. However, if all the connections of node *i* are distributed between the rest of the modules, *<sup>j</sup>* approaches one (Guimerà and Amaral, 2005).

### *Statistical analysis*

We tested for significant differences between the pre- and post-motor task RSN measurements using the non-parametric Wilcoxon statistical hypothesis test when comparing related groups, at identical thresholds for each network measure. Additionally, we corrected for multiple comparisons to identify within group (corrected for the 116 nodes) and between-group differences (corrected of the four groups) across all network measures (Zalesky et al., 2010).

## **RESULTS**

Behavioural results indicate that motor performance significantly improved across all three learning tasks [visuo-motor data was reported in Albert et al. (2009)]. Hence learning was induced in each case, but the task differences imply that we would expect different network changes underlying this change in performance. The performance in the non-learning visuo-motor control group did not change (Albert et al., 2009).

For the SRTT both groups showed procedural learning following the training exposure phase of the SRTT task. Comparing pre- and post-training for the implicit task, there was a significant reduction in reaction times (*p <* 0*.*05), while the explicit task showed a greater difference *(p <* 0*.*01*)*. There was also a marked difference between the performance of these two groups, with significantly reduced reaction times for the explicit group *(p* = 0*.*01*)*. This is an expected outcome due to the more rapid sequence acquisition of participants with explicit awareness (i.e., in the "explicit group").

## **GLOBAL CHANGES IN** *STRENGTH*

To define global changes in the resting state networks after learning we measured the correlation coefficient calculated on RSN-specific low frequency components of the BOLD signal. For each of 116 anatomically defined brain regions, we estimated the *strength* (Rubinov and Sporns, 2010) of its functional connectivity to the rest of the brain in each individual dataset. In all 3 of the motor learning tasks functional connectivity *strength* was significantly greater in the 2nd rest period post learning (*p <* 0*.*001; see **Figures 3A–C**). In contrast overall strength of connectivity was significantly reduced in the visuomotor performance task (*p <* 0*.*01; **Figure 3D**). *Strength* also varied widely over different brain regions, as indicated by overall ranking of strength across the 116 regions, and by the local differences in the amount of change in strength between the two rest sessions separated by learning (as indicated by the jagged pre-learning ordered data in blue, in **Figure 3**, compared to the red post-learning data).

### **LOCAL CHANGES IN** *STRENGTH*

For the sequence learning tasks (task 1 and 2), the global changes in *strength* also showed specific local network changes that were persistently higher in the frontal and visual regions for explicit SRTT task contrasted to the implicit SRTT task (*p <* 0*.*01). Given the different nature of visuo-motor rotation tasks (task 3 and 4) we expected different network responses; indeed, in the learning group (task 3) the most significantly affected nodes were the amygdala and the hippocampus (*p <* 0*.*01), while for the performance group, there were no significant effects in these brain areas.

To complement these results based on analysis of continuous *strength* measures of association between regions, we also measured the topological properties of the binary (unweighted and undirected) graphs derived by thresholding the Pearson's correlation coefficient of the individual functional connectivity matrices. At each threshold we compared the observed values of *degrees, correlation coefficients*, and *path length* in brain networks to their distributions in comparable random graphs with the same number of nodes and degree distributions.

## **GLOBAL CHANGES IN** *DEGREES*

All three learning groups showed a significant increase in *degrees* after learning, at all costs or threshold levels (see **Figures 4A–C**). For the explicit SRTT task 1, *degrees* were the most significantly increased over all costs (*p <* 0*.*0001) this also indicates a large effect size. This was followed by the visuo-motor learning group (task 3), while the implicit sequence group (task 2) showed the least significant increase across costs among the three learning conditions. In contrast, the visuo-motor performance group (task 4) showed a significant decrease across all 4 cost levels (*p <* 0*.*001; see **Figure 4D**).

**Figure 5** shows the broad scale *degree* distributions consistent with the existence of *hubs*. The figure also highlights the increase in hubs only in the learning groups (see **Figures 5A–C**) while

**FIGURE 3 | Group mean connectivity** *strength* **for each of the 116 regions before and following all four tasks, rank ordered by mean regional** *strength* **measured in rest 2 (pre-task rest 1 data are shown in**

**blue, post-task rest 2 data in red).** The solid line represents the mean while shaded area indicates SEM. **(A)** Explicit SRTT task 1, **(B)** Implicit SRTT task 2, **(C)** Visuo-motor learning task 3, and **(D)** Visuo-motor performance task 4.

learning (red).

the visuo-motor performance group (task 4) showed a decrease (see **Figure 5D**).

**and REST2 (red), (A) explicit SRTT (B) implicit SRTT task (C)**

## **LOCAL CHANGES IN** *DEGREES*

For task 1 the main nodes showing significant increases in *degrees* were in the frontal orbital cortex including inferior triangular middle occiptal gyrus (*p <* 0*.*01), which also showed significant increases in *strength*. However, unlike *strength*, for *degrees*, the right superior partietal gyrus also showed a significant increase in the implicit SRTT task (task 2). Although a similar overall pattern of global increase in degrees was seen in the visuo-motor learning task (task 3; *p <* 0*.*001) the most pronounced local effects were in entirely different regions. The significantly affected nodes include the right amygdala and left cerebellum (lobule III) (*p <* 0*.*01) while the visuo-motor adaptation performance group (task 4) showed a significant decrease over the left cerebellum and basal ganglia (*p <* 0*.*01).

## **GLOBAL CHANGES IN** *LOCAL EFFICIENCY*

The measure of *local efficiency* showed similar post-learning increases across all costs in the learning groups (tasks 1, 2, and 3; *p <* 0*.*001; see **Figures 6A–C**). The visuo-motor performance group (task 4) consistently showed a decrease in *local efficiency* in rest 2 across costs (**Figure 6D**). However, these decreases were non-significant (*p >* 0*.*05).

## **LOCAL CHANGES IN** *LOCAL EFFICIENCY*

Furthermore, topological brain network images highlight the fact that different anatomical networks are affected by the different tasks. The explicit SRTT group (task 1) showed significant increases in *local efficiency* (*p <* 0*.*05) in the frontal orbital regions and the right angular gyrus and the right medial temporal cortex while the implicit group (*p <* 0*.*05) (task 2) showed increases in the left precentral gyrus, SMA and the thalamus (see **Figures 7**, **8**). The opposing effects on *strength* seen between the visuo-motor learning and performance groups (task 3 and 4) were also evident for this measure of *local efficacy*: the learning group (task 3) revealed significant increase the right cerebellum (*p <* 0*.*05) (lobule 9) (see **Figure 9**), in the left caudate nucleus of the basal ganglia and the left hippocampus (*p <* 0*.*05), while the performance group (task 4) revealed significant decreases in the right inferior parietal (*p <* 0*.*01; see **Figure 10**).

## **GLOBAL CHANGES IN** *PATH LENGTH*

Another measure that showed significant decreases across all costs for all learning conditions was *path length* (*p <* 0*.*001), while the performance group (task 4) showed a significant increase (*p <* 0*.*001) across all thresholds except at the threshold *r* = 0*.*6 which showed a more subtle increase (*p <* 0*.*05).

## **LOCAL CHANGES IN** *PATH LENGTH*

Here the explicit SRTT group (task 1) showed significant and widespread regional decreases in *path length* in the orbital frontal regions, left inferior triangular gyrus, right post central gyrus, left middle occipital cortex, right basal ganglia, and right cerebellum crus II (*p <* 0*.*05). The implicit SRTT group (task 2) showed increased effects *path length* in the left hippocampus and the left parahippocampus (*p <* 0*.*05). The visuo-motor learning group (task 3) showed decreases in the precunus, the left amygdala, and the cerebellum while there was also a single increase in the

left inferior opercular frontal cortex (*p <* 0*.*05). The performance group (task 4) did not show any significant changes in *path length* at the node level despite a significant overall increase.

### **CHANGES IN** *SMALL WORLDNESS*

In order to calculate the small worldness coefficient, sigma, we also calculated the clustering coefficient for all the four tasks this produced near identical results to the local efficiency measure (see above). An additional measure that is required for the calculation of small worldness is path length (see above).

At a global level, all measures of functional networks expressed some key organizational properties consistently across both groups. All resting state networks including pre task networks showed small world characteristics. At each cost level in the smallworld regime, we sampled 1000 random graphs and estimated the mean and SD of each parameter so that we could then calculate. *Small worldness* did not show any significant change (*p >* 0*.*05; see **Figure 11**).

## **GLOBAL CHANGES IN** *BETWEENNESS CENTRALITY*

*Betweenness centrality* showed the opposite trend showing an overall global decrease in all the learning groups (task 1, 2, and 3) while showing a global increase in the performance group (task 4, *p <* 0*.*001).

## **LOCAL CHANGES IN** *BETWEENNESS CENTRALITY*

More specific significant nodal changes for *betweenness centrality* were seen in the explicit SRTT group (task 1) including decreases in the left precental gyrus, the right angular gyrus, left thalamus and right cerebellum crus I, while the implicit SRTT group (task 2) only showed decreases in the left post central gyrus and left caudate (*p <* 0*.*01). As for the visuomotor learning group (task 3), they also showed a general decrease in the right inferior triangular gyrus and right middle occipital gyrus, cerebellum crus II left (*p <* 0*.*01), while the performance group (task 4) showed overall increases for this measure in the right precental gyrus and right SMA and a decrease in the cerebullum (*p <* 0*.*01).

## **GLOBAL EFFICIENCY**

Overall *global efficiency* showed a non-significant increase for all the learning groups [task 1 (*p* = 0*.*23), task 2 (*p* = 0*.*53) and task 3 (*p* = 0*.*46)] over all costs while the performance group (task 4) showed a non-significant decrease (*p* = 0*.*42).

## **MODULARITY**

Another global measure is that of *modularity* in the form of Q value (see Materials and Methods). This showed opposite effects to *global efficiency*, with non-significant decreases for the learning groups and a non-significant increase for the visuo-motor performance group (*p >* 0*.*05). However, *modularity* exposed a different network distribution between the two SRTT tasks and the two visuo-motor tasks (see **Figures 12**–**15**). Additionally, **Figure 14** highlights the segregation of cerebellum shown as a separate cluster in the visuo-motor learning task (task 3).

#### **PARTICIPATION COEFFICIENT**

**Table 1** shows significant increases in *participation coefficient* for all three motor learning tasks, more specifically the explicit SRTT (task 1) showed widespread cortical increases over frontal, parietal, visual and sub-cortical regions while the implicit SRTT

(task 2) increased over sensory motor and sub-cortical regions (*p <* 0*.*05). The visuo motor learning group (task 3) also showed increases in the frontal cortex, precuneus, temporal gyrus, and multiple areas in the cerebellum, while the performance group showed no significant changes (*p >* 0*.*05).

## **DISCUSSION**

Our graph analytic results highlight regular patterns in the changes across four resting state functional connectivity data sets. In each case we tested for between pre- and post-motor task changes, showing comparable global topological patterns following the three motor learning tasks, although the different tasks affect different nodes and sub-networks. Moreover, the group performing a "non-learning" visuo-motor task revealed a different global topological pattern in comparison with the three learning groups. The current graph theoretic analysis also emphasizes that motor learning leads to rapid functional reorganization that is maintained during post-learning resting state activity as indicated by emergence of new functional network relationships as a result of training.

Our resting state BOLD results followed an analogous pattern, showing identical changes in all of the key measures aspects of the network topology in comparison to Heitger et al. (2012) graph theoretical results from task-based acquisitions.

Although behavioral differences existed between the tasks, performance differences due to task duration are unlikely to have

affected the outcome of the graph analysis results as all the tasks lasted ∼10 min.

Task differences showed the expected differential local network changes. Generally a large number of network measures showed that the explicit tasks i.e., task 1 and task 3 affected the prefrontal cortex. These effects were not seen in the implicit condition (task 2). This dissociation between implicit and explicit conditions has also been shown in task based imaging data (Destrebecqz et al., 2005; Fletcher et al., 2005; Ghilardi et al., 2009).

Additionally, our graph analytic RSN results support the hypothesis—based on a recent meta-analysis of task-based fMRI literature—that experience in visuo-motor tasks will show stronger cerebellar changes while the procedural sequencelearning tasks will show more widespread cerebral cortical activity (Hardwick et al., 2012).

This increase in cerebellar activity for the visuo-motor task is particularly distinct in terms of degrees, local efficiency, and participation coefficients highlighting to an increase in both short range local and long distance inter-modular processing. Furthermore, as the SRT tasks were performed with right hand and the visuo-motor were performed with the left a further distinction can be revealed due to handedness these tasks with a greater right hemispheric activation in the case of the visuomotor tasks.

An added benefit of using graph network measure compared with other standard techniques is that it highlights how different network elements play different roles within the network e.g., some nodes may provide improved local information transfer due to increased local computational demand while other nodes may play a greater role in the longer distance transfer of the information as indicated by *path length* and *betweenness centrality* or may in fact in some cases do both.

As expected the *strength* measure revealed regular enhancement between the pre- and post-exposure measurements for the learning groups, while demonstrating that network connectivity increased most significantly in the explicit SRTT task (see **Figure 3**). The collective significant increases observed across

represents significant decreases in local efficiency between the two rest conditions are highlighted in blue.

several graph analytic measures including global *strength*, *degrees*, *correlation coefficients,* and *local efficiency* are all indicative of increased local connectivity in the network. The increases in three of these measures were also observed by Heitger et al. (2012) in task-related BOLD, in participants following a 4-day bimanual coordination training regime with either visual or auditory feedback. Furthermore, the two graph analytic measures of *path length* and *betweenness centrality* confirmed the previously reported decreases following motor learning (Heitger et al., 2012). Reductions in these two measures indicate more direct communication pathways, with fewer intermediate nodes.

These decreased graph measures are likely to affect the global communication patterns, and in support of this, *global efficacy* showed a regular yet non-significant increase across the three learning experiments.

Small-world networks are characterized by a short average path length linking nodes together with a high clustering coefficient (Watts and Strogatz, 1998). This *small worldness* property has been repeatedly shown in both structural and functional neuroimaging over a broad range of spatial and temporal scales detected by a variety of modalities including EEG and MEG (Stam, 2004) and suggests that brain networks are characterized by dense local networks, and by long range connections

between these local clusters. However, it has been shown that small-world network properties break down in neuropsychiatric and epileptic patients, making it an important indicator of abnormalities. In our data following motor learning in healthy participants, *small world* properties were maintained but slightly reduced (see **Figure 4**). This indicates an uneven increase between local and global efficiency as small worldness can also be seen as a ratio between these efficacy measures. It also implies that learning only minimally affects the brains' normal operational boundaries.

Although *small-worldness* provides a useful network topological descriptor for both global and local levels of connectivity, it does not give any information about the sub-network organization, which is instead captured by the *modularity* of the network.

Modularity describes densely connected regions of a community or sub-networks within the same module but sparsely linked to regions in other modules (see **Figures 1** and **12**–**15**). Recent studies investigating resting-state BOLD data have found that *modularity* shows meaningful decompositions of the network into related functional sub-networks across a wide range of populations and experimental conditions (Fair et al., 2009; Meunier et al., 2010). Furthermore, *modularity* has been used to highlight associations between functional and structural subnetworks (Hagmann et al., 2008).

Due to these regional increases in density within the same module compared to random graphs of the same size and connection density, there was a positive *Q* value for *modularity*

(see Materials and Methods) for rest conditions. However, the decrease in these *Q* values following learning is likely to be due to the increase in the number of nodes participating in a greater number of modules, as indicated by the *participation coefficient*. Intra-modular connectivity therefore showed a significant increase in the number of connector nodes following motor learning (tasks 1, 2, and 3) in the fronto-parietal and hippocampal networks, while the performance group (task 4) showed very minor decreases. *Hub* measures for all three learning tasks were also significantly increased. Among the motor learning tasks the explicit serial reaction time task showed the greatest difference in *connectivity degree, betweeness centrality, mean path length,* and connection *strength*. This was followed by the visuo-motor adaptation task and finally the implicit serial reaction time task. The visuo-motor learning task (task 3) was difficult and very obvious to the participants. Hence it may have considerable explicit components. This suggests that this *hub* outcome could be partly due to the additional areas recruited by these two different explicit tasks (the sequence task 1 and the visuo-motor learning task 3). This is then analogous to the results of Heitger et al. (2012) who also showed that their visual feedback group had a more favorable outcome on all the above measures.

Greater *efficiency* and shorter *path length* of functional links between the nodes of a neural network will probably lead to more rapid transmission times and reduced noise degradation. This

**FIGURE 15 | Network modularity following the visuo-motor performance task, in the same format at Figure 12 and same Color scheme same as Figure 14.**

**Table 1 | Brain regions with increased participation coefficients.**


*Summary of significantly increased brain regions measured in terms of participation coefficients in all three learning tasks at a threshold of r* = *0.6. Significant increases are labeled by In\*.*

increased efficiency also implies that these strengthened functional connections form new "virtual" networks, reducing the need for the equivalent dedicated structural networks, and thus also avoiding the added incremental metabolic costs in terms of modifying physical connections. As such, this may underlie a general brain optimization strategy that may support consolidation of these motor memories, as the brain areas affected following immediate task based changes also play a role in consolidation (Ma et al., 2010; Bullmore and Sporns, 2012; Penhune and Steele, 2012). However, there is likely to be a trade off in longer-term motor learning to be negotiated between generality, efficiency and wiring cost in the optimal configuration of brain networks (Taubert et al., 2011).

## **CONCLUSIONS**

This work has used a number of graph theoretical methods to assess functional connectivity changes in resting state networks following motor learning. Our findings of changes in resting state activity following motor learning tasks are consistent with prior observations of changes in graph metrics that were based on task-related BOLD recordings. This adds further credence to the growing view that resting state network analysis can identify

## **REFERENCES**


sequence learning: interacting networks revealed by the process dissociation procedure. *Learn. Mem.* 12, 480–490.


changes in functional connections that are both task-relevant and likely to support longer-term consolidation of these motor memories. An additional finding is that we show for the first time using graph analysis a clear distinction between network changes in groups challenged with motor learning compared to a motor performance group.

Taken together with the other network measures like *local efficiency* these results imply that motor learning results in more direct information transfer across the relevant networks, while motor performance alone either decreased or maintained the status quo.

## **ACKNOWLEDGMENTS**

This work has been supported by the Wellcome Trust, WT087554. The data analysed here was collected from experiments performed in collaboration with Edwin Robertson, and we are grateful to him for his contribution to the design of the tasks.

cortex. *PLoS Biol.* 6:e159. doi: 10.1371/journal.pbio.0060159


connectivity during 4 weeks motor learning. *Brain Res.* 1318, 64–76.


of complex functional brain networks. *Neuroimage* 56, 2068–2079.


*Proc. Natl. Acad. Sci. U.S.A.* 106, 13040–13045.


brain networks. *Neuroimage* 53, 1197–1207.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 12 January 2013; accepted: 15 April 2013; published online: 15 May 2013.*

*Citation: Sami S and Miall RC (2013) Graph network analysis of immediate motor-learning induced changes in resting state BOLD. Front. Hum. Neurosci. 7:166. doi: 10.3389/fnhum.2013.00166 Copyright © 2013 Sami and Miall. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

## Cerebellar contributions to visuomotor adaptation and motor sequence learning: an ALE meta-analysis

#### *Jessica A. Bernard1 \* and Rachael D. Seidler 2,3,4*

*<sup>1</sup> Department of Neurology, University of Colorado Denver School of Medicine, Aurora, CO, USA*

*<sup>2</sup> Department of Psychology, University of Michigan, Ann Arbor, MI, USA*

*<sup>3</sup> School of Kinesiology, University of Michigan, Ann Arbor, MI, USA*

*<sup>4</sup> Neuroscience Graduate Program, University of Michigan, Ann Arbor, MI, USA*

#### *Edited by:*

*Sean K. Meehan, University of Michigan, USA*

#### *Reviewed by:*

*Lutz Jäncke, University of Zurich, Switzerland S. H. Annabel Chen, Nanyang Technological University, Singapore*

#### *\*Correspondence:*

*Jessica A. Bernard, Department of Neurology, University of Colorado Denver School of Medicine, 13001 E. 17th Place, Building 500, N3200, Aurora, CO 80045, USA. e-mail: jessica.bernard@ ucdenver.edu*

Cerebellar contributions to motor learning are well-documented. For example, under some conditions, patients with cerebellar damage are impaired at visuomotor adaptation and at acquiring new action sequences. Moreover, cerebellar activation has been observed in functional MRI (fMRI) investigations of various motor learning tasks. The early phases of motor learning are cognitively demanding, relying on processes such as working memory, which have been linked to the cerebellum as well. Here, we investigated cerebellar contributions to motor learning using activation likelihood estimation (ALE) meta-analysis. This allowed us to determine, across studies and tasks, whether or not the location of cerebellar activation is constant across differing motor learning tasks, and whether or not cerebellar activation in early learning overlaps with that observed for working memory. We found that different regions of the anterior cerebellum are engaged for implicit and explicit sequence learning and visuomotor adaptation, providing additional evidence for the modularity of cerebellar function. Furthermore, we found that lobule VI of the cerebellum, which has been implicated in working memory, is activated during the early stages of explicit motor sequence learning. This provides evidence for a potential role for the cerebellum in the cognitive processing associated with motor learning. However, though lobule VI was activated across both early explicit sequence learning and working memory studies, there was no spatial overlap between these two regions. Together, our results support the idea of modularity in the formation of internal representations of new motor tasks in the cerebellum, and highlight the cognitive processing relied upon during the early phases of motor skill learning.

**Keywords: cerebellum, sequence learning, visuomotor adaptation, working memory, meta-analysis**

## **INTRODUCTION**

Individuals are able to learn to use new tools and can turn novel movements into accomplished skills through practice. This process recruits a diverse network of cortical and subcortical brain regions (Jenkins et al., 1994; Imamizu et al., 2000; Doyon et al., 2002; Lehéricy et al., 2005; Seidler et al., 2006), though the neural substrates vary somewhat based on task type (c.f. Rauch et al., 1995; Honda et al., 1998; Schendan et al., 2003). Several different paradigms have been used to investigate motor skill learning. These commonly include visuomotor adaptation and motor sequence learning. Visuomotor adaptation requires individuals to adapt movements to distorted visual feedback (e.g., Imamizu et al., 2000; Seidler et al., 2006). The sensory information provided to the participant does not match the movement they have made, and as such the participant needs to modify their movement to produce the appropriate result. Motor sequence learning requires individuals to learn novel patterns of movements, often made with the fingers (**Figure 1**). Based on cues provided to the individual, a new movement sequence is practiced and learned. Within the domain of motor sequence learning both implicit and explicit paradigms are used (e.g., Schendan et al., 2003).

During implicit sequence learning, the goal of learning a new sequence is unknown to the participants, and the sequence is often embedded within other movements. Conversely, during explicit sequence learning, the goal of learning the sequence is made clear at the outset of the task. One brain region that has been consistently implicated in motor learning is the cerebellum. Cerebellar activation has been observed in a variety of motor learning tasks including visuomotor adaptation (Imamizu et al., 2000, 2003; Anguera et al., 2010) and both implicit and explicit motor sequence learning (Jenkins et al., 1994; Grafton et al., 2001; Lehéricy et al., 2005; Orban et al., 2010). Cerebellar circuits have also been implicated in associative learning paradigms such as eye-blink conditioning (Woodruff-Pak et al., 2000, 2001; Cheng et al., 2008).

In particular, the cerebellum is thought to play a role in the formation of internal representations of actions that allow for the smooth execution of motor skills (Ramnani, 2006; Ito, 2008). Learning and formation of these representations is thought to rely on error signals based on feedback from prior performance (Ito, 2000). The result of this learning is a new internal model of a particular task. Indeed, the engagement of the cerebellum

during the learning of a new motor task changes as the course of learning progresses (Imamizu et al., 2000). During a locomotor adaptation task, cerebellar excitability is decreased over the course of the task as measured by the degree of cerebellar brain inhibition of the motor cortex (Jayaram et al., 2011). Furthermore, the degree to which cerebellar brain inhibition decreased was strongly associated with learning of the locomotor adaptation task such that those with the greatest decreases in cerebellar excitability learned best. This decrease in excitability was suggested to be related to synaptic long-term depression (Jayaram et al., 2011). Also using non-invasive brain stimulation it has been demonstrated that the cerebellum is associated with the learning of a visuomotor adaptation task, while the primary motor cortex is associated with retention of learning (Galea et al., 2011). Relatedly, different neural substrates are engaged during performance of a task shortly after learning, including the cerebellum (Shadmehr and Holcomb, 1997). This is indicative of changes in and consolidation of the internal model of a particular action.

With that in mind, one is likely to learn multiple motor skills. The question then becomes whether or not the cerebellum then forms distinct internal models for these different motor skills. It has previously been suggested that multiple internal models are present in the cerebellum. This has been conceptualized in the computational model know as MOdular Selection And Identification Controller (MOSAIC; Wolpert and Ghahramani, 2000; Imamizu et al., 2003). Imamizu et al. (2003) tested this idea by having individuals learn to use a computer mouse under two novel visuomotor mappings. The visual feedback of the mouse was rotated, and in a separate condition, the velocity of the feedback was also manipulated. Over the course of learning in these two conditions, they found distinct regions of cerebellar activity, supporting modular internal models in the cerebellum (Imamizu et al., 2003). Imamizu and colleagues (2003) noted that this work serves as an extension of the MOSAIC theory in that the regions of cerebellar engagement associated with the internal models of these two conditions are in lateral regions of the cerebellum more associated with cognitive functions. Regardless, the MOSAIC theory can be further tested in the motor domain through the use of meta-analysis, as there are now numerous studies of motor learning across a variety of motor task domains. However, given the potential storage capacity issues with strictly modular representations of internal models, there may be overlapping cerebellar regions associated with motor tasks that require similar types of cognitive processing for learning, or are similar in task domain (for example, implicit and explicit sequence learning).

In addition to investigating the MOSAIC theory through the use of meta-analysis, this method also allows us to investigate the potential cognitive contributions of the cerebellum to motor learning. Compared to the more automatic performance that occurs in late learning, the early stage of learning is thought to be cognitively demanding (Fitts and Posner, 1967). Indeed, the rate of early learning during a visuomotor adaptation task has been correlated with individual differences in spatial working memory ability, as measured using the card rotation task (Anguera et al., 2010). Furthermore, this work demonstrated that in this early learning phase, there is engagement of prefrontal and parietal brain regions that are also associated with the performance of a working memory task involving mental rotation (Anguera et al., 2010). Relatedly, visuospatial and verbal working memory have also been implicated in motor sequence learning. Visuospatial working memory capacity is correlated with explicit motor sequence learning and the formation of motor chunks (Bo and Seidler, 2009; Bo et al., 2009). In implicit sequence learning paradigms, both visuospatial and verbal working memory are correlated with improved performance (Bo et al., 2011, 2012). Additionally, individuals with high working memory capacity learn sequences better when executive attention is required relative to those with low working memory capacity (Unsworth and Engle, 2005).

The posterior and lateral regions of the cerebellum have been associated with the performance of working memory tasks (Chen and Desmond, 2005a,b; Kirschen et al., 2005, 2010; Stoodley and Schmahmann, 2009; Stoodley et al., 2010, 2012). While these regions have been investigated using working memory paradigms, it remains unknown whether the same sites are also engaged during the learning of new motor skills. Given that prefrontal and parietal regions associated with working memory are also engaged during early visuomotor adaptation learning (Anguera et al., 2010), the same may be true for the cerebellum. Though more lateral regions of the cerebellum have been recently implicated in complex motor tasks (Schlerf et al., 2010), perhaps due to the cognitive demands of those tasks, there have been no investigations of whether the same cerebellar regions are engaged for both working memory and motor skill learning. Again, meta-analysis allows for assessment of this question.

Here our goal was to investigate the cerebellar contributions to both sensorimotor adaptation and sequence learning. Cerebellar activation has been seen in implicit and explicit sequence learning and visuomotor adaptation, along with both spatial and verbal working memory (Hazeltine et al., 1997; Thomas et al., 1999; Daselaar et al., 2003; Haaland et al., 2004; Krakauer et al., 2004; Chen and Desmond, 2005a,b; Lehéricy et al., 2005; Seidler et al., 2006; Anguera et al., 2007; Schendan and Stern, 2007; Stoodley et al., 2010). Given that working memory capacity is correlated with these three types of learning (Bo and Seidler, 2009; Bo et al., 2009, 2011, 2012; Anguera et al., 2010, 2011), it may be the case that a single cerebellar modular region underlies all three types of learning. Though the cerebellum and basal ganglia show dissociated activity in the later stages of learning, both are active in the earlier stages of learning for both sequence learning and visuomotor adaptation (Doyon and Benali, 2005). One possibility is that the overlapping neural substrates of learning in the cerebellum

may be due to the involvement of the cerebellum in working memory processes, particularly given that working memory is important for both sequence learning and visuomotor adaptation (Bo and Seidler, 2009; Bo et al., 2009, 2011, 2012; Anguera et al., 2010, 2011). However, because cerebellar engagement changes over the time course of learning (Imamizu et al., 2000, 2003), it may be oversimplified to look at just general overlap across these task types. Thus, we will investigate overlap in cerebellar activation across studies for working memory tasks with that of explicit sequence learning, implicit sequence learning, and visuomotor adaptation, taking into account the stages of learning (early vs. late) whenever possible. This approach will help to refine our view of cerebellar functions and modularity for cognitive and motor behaviors. In particular, investigations of the early and late stages of learning will provide further insight into the formation of internal models and allow for an additional test of the MOSAIC theory (Wolpert and Ghahramani, 2000; Imamizu et al., 2003) in the motor domain.

We used activation likelihood estimation (ALE) meta-analysis (Turkeltaub et al., 2002; Laird et al., 2005; Eickhoff et al., 2009), implemented using the GingerALE software package, to investigate the cerebellar regions involved in both motor sequence learning and visuomotor adaptation as well as working memory. Given the number of task domains, and the time necessary to assess learning, it would be extremely challenging to investigate all of these tasks in one functional neuroimaging study in order to answer the questions at hand. Meta-analysis, however, provides a reasonable solution. ALE meta-analysis pools coordinates in standard space across studies, and treats them as spatial probability distributions. Overlap among these regions is assessed through permutation testing, and the result is an ALE statistic for regions across studies with significant overlap (thresholded and corrected for multiple comparisons; Turkeltaub et al., 2002; Laird et al., 2005; Eickhoff et al., 2009). Studies demonstrating cerebellar activation during visuomotor adaptation, explicit and implicit sequence learning, as well as both spatial and verbal working memory tasks were combined in this meta-analysis. We hypothesized that visuomotor adaptation and motor sequence learning would engage similar motor regions of the cerebellum during early learning, but additional distinct regions associated with spatial and verbal working memory processes, respectively, would be engaged as well. We further hypothesized that distinct regions of the cerebellum would be involved in the later stages of learning and the formation of internal models, consistent with the MOSAIC theory (Wolpert and Ghahramani, 2000; Imamizu et al., 2003) which suggests a modular organization of representations in the cerebellum.

## **METHODS LITERATURE REVIEW**

Papers were identified through three PubMed (http://www*.*ncbi*.* nlm*.*nih*.*gov/pubmed/) searches. Searches for papers investigating visuomotor adaptation, motor sequence learning, and working memory were conducted separately using the following search terms: "sensorimotor adaptation AND imaging," "motor sequence learning AND imaging," and "working memory AND imaging." Additionally, the searches used the limits "Humans," "English," and "Adult 19-44 years." These searches resulted in 45, 149, and 1997 papers, respectively. We also consulted a recent review of motor learning and included related work on sensorimotor adaptation not found in our PubMed search (Seidler, 2010). We followed the same exclusion criteria as reported by Stoodley and Schmahmann (2009). That is, we excluded papers that did not use functional imaging techniques, did not report any coordinates in the cerebellum, did not report coordinates in either Montreal Neurological Institute (MNI; Collins et al., 1998) or Talairach (Talairach and Tournoux, 1988) space, investigations with incomplete coverage of the cerebellum, those using only region of interest analyses, and clinical or aging studies that did not report a healthy young adult control group. Additionally, we excluded studies where the learning of the adaptation task or sequence was completed outside of the scanner (that is, the early learning phase was not scanned), and those that did not have subjects overtly perform the task (e.g., studies that investigated mental rehearsal of a sequence and the resultant learning outcomes), along with studies that did not use standard contrast analyses (for example, those using independent components analysis). Studies of working memory were limited to the spatial and verbal domains, consistent with tasks found to be associated with motor learning (Bo and Seidler, 2009; Bo et al., 2009, 2011, 2012; Anguera et al., 2010, 2011). Thus, we excluded studies with emotional, auditory, and visual manipulations. After excluding studies that did not meet our criteria, 5 studies of visuomotor adaptation, 18 studies of sequence learning, and 44 studies of working memory remained (9 of spatial working memory, and 35 of verbal working memory). Finally, for our analyses of sequence learning, we divided our studies into those investigating implicit sequence learning (7 studies) and those investigating explicit sequence learning (the remaining 11 studies). Studies of explicit sequence learning were further divided, grouping those investigating early and late learning separately (5 studies in each category). The number of studies included in each of our task domains is consistent with the number of studies used in similar recent ALE meta-analyses of cerebellar function (Stoodley and Schmahmann, 2009; E et al., in press). These investigations included between 2 studies (somatosensory task domain; Stoodley and Schmahmann, 2009) and 26 studies (working memory domain; E et al., in press). Though our initial goal was to compare early and late learning across all three motor tasks, none of the studies meeting our criteria for both visuomotor adaptation and implicit sequence learning included analyses based on learning stage, and we were therefore unable to complete the analysis of learning stage on these two task domains. For explicit motor sequence learning, early and late learning were typically defined within a single practice session. The first half of learning was compared to the second half of learning. However, in one instance (van der Graaf et al., 2004) learning was compared across two sessions with practice occurring for several days in between the two sessions. **Table 1** presents the studies included in our analyses, along with the sample size, imaging modality, the number of cerebellar foci, and a brief description of the tasks and contrasts resulting in those foci for each study.

The sequence learning tasks required subjects to learn novel sequences of movements, typically through finger button presses. However, Albouy and colleagues (2008) investigated the implicit learning of a sequence of eye movements. In the implicit conditions, action sequences were often embedded in a larger set of movements so as to block explicit awareness of the task. Decreases in reaction time are indicative of learning during sequence blocks, relative to blocks where all button presses were random. Relatedly, a secondary task was also at times employed to further prevent participants from gaining explicit awareness of the sequence (Grafton et al., 2002). Under explicit learning conditions, participants were instructed that they would be learning a sequence and were aware of the task goals.

Visuomotor adaptation paradigms take two main forms. Most commonly, participants manipulate a hand-held joystick in order to move a small object to a target location. After a practice period, the visual feedback is rotated such that the feedback on the screen does not match the movements of the joystick (c.f. Anguera et al., 2007). Alternatively, participants may also be instructed to make pointing movements to a target while wearing prism distortion goggles (Luauté et al., 2009). In both cases, the visual feedback of movement is distorted.

In both verbal and spatial working memory tasks participants have to hold and manipulate information in mind over a span of a few seconds. Two of the most commonly used tasks are the n-back task and the Sternberg working memory task. The n-back task can be administered using either verbal or spatial stimuli (c.f. Thomas et al., 1999; Kim et al., 2003). In verbal tasks, letters (or numbers) are presented individually and subjects have to indicate whether the current letter matches what was presented "n" trials previously. In spatial tasks participants are asked to compare locations of stimuli across successive presentations. Also commonly used is the Sternberg working memory task (Sternberg, 1966). In this paradigm groups of letters are presented. After a delay period participants are presented with a letter and are asked to indicate whether or not that letter was part of the previously viewed set. Additionally, the included studies also employed tests of mental rotation (spatial working memory) as well as paced addition tasks (verbal working memory).

Importantly, across these task domains, participants were required to make their responses with the fingers and hand. The one exception was implicit sequence learning of eye movements (Albouy et al., 2008). In general, across domains the effectors used during the learning paradigms did not vary significantly. This is particularly important given the somatotopy within the cerebellum (Nitschke et al., 1996; Wiestler et al., 2011). Any differences in activation across these motor tasks cannot be attributed to differences in the effectors used in each task domain. With respect to working memory, all of the responses were made with the hands and fingers across tasks, although all of the studies included in our analyses also controlled for the motor responses.

#### **ALE META-ANALYSIS**

All analyses were completed using GingerALE 2.3 (www*.* brainmap*.*org/ale; Laird et al., 2005; Eickhoff et al., 2009). Foci within the cerebellum for each task type were combined into individual text files. Because all of the foci need to be in the same space, foci in Talairach space that were transformed using the Brett transform (mni2tal) were converted back to MNI

#### **Table 1 | Studies included in the meta-analysis, organized by category.**


#### **Table 1 | Continued**


space using the inverse of the Brett transform. Those that were transformed into Talairach space using the Lancaster transform (Lancaster et al., 2007; icbm2tal) were transformed back into MNI space, also using the inverse of this transform. Finally, in cases where there was no transform specified, or where data were initially normalized into Talairach space, the Lancaster icbm2tal transform was used to bring these foci into MNI space. Importantly, the icbm2tal is a newer transformation (Lancaster et al., 2007) and we were careful to ensure that this was used only on studies where it would have been initially available. Older work transformed with icbm2tal was restricted to studies that were initially normalized directly into Talairach space. These transformations were completed using the "convert foci" tool in GingerALE. Foci in MNI space within the cerebellum for each task type were combined into individual text files.

The text files were then entered into GingerALE. GingerALE automatically computes the ALE values for every voxel in the brain, and does so using an automatically determined full-width half-maximum (FWHM) value (Eickhoff et al., 2009). However, upon completion of the analyses, the FWHM value of each focus was reported to be between 9 and 10 mm. The ALE value was computed using permutation testing (5000 permutations) against the null-distribution of random spatial associations of foci across experiments (Eickhoff et al., 2009). The ALE scores resulting from this permutation testing are then used to assign *p*-values to the actual values of the input data. We used a false discovery rate of *p <* 0*.*05 to correct for multiple comparisons. Additionally, all clusters were set to a minimum of 50 mm3. We completed ALE analyses for visuomotor adaptation, explicit sequence learning, implicit sequence learning, spatial working memory, and verbal working memory. We completed additional ALE analyses on the subset of explicit motor sequence learning studies that looked at activation during the early and late stages of learning. Notably, because we were generally interested in the regions involved in motor learning, areas that showed decreases in activation over the course of learning were considered with those that showed increases in activation. While most studies specifically investigated increases in activation, there were a few investigations of decreases in activation, though there were not a sufficient number of foci to investigate these decreases separately.

GingerALE also allows for statistical comparisons between the ALE maps of two distinct sets of foci. We used this method to investigate areas of overlap between task domains. We were particularly interested in the conjunction analyses across different tasks. Specifically, we investigated potential regions of overlap between visuomotor adaptation and all studies of sequence learning (combining both explicit and implicit studies), visuomotor adaptation and spatial working memory, explicit sequence learning and verbal working memory, all sequence learning and verbal working memory, and the early and late phases of learning during explicit sequence learning. This was computed using 5000 permutations, and we again used a false discovery rate of *p <* 0*.*05 and minimum cluster size of 50 mm3.

The results were visualized using MRICron (http://www*.* mccauslandcenter*.*sc*.*edu/mricro/mricron/index*.*html) and overlaid on an MNI template brain. The peaks of the ALE clusters were localized using the (Schmahmann et al., 1999) atlas of the human cerebellum. Because we were combining studies using standard normalization procedures, we were unable to use the recently developed SUIT atlas (Diedrichsen, 2006; Diedrichsen et al., 2009). The implications of older cerebellar templates and standard normalization procedures are addressed further in the discussion.

## **RESULTS**

## **ALE PEAKS FOR MOTOR LEARNING AND WORKING MEMORY TASKS**

**Table 2** presents the peak coordinates, weighted centers, cluster sizes, and anatomical locations for the significant ALE maxima across each task domain. **Figure 2** presents the ALE activation maps for visuomotor adaptation, explicit and implicit sequence learning, and spatial and verbal working memory. **Figure 3** presents the ALE activation maps for early and late explicit sequence learning.

Though we were unable to investigate early vs. late learning activation in the visuomotor adaptation task, analysis of activations across the entire learning period resulted in one significant cluster in the anterior cerebellum, localized in lobule IV. Also located in the anterior lobe was a significant cluster associated with implicit sequence learning. However, this cluster was located along the midline in the vermis region of lobule V.

When combining across all studies and phases of learning, explicit sequence learning was associated with a large cluster in the vermis region of lobule VI, extending into lobule VI itself. During the early phase of explicit sequence learning there were two significant ALE clusters. One cluster was located more medially in the vermis region of lobule VI and extended laterally into lobule VI. The second cluster was located more laterally, and was inferior to the first cluster in lobule VI. During the late phase of explicit sequence learning, the activation was again more medial in the vermis of lobule VI and extending into lobule VI itself.

Finally, we investigated both spatial and verbal working memory. Spatial working memory processing activated a cluster in the left cerebellum in lobule VI, while verbal working memory processing activated a large cluster in the right cerebellum on the border between lobule VI and Crus I. These findings closely replicate those described by the meta-analysis performed by Stoodley and Schmahmann (2009), and are also consistent with the functional topography of the cerebellum that has been demonstrated using functional neuroimaging (Stoodley et al., 2010, 2012). Notably, there were no clusters in the inferior regions of the cerebellum as reported in recent meta-analyses (Stoodley and Schmahmann, 2009; E et al., in press). In part, this may be due to the number of additional studies included in our analysis [44 working memory studies, compared to the 8 and 26 used by Stoodley and Schmahmann (2009) and E et al. (in press), respectively]. Furthermore, this inferior region was associated most strongly with the Sternberg task (E et al., in press), as evidenced by comparisons across working memory tasks. Though many of the studies in our analyses employed variants of the Sternberg task, there were additional working memory tasks included, perhaps resulting in our null finding in the inferior cerebellum.



#### **ANALYSIS OF OVERLAP ACROSS TASKS**

Conjunction analyses across sets of foci were carried out to investigate overlapping regions of the cerebellum across tasks. We investigated overlap between visuomotor adaptation and sequence learning (collapsing across all implicit and explicit studies), visuomotor adaptation and spatial working memory, all sequence learning and verbal working memory, explicit sequence learning and verbal working memory, and the early and late stages of explicit sequence learning. There was no significant overlap between any of these sets of foci with the exception of the early and late stages of explicit sequence learning. There was a significant cluster of overlap in the vermis region of lobule VI associated with both early and late explicit sequence learning (**Table 3**, **Figure 3**). However, there was no overlap between late learning and the more lateral lobule VI cluster associated with early explicit sequence learning.

## **DISCUSSION**

Using ALE meta-analysis, we investigated cerebellar involvement in multiple motor learning tasks, including visuomotor adaptation and both explicit and implicit motor sequence learning. We further investigated cerebellar regions associated with working memory processes and their potential involvement in motor learning. Our results provide evidence consistent with the role of the anterior cerebellum in motor tasks, though our findings did not indicate overlapping engagement of cerebellar regions for both working memory processes and motor learning. The anterior cerebellum, particularly along the midline, was active across studies of explicit and implicit sequence learning, with an additional anterior region associated with visuomotor adaptation. The distinct regions associated with these motor tasks provide conceptual support for the MOSAIC theory (Wolpert and Ghahramani, 2000; Imamizu et al., 2003) of modular internal models in the cerebellum. Additionally, we provide support for the involvement of more lateral and posterior regions of the cerebellum in explicit sequence learning. This is consistent with prior work indicating an additional homunculus in this region associated with the performance of complex motor tasks (Schlerf et al., 2010). However, notably, we found no overlap between regions associated with spatial and verbal working memory processes and any of the motor learning tasks we investigated, despite our previous work demonstrating correlations between an individual's working memory capacity and their motor learning of these tasks (Bo and Seidler, 2009; Bo et al., 2009, 2011, 2012; Anguera et al., 2010, 2011).

## **THE CEREBELLUM AND INTERNAL MODELS OF ACTION**

It has been proposed that the cerebellum is important for the formation of internal models of actions (Miall et al., 1993; Miall and Wolpert, 1996; for reviews see Ramnani, 2006; Ito, 2008). According to the MOSAIC theory, these internal models are modularly represented in the cerebellum for motor actions as well as cognitive processes (Wolpert and Ghahramani, 2000;

**type are presented on coronal (left) and axial (right) slices of the cerebellum.** All clusters are thresholded and corrected for multiple

adaptation; ISL, implicit sequence learning; ESL, explicit sequence learning; VWM, verbal working memory; SWM, spatial working memory; CRI, Crus I.

Imamizu et al., 2003). Supporting this theory, we found that cerebellar activation was distinct for each of multiple motor learning task types.

With that said, it is important to note that in both implicit sequence learning and visuomotor adaptation, we were unable to subdivide the collected foci into the early and late phases of learning. We were therefore unable to investigate differences in the activated regions that would be indicative of the formation of new internal models of the learned skills. One alternative possibility is that the different regions of activation across tasks were due to the motor somatotopy within the anterior cerebellum (Nitschke et al., 1996; Buckner et al., 2011; Wiestler et al., 2011). There is a general body representation within this region; even individual finger representations can be discerned (Wiestler et al., 2011). The distinct regions may be due to the overall motor demands of the learning tasks, and variability may be associated with different effector usage for task performance. For example, sequence learning tasks typically involved tapping with individual fingers, whereas visuomotor adaptation often required the manipulation

of a joystick with either several fingers or the whole hand. As such, distinct anterior cerebellar regions may have been engaged.

Lastly with respect to the localization of these activations, across these motor tasks activity across studies was generally confined to anterior regions of the cerebellum. This is consistent with the functional topography of the cerebellum wherein motor representations are located in the anterior cerebellum, as well as in lobules VIIIa and VIIIb in the posterior cerebellum (Schmahmann and Sherman, 1998; Gerwig et al., 2003; Stoodley and Schmahmann, 2009; Stoodley et al., 2010, 2012). Though we did not see any activation clusters in the secondary, more posterior motor representation, it has recently been suggested that the function of the posterior region is different than that of the anterior motor representation, and it may be less important for motor control (Donchin et al., 2012). Additionally, our midline clusters associated with both implicit and explicit sequence learning are in a cerebellar region where gray matter volume has been linked to the degree to which individuals learn a new motor sequence (Steele et al., 2012).



## **WORKING MEMORY AND MOTOR LEARNING IN THE CEREBELLUM**

As the early stage of learning is thought to rely on cognitive processes (Fitts and Posner, 1967; Anderson, 1982), we predicted that there would be overlap between areas engaged in spatial and verbal working memory and those associated with motor learning. However, this was not supported by the results. This is somewhat surprising given the relationship between working memory capacity and both sequence learning and visuomotor adaptation (Bo and Seidler, 2009; Bo et al., 2009, 2011, 2012; Anguera et al., 2010), and the recruitment of neural resources associated with working memory during the early phase of visuomotor adaptation (Anguera et al., 2010).

Lateral and posterior cerebellar regions are thought to communicate with the prefrontal cortex through closed loop circuits (Ramnani, 2006). These regions are also implicated in both spatial and verbal working memory tasks as demonstrated in our analyses, consistent with prior work (Chen and Desmond, 2005a,b; Stoodley and Schmahmann, 2009; Stoodley et al., 2010, 2012). One may then imagine that if working memory circuits are engaged during the early phases of motor learning, the cerebellar components of those circuits may also be brought online. In fact, in learning novel skills that may require more cognitive resources, new internal models are formed, but they seem to be in more lateral regions of the cerebellum (Imamizu et al., 2000, 2003). Perhaps, because we were unable to differentiate between the early and late learning phases in the visuomotor adaptation task and in implicit sequence learning, we were unable to differentiate regions that may be associated with more general motor execution from those associated with the formation of a new internal model. Similarly, in our analyses we treated regions that showed decreases in activation over the course of learning in the same way as those that showed parametric increases in activation over the course of learning. A greater number of foci in each category would allow for differentiation and may indicate that areas of decrease are associated with the cognitive demands of early learning (Anguera et al., 2010), while those that exhibit increases may be more associated with the formation of new internal models.

We were able to investigate the early and late phases of explicit sequence learning. During the early phase of learning there were two significant cerebellar clusters, one of which was more lateral and inferior in lobule VI. Though there was no overlap with regions associated with either spatial or verbal working memory, this region is consistent with an area reported to show increased activation during the performance of more complex motor tasks (Schlerf et al., 2010). In this investigation during the complex task participants executed sequences of finger flexion and extension. This was compared to a simple task requiring the simultaneous flexion and extension of multiple digits at once. In our data, as with those of Schlerf and colleagues (2010), activity was localized in lobule VI. Lobule VI has been implicated in working memory task performance (Chen and Desmond, 2005a,b), and the resting state networks of this region include correlations with both pre-motor and lateral prefrontal cortical regions (Bernard et al., 2012). Thus, though the activation in lobule VI associated with early explicit sequence learning does not directly overlap with those associated with verbal or spatial working memory, lobule VI does seem to be involved in higher cognitive processing. However, given that we averaged across multiple studies and foci, there may be some overlap on an individual study level. Our cluster in this region associated with early explicit sequence learning may therefore reflect some of the cognitive demands associated with this stage of motor skill learning. Finally, the more superior and medial early learning cluster overlapped with that of late learning. This may be more indicative of a newly formed internal model.

## **LIMITATIONS**

The use of meta-analysis to investigate activations across studies has some limitations. First is that of study selection. While we defined our study selection criteria based on age and study parameters to eliminate any potential bias, there may be additional available studies that merit inclusion but were not found based on our search terms. Our results are limited to those studies that are available in Pubmed within our given search parameters. Furthermore, a variety of different tasks have been used to investigate working memory and motor learning. For example, verbal working memory may be measured using an n-back task, a Sternberg task, or the paced auditory serial addition task. Because we were interested in the general processes, and not the specific tasks themselves we collapsed across these task types. Notably, there was less variability across sequence learning tasks and visuomotor adaptation paradigms, but this may still impact our results.

Second, combining multiple studies means that data are often normalized to different brain templates, or normalized and transformed from one template to another. Though algorithms are available to bring data across several studies into the same anatomical space, perfect registration across subjects cannot be guaranteed. Relatedly, the acquisition and other processing parameters vary across these studies. Indeed, because we included both PET and fMRI results in our analysis, this is particularly pertinent. Importantly however, the ALE algorithm employed here includes random-effects modeling designed to account at least in part for these limitations (Eickhoff et al., 2009).

Lastly, it is worth noting that the studies included in this meta-analysis relied primarily on standard affine transformations for normalization. These methods implemented in common neuroimaging packages often result in poor alignment between cerebellar regions (Diedrichsen, 2006). Recently, Diedrichsen and colleagues have created a spatiallyunbiased atlas and updated normalization procedure to improve cerebellar registration (Diedrichsen, 2006; Diedrichsen et al., 2009). Because of the relative novelty of this normalization procedure and the span of time over which our studies originate, use of this procedure was rare in the studies we sampled. Most of the investigations we included used more standard normalization parameters and templates. Thus, our results should be interpreted with some caution as the actual locations may vary slightly due to normalization procedures.

## **CONCLUSIONS**

Here, we investigated the role of the cerebellum in motor skill learning using ALE meta-analysis. We combined foci across studies investigating visuomotor adaptation, motor sequence learning (explicit and implicit), and verbal and spatial working memory. We demonstrated that distinct motor tasks engaged differing regions of the cerebellum, providing further evidence for the notion that the internal models of the cerebellum are formed in a modular manner. Furthermore, these regions were generally limited to the anterior portion of the cerebellum, consistent with its general functional topography. Additionally, we also found that although the cerebellum seems to engage regions associated with the lateral prefrontal cortex and working memory performance during the early stage of explicit sequence learning, this region did not overlap with any of the significant ALE clusters associated with the working memory domains (verbal and spatial) that we investigated here. In general, this provides support for the role of the cerebellum in processing the cognitive demands of the early phases of sequence learning, but further investigations are needed to see if this generalizes to other domains of motor skill learning. In particular, more fine-grained studies investigating cerebellar functional modularity across tasks and their associated timecourses are warranted.

## **ACKNOWLEDGMENTS**

Jessica A. Bernard was supported by NIA T32 AG000114 (S. Pletcher, PI) and is currently supported by NIA T32 AG000279 (R. Schwartz, PI). Jessica A. Bernard (alumna) and Rachael D. Seidler (faculty) are members of the International Max Planck Research School on the Life Course (LIFE, www*.* imprs-life*.*mpg*.*de; participating institutions: MPI for Human Development, Humboldt-Universität zu Berlin, Freie Universität Berlin, University of Michigan, University of Virginia, University of Zurich). We also thank Joshua Carp for helpful comments on this manuscript.

## **REFERENCES**


Wright, M. J. (2011). Heritability of working memory brain activation. *J. Neurosci.* 31, 10882–10890.


and verbal working memory tasks. *Neuroimage* 24, 332–338.


contributions to higher cognition from PET and fMRI studies. *Hum. Brain Mapp.*


(2003). Functional disconnection between the prefrontal and parietal cortices during working memory processing in Schizophrenia: a [15O] H2O PET study. *Am. J. Psychiatry* 160, 919–923.


P., and Wu, A. D. (2011). Brainbehavioral correlates of optimizing learning through interleaved practice. *Neuroimage* 56, 1758–1772.


A high-resolution functional MRI study. *Brain* 119, 1023–1029.


fMRI study. *J. Psychiatr. Res.* 42, 469–476.


basal ganglia activation associated with sensorimotor adaptation. *Exp. Brain Res.* 175, 544–555.


investigating prefrontal function. *Psychiatry Res.* 140, 73–83.


*Received: 03 December 2012; paper pending published: 02 January 2013; accepted: 22 January 2013; published online: 07 February 2013.*

*Citation: Bernard JA and Seidler RD (2013) Cerebellar contributions to visuomotor adaptation and motor sequence learning: an ALE meta-analysis. Front. Hum. Neurosci. 7:27. doi: 10.3389/ fnhum.2013.00027*

*Copyright © 2013 Bernard and Seidler. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

## Probabilistic sequence learning in mild cognitive impairment

#### *Dezso Nemeth1 \*†, Karolina Janacsek1†, Katalin Király2†, Zsuzsa Londe3†, Kornél Németh4†, Kata Fazekas4†, Ilona Ádám5, Király Elemérné5 and Attila Csányi <sup>5</sup>*

*<sup>1</sup> Department of Clinical Psychology and Addiction, Institute of Psychology, Eötvös Loránd University, Budapest, Hungary*

*<sup>2</sup> Institute of Psychology, University of Szeged, Szeged, Hungary*

*<sup>3</sup> American Language Institute, University of South-California, Los Angeles, USA*

*<sup>4</sup> Department of Cognitive Sciences, Budapest University of Technology and Economics, Budapest, Hungary*

*<sup>5</sup> Aladár Petz County Research Hospital, Gyor, Hungary*

#### *Edited by:*

*Rachael D. Seidler, University of Michigan, USA*

#### *Reviewed by:*

*Adrian M. Haith, Johns Hopkins University School of Medicine, USA Rebecca Spencer, University of Massachusetts, Amherst, USA*

#### *\*Correspondence:*

*Dezso Nemeth, Department of Clinical Psychology and Addiction, Institute of Psychology, Eötvös Loránd University, Budapest, Izabella utca 46, 1064 Budapest, Hungary*

*e-mail: nemethd@gmail.com*

*†These authors have contributed equally to this work.*

Mild Cognitive Impairment (MCI) causes slight but noticeable disruption in cognitive systems, primarily executive and memory functions. However, it is not clear if the development of sequence learning is affected by an impaired cognitive system and, if so, how. The goal of our study was to investigate the development of probabilistic sequence learning, from the initial acquisition to consolidation, in MCI and healthy elderly control groups. We used the Alternating Serial Reaction Time task (ASRT) to measure probabilistic sequence learning. Individuals with MCI showed weaker learning performance than the healthy elderly group. However, using the reaction times only from the second half of each learning block—after the reactivation phase—we found intact learning in MCI. Based on the assumption that the first part of each learning block is related to reactivation/recall processes, we suggest that these processes are affected in MCI. The 24-h offline period showed no effect on sequence-specific learning in either group but did on general skill learning: the healthy elderly group showed offline improvement in general reaction times while individuals with MCI did not. Our findings deepen our understanding regarding the underlying mechanisms and time course of sequence acquisition and consolidation.

**Keywords: mild cognitive impairment, offline learning, statistical learning, implicit learning, skill learning, consolidation, automaticity**

## **INTRODUCTION**

Mild cognitive impairment (MCI) is a transition stage between normal age-related cognitive decline and the more serious symptoms of dementia caused by, for example, Alzheimer's disease. According to the American College of Physicians, MCI affects about 20% of the population over 70 years of age. Many who develop MCI eventually develop Alzheimer's disease, although some will remain stable or might even return to normal (Roberts et al., 2008). Of those with MCI, 12–15% will develop the signs of dementia within a year and about 50% will progress to dementia within 5 years (Gauthier et al., 2006). The characteristic symptoms of MCI are impaired memory functions during learning or recall, impaired attention and information processing evidenced by the speed with which these functions are executed, flawed executive functions, and perceptual motor-skill and languageexpression disturbances (e.g., word finding). MCI is diagnosed if at least two of these symptoms are present for at least 2 weeks (Tariska et al., 1990; Petersen et al., 1999; Grundman et al., 2004; Portet et al., 2006). MCI produces greater than age appropriate memory impairment but in all other aspects the individual functions well. Most often, learning skills and the ability to recall new information are affected to the highest extent. Brain imaging research shows dysfunction in the medial temporal lobe (MTL), including the hippocampal formation in MCI (Jack et al., 1997; Dickerson and Sperling, 2008; Nickl-Jockschat et al., 2012; Szamosi et al., 2013) but other areas might also be affected (Rombouts et al., 2009). Memory tests have established that certain forms of explicit memory and learning, such as delayed recall and list learning, decline in MCI (Petersen et al., 1999; Grundman et al., 2004; Leube et al., 2008). However, the question of how implicit learning is affected by MCI has received less attention (Nagy et al., 2007; Negash et al., 2007b). Properties of implicit learning and its consolidation could be useful in the dissociation of MCI from healthy age-related changes and also could contribute to a better understanding of the formation and consolidation of sequence acquisition, specifically the role of the MTL and hippocampus in these processes.

Explicit or declarative memory is accessible to conscious recollection, including facts and episodes (for example remembering events explicitly). It is defined by voluntary mechanisms which rely more on attentional resources. Non-declarative memory relies more on automatic, non-conscious/implicit processes including habituation, conditioning, motor and perceptual skills (for example playing piano). According to Squire and his colleagues, explicit or declarative memory can be linked to the brain's medial-temporal area, while the implicit or non-declarative processes fall outside these areas (Squire and Zola, 1996; Squire, 1998). Nevertheless, others showed that areas in the MTL including hippocampus also play a role in implicit learning (Chun and Phelps, 1999; Albouy et al., 2008; for critics, see Manns and Squire, 2001; Poldrack and Rodriguez, 2003).

The focus in our study is primarily on implicit sequence learning which underlies the acquisition of not only motor but also cognitive and social skills (Lieberman, 2000; Nemeth et al., 2011; Romano Bergstrom et al., 2012). Most models of sequence learning (Hikosaka et al., 1999, 2002; Doyon et al., 2009a) emphasize the role of the frontal-striatal-cerebellar system, while the role of the MTL and related structures (e.g., hippocampus) remains inconclusive (Schendan et al., 2003; Albouy et al., 2008; Simon et al., 2012). Negash et al. (2007b) have conducted the first and only research to address this topic so far, in which they investigated the effect of MCI on implicit learning. They used two implicit learning paradigms: the Serial Reaction Time (SRT; Nissen and Bullemer, 1987) to measure sequence learning, and the Contextual Cueing Task (Chun and Jiang, 1998) to measure visuospatial configuration learning. Despite the similarity in implicitness of these tasks, they call on two different neural systems; previous studies showed greater involvement of MTL in the Contextual Cueing (Chun and Jiang, 1998; Manns and Squire, 2001) compared to the SRT task, which is primarily mediated by the previously mentioned frontal-striatal-cerebellar system (Curran, 1998; Honda et al., 1998; Gomez-Beldarrain et al., 1999; Willingham et al., 2002). Negash et al.'s results revealed that individuals with MCI, although generally slower, showed similar sequence learning to the controls; however, learning was impaired in the Contextual Cueing task. These findings implicate that the MTL system, including the hippocampal formation is involved in MCI, while the frontal-striatal-cerebellar system is involved to a lesser extent (Negash et al., 2007b).

While Negash et al. (2007b) used a deterministic 8-element sequence, we take the task one step further. Here we use a modified version of the SRT task, the Alternating Serial Reaction Time (ASRT) task (Howard and Howard, 1997), which enables us to separate general skill learning and sequence specific learning. General skill learning refers to the increase in speed as the result of practice and it is relatively independent from sequence structure, while sequence-specific learning refers to the acquisition of sequence-specific knowledge, which results in relatively faster responses for events that can be predicted from the sequence structure vs. those that cannot. Most research, including the Negash et al.'s (2007b) study cited above, has not distinguished these because the tasks used make it difficult to do so. In classical SRT tasks used also by Negash et al. (2007b), the structure of a sequence is deterministic, with the stimuli following a simple repeating pattern as in the series 213412431423, where numbers refer to distinct events. In contrast, in the ASRT task (Howard and Howard, 1997; Remillard, 2008), repeating events alternate with random elements. This means that the location of every second stimulus on the screen is determined randomly. If, for instance, the sequence is 1234, where the numbers represent locations on the screen, in ASRT the sequence of stimuli will be 1r2r3r4r, with r representing a random element. The sequence is thus 'better hidden' than in the deterministic SRT task and it is also possible to track sequence-specific learning continuously by comparing responses to the random and sequence elements. This structure is called probabilistic second-order dependency (Remillard, 2008) because to predict element 'n' we need only to know element n-2, regardless of element n-1. In this way, the representations of the probabilistic sequences are more abstract and the acquisition of the sequences is also a statistical learning process. One of the outstanding questions in the literature of implicit learning is if there are functional differences in how implicit learning develops in motor vs. cognitive tasks (Foerde et al., 2008; Ashby et al., 2010). The fact that probabilistic sequences with their statistical properties are more ambiguous due to certain transitions being dictated by a context defined by remote events (Remillard, 2008) suggests that learning these sequences might result in more abstract representations than in deterministic sequence learning tasks (for another view see Keele et al., 2003). Moreover, several studies showed that probabilistic sequence learning is related not only to motor, but also to perceptual processes (Song et al., 2008; Nemeth et al., 2009; Hallgató et al., 2013). Based on these considerations, probabilistic sequence-specific learning is presumed to be related relatively more to cognitive skills, while general skill learning is presumed to be related relatively more to motor skills in this specific design. It is a particularly interesting issue how MCI affects the performance on these two aspects of learning.

In the development and stabilization of memory representation for sequences, the processes of consolidation and reconsolidation, are particularly important (Walker et al., 2003; Rickard et al., 2008; Tucker et al., 2011). During the acquisition of sequences we are learning and recalling and reactivating the sequence elements continuously. Recalling or reactivating a previously consolidated memory makes it once again fragile and susceptible to interference, therefore requiring periods of reconsolidation (Walker et al., 2003). These circle processes make possible the continued refinement and reshaping of previously learned motor or cognitive skills in the context of ongoing experience. In experimental designs (fingertapping or SRT tasks) and partly in real-life situations, we are learning sequences arranged in blocks which are separated by shorter or longer time periods. In the beginning of the blocks we reactivate the already consolidated memory traces. Rickard et al. (2008) and Brawn et al. (2010) showed that the separate analysis of the different parts of the learning blocks is crucial in understanding the consolidation of sequence learning. For example, if we analyze only the first part of each of the learning blocks, we can find greater sequence learning effects by controlling the reactive inhibition [i.e., the inhibiting effect of fatigue on learning (Rickard et al., 2008)]. These effects can be particularly relevant in a cognitive impaired population such as MCI. It is important to highlight, however, that Rickard et al. (2008) and Brawn et al. (2010) used explicit and not implicit sequence learning. Thus, the question can be raised whether the pattern of results is the same for implicit learning. We hypothesize dissociation between explicit and implicit sequence learning because several factors, such as fatigue and attentional resources, affect the two types of learning differently (Nissen and Bullemer, 1987; Squire and Zola, 1996; Janacsek and Nemeth, 2013).

It is also a relevant issue that sequence learning does not occur only during practice—online periods—but also between practice periods—during offline periods. The process that occurs during the offline periods is referred to as consolidation and is typically revealed either by increased resistance to interference and/or by improvement in performance, following an offline period (Krakauer and Shadmehr, 2006). The nucleus caudate and ventricle putamen, which are part of the fronto-striato-cerebellar network, play important roles in sequence consolidation (Doyon et al., 2002, 2009b; Doyon and Benali, 2005; Lehericy et al., 2005; Albouy et al., 2008; Debarnot et al., 2009). More recent studies also emphasize the role of the hippocampus in the consolidation of sequence knowledge: for example, Albouy and colleagues (2008) found hippocampus activity using a 24-h delay interval between the learning and testing session. MCI is an ideal avenue to solve the puzzle of sequence consolidation because of the above mentioned neurocognitive background of this cognitive impairment. Although there are several studies focusing on the consolidation of explicit processes in MCI (e.g., Westerberg et al., 2012), to our knowledge no study has investigated the effect of a 24-h offline period on implicit sequence learning in this population so far.

In this study, we investigated sequence-specific and general skill learning in individuals with MCI. In this way we could indirectly investigate the role of the hippocampus and related MTL structures in this learning mechanism. A probabilistic sequence learning task was set up in a prolonged way in order to map the development and consolidation of memories for sequences. We had two main questions here: (1) to which extent can the individuals with MCI learn raw probabilities implicitly, (2) how withinblock effects contribute to sequence learning performance. For the second question we hypothesized that the beginning of the learning blocks reflects the processes in which we are picking up high and low frequency triplets and reactivating/recalling the sequence information learned in the previous blocks. As reactivation/recall processes are shown to be related to the hippocampus and related structures (e.g., Gelbard-Sagiv et al., 2008; Xue et al., 2010), we expected weaker learning performance in MCI based on the first half of the blocks compared to the second half of the blocks.

## **MATERIALS AND METHODS**

## **PARTICIPANTS**

Seventeen MCI patients and 17 healthy elderly controls participated in the experiment. Diagnoses of MCI were established via a consensus meeting of at least two clinical neurologists and a neuropsychologist using various examinations and tests (e.g., basic laboratory tests, brain MRI, clinical evaluation, Mini Mental State Examination—MMSE). Controls were individuals who: (1) were independently functioning community dwellers, (2) did not have active neurological or psychiatric conditions, (3) had no cognitive complaints, (4) demonstrated a normal neurological behavior, (5) were not taking any psychoactive medications (Negash et al., 2007b).

The MCI and the control group were matched on age (*M*MCI = 61*.*82, *SD*MCI = 7*.*70; *M*control = 57*.*82, *SD*control = 8*.*47), years of education (*M*MCI = 13*.*35, *SD*MCI = 2*.*21; *M*control = 14*.*18, *SD*control = 2*.*38) and gender (14 and 15 females, respectively). The groups differed in performance on the MMSE [*t(*32*)* = −6*.*31, *p <* 0*.*001]: the mean score was 26.91 (*SD* = 1*.*69, range 25–28) for the MCI group and 29.69 (*SD* = 0*.*48, range 29–30) for the controls. All participants provided signed informed consent agreements and received no financial compensation for their participation. The examinations were conducted at the neuropsychiatric office of the Aladár Petz County Research Hospital.

## **PROCEDURE**

The ASRT task was administered in two sessions separated by a 24-h interval. Participants were informed that the main aim of the study was to find out just how extended practice affected performance on a simple reaction time task. Therefore, we emphasized performing the task as fast and as accurate as they could. They were not given any information about the regularity that was embedded in the task.

In the first session the ASRT consisted of 20 blocks. As one block took about 1.5–2 min, the first session took approximately 30–40 min. Between blocks, participants received feedback on the screen about their overall reaction time and accuracy, then had a rest of between 10 and 20 s before starting a new block. Session 2 lasted approximately 22–30 minutes, as the ASRT consisted of 15 blocks.

The computer program selected a different ASRT sequence for each participant based on a permutation rule, such that each of the six unique permutations of the four possible stimuli occurred. Consequently, six different sequences were used across participants while the sequence within participants was identical during Session 1 and Session 2 (Howard and Howard, 1997; Nemeth et al., 2010).

## **THE ALTERNATING SERIAL REACTION TIME (ASRT) TASK**

Sequence learning was measured by the "Catch the dog" version (Nemeth et al., 2010) of the ASRT task (Howard and Howard, 1997). In this ASRT task, a stimulus (a dog's head) appears in one of four empty circles on the screen and participants have to press the corresponding button when it occurs. The computer is equipped with a special keyboard with four heightened keys (Y, C, B, and M on a Hungarian keyboard; equivalent to Z, C, B, M on a US keyboard), each corresponding to the circles in a horizontal arrangement.

Unbeknownst to participants, the appearance of stimuli follows a predetermined order. As stimuli are presented in blocks of 85 stimuli, the first five button pressings are random for practice purposes, then an 8-element alternating sequence (e.g., 2r3r1r4r, where numbers represents the four circles on the screen and r represents random elements) repeats ten times. Because of this structure, some triplets or runs of three consecutive events occur more frequently than others. For example, in the above illustration, 1\_4, 2\_3, 3\_1, and 4\_2 (where "\_" indicates the middle element of the triplet) would occur often because the third element (bold numbers) could be derived from the sequence or could also be a random element. In contrast, 1\_3 or 4\_1 would occur less frequently because in this case the third element could only be random. Following previous studies, we refer to the former as high-frequency triplets and the latter as lowfrequency triplets. Note that the final event of high-frequency triplets is therefore more predictable from the initial event when compared to the low-frequency triplets [also known as nonadjacent second-order dependency (Remillard, 2008)]. Therefore, for each stimulus we determined whether it was the last element of a high- or low-frequency triplet.

There are 64 possible triplets (43, 4 stimuli combined for three consecutive events) in the task. Out of these triplets, 16 are high frequency triplets, each of them occurring on approximately 4% of the trials, about five times more often than the low-frequency triplets. Thus, approximately 64% of all trials are high-frequency triplets and the remaining 36% of trials are low-frequency ones.

Previous studies have shown that as people practice the ASRT task, they come to respond more quickly to the high- than lowfrequency triplets, revealing sequence-specific learning (Howard and Howard, 1997; Howard et al., 2004; Song et al., 2007). In addition, general skill learning is revealed in the ASRT task in the overall speed with which people respond, regardless of the triplet types. Thus, we are able to obtain measures of both sequence-specific and general skill learning in the ASRT task.

## **STATISTICAL ANALYSES**

To facilitate data processing, the blocks of ASRT were organized into epochs of five blocks. The first epoch contains blocks 1–5, the second blocks 6–10, etc. (Bennett et al., 2007; Barnes et al., 2008). As participants' accuracy remained very high (98.1% for the MCI and 99.2% for the control group) throughout the test (similarly to previous studies, e.g., Howard and Howard, 1997; Nemeth et al., 2010), we focused on reaction time (RT) for the analyses reported. For RTs, we calculated medians for correct responses only, separately for high and low frequency triplets and for each participant and each epoch.

To compare the overall learning between the groups, RTs were analyzed by a mixed design ANOVA on the 7 epochs of Session 1 and 2 with TRIPLET (2: high vs. low) and EPOCH (1–7) as within-subjects factors and GROUP (MCI vs. control) as a between-subjects factor. For exploration of offline changes in the 24-h delay period, a similar ANOVA was conducted including only the last epoch of Session 1 and the first epoch of Session 2. All significant results are reported together with the η<sup>2</sup> *<sup>p</sup>* effect size and Greenhouse–Geisser ε correction factors where applicable. Planned comparisons and *post-hoc* analyses were conducted by Fisher's LSD pairwise comparisons.

## **RESULTS**

## **DO THE MCI AND THE CONTROL GROUP DIFFER IN OVERALL SEQUENCE LEARNING?**

The ANOVA revealed significant *sequence-specific learning* [indicated by the significant main effect of TRIPLET: *F(*1*,* <sup>32</sup>*)* = 18*.*50, η2 *<sup>p</sup>* = 0*.*37, *p <* 0*.*001] such that RTs were faster on high than on low frequency triplets (**Figure 1A**). The groups differed in the extent of this sequence-specific learning [shown by the significant TRIPLET × GROUP interaction: *F(*1*,* <sup>32</sup>*)* = 8*.*31, η<sup>2</sup> *<sup>p</sup>* = 0*.*21, *p* = 0*.*007]: the MCI group was 2.80 ms faster on high than on low frequency triplets (*p* = 0*.*32) while this difference was 14.20 ms for the controls (*p* = 0*.*001). Thus, only the controls acquired the sequence-specific knowledge overall.

The ANOVA also revealed *general skill learning* [shown by the significant main effect of EPOCH: *F(*6*,* <sup>192</sup>*)* = 42*.*70, η<sup>2</sup> *<sup>p</sup>* = 0*.*57, *p <* 0*.*001], such that RTs decreased across epochs, irrespective of the triplet type. This decrease was slightly different for the groups [EPOCH × GROUP interaction: *F(*6*,* <sup>192</sup>*)* = 2*.*33, η<sup>2</sup> *<sup>p</sup>* = 0*.*07, *p* = 0*.*078]: RTs decreased steeper in the MCI group (153 ms from the first epoch to the last epoch) than in the controls (95 ms). This difference was mainly caused by the MCI group's relatively slower RTs in the first epoch compared to that of the controls (790 vs. 692 ms, *p* = 0*.*07). This difference diminished for the last epoch (647 vs. 607 ms, *p* = 0*.*41). Other interactions were not significant (*p*s *>* 0*.*17).

Although the MCI and the control group performed with similar RTs [main effect of GROUP: *F(*1*,* <sup>32</sup>*)* = 1*.*99, *p* = 0*.*17], we re-ran our analyses using z-transformed RTs to confirm our findings. The ANOVA revealed sequence-specific learning [significant main effect of TRIPLET: *F(*1*,* <sup>32</sup>*)* = 43*.*77, *p <* 0*.*001] with significantly smaller learning for the MCI than for the control group [TRIPLET × GROUP interaction: *F(*1*,* <sup>32</sup>*)* = 4*.*01, *p* = 0*.*05]. After the z-transformation, the EPOCH × GROUP interaction was not significant [*F(*6*,* <sup>192</sup>*)* = 1*.*26, *p* = 0*.*31], suggesting a similar level of general skill learning in the two groups.

## **IS THERE ANY WITHIN-BLOCK EFFECT ON LEARNING? ARE THESE EFFECTS DIFFERENT IN THE MCI AND THE CONTROL GROUP?**

A fine-grained analysis of the data can give us a deeper insight into the mechanisms of the development of sequence representation; therefore, it can also help to better understand the above reported sequence-learning deficit in MCI compared to controls. Analyzing the learning data by splitting each block into two halves is an excellent approach for exploring these questions. Therefore, we conducted a mixed design ANOVA on the data shown in **Figures 1B**,**C** with TRIPLET (2: high vs. low frequency), EPOCH (7: 1–7) and PART (2: first vs. second half of blocks) as within-subject factors and GROUP (2: MCI vs. control) as a between-subject factor.

The ANOVA revealed significant sequence-specific learning overall [main effect of TRIPLET: *F(*1*,* <sup>32</sup>*)* = 18*.*27, η<sup>2</sup> *<sup>p</sup>* = 0*.*36, *p <* 0*.*001] with smaller learning for the MCI group compared to controls [4 vs. 14 ms; TRIPLET × GROUP interaction: *F(*1*,* <sup>32</sup>*)* = 5*.*62, η<sup>2</sup> *<sup>p</sup>* = 0*.*15, *p* = 0*.*02; **Figure 1D**]. Interestingly, taking the PART of the blocks into account, we found a significant TRIPLET × PART interaction [*F(*1*,* <sup>32</sup>*)* = 4*.*43, η<sup>2</sup> *<sup>p</sup>* = 0*.*12, *p* = 0*.*04]: the sequence-specific learning was greater in the second part of the blocks compared to the first part (6 vs. 12 ms). Although the TRIPLET × PART × GROUP interaction did not reach significance [*F(*1*,* <sup>32</sup>*)* = 2*.*62, η<sup>2</sup> *<sup>p</sup>* = 0*.*08, *p* = 0*.*12], planned comparisons revealed that the controls showed a similar extent of sequence-specific learning in the first and the second part of the blocks (13 and 14.5 ms, *p* = 0*.*73). In contrast, the MCI group showed higher sequence-specific learning in the second part of blocks than in the first part (1.7 vs. 9.6 ms, *p* = 0*.*01). All of these learning measures were significant (*p*s *<* 0*.*004), except for the first part of the blocks in the MCI group (*p* = 0*.*68). Thus, the group difference in sequence learning that we found in the previous analysis was driven mainly by the first part of the blocks (**Figure 2**), where the extent of sequence-specific learning was different between groups (*p* = 0*.*01), while they were similar in the second part of the blocks (*p* = 0*.*22).

**FIGURE 1 | (A)** Sequence learning across 7 epochs (35 blocks) for the MCI and control group. Circles represent RTs for high-frequency triplets and squares represent RTs for low-frequency triplets. **(B)** Learning curves for the first part of each block and **(C)** the second part of each block. **(D)** Sequence-specific learning (measured by the RTs for the low- minus high-frequency triplets) for the MCI and control group is plotted for the overall, first block-part and second block-part learning measures. Overall, the MCI group did not show significant sequence-specific learning, which was caused mainly by the learning performance in the first part of the blocks. The learning performance in the second part of the blocks was

similar in the groups. **(E)** General reaction times are plotted for the first and second parts of the blocks for the MCI and control group, separately. The MCI group was slower in the second parts of the blocks compared to the first parts of the blocks, but only in Session 1. The control group showed a similar pattern, but in Session 2. **(F)** Offline general skill changes (measured as the RT difference between Epoch 4 and Epoch 5, irrespectively of the triplet types) over the 24-h delay are plotted for the MCI and the control group with significant offline improvement for the controls only. Error bars represent standard error of mean. ns, non-significant, <sup>∗</sup>*p <* 0*.*05, ∗∗*p <* 0*.*01, ∗∗∗*p <* 0*.*001.

The ANOVA computed on z-transformed data confirmed our findings, as the TRIPLET × PART × GROUP interaction was significant [*F(*1*,* <sup>32</sup>*)* = 5*.*93, *p* = 0*.*02]. The MCI group showed significant sequence-specific learning only in the second halves of the blocks (*p <* 0*.*001) but not in the first halves (*p* = 0*.*29). In contrast, the controls exhibited significant sequencespecific learning both in the first and second parts of the blocks (*p*s *<* 0*.*001).

In the case of general skills, the ANOVA showed a significant improvement across epochs [main effect of EPOCH: *F(*6*,* <sup>192</sup>*)* = 42*.*42, η<sup>2</sup> *<sup>p</sup>* = 0*.*57, *p <* 0*.*001], with a trend toward group differences [EPOCH × GROUP interaction: *F(*6*,* <sup>192</sup>*)* = 2*.*46, η<sup>2</sup> *<sup>p</sup>* = 0*.*07, *p* = 0*.*06]. This latter effect is similar to the results of the previous analysis finding that RTs decreased steeper in the MCI group (149 ms from the first epoch to the last epoch) than in the controls (87 ms). This difference, however, diminished when analyzing z-transformed data: [EPOCH × GROUP interaction: *F(*6*,* <sup>192</sup>*)* = 1*.*23, *p* = 0*.*32].

There was also a trend for different degrees of general skill improvement in the first and second part of the blocks [EPOCH × PART interaction: *F(*6*,* <sup>192</sup>*)* = 1*.*91, η<sup>2</sup> *<sup>p</sup>* = 0*.*06, *p* = 0*.*08]: the speed-up from the first to the last epoch was

123 ms when analyzing only the first parts of the blocks, while it was slightly smaller in the case of the second parts of the blocks (106 ms). This was caused mainly by being faster in the second half of the blocks at the beginning of the task (737 vs. 746 ms in the first epoch), with a reverse pattern for the end of the task (631 vs. 624 ms in the last epoch). This effect remained and even became stronger after z-transforming the RTs [EPOCH × PART interaction: *F(*6*,* <sup>192</sup>*)* = 6*.*80, *p <* 0*.*001].

Groups further detailed this picture [significant EPOCH × PART × GROUP interaction: *F(*6*,* <sup>192</sup>*)* = 2*.*22, η<sup>2</sup> *<sup>p</sup>* = 0*.*07, *p* = 0*.*04; **Figure 1E**] as the MCI group was 12 ms faster in the first parts of the blocks compared to the second parts in Session 1 (*p* = 0*.*004) but showed similar RTs in Session 2 (1 ms difference between the RTs of the first and second parts of the blocks, *p* = 0*.*73). In contrast, the control group performed the task with similar RTs in Session 1 (2.6 ms difference, *p* = 0*.*51) but was 8.6 ms faster at the beginning of the blocks compared to the second parts in Session 2 (*p* = 0*.*01). This difference, however, disappeared when using z-transformed data [EPOCH × PART × GROUP interaction: *F(*6*,* <sup>192</sup>*)* = 0*.*02, *p* = 0*.*33]. No other main effects of interactions were significant (*p*s *>* 0*.*21).

#### **IS THERE ANY CHANGE IN LEARNING IN THE 24-h DELAY?**

For the exploration of the offline changes in the 24-h delay period, ANOVA was conducted with TRIPLET (2: high vs. low frequency) and EPOCH (2: the last epoch of Session 1 and the first epoch of Session 2) as within-subject factors and GROUP (2: MCI vs. control) as a between-subject factor.

The ANOVA revealed sequence-specific learning [indicated by the significant main effect of TRIPLET: *F(*1*,* <sup>32</sup>*)* = 19*.*68, η2 *<sup>p</sup>* = 0*.*38, *p <* 0*.*001] which was retained across the sessions [TRIPLET × EPOCH interaction: *F(*1*,* <sup>32</sup>*)* = 0*.*51, η<sup>2</sup> *<sup>p</sup>* = 0*.*02, *p* = 0*.*48]. The groups did not differ either in overall sequencespecific knowledge [TRIPLET × GROUP interaction: *F(*1*,* <sup>32</sup>*)* = 0*.*19, η<sup>2</sup> *<sup>p</sup>* = 0*.*01, *p* = 0*.*67] or in the offline change of this knowledge between the sessions [TRIPLET × EPOCH × GROUP: *F(*1*,* <sup>32</sup>*)* = 1*.*63, η<sup>2</sup> *<sup>p</sup>* = 0*.*05, *p* = 0*.*21].

In contrast, there was an offline improvement in general skills [main effect of EPOCH: *F(*1*,* <sup>32</sup>*)* = 5*.*32, η<sup>2</sup> *<sup>p</sup>* = 0*.*14, *p* = 0*.*028], with faster RTs in the first epoch of Session 2 compared to the last epoch of Session 1 (**Figure 1F**). This change was slightly different between groups [EPOCH × GROUP interaction: *F(*1*,* <sup>32</sup>*)* = 3*.*69, η<sup>2</sup> *<sup>p</sup>* = 0*.*10, *p* = 0*.*064]: the MCI group showed no betweensession speed-up (3 ms, *p* = 0*.*79) while the controls did (34.7 ms, *p* = 0*.*005). The ANOVA on z-transformed RTs confirmed this result, showing a weaker consolidation of general skills for the MCI than for the control group [marginally significant EPOCH × GROUP interaction: *F(*1*,* <sup>32</sup>*)* = 3*.*85, *p* = 0*.*06]. Other interactions involving the GROUP were not significant (*p*s *>* 0*.*71).

We also conducted a consolidation analysis taking the first and second parts of the blocks into account and found similar results, with significant group differences in offline general skill changes [EPOCH × GROUP interaction: *F(*1*,* <sup>32</sup>*)* = 4*.*30, η2 *<sup>p</sup>* = 0*.*12, *p* = 0*.*046]. The offline change in general skills was significant for the control group (35.8 ms faster at the beginning of Session 2 compared to the end of Session 1, *p* = 0*.*004) but not significant for the MCI group (1.65 ms difference, *p* = 0*.*89).

## **DISCUSSION**

Our goal was to investigate the acquisition of sequence knowledge in Mild Cognitive Impairment. We used a task that allows differentiating between sequence-specific and general skill learning. At first, based on the standard ASRT analysis we found that individuals with MCI showed weaker implicit probabilistic sequence learning than the healthy aged group. However, once we dug deeper and considered only the second half of each learning block, we found similar learning performances in the MCI as in the healthy aged group. Thus, the overall sequence-specific learning in MCI depends on which part of each learning block is considered. In the case of general reaction time, the MCI group was faster in the first part of the blocks compared to the second part in Session 1. The healthy aged group showed a similar pattern, except in Session 2. We were able to demonstrate that general skill consolidation over a 24-h delay period was different in MCI and in the healthy aged group. The latter group showed offline improvement in general reaction time while the MCI group did not show this speed-up effect. We believe our study to be the first one that uses an implicit sequence learning task with second-order dependency in individuals with MCI.

Our results partly contradict but partly support the findings of Negash and his colleagues (2007b), who showed learning with a deterministic SRT task in MCI but not in the Contextual Cueing task (Chun and Jiang, 1998). The impaired sequence learning that we found in MCI could be due to the more difficult and more complex sequence structure in our task, compared to the one used by Negash et al. (2007b). Another possibility is that deterministic and probabilistic sequence learning tasks are qualitatively different: the latter with their statistical properties are more ambiguous due to higher order associations in which a current event is predicted not by the preceding event but by the context of more remote events (Cohen et al., 1990; Keele et al., 2003). Thus, our result of impaired sequence learning in MCI is more similar to the results of the Contextual Cueing task in Negash et al.'s study. The Contextual Cueing task relies on visual search (e.g., find a horizontal T on the screen), which is generated within a background of some repeated distractor configuration (unknown to participants) providing a contextual cue to the location of the target. As a result of practice, the participants detect the target-stimulus in repeated configurations faster than in random configurations, even though they are not aware of the repeated distractors. This task calls on different neural systems than the SRT task (MTL-hippocampus vs. the frontal-striatalcerebellar system; Curran, 1998; Honda et al., 1998; Chun and Jiang, 1999; Gomez-Beldarrain et al., 1999; Manns and Squire, 2001; Willingham et al., 2002). Despite these differences in the involvement of different neural systems, our results suggest that the MTL and the hippocampal formation are also somehow involved in probabilistic sequence learning measured by the ASRT task. The within-block analysis can help us specify the nature of this involvement.

The result that the overall sequence-specific learning depends on whether we consider the first part or the second part of each learning block supports the suggestion of Rickard et al. (2008), who stressed the importance of the within-block position effect. However, we did not find a fatigue effect within the block in either group. Moreover, in the MCI group we showed significant overall sequence-specific learning when only taking the second part of the learning blocks into account, suggesting a warm-up or priming effect (cf. **Figure 2**). The fact that the MCI group exhibited significant sequence-specific learning in the second part of the blocks but not in the first part, suggests that the processes are qualitatively different between the first and the second part of the learning blocks. In the beginning of the blocks we have to recall and reactivate the sequence structure partly learned already in the previous blocks. The second part of each block might be responsible for the utilization and/or proceduralization of the sequence knowledge. Based on these assumptions, we claim that the detection of probabilities in the reactivation/recall phase is somehow impaired in MCI. In addition, as MTL structures, including the hippocampus are primarily affected in MCI (Jack et al., 1997; Dickerson and Sperling, 2008) and we found impaired sequence learning in the first part of learning blocks, the reactivation/recall of the sequence knowledge in the beginning of the blocks might be more MTL-dependent than in the second part. However, more studies are needed to confirm this suggestion.

These within-block effects also open a window to the similarities and dissimilarities between learning performance on the ASRT and the Contextual Cueing task. Although several neuropsychological studies have showed dissociation on the performance of these tasks, showing evidence of the different neurocognitive background (Howard et al., 2006; Negash et al., 2007a; Barnes et al., 2010; Simon et al., 2011), our results suggest that these two tasks somehow involve similar processes but only in the first part of the ASRT blocks. In this part of the blocks the reactivation/recall of the previously learned regularities is prominent. Moreover, in order to recover the previously acquired sequence memories, picking up the context information of the items at the beginning of each block is essential. As previous studies showed, these processes are linked to the hippocampus and related MTL structures (Wood et al., 2000; Gelbard-Sagiv et al., 2008; Xue et al., 2010). In sum, learning performance in specific parts of the ASRT seems to rely on the involvement of the hippocampus and related MTL structures.

Regarding general reaction times, we found that in Session 1 the MCI group was faster in the first part of the learning blocks compared to the second part, while this pattern was present for the control group in Session 2. Generally, slower RTs at the end of learning blocks than at the beginning suggest a build-up of fatigue within each block. This fatigue effect emerges later for the controls than for the MCI group. These results partly support the findings of Rickard and his colleagues (2008), who showed this fatigue effect masking some aspects of learning performance in a fingertapping task. Since the MCI group showed significant sequence-specific learning in the second half of the blocks, in spite of the fact that they were generally slower due to fatigue, we can claim that the impaired sequence-specific learning in the MCI group is not caused by this fatigue effect in our study.

Previous studies argue that the caudo-ventral putamen (Doyon and Benali, 2005; Debarnot et al., 2009) and the hippocampus (Albouy et al., 2008) can both play a role in the consolidation of sequence learning. Since MTL structures, including the hippocampus, are mostly affected by MCI (Dickerson and Sperling, 2008), our results that the MCI group did not forget the sequence in the 24-h delay period might suggest that these structures are not essential for the consolidation of sequence-specific knowledge, though they might affect the consolidation of general skill learning. This latter finding is in line with previous studies using fingertapping tasks (e.g., Walker et al., 2003), suggesting that general skill learning in our design might share similar neurocognitive background with motor learning. However, future studies need to clarify these similarities.

In sum, our findings that the detection of probabilities in the reactivation/recall phases of the learning is impaired in MCI draw attention to the importance of the hippocampus

**REFERENCES**


doi: 10.1523/JNEUROSCI.3295-10. 2010


and the related MTL structures in the development of sequence memory representation. Our results add detail to the picture regarding background processes of sequence acquisition and consolidation and refine Negash et al.'s (2007b) final conclusion that adapting to environment is preserved in MCI. Based on our findings, we believe that the reactivation phase of the detection of probabilities is impaired in MCI. If further studies with different methods, including functional brain mapping, confirm this view, it could lead to the development of more focused and more effective prevention and rehabilitation programs for minor and major cognitive disorders.

## **ACKNOWLEDGMENTS**

Thanks to István Winkler for the comments on the previous version of the manuscript. Thanks to Clare Allyce Cravey Tucker for the English proofing. This research was supported by OTKA MB08A 84743 and OTKA NF 105878.

46, 1624–1635. doi: 10.1016/ j.neuropsychologia.2007.11.030


cognitive impairment. *Lancet* 367, 1262–1270.


with MCI and Alzheimer's disease -a VBM study. *Int. J. Geriatr. Psychiatry* 23, 1114–1118. doi: 10.1002/gps.2036


measures. *Cogn. Psychol.* 19, 1–32. doi: 10.1016/0010-0285(87)90002-8


H. Jr., and Howard, D. V. (2011). Dopamine transporter genotype predicts implicit sequence learning. *Behav. Brain Res.* 1, 252–257. doi: 10.1016/j.bbr.2010. 08.043


*Neuropsychol.* 18, 490–500. doi: 10.1017/S135561771200001X


episodes occurring in the same location. *Neuron* 27, 623–633. doi: 10.1016/S0896-6273(00) 00071-4

Xue, G., Dong, Q., Chen, C., Lu, Z., Mumford, J. A., and Poldrack, R. A. (2010). Greater neural pattern similarity across repetitions is associated with better memory. *Science* 330, 97–101.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 02 January 2013; accepted: 10 June 2013; published online: 01 July 2013.*

*Citation: Nemeth D, Janacsek K, Király K, Londe Z, Németh K, Fazekas K, Ádám I, Elemérné K and Csányi A (2013) Probabilistic sequence learning in mild cognitive impairment. Front. Hum.* *Neurosci. 7:318. doi: 10.3389/fnhum. 2013.00318*

*Copyright © 2013 Nemeth, Janacsek, Király, Londe, Németh, Fazekas, Ádám, Elemérné and Csányi. This is an openaccess article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

## Directionality in distribution and temporal structure of variability in skill acquisition

## *Masaki O. Abe1 and Dagmar Sternad2,3,4,5\**

*<sup>1</sup> Research Center for Advanced Science and Technology, The University of Tokyo, Tokyo, Japan*

*<sup>2</sup> Department of Biology, Northeastern University, Boston, MA, USA*

*<sup>3</sup> Department of Electrical and Computer Engineering, Northeastern University, Boston, MA, USA*

*<sup>4</sup> Department of Physics, Northeastern University, Boston, MA, USA*

*<sup>5</sup> Center for the Interdisciplinary Research on Complex Systems, Northeastern University, Boston, MA, USA*

#### *Edited by:*

*Rachael D. Seidler, University of Michigan, USA*

#### *Reviewed by:*

*Young-Hui Chang, Georgia Institute of Technology, USA Julius Verrel, Max Planck Society, Germany*

#### *\*Correspondence:*

*Dagmar Sternad, Departments of Biology, Electrical and Computer Engineering, and Physics, Northeastern University, 134 Mugar Life Science Building, 360 Huntington Ave, Boston, MA 02115, USA e-mail: dagmar@neu.edu*

Observable structure of variability presents a window into the underlying processes of skill acquisition, especially when the task affords a manifold of solutions to the desired task result. This study examined skill acquisition by analyzing variability in both its distributional and temporal structure. Using a virtual throwing task, data distributions were analyzed by the Tolerance, Noise, Covariation-method (TNC); the temporal structure was quantified by autocorrelation and detrended fluctuation analysis (DFA). We tested four hypotheses: (1) Tolerance and Covariation, not Noise, are major factors underlying long-term performance improvement. (2) Trial-to-trial dynamics in execution space exhibits preferred directions. (3) The direction-dependent organization of variability becomes more pronounced with practice. (4) The anisotropy is in directions orthogonal and parallel to the solution manifold. Results from 13 subjects practicing for 6 days revealed that performance improvement correlated with increasing Tolerance and Covariation; Noise remained relatively constant. Temporal fluctuations and their directional modulation were identified by a novel rotation method that was a priori ignorant about orthogonality. Results showed a modulation of time-dependent characteristics that became enhanced with practice. However, this directionality was not coincident with orthogonal and parallel directions of the solution manifold. A state-space model with two sources of noise replicated not only the observed temporal structure but also its deviations from orthogonality. Simulations suggested that practice-induced changes were associated with an increase in the feedback gain and a subtle weighting of the two noise sources. The directionality in the structure of variability depended on the scaling of the coordinates, a result that highlights that analysis of variability sensitively depends on the chosen coordinates.

**Keywords: motor learning, variability, noise, skill, time series analysis, computational model**

## **INTRODUCTION**

The past decade has seen a number of studies on motor control and learning that used variability as a window into the underlying processes of skill acquisition. This approach is particularly promising when the task is redundant and affords a manifold of solutions that achieve the desired task result. Such mathematically infinite set of equivalent solutions may be advantageous as the complex sensorimotor system abounds with noise arising at all levels, ranging from variations in ion channel kinetics to amplitudes of action potentials (Faisal et al., 2008). As long as these variations remain within the space of equivalent solutions, the task goal can be achieved.

As early as 1933, Stimpel reported in a throwing task that the release variables showed covariation, such that the throwing precision was better than expected from the individual variables' variability (Stimpel, 1933). More recently, several lines of research have presented support that the sensorimotor system exploits the redundancy of the task by channeling variability into the directions that have no detrimental effect on the task goal (Scholz and Schöner, 1999; Müller and Sternad, 2004, 2009; Todorov, 2004; Cusumano and Cesari, 2006; Cohen and Sternad, 2009; Sternad et al., 2011). For example, using the well-established mathematical concept of null space, the Uncontrolled Manifold (UCM) approach showed that variations in direction parallel to the solution manifold, that are deemed irrelevant to task achievement and, hence, "uncontrolled," were larger than variability in direction orthogonal to the manifold (Scholz and Schöner, 1999). Hence, the ratio of variances in the two directions expresses the motor system's sensitivity to the solution manifold. A related mathematical approach by Cusumano and Cesari showed similar results (Cusumano and Cesari, 2006). The same concept has been part of the stochastic optimal feedback control framework, where only errors in directions irrelevant for task achievement are penalized by the cost function (Todorov and Jordan, 2002; Todorov, 2004).

Sternad and colleagues developed mathematically different tools in their Tolerance, Noise, Covariation approach (TNC) evaluating variability in terms of its cost to the result, rather than by its covariance in the space spanned by execution variables (Müller and Sternad, 2004, 2009; Cohen and Sternad, 2009). Tolerance evaluates sensitivity to noise in result space, Covariation evaluates the covariation between execution variables, and Noise quantifies the stochastic portion. Parsing the variability into the three components showed that all three contributed to performance improvement, albeit in different degrees: Tolerance improved fastest, while Covariation and Noise had significantly longer time scales (Cohen and Sternad, 2009). Note that unlike the covariance-based approaches, the TNC-analysis differentiates between changes in the overall magnitude of variability or noise and the extent of anisotropy or covariation. It also evaluates changes in the mean, which are outside the scope of covariance-based approaches. The current study complements the TNC-approach by an analysis of directionality in the temporal fluctuations of the data.

Some recent studies added further support to the hypothesis that the CNS channels its excess noise into "do-not-care" directions by examining the temporal structure of data. Projecting individual data in execution space into the directions parallel and orthogonal to the manifold, Dingwell and colleagues showed that the sequential structure in the data showed correlations, i.e., persistence and anti-persistence that differed in the two directions. In their study on treadmill walking, the execution space was defined by stride length and duration with constant (treadmill) speed defining the solution manifold (Dingwell et al., 2010). As hypothesized, the stride-to-stride fluctuations showed anti-persistence orthogonal to the manifold, a finding that was interpreted as error corrections. In a virtual reaching task the same group corroborated the directional differences, but showed persistence in both directions (Dingwell et al., 2012). A recent study on bipedal standing demonstrated higher temporal correlations of postural variability in task-equivalent directions (Verrel et al., 2012). Lastly, van Beers and colleagues reported that in a simple reaching task lag-1 autocorrelations were positive in the task-irrelevant direction, while they were zero in the task-relevant direction (van Beers et al., 2013).

While these studies provided evidence that humans are sensitive to task-relevant directions, several others examined whether this sensitivity is a result of practice. However, surprisingly, the results were not as consistent as expected. For example, Latash (2010) reviews results on UCM-based studies and reports that changes in anisotropy with practice were brought about by a decrease of variability in the orthogonal direction, increase in the parallel direction, or both. Dingwell's temporal analysis of directionality in reaching could not identify changes across two days of practice. As possible causes for these inconsistencies the researchers invoke insufficient duration of practice, or taskrelated differences, even though task complexity is not a very satisfying explanation. To address the issue of practice duration, the present study will examine performance in the skittles task over 6 days of practice, encompassing familiarization to perfection.

One further possibility for these evident differences in the results may be found in methodological issues that ultimately lead to a conceptual problem. Common to the analyses of data distributions and their fluctuations over time is that the analyses are performed in the space of execution variables. For example, analysis of multi-joint coordination with respect to a single target of the endpoint is analyzed in the space of joint angles; variability in gait speed is analyzed in the space of stride amplitude and duration; throwing accuracy is analyzed in the space spanned by position and velocity at ball release. The underlying assumption is that this space is the space in which the CNS "makes decisions." This is a daring assumption, as scientists do not yet know the coordinates of the CNS. Sternad and colleagues recently highlighted that the analysis of variability with respect to a null space is highly sensitive to the coordinates that the analysis is conducted in (Sternad et al., 2010). For example, for the UCM-based identification of anisotropy in joint space, the results depend on whether joint angles are defined in relative or absolute coordinates. As it remains unresolved which coordinates the CNS "cares about," an analysis that depends sensitively on a choice of coordinates may be misguided. Further, if the execution space does not have a metric, orthogonality is not defined. Hence, if directions are not pre-defined, the analysis of directionality is tenuous. This study presents a novel method that identifies the direction of maximal structure of variability in a given space, without an a priori assumption about what is orthogonal to the solution manifold. We will further show by example how rescaling of coordinates can change the results.

In overview, this study will examine skill learning by analysis of variability in both its distributional and temporal structure. We test four hypotheses: (1) Tolerance and Covariation, not Noise, are the major factors underlying long-term performance improvement. (2) Trial-to-trial dynamics in execution space has preferred directions with respect to the solution manifold. (3) This direction-dependent organization becomes more pronounced with practice. (4) The anisotropy in the distributional and temporal structure is in directions orthogonal and parallel to the solution manifold. For the identification of preferred directions in execution space, we will introduce a novel method that is a priori independent of orthogonality.

## **METHOD**

## **PARTICIPANTS**

Thirteen healthy participants (10 males and 3 females, 23–48 years) performed the experimental task after having been given informed consent in accordance with the Institutional Review Board of the Pennsylvania State University. They were righthanded according to the Edinburgh inventory for handedness (Oldfield, 1971). None of the participants had any disorders or injuries in their right limb motor function and they had normal or corrected vision.

## **EXPERIMENTAL SETUP**

The experimental task emulated the ball game skittles or tetherball where players throw a ball that is suspended on a string from a vertical post to hit a target skittle on the other side of the post (**Figure 1A**). The experimental set-up rendered this task in a virtual environment where the participant performed a real forearm movement and initiated the release of a ball by releasing the index finger from a contact switch; the ball only existed virtually (**Figure 1B**). The ball's trajectory traversed a virtual workspace

**FIGURE 1 | (A)** Real skittles task. **(B)** The virtual skittle task: participants operated a manipulandum in the horizontal plane that is shown online as a rotating paddle on the visual display. Ball release is triggered by releasing a switch with the index finger. **(C)** Visual display showing the paddle, the ball trajectory, the center post, and the target. The task result was defined as the distance between the trajectory and the target skittle, shown after execution in a zoom window. **(D)** The target color turned red when the distance error was below 1.2 cm.

that was projected for the participant onto a back-projection screen showing a top-down view (**Figure 1C**). The participant stood ∼0.6 m in front of the projection screen (width: 2.50 m, height: 1.80 m). The real-time display showed both the movements of the manipulandum and the ball traversing the center post toward the target. The participant was instructed to hit the center of the target. The error was defined as the shortest distance between the ball trajectory and the center of the target. At the end of a trial the ball's trajectory close to the target was shown in an enlarged window for 1 s after the throw to display the accuracy of the throw (**Figures 1C**,**D**). The post in the center of the workspace was represented by a circle of 16 cm diameter. The circular target had a radius of 1.5 cm and was located 50 cm above and 20 cm to the right of the center post. The participant's forearm movement was represented by a solid bar of 12 cm length that was fixed at one end, 50 cm below the center of the post. A circle of 1.5 cm radius representing the ball was "held" and "released" at the free end of the virtual arm by pressing the finger on the contact switch. The display was generated in Visual C++ and projected via an LCD projector (TLP 680U, Toshiba) onto the back-projection screen. The visuomotor delay between the movement and the online display was measured to be 22 ± 0*.*5 ms.

Participants placed their right forearm on a horizontal manipulandum padded with foam; the participant's forearm was fixed to it with Velcro straps. The height of the manipulandum was adjusted to be comfortable for each participant so that his/her upper arm was at ∼45◦ and the forearm was horizontal. The rotating end of the manipulandum was fixed to a vertical support with its axis of rotation directly underneath the elbow joint. The angular position of the manipulandum was recorded via a 3-turn potentiometer attached to the axis of rotation and was recorded at a sampling frequency of 700 Hz and displayed in real time on the screen (75 Hz update rate).

At the free end of the manipulandum a wooden ball, the size of a tennis ball, was attached. The participant grasped the ball with his/her right hand. A force transducer was attached to the ball located underneath the index finger. To simulate the throw of the ball, the participant moved the arm in an outward horizontal motion and released the ball by extending the index finger, thereby decreasing the force on the sensor. The arm movement resembled that of a Frisbee toss. Both the movements of the arm and the simulated trajectory of the ball were displayed on the screen. The ball's trajectory, as determined by the simulated physics of the task, traversed an elliptic path around the center post as determined by the model equations (see Cohen and Sternad, 2012 for details). This trajectory was not immediately intuitive to participants, and they had to learn the mapping between the real arm movements and the ball's trajectories in the projected workspace. Hence, the task was novel, even for those participants that had experience and skill in throwing.

The ball trajectories were simulated online based on the measured angle and derived velocity at the moment of release. To get the best possible online reading of release velocity and reduce contamination from measurement noise, the last 10 samples of the angular position before the moment of release were fitted with a straight line. This regression slope was used as estimate of the angular velocity at the release moment. This calculation added minimal delay to the display (in the order of 1 μs). To evaluate the error, the minimum distance between the trajectory and the center of the target was calculated.

The elliptic trajectories of the ball were generated by a twodimensional model in which the ball was attached to two orthogonal massless springs at the origin of the coordinate system (*x* = 0; *y* = 0 in the middle of the post), generating a restoring force proportional to the distance between the ball and the center post. Due to the restoring forces, the ball was accelerated around the center post. At time *t*, the equations for the position of the ball in *x*- and *y*-directions were:

$$\mathbf{x}(t) = A\_{\mathbf{x}} \sin \left(\alpha t + \varphi\_{\mathbf{x}}\right) e^{-\frac{1}{\mathbf{r}}} \tag{1}$$

$$\mathcal{Y}(t) = A\_{\mathcal{Y}} \sin \left(\alpha t + \varphi\_{\mathcal{Y}}\right) e^{-\frac{1}{t}} \tag{2}$$

The amplitudes *Ax* and *Ay* and the phases ϕ*<sup>x</sup>* and ϕ*<sup>y</sup>* of the sinusoidal motions of the two springs were calculated from the ball's *x–y* position and velocity at release, which were converted into angle and velocity with respect to the center post. The motions were lightly damped to approximate realistic behavior, with the parameter τ describing the rate of decay for the trajectory (for more detail, see Müller and Sternad, 2004).

## **EXPERIMENTAL PROTOCOL**

For this study participants stood with their shoulder axis at a right angle to the screen, the right shoulder close to the screen. The experimenter instructed participants to throw the ball in clockwise direction performing a forearm rotation as in a Frisbee backhand (see exemplary ball trajectory in **Figure 1C**). The position of the subject was chosen to make the forearm movement as comfortable as possible to avoid any biomechanical constraints. Aside from the zoomed image of target and trajectory, no explicit quantitative feedback was given. However, if the trajectory passed within 1.2 cm of the center of the target, the target color changed from yellow to red to give a reward signal for successful performance (**Figure 1D**). The experimenter encouraged participants to achieve as many of these hits as possible. Note that the error distance was always positive, similar to a darts board where the bull's eye is surrounded by iso-error circles.

Participants performed 180 throws per day. After each set of 60 throws, participants were allowed to take a short break. The sequence of throws was sufficiently engaging and the participants reported neither physically nor psychologically fatigued. Each participant performed 180 throws on each of the 6 days. The intervals between collection days were one or two days.

## **ANALYSIS OF DATA DISTRIBUTIONS: TOLERANCE, COVARIATION, AND NOISE**

With the goal to quantify how skill changes with practice, the TNC analysis was applied that parses variability into three components. The three components are expressed as costs, quantifying how much of the observed performance error could be improved by a change of Tolerance, Noise, and Covariation (for details see Cohen and Sternad, 2009). Tolerance or T-Cost evaluates how much performance could be improved if the same data distribution were in a more error-tolerant location in execution space. It is calculated by shifting the data in execution space to determine the best location with smallest performance error. Noise or N-Cost is a measure of how random scatter around the mean execution affects performance. It is calculated by shrinking the amplitude of the dispersion toward its mean to determine the scatter that produces minimum error. Covariation or C-Cost quantifies to what degree covariation among execution variables takes advantage of the orientation of the solution manifold. It is calculated by recombining the observed data in execution space and evaluating any improvement in the average results.

## **ANALYSIS OF DIRECTIONALITY IN EXECUTION SPACE**

**Figure 2A** shows the data distributions of 3 days plotted in execution space spanned by angular position and velocity of the paddle; the color shades code the magnitude of error for all position-velocity combinations if the ball were released at this position-velocity combination. The set of zero-error solutions defines the solution manifold, which is a one-dimensional set shown in white. The black areas indicate position-velocity combinations, i.e., ball releases, where the ball would hit the center post. The blue symbols are the 180 throws per day showing a small decrease in scatter with practice, concentrating increasingly more on the light-colored area, where errors are small or 0; on day 6 a more elliptic distribution in alignment with the solution manifold is visible. For the time series analysis the data of the six practice days were first pooled into three blocks to ensure a sufficient number of data: Block 1: day 1 and 2, Block 2: day 3 and 4, Block 3: day 5 and 6. **Figure 2B** shows the associated time series of the execution variables (position and velocity) and the result variable (error) across the entire 6 days (or three blocks) with 1080 throws.

The time series analyses on the directionality of changes from trial to trial were conducted in execution space spanned by angular position and velocity. However, due to the different units of position and velocity, distance, and orthogonality are not defined in this space (Sternad et al., 2010). A commonly used procedure to overcome this problem is to normalize the units by dividing the variables by their standard deviations:

$$\times\_{\rm l}(i) = (p\,(i) - \overline{p}) / \sigma\_{\rm p} \tag{3}$$

$$\propto\_2(\hat{\imath}) = (\nu(\hat{\imath}) - \overline{\nu}) / \sigma\_{\hat{\imath}} \tag{4}$$

where *x*1*(i)* and *x*2*(i)* denote normalized position and velocity, *i* is the trial number, *p*(*i*) and *v*(*i*) position and velocity, *p* and *v* are means of one block, and σ*<sup>p</sup>* and σ*<sup>v</sup>* are standard deviations of position and velocity of the same block. Note that this procedure assumes that covariance can be used to define a metric and that the metric only has diagonal entries. This normalization was performed for each participant and each block separately.

To assess whether the trial-to-trial changes had a directional preference, the data of one block were projected onto a line through the center of the data set:

$$x\_{\theta} \left( \right) = x\_1 \left( \right) \cos \theta + x\_2 \left( \right) \sin \theta \tag{5}$$

where *x*θ*(i)* denotes the new variables after projection onto the line. The angle θ of this line was defined as 0 when parallel to the x-axis or position direction; θ = 0*.*5π rad when parallel to the y-axis or velocity direction (**Figure 3A**). The direction parallel to the solution manifold was defined as θpar for each individual; the direction orthogonal to the solution manifold was defined as θort. The center of the data was defined by the median of the position and the median of the velocity data for each block of each individual (the median was chosen to avoid any bias from outliers). This line was then rotated through 0 *<* θ *<* π rad, in 100 steps of 0.01∗π rad. At each rotation angle, the data were projected onto the line (Equation 5) and the time series of the projected data was evaluated as described next. Note

time series *x*θ*(i)* is obtained by projecting the variables onto the rotation axis as expressed in the equation. **(B)** The time series at the top shows *x*θ*(i)* for the rotation angle θ with the maximum autocorrelation value, the bottom time series shows *x*θ*(i)* for the rotation angle θ with the smallest autocorrelation.

that this point of rotation was close, but not exactly on the solution manifold, especially early in practice. (The average distance from the solution manifold measured in terms of error was 1.21 ± 1.0 cm.)

#### **ANALYSES OF TIME SERIES**

We evaluated the temporal structure of *x*θ*(i)* for each rotation angle θ of the line through the center of the data set. At each angle both autocorrelation and detrended fluctuation analysis (DFA) were computed. From the autocorrelation analysis, only the lag-1 coefficient (abbreviated as AC1) was reported. To assess temporal structure beyond lag-1 the DFA was evaluated. DFA is a modification of the root-mean square analysis of a random walk (Feder, 1988) but is less sensitive to non-stationarities and noise in the data.

For the DFA analysis, the time series was first cumulatively summed to obtain an integrated time series; this integrated series was linearly detrended within a given window *n*. The root mean square value of the detrended time series *F*(*n*) was calculated for different window sizes *n*. Plotting *F*(*n*) over *n* on a log-log scale the scaling index SCI was obtained from the slope of the linear regression of log*F*(*n*) over log(*n*). This scaling index quantifies the long-range correlations of the time series. If SCI = 0.5, the time series has no time correlation as in white noise. If 0.5 *<* SCI *<* 1.0, the time series is categorized as a stationary signal with fractal noise (Eke et al., 2002). In this case, he increasing and decreasing tendency of the time series persists. Using sets of 60 trials the slope was calculated for window sizes between 6 and 20 trials. Although this size of the samples is relatively short compared to other applications of the DFA analysis, we opted for this size to avoid discontinuities that may arise from subjects taking short breaks. We calculated both AC1 and SCI in the time series of angular position, velocity, and error. **Figure 3B** shows two time series of the projected data for the two directions θ that showed the minimum and maximum values of the autocorrelation analysis (which was very close to the minimum and maximum of the DFA). The difference in fluctuation profile is visible by eye. For comparison, autocorrelation and DFA analyses were also performed on surrogate data. These surrogate data were produced by randomly shuffling the time series. These analyses were conducted 20 times to obtain mean results and standard deviations similar to the data.

#### **STATISTICAL ANALYSES**

The changes in error and T-Cost, C-Cost, and N-Cost across practice were fitted by exponential functions to assess the different time scales of change. Pearson correlations between TNC-Costs and error revealed contributions of the costs to error. The directionality analysis of AC1 and SCI, specifically its maximum and minimum values, θmax and θmin, were analyzed with *t*-tests and Smirnov-Grubbs tests to compare them against directions of the solution manifold and the angle and velocity direction. Two-way repeated-measures ANOVAs were used to assess changes with practice. The significance level was set to 0.05. Analyses were conducted with SPSS v16.

## **RESULTS**

### **PERFORMANCE IMPROVEMENT**

Before analyzing variability as a function of practice we first assessed whether participants indeed showed the expected performance improvement. The average error, or distance to the target, in sets of 60 trials was plotted to establish that participants showed the expected learning. The individual error profiles of the 13 subjects were fitted by exponential functions and are summarized in **Figure 4A** (to avoid clutter, data points are not shown). While 3 participants performed with small error from the beginning of practice and showed no improvement (P2, P6, P12, shown by red lines), 10 individuals showed a visible decrease in error; the *R*2-values of their exponential fits were between 0.26 and 0.95.

**FIGURE 4 | (A)** Performance errors for all 13 participants across 18 sets of 60 trials each. The blue and red lines represent the exponential fits for all 13 individuals. Red lines indicate the three subjects that did not show any improvement across practice. The insert shows the average error across all 13 subjects; the error bars denote standard errors across participants. **(B)** Participant averages of T-Cost, C-Cost, and N-Cost across all 18 sets in the 6 days of experiments.

The inset of **Figure 4A** shows the average decrease of error of all 13 participants across the 18 sets. From an initial 9.19 cm in the first set, the average error declined exponentially to 2.98 cm in the final set; the *R*<sup>2</sup> of the exponential fit was 0.96, the time constant was 1.07. Subsequent analyses were conducted with both inclusion and exclusion of the three individuals who showed no improvement, but the statistical results were not affected.

#### **TOLERANCE, COVARIATION, AND NOISE**

**Figure 2A** showed an exemplary participant's data distributions in execution space on Days 2, 4, and 6. As could be seen, the relatively isotropic data distributions on Day 2 and Day 4 showed a visible change on Day 6, where the data started to cluster along the solution manifold. Interestingly, the data showed little decrease in the overall amplitude of dispersion. This observation

**Table 1 | Results of correlations between error and T-Cost, C-Cost, and N-Cost (Pearson correlation coefficients** *r***).**


*Participants 2, 6, and 12 are the ones that did not show any improvements in the error measure (Figure 4).*

*\*p < 0.05; \*\*p < 0.01; \*\*\*p < 0.001.*

was quantified by T-Cost, C-Cost, and N-Cost. **Figure 4B** shows the three costs averaged over all participants across the 18 practice sets. T-Cost shows a rapid decline and reaches a plateau at set 3, which corresponds to the end of day 1. C-Cost also shows a visible decline which lasts over the first 6 sets, leading to a plateau thereafter. N-Cost did not show any obvious improvement. To directly test whether error was reduced via changing C-Cost, Pearson correlations were performed between error and T-Cost, C-Cost, and N-Cost for each participant. **Table 1** summarizes these results: as expected, both T-Cost and C-Cost showed significant positive correlations with error in most participants. While causal conclusions cannot be made, these results nevertheless strongly suggest that Tolerance and Covariation in the execution variables lead to the observed reduction in error. In contrast, N-Cost only showed 4 significant positive correlations. Three of these correlations were seen in the 3 participants that showed low error scores right from the beginning (P2, P6, P12). This suggests that small changes in N-Cost may still account for some of the changes across the trial sets in the three very good subjects. In sum, these results were consistent with Hypothesis 1.

#### **AUTOCORRELATION AND SCALING INDEX**

Exemplary time series at the minimum and maximum value of AC1 were already presented in **Figure 3B** to visualize that the structure of their fluctuations was different. **Figure 5** summarizes the results of AC1 and SCI as a function of direction θ. Note that θ = 0 rad was defined as parallel with the x- or position-axis and θ = π*/*2 rad was parallel with the y-axis or velocity in execution space. Hence, the orthogonal and parallel direction, indicated by the green vertical lines, differed for each subject as they centered their data at slightly different locations with respect to the solution manifold. The six panels show the average AC1 and SCI

over all participants for each direction θ across the three blocks; the shaded areas around the solid red line indicate one standard deviation across all participants.

While AC1 was predominantly positive, the values also showed a clear modulation with θ, especially in Blocks 2 and 3. Similarly, SCI was consistently between 0.5 and 1.0 and showed an equivalent modulation with θ. The directions at which AC1 and SCI reached their minima and maxima, θmin and θmax, are indicated by triangles. The blue lines show the results for the time-shuffled surrogate data with each value representing an average from 20 repeated shuffles. As expected, these results did not show any modulation across θ and were close to 0 and 0.5, respectively. Hence, the data showed persistence in all directions but of varying degree, as stated in Hypothesis 2.

**Figure 6** summarizes the changes of the AC1 and SCI minima and maxima across the three blocks showing the means across participants and their standard deviations. Both extrema of AC1 at θmax and θmin were subjected to a 3 (block) × 2 (variable) repeated-measures ANOVA. It rendered a significant interaction, *F(*2*,* <sup>24</sup>*)* = 4*.*69, *p* = 0*.*019, and both main effects were significant: block, *F(*2*,* <sup>24</sup>*)* = 7*.*43, *p* = 0*.*003, and variable, *F(*1*,*12*)* = 264*.*96, *p <* 0*.*001. *Post-hoc* tests showed that AC1 at θmin decreased significantly from Block 1 to Block 2 and to Block 3 (*p <* 0*.*05). These observations were consistent with Hypothesis 3. In contrast, AC1 at θmax did not show significant changes across blocks. The same ANOVA for SCI showed equivalent results: the interaction was significant, *F(*2*,* <sup>24</sup>*)* = 7*.*85, *p* = 0*.*002, as were the main effects for block, *F(*2*,* <sup>24</sup>*)* = 8*.*89, *p* = 0*.*001, and variable, *F(*1*,* <sup>12</sup>*)* = 202*.*23, *p <* 0*.*001. The values of θmin changed significantly from Block 1 to Block 2 and to Block 3 (*p <* 0*.*05), while θmax did not show any significant differences among blocks.

In Hypothesis 4 we stated that long-range correlations should be maximal in the direction parallel to the solution manifold and minimal in the direction orthogonal to the solution manifold. Returning to **Figure 5** shows *SM*par and *SM*ort averaged across all participants depicted by the green vertical lines; the thin lines indicate one standard deviation across all participants. To test Hypothesis 4 the angles of *SM*par and θmin and of *SM*par and θmax were computed for each subject and compared by pairwise *t*-tests. The results were only partially consistent with this hypothesis: the minima were close to *SM*ort, while the maxima significantly differed from *SM*par. The average angle differences between *SM*ort and θmin across all subjects and all three blocks were: 0*.*24 ± 0*.*39 rad for AC1 and 0*.*23 ± 0*.*42 rad for SCI. The average differences between *SM*par and θmax across all three blocks were 0*.*55 ± 0*.*45 rad for AC1 and 0*.*36 ± 0*.*65 rad for SCI. These differences were statistically significant from zero (*p <* 0*.*01) and did not show any changes across blocks. These results were not consistent with Hypothesis 4.

To further assess whether the observed extrema indicated sensitivity to the solution manifold as hypothesized, or whether they were merely coincident with the measured variables angular position and velocity, Smirnov-Grubbs tests evaluated whether θmin and θmax differed from the position or velocity direction, 0 or

π/2 rad, respectively. Results showed that for both AC1 and for SCI θmin was not significantly different from π/2 rad (velocity) in Block 1, but differed in Block 2 and Block 3 (*p <* 0*.*01). The autocorrelations at θmax were not significantly different from AC1 in the position direction or 0 rad in all blocks (*p >* 0*.*05). In sum, the direction of maximum persistence was observed in the angle direction.

## **MODELING**

The observed results showed significant changes in the structure of variability, both in distributions and in their temporal fluctuations. However, several aspects in the time series analyses also deviated from the expectations formulated in Hypotheses 2 and 4: the autocorrelations were overall positive (counter Hypothesis 2), and the maxima and minima in the temporal structure deviated from the parallel and orthogonal directions defined by the solution manifold (counter Hypothesis 4). A simplified model of skittles was used to shed light on these unexpected observations and suggest possible mechanisms that account for them. The model will not only reproduce the positive autocorrelations and its changes with practice, it will also show how a rescaling of the execution coordinates may sensitively skew the results, highlighting that reliance of the analysis on a pre-defined orthogonality in execution space may be misguided.

The model was kept as simple as possible, yet captured the essential component of the skittles task—redundancy. The simplification made the task similar to a line-reaching task: There

were two execution variables *x*<sup>1</sup> and *x*<sup>2</sup> (like position of an endpoint in the plane) and there was one result variable, the error *e*, or distance from reaching the line (**Figure 7**). The task was to change execution (*x*1, *x*2) to be on the line, defined by *x*<sup>1</sup> – *x*<sup>2</sup> = 0. The error *e* or result variable was defined as the minimum distance between the execution (*x*1, *x*2) to the solution manifold. Note that this error definition simplifies the skittles task as it excludes the dynamics of the ball trajectory. In skittles, the ball dynamics creates an approximately parabolic increase of the error orthogonal to the solution manifold; the linearization in this simplified model is acceptable for a sufficiently small neighborhood. Importantly, the model has redundancy, mapping two execution variables into one result variable, analogous to the skittles task.

To simuate trial-by-trial learning and the increasing anisotropy in the data distributions with practice, the main assumption was that the execution variables **x***(i)* = [*x*1, *x*2] *<sup>T</sup>* were updated based on the previous states. The error was defined as **e***(i)* = **x***(i)*–**x**Target, where **x**(*i*) was the output state and **x**Target was the target state defined in the workspace. The target state defined the point closest to **x**(*i*) on the line or solution manifold:

$$\mathbf{x}\_{\text{Target}} = \left[\frac{\mathbf{x}\_1 + \mathbf{x}\_2}{2}, \frac{\mathbf{x}\_1 + \mathbf{x}\_2}{2}\right]^T \tag{6}$$

Consequently, the error was defined as:

$$\mathbf{e}(i) = \mathbf{x}(i) - \mathbf{x}\_{\text{Target}} = \left[\frac{\boldsymbol{\varkappa}\_1 - \boldsymbol{\varkappa}\_2}{2}, \frac{\boldsymbol{\varkappa}\_2 - \boldsymbol{\varkappa}\_1}{2}\right]^T \tag{7}$$

The motor command **u***(i* + 1*)* was obtained by subtracting the error signal *e*(*i*) from the previous *i*th motor command **u**(*i*). The error was weighted by the feedback gain *B*. Two sources of additive noise were included: *rE* was added to the motor command **u**(*i*) at the execution level; at the planning level *rP* was added to obtain the updated command **u***(i* + 1*)* (van Beers, 2009). Both noise sources were independently drawn from a Gaussian distribution with 0 mean and unit amplitude η*(i)*. The coefficient ω defined the relative magnitude of the two noise sources *rP* and *rE*. The model is summarized as follows:

$$\begin{aligned} \mathbf{x}(i) &= \mathbf{u}(i) + \mathbf{r}\_{\mathbb{E}}(i) \\ \mathbf{e}(i) &= \mathbf{x}(i) - \mathbf{x}\_{\text{Target}} \end{aligned} $$

$$\begin{aligned} \mathbf{u}(i+1) &= \mathbf{u}(i) - Be(i) + \mathbf{r}\_{\mathbb{P}}(i+1) \\ \mathbf{r}\_{\mathbb{P}}(i) &= \omega \eta\_1(i) \end{aligned} \tag{8}$$

$$\begin{aligned} \mathbf{r}\_{\mathbb{E}}(i) &= (1-\omega)\eta\_2(i) \end{aligned} \tag{9}$$

In forward simulations, 50 values for the feedback gain *B* (between 0 and 0.5), and 20 values for the relative noise magnitude ω (between 0 and 1) were tested. For each of the 1000 parameter combinations we simulated 100 runs with different initial values for the noise sources *rE* and *rP*; the initial value for **u**(0) was always (0, 0). For each simulation output, autocorrelations AC1 were calculated for all direction angles, using the same procedure as for the experimental data. Given that the autocorrelation analysis and the DFA rendered consistent results in the experimental data, the analyses were confined to the autocorrelations.

## **SIMULATION RESULTS**

Exemplary data distributions and time series in the principal directions for three different parameter combinations are presented in **Figure 8**. The first simulation result with *B* = 0 and ω = 0 illustrates the case where planning noise *rP* was 0 and there was no error fed back to the update of **u***(i)*. Not surprisingly, the distribution in *x*1–*x*2-execution space was isotropic and the time profiles over 1000 iterations of the error signal in both parallel and orthogonal directions were random, as indicated by the AC1 values close to 0. The second row illustrates how the presence of the second noise source changed the distribution and the temporal structure of the noise: the distribution became larger and the autocorrelations in the two directions became positive. Note that the feedback gain *B* was still 0. The third row illustrates the case where both noise sources (ω = 0*.*10) and error feedback (*B* = 0*.*20) were present: the distribution shows covariation and the autocorrelations parallel to the solution manifold were positive, while they were close to 0 in the orthogonal direction. Despite these significant differences in distribution and temporal structure in the three parameterizations, the overall magnitude of the variability was similar.

**Figure 9** summarizes the simulation results for selected parameter combinations in the same format as the data summary in **Figure 5**. Setting ω = 0*.*10 as in **Figure 8**, **Figure 9A** illustrates the values of AC1 across all direction angles θ for three different feedback gains *B*. The parallel and orthogonal directions with respect to the solution manifold were symmetric at 0.25π and 0.75π rad. The simulations revealed that the magnitude of

series of error in two directions in execution space.

*B* selectively affected AC1 in *SM*ort: zero feedback gain led to positive AC1 values in directions *SM*par and *SM*ort; for increasing feedback gains the modulation of AC1 at *SM*ort became more pronounced and AC1 became slightly negative. This is intuitive and reflects the increasing influence of corrections that minimize the error. **Figure 9B** depicts the effect of the relative noise amplitudes ω on AC1, keeping *B* fixed at 0.20. The modulation of AC1 at *SM*par showed a decrease of AC1 for smaller ω. Hence, the experimentally observed modulations across direction angles and the signs of AC1 reflect the relative magnitude of the noise and feedback parameters.

A different summary of the AC1 results for all *B* and ω parameter combinations is shown in **Figure 10A**, results at *SM*par are shown in the left panel, at *SM*ort in the right panel. The magnitude of AC1 is represented by color, with red showing positive values and blue showing negative values. At *SM*par AC1 was mainly affected by the noise ratio ω; at *SM*ort AC1 was affected by both variables ω and *B*. As is to be expected, the larger the feedback gain *B*, the more negative the autocorrelations in the orthogonal direction. For small feedback gains, the noise ratio has a significant effect on AC1, which disappears at higher values of *B*.

**Figure 10B** shows the magnitude of AC1 at *SM*par and *SM*ort for a constant ω = 0*.*08. The specific ω-value was chosen because it generated similar AC1 results as seen in the experimental data. When AC1 was plotted as a function of feedback *B*, the figure shows that AC1 at *SM*ort decreased, while AC1 at *SM*par maintained almost the same value throughout. This pattern was qualitatively and quantitatively similar to the change of AC1 at θmax and θmin in the experimental results (**Figure 6**). It suggests that changes in performance were mainly brought about by changes in the feedback gain.

One important observation is that, different from the experimental results, the minima and maxima of AC1 in **Figure 9** were exactly at 0.25π and 0.75π rad. This is to be expected for the linear manifold that is defined at 0.25π rad (45◦) in execution space. Furthermore, the simple model assumed an execution space with two variables of the same units such that the space had a metric and orthogonality was defined. This contrasts with the

experimental case where the two execution variables had different units and normalization was applied to allow for a distance measure and definition of angle. However, this normalization is necessarily a crutch as we do not know the true metric of the variables inside the nervous system.

To illustrate how a scaling of the variables may thwart orthogonality and thereby the minima and maxima of the temporal structure, we performed model simulations with different types of rescaling of the execution variables. To emulate the case where the state variables may be rescaled "inside the CNS", we conducted simulations where **x**(t) was rescaled at each iteration. Specifically, we included a rescaling of *x*1: *x* <sup>1</sup>*(i)* = α*(u*1*(i)* + *rE)*, where α is the scaling factor. Setting the system parameters to *B* = 0*.*10 and ω = 0*.*08, we performed the simulations with α = 2 and α = 0*.*5. In a first set of simulations the solution manifold was not changed. This case emulated the interpretation that the solution manifold was defined in external physical space, where the units are given. In a second set of simulations, the solution manifold was adapted to the rescaling of variables.

**Figure 11** summarizes the results: the panels on the left show the data from 1000 runs in execution space together with the linear solution manifold (black line). The panels on the right display AC1 of the time series as a function of direction, in the same format as the experimental and model data in **Figures 5**, **9**, respectively. The red line represents the mean of 1000 runs for each of the 100 directions, the green vertical lines denote the parallel and orthogonal directions of the solution manifold. Panels **(A)** and **(B)** show the simulation results with SM: *x*<sup>1</sup> = *x*2, α = 0*.*5; panels **(C)** and **(D)** shows results with SM: *x*<sup>1</sup> = *x*2, α = 2; panels **(E)** and **(F)** show the case of SM: *x*<sup>1</sup> = 2*x*<sup>2</sup> and α = 2. Note that in **Figure 11D**, the parallel and orthogonal directions of the solution manifold were unchanged, while they were shifted in **Figure 11F**. The minima and maxima of AC1 are highlighted by the triangles as in the experimental results in **Figure 5**.

The results for both rescalings exhibited a modulation similar to what was shown in **Figure 9**: however, the maxima and minima were no longer at *SM*par and *SM*ort. Comparing these results with the experimental data in **Figure 5** shows that the skewing in the scaling where α = 2 was very similar to the data. The maximum is to the left of *SM*par and the minimum is close to *SM*ort. This skewing was relatively unaffected by the concomitant scaling of the solution manifold. Additional simulations were run where we rescaled all data after the simulations were completed showed similarly skewed modulations. These modeling results suggest that the experimental deviations from the hypothesized pattern can be ascribed to such scaling in the variables at one stage of the processing. However, as the model is a simplification of the actual system dynamics, we do not venture to equate this model exercise with the actual variable scaling in the central nervous system.

## **DISCUSSION**

The hypothesis that humans are sensitive to the direction of the solution manifold has found support in several lines of research that examined variability with respect to task-relevant and irrelevant dimensions. Using the skill of goal-directed throwing, our experimental and modeling work presents new results that reveal how practice changes both the distributional and temporal structure of data. Further, our new analysis method highlights an important issue: variability analysis is sensitive to the coordinates. As we do not know the coordinates that the CNS operates in, results may be skewed.

We summarize our results with respect to the four hypotheses: (1) Tolerance and Covariation increased with practice, and T-Cost and C-Cost correlated with the decreasing error; Noise,

**FIGURE 11 | Model analysis with one of the two execution variables scaled.** The left panels show the data distributions of 1000 simulation runs plotted in execution space. The black line denotes the solution manifold. The right panels summarize the results of the autocorrelation analysis plotted as a function of direction. The red lines with the shaded bands represent the mean and standard deviations across 1000 simulations for each of the 100 directions; the green vertical lines denote the parallel and orthogonal direction of the solution manifold. Panels **(A)** and **(B)** show the simulation results with SM: *x*<sup>1</sup> = *x*2, α = 0*.*5; panels **(C)** and **(D)** shows results with SM: *x*<sup>1</sup> = *x*2, α = 2; panels **(E)** and **(F)** show the case of SM: *x*<sup>1</sup> = 2*x*<sup>2</sup> and α = 2. In all cases, the minima and maxima of the autocorrelations, marked by triangles, have shifted away from the orthogonal and parallel directions of the solution manifold. The simulations in panel **(D)** show very similar results to the experimental data in **Figure 5**.

estimated as N-Cost, remained constant. (2) The temporal dynamics of the trial-by-trial data exhibited preferred directions; the structure showed mostly persistence, as quantified by positive autocorrelations and a scaling index greater than 0.5. (3) Six days of practice not only led to improvement in overt performance, but also to an increasing directionality in the temporal structure in execution variable. Model results suggest that this change can be ascribed to increases in the feedback gain. (4) The directions with maximum and minimum structure in the fluctuations were not coincident with the directions orthogonal and parallel to the solution manifold. Simulations of a simplified model of the skittles task helped to interpret these findings. Similar deviation were obtained when applying a simple linear rescaling to one of the state variables.

## **TOLERANCE, COVARIATION, AND NOISE**

The decomposition of variability into Tolerance, Covariation, and Noise revealed that the main contribution to practice-induced decrease of error stemmed from Tolerance and Covariation, as estimated by T-Cost and C-Cost. Noise or N-Cost remained constant throughout the 6 days. These differential results for the three components highlighted that practice-induced decreases in variability, commonly quantified by decreasing standard deviations of error or other performance variables, should not be immediately equated with a reduction of stochastic processes. While the present data suggested that stochastic processes were not affected by practice, previous results on 15 practice sessions gave evidence that reduction in noise processes may just have a very slow time scale (Cohen and Sternad, 2009). As in previous studies, Tolerance was a significant factor contributing to error reduction and dropped early in practice. T-Cost quantifies how the data mean, or location in execution space changed with practice. To account for this change in the mean, the directionality analysis was centered for each individual and each block. Covariation had a slower time scale but also significantly contributed to performance improvement. The different time scales of the three components probably reflect the multiple time scales of plastic changes in the nervous system (Kiebel et al., 2008). Note that this parsing of variability into Tolerance, Covariation, and Noise is unique to the TNC-approach. Analyses that focus on the anisotropy using covariance-based methods with respect to mean performance cannot parse the overall decrease in noise, nor detect a possible bias (Latash et al., 2002; Latash, 2008). The fact that Covariation became more pronounced provided the basis for the analysis of temporal fluctuations in different directions.

## **DIRECTIONALITY AND PERSISTENCE IN TEMPORAL DYNAMICS**

The trial-to-trial dynamics in the directional execution variables showed a clear modulation of structure in different directions, supporting the overall hypothesis that humans are sensitive to the orientation of the solution manifold. This result is consistent with Dingwell's and van Beers' results, although the studies differ in the kind of structure seen in orthogonal and parallel directions. Initially, negative correlations were expected orthogonal to the solution manifold, compared to persistence in the goal-irrelevant direction, as was reported by Dingwell and colleagues in their study on treadmill walking (Dingwell et al., 2010). In contrast, our study revealed positive autocorrelations in both orthogonal and parallel directions, similar to what van Beers et al. (2013) report for three different tasks. One possible reason for Dingwell's results could be that successive strides are not independent, and the temporal sequence of strides can induce negative autocorrelations. For example, any small measurement error in temporally adjacent variables, such as overestimating one stride length, has the inverse effect on the next stride and underestimates the next stride. Similarly, inertial "carry-over" effects can also enhance this observation. See also the "clock-motor" model on rhythmic timing by Wing and Kristofferson where the effect of noise creates negative lag-1 autocorrelations into the sequence of inter-response intervals. As the authors point out, these negative autocorrelations are simply due to the temporal adjacency of intervals in the presence of a noisy "clock," not corrective feedback processes (Wing and Kristofferson, 1973a,b).

As the model simulations made explicit, the effect of two added noise sources could lead to positive autocorrelation obscuring the effect of possible corrections. Negative autocorrelations only emerged when the feedback gain became relatively large. One other potential account for the persistence in the data is that subjects did not have direct error information. One challenge in the skittles task is that the visible error is non-linearly mapped onto the execution variables position and velocity at ball release. Hence, subjects may try a "blind" gradient descent to find the best release parameters. Previous studies suggested that when knowledge of results was withheld or when visual information was occluded, the temporal structure of the task output was not white noise but had persistent characteristics (Blackwell and Newell, 1996; Baddeley et al., 2003; Miyazaki et al., 2004). The fact that in our study the scaling index and the autocorrelations showed a consistent pattern gives evidence that there were both short-range and long-range correlations, the latter reflecting system-inherent "memory processes" (Hausdorff et al., 1995). However, without further modeling, the exact nature of these processes remains elusive.

## **PRACTICE-INDUCED CHANGES IN TEMPORAL DYNAMICS**

Our study is the first to show that the directional structure in trial-to-trial dynamics changed with practice. The recent study by Dingwell et al. (2012) on learning a virtual reaching task with two different solution manifolds, defined by the product and ratio of reach time and distance, reported a learning effect across 2 days only in the overt error and variance, not in the directionality of temporal structure. This may be due to the fact that the GEM analysis was only performed across 2 days, excluding the initial practice period. Using the rotation analysis, our study showed that the directional modulation in both autocorrelation and scaling index became more pronounced with practice. The initial lack of modulation reflects that subjects did not yet know the directionality of execution space. This is not surprising, as in the skittles task the solution manifold is not visible to the performer but is defined by the mediating dynamics of the ball trajectory. Without knowledge of the orientation of the solution manifold, exploration is needed that may occur in a gradient-descent-like fashion that leads to the persistent structure, as mentioned above. After this exploratory stage, trial-to-trial dynamics became more directionally sensitive and the structure in the orthogonal direction changed from initially positive autocorrelations to white noise and eventually very small negative values.

This result could be replicated with the simple model by a suitably chosen noise ratio and feedback gain. Given that the noise component in the experimental data was constant throughout the 6 days, the noise ratio was fixed to 0.08; assuming further an initially small or zero feedback gain, an increase in the gain to ∼0.20 reproduced the experimental modulation of temporal dynamics. Both the decrease in AC1 in the orthogonal direction and the relative invariance in the parallel direction could be replicated in the model results.

## **DIRECTIONALITY OF TEMPORAL STRUCTURE AND SENSITIVITY TO COORDINATES**

One important caveat for many approaches that analyze structure of variability is that these analyses are fundamentally sensitive to the chosen coordinates (Müller et al., 2007; Smeets and Louw, 2007; Sternad et al., 2010; Campolo et al., 2013). As demonstrated in our earlier study, variability analyses that rely on the covariance matrix are highly sensitive to the definition of the variables that span the space. This caveat holds for the large array of wellestablished methods, ranging from principal component analysis to isomap and others. While the mathematical tools are not questioned, when applying these methods to analyze hidden variables used by the nervous system, potential pitfalls arise. How easily the results can be thwarted was highlighted at the example of a UCM analysis of a multi-joint pointing task (Sternad et al., 2010). This study illustrated that results from two different, but equally valid mathematical definitions of joint angles—which are related by a simple linear transformation—differed: a synergy was indicated by anisotropy in one joint space, while not in the other. As shown in our study, the TNC-analysis is also not immune to this problem, but the sensitivity of the three components is less severe, due to the fact that structure of variability is evaluated in result space defined by the task (Campolo et al., 2013).

A second limitation of a covariance-based decomposition of variance in execution space is that they can only be applied in a space that has a defined metric, and thereby, orthogonality. The execution space in the skittles task is defined by angle and velocity, which have different units and, hence, no metric. This is similar to the GEM-analysis of walking, where the space was spanned by stride length and period. A straightforward remedy is to normalize the variables by their variance, as was done by Dingwell and colleagues and also in our skittles analysis. However, this correction by no means guarantees the right metric from which to define orthogonality. The fundamental issue is that the analyses rely on the assumption that the chosen execution variables span the space that is relevant for the nervous system. Until we know the coordinates used by the nervous system, this remains a tenuous assumption (Lacquaniti and Maioli, 1994; Fasse et al., 2000).

Clearly, there is no easy remedy. For the analysis of our experimental data we first normalized the coordinates by their variance before conducting time series analyses. We then introduced a rotation analysis that did not a priori depend on the definition of orthogonality but scouted the data for the direction with structure that may be relevant for the controller. The results showed that the direction of maximum persistence was not exactly parallel to the solution manifold and also did not significantly differ from the angle direction. The direction of minimum structure was coincident with the orthogonal direction and did differ from velocity, although only after some practice. These deviations from the straightforward expectations may be accounted for by the fact that the variables measured in external coordinates do not have to map onto the variables used by the nervous system.

#### **REFERENCES**


knowledge of results in motor learning. *Acta Psychol.* 92, 119–129. doi: 10.1016/0001-6918(95)00013-5

Campolo, D., Widjaja, F., Xu, H., Ang, W. T., and Burdet, E. (2013). Analysis of accuracy in pointing with redundant hand-held tools: a geometric approach to the uncontrolled manifold method. *PLoS*

To demonstrate such possible distortions, we used the simple model and introduced a linear rescaling of one of the state variables to skew the directions of maximum and minimum temporal correlations. Importantly, such rescaling can take place at many stages of the system: It can be applied at each iteration loop inside the system, it can happen independent of a concomitant rescaling of the solution manifold, it can include or exclude the noise, or it may only be applied on the data distributions. We modeled some select possibilities. The results showed that a linear rescaling of one variable indeed produced a skewing of the directionality of the data. Interestingly, this rescaling closely replicated the observed distortions in the pattern of modulation in the experimental data. While we did not intend to quantitatively model the experimental data, the results illustrate that the observed deviations in the directionality of the temporal structure may be caused by such internal rescaling. We venture the speculation that such results may provide clues about the relative scaling of the coordinates inside the nervous system.

## **CONCLUSIONS**

In summary, this experimental and modeling work demonstrated that the acquisition of a complex motor skill with a redundant task space is associated with an increasing anisotropy in data distributions and a corresponding increase in directionality in their temporal structure. The experimental results showed that the Tolerance, Covariation, and Noise of the data distributions changed with different time scales. The time-dependent characteristics in execution variables give further support that trial-to-trial dynamics is structured in directions specific to the solution manifold. A new analysis method highlighted that an a priori assumption of orthogonality in execution space may thwart the results. Model simulations suggested that the performance improvement is largely accounted for by changes in one essential system parameter—feedback gain. Further, analysis of the model highlighted how a rescaling of the variables can thwart the directionality of the maximum temporal correlations. These results may encourage future studies on variability to be less reliant on predefined directions. Rather, the search for directionality could help to reveal the coordinates important to the central nervous system.

## **ACKNOWLEDGMENTS**

This research was supported by grants from the National Institutes of Health, R01-HD045639, and the National Science Foundation, NSF-DMS0928587. We would like to express our thanks to Hermann Müller, Tjeerd Dijkstra, and Neville Hogan for many insightful discussions.

*Comput. Biol.* 9:e1002978. doi: 10.1371/journal.pcbi.1002978


timing: exploiting task redundancy to reduce sensitivity to timing. *J. Neurophysiol.* 107, 618–627. doi: 10.1152/jn.00568.2011

Cusumano, J. P., and Cesari, P. (2006). Body-goal variability mapping in an aiming task. *Biol. Cybern.* 94, 367–379. doi: 10.1007/s00422-006- 0052-1


dimensions. *J. Neurophysiol.* 109, 969–977. doi: 10.1152/jn.00706. 2012


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 28 January 2013; accepted: 10 May 2013; published online: 06 June 2013.*

*Citation: Abe MO and Sternad D (2013) Directionality in distribution and temporal structure of variability in skill acquisition. Front. Hum. Neurosci. 7:225. doi: 10.3389/fnhum.2013.00225 Copyright © 2013 Abe and Sternad. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

## Mind wandering and motor control: off-task thinking disrupts the online adjustment of behavior

#### *Julia W. Y. Kam1 \*, Elizabeth Dao2, Patricia Blinn1, Olav E. Krigolson3, Lara A. Boyd4 and Todd C. Handy1*

*<sup>1</sup> Attentional Neuroscience Lab, Department of Psychology, University of British Columbia, Vancouver, BC, Canada*

*<sup>2</sup> Aging, Mobility and Cognitive Neuroscience Laboratory, Department of Physical Therapy, University of British Columbia, Vancouver, BC, Canada*

*<sup>3</sup> Neuroeconomics Laboratory, Department of Psychology and Neuroscience, Dalhousie University, Halifax, NS, Canada*

*<sup>4</sup> Brain Behavior Laboratory, Department of Physical Therapy, University of British Columbia, Vancouver, BC, Canada*

#### *Edited by:*

*Rachael D. Seidler, University of Michigan, USA*

#### *Reviewed by:*

*Rachael D. Seidler, University of Michigan, USA Donatella Spinelli, Università di Roma 'Foro Italico', Italy*

#### *\*Correspondence:*

*Julia W. Y. Kam, Attentional Neuroscience Lab, Department of Psychology, University of British Columbia, 2136 West Mall, Vancouver, BC V6T 1Z4, Canada. e-mail: kamjulia@gmail.com*

Mind wandering episodes have been construed as periods of "stimulus-independent" thought, where our minds are decoupled from the external sensory environment. In two experiments, we used behavioral and event-related potential (ERP) measures to determine whether mind wandering episodes can also be considered as periods of "response-independent" thought, with our minds disengaged from adjusting our behavioral outputs. In the first experiment, participants performed a motor tracking task and were occasionally prompted to report whether their attention was "on-task" or "mind wandering." We found greater tracking error in periods prior to mind wandering vs. on-task reports. To ascertain whether this finding was due to attenuation in visual perception *per se* vs. a disruptive effect of mind wandering on performance monitoring, we conducted a second experiment in which participants completed a time-estimation task. They were given feedback on the accuracy of their estimations while we recorded their EEG, and were also occasionally asked to report their attention state. We found that the sensitivity of behavior and the P3 ERP component to feedback signals were significantly reduced just prior to mind wandering vs. on-task attentional reports. Moreover, these effects co-occurred with decreases in the error-related negativity elicited by feedback signals (fERN), a direct measure of behavioral feedback assessment in cortex. Our findings suggest that the functional consequences of mind wandering are not limited to just the processing of incoming stimulation *per se*, but extend as well to the control and adjustment of behavior.

**Keywords: mind wandering, experience sampling, motor control, visuomotor tracking task, fERN, performance monitoring, time-estimation**

## **INTRODUCTION**

Mind wandering, or those transient periods of time during which our attention momentarily drifts away from our on-going task and perceptual milieu, is fundamental to human neurocognitive function. In terms of neural architecture, mind wandering episodes have been strongly associated with activation of the brain's default mode network (e.g., Mason et al., 2007; Christoff et al., 2009; Kirschner et al., 2012), while in terms of cognitive processes, mind wandering has been tied to fluctuations in executive control (e.g., McVay and Kane, 2009, 2012). Such findings have supported the hypothesis that regular oscillations in the depth of our neurocognitive engagement with the external environment is normative to healthy human brain function (e.g., Smallwood and Schooler, 2006; Schooler et al., 2011; Smallwood, in press), and that a variety of clinical and sub-clinical cognitive pathologies may be linked to altered patterns of mind wandering (e.g., Shaw and Giambra, 1993; Helton, 2009; Smallwood et al., 2009; Killingsworth and Gilbert, 2010; Elua et al., 2012).

Given that mind wandering is central to our neurocognitive make-up, there has been growing interest in understanding the practical consequences of slipping into a mind wandering state. For example, when we mind wander, we now know that there is a systematic reduction in the extent to which we process external stimulus events at both the sensory and cognitive levels (e.g., Smallwood et al., 2008; O'Connell et al., 2009; Smilek et al., 2010; Kam et al., 2011; Hu et al., 2012), effects that can arise regardless of whether the events are task-related or not (e.g., Barron et al., 2011). In a corresponding manner, behavioral motor performance reliably shifts to a more automatic and/or degraded state (e.g., Schooler et al., 2004; Cheyne et al., 2006; Weissman et al., 2006; Carriere et al., 2008; Smallwood et al., 2008; Reichle et al., 2010), such that reaction times (RTs) tend to speed up and error rates are higher during mind wandering vs. on-task states (Smallwood et al., 2004; Franklin et al., 2011).

Yet despite such findings, our understanding of how mind wandering impacts motor behavior remains incomplete at best. Considered from a motor perspective, the range of potential mind wandering effects on behavioral control concerns more than just the speed and accuracy of response selection and the degree of response automaticity. In addition, the normal control of movement also involves the ability to adaptively monitor and adjust our motor outputs on a moment-to-moment basis as needed. Given that mind wandering attenuates the sensory and cognitive processing of external stimulus inputs, the goal of our study was to determine whether this may have a corresponding effect on our ability to dynamically adjust our motor behavior on-line in response to shifting, unpredictable environmental conditions.

In our first experiment we addressed the question using a canonical visuomotor tracking task that allowed us to measure the magnitude of continuous tracking error as a function of whether or not participants were in a mind wandering state. Tracking error did in fact increase during mind wandering. In our second experiment we examined whether this effect of mind wandering on behavior would generalize to a qualitatively distinct form of response monitoring and control—namely, feedback learning in the context of a time-estimation task. We again found behavioral evidence of the impact of mind wandering on the dynamic control of motor outputs, an effect that co-occurred with attenuations in direct, event-related potential (ERP) measures of performance monitoring processes in cortex.

In both experiments we relied on "experience sampling" as a means of determining the attention state of our participants over time (e.g., Schooler et al., 2011). Considered to be a "direct" measure of mind wandering, experience sampling relies on the fact that if prompted, we can reliably report on the content of our thoughts at any given moment and further, determine whether they center on the on-going task being performed (referred to as an "on-task" state), or alternatively, whether they have drifted off to other times, places, or issues (referred to as an "off-task" or "mind wandering" state) (for a review, see Gruberger et al., 2011). Although the act of reporting on one's thought state interferes with the content of consciousness itself (e.g., Filler and Giambra, 1973), by using the report to categorize a participant's attentional state in the 10–15 s immediately prior to the report, the methodology has been used to demonstrate reliable and replicable differences in neurocognitive functioning between "on-task" and "off-task" or "mind wandering" states (e.g., Smallwood et al., 2004; McKiernan et al., 2006; Mason et al., 2007; Smallwood et al., 2008; Christoff et al., 2009; Franklin et al., 2011; Kam et al., 2011; Stawarczyk et al., 2011; Kirschner et al., 2012). As such, in adopting this methodology here, our approach to defining attentional states aligned with widely-accepted norms in the field of mind wandering research.

## **EXPERIMENT 1**

In the first experiment, participants performed a visuomotor tracking task. They were stopped at unpredictable intervals and asked to report on whether their attention at that moment was "on-task" or whether they were "mind wandering." To examine the influence of mind wandering on motor control, we compared the error in tracking performance between on-task and mind wandering states. Given that disruptive effects of mind wandering extend beyond perceptual and cognitive processes to response selection, we predicted there would be more errors during mind wandering relative to on-task states.

## **METHODS**

## *Participants*

Twenty-two participants completed the experiment in exchange for one course credit. They were all right handed, with no history of neurological problems and had normal or corrected-to-normal vision. Participants provided written informed consent to the experimental procedure. The Clinical Research Ethics Board at the University of British Columbia approved this study.

## *Task paradigm and procedures*

Participants performed a visuomotor tracking task (Boyd and Winstein, 2004; Boyd and Linsdell, 2009), in which they continuously tracked a target moving in sine-cosine waveform on a computer monitor by controlling the position of a cursor using a joystick. The target appeared as an open white circle and participant's movements were represented as a filled red dot on the monitor. The paradigm is shown in **Figure 1**. The task was to track the vertical path of the target with the joystick as accurately as possible. Joystick position sampling and stimuli presentation were both at 60 Hz, using custom software developed on the LabView platform (v. 7.1; National Instruments Co.).

There were 14 blocks of varying duration; lasting from 48 to 192 s. Each trial was 32 s long, tracking the target from left to right across a 17- computer screen. Trials contained a 2 s baseline and 30 s of tracking a unique sine-cosine segment; each 30 s waveform was unique and could not be learned, thus participants were required to attend to visual stimuli in order to track accurately. The pattern of target movement was predefined and modified from Wulf and Schmidt's method (1997). Waveforms were generated using the polynomial equation with the following general form (cf. Wulf and Schmidt's, 1997), using randomly inserted coefficients ranging from −5 to 5:

$$f(\mathbf{x}) = b\_0 + a\_1 \sin(\mathbf{x}) + b\_1 \cos(\mathbf{x}) + a\_2 \sin(2\mathbf{x})$$

$$+ b\_2 \cos(2\mathbf{x}) + \dots + a\_6 \sin(6\mathbf{x}) + b\_6 \cos(6\mathbf{x}).$$

Importantly, neither the target or participants' movements left a trail, thus participants could not visualize the entire target

joystick.

pattern. To control of waveform difficulty across participants, each practiced the same set of random waveforms.

Our primary behavioral measure was the changes in root mean squared error (RMSE), which reflects the overall tracking error in the kinematic pattern. It is the average difference between the target pattern and participant movements (cf. Boyd and Winstein, 2004). The RMSE is calculated as follows:

$$\text{RMSE} = \sum\_{i=1}^{n} \left\{ (\mathbf{x}\_i - T\_i)^2 / n \right\}^{1/2}.$$

## *Task-related attention*

To measure task-related attention, participants were instructed to report their "attentional state" at the end of each block. Specifically, they were asked to identify their state immediately prior to the block termination as either being "on-task" (fully attentive to task performance), or "mind wandering" (unattentive to the task) at the block's end. Importantly, participants were provided with descriptions and examples of these two attentional states prior to the testing session. "On-task" states were defined as when one's attention was firmly directed toward the task, whereas "mind wandering" states were described as when one is thinking about other things than just the task. Attentional reports were recorded at the conclusion of each block, and these reports were then used to sort behavioral data based on "on-task" vs. "mind wandering" states. As mentioned above, block duration was randomly varied between 48 and 192 s in order to minimize predictability of block completion and maximize variability of attentional state at the time of block completion. The duration of the task itself was approximately 30 min.

## *Statistical analysis*

In terms of comparing on-task vs. mind wandering states, the periodicity of shifts in these attentional states tends to approximate 10–15 s (e.g., Sonuga-Barke and Castellanos, 2007; Christoff et al., 2009). We thus examined the movement data in the last 12 s prior to the subjective report of each attentional state prompted by the probes (cf. Smallwood et al., 2008; Kam et al., 2011). Specifically, we conducted paired-samples *t*-tests to compare the RMSEs by averaging together data in the 12 s preceding each of the two attentional states (on-task vs. mind wandering) report. Although we had no knowledge as to how long participants had actually been in a particular attentional state at the time a subjective report was given, our analyses were based on the assumption and recent evidence (Sonuga-Barke and Castellanos, 2007; Christoff et al., 2009) that the 12 s prior to each report would, on average, reliably capture the given attentional state.

## **RESULTS**

## *Tracking performance*

Participants completed 14 trial blocks, of which 43% were reported as "on-task" and 57% as "mind wandering"—a typical breakdown of attentional states (Smallwood et al., 2008; Kam et al., 2011). The motor tracking performance, indexed by the RMSE, was examined as a function of participants' attentional states. The RMSE preceding reports of mind wandering (*M* = 4*.*71, *SD* = 1*.*90) appeared to be much greater than

those preceding on-task reports (*M* = 3.93, *SD* = 0*.*70). This was confirmed by a significant paired-samples t-test (*t(*21*)* = −2*.*23, *p* = 0*.*03).

## **DISCUSSION**

In Experiment 1, we found greater error in motor tracking just preceding reports of mind wandering relative to reports of ontask. This suggests that mind wandering does impair the precision at which we control our motor behavior on a moment-tomoment basis. Given the lack of external feedback on the participants' performance, however, it is unclear whether the increased tracking error during mind wandering was due to visual sensory attenuation *per se* (Kam et al., 2011), or whether mind wandering can also down-regulate behavioral/performance monitoring. We addressed this question in Experiment 2.

## **EXPERIMENT 2**

We recorded participants' EEG as they performed a simple timeestimation task during which they received trial-by-trial feedback on the accuracy of their responses and were occasionally asked to report their attention state at that moment as "on-task" or "mind wandering." To determine the impact of mind wandering on performance monitoring, we measured the feedback error-related negativity (fERN) elicited by task feedback in the intervals immediately preceding "on-task" vs. "mind wandering" reports. In particular, the fERN is an endogenously-evoked ERP component that indexes the extent to which we are monitoring the accuracy of our responses, such that its amplitude positively covaries with the magnitude of behavioral assessment (Miltner et al., 1997; Holroyd and Krigolson, 2007; Krigolson et al., 2009). If mind wandering attenuates feedback monitoring, then it predicted that the fERN would be lower in amplitude during periods of mind wandering vs. on-task attentional states.

## **METHODS**

## *Participants*

Fifteen participants (9 females; *M* = 24.8 years, *SD* = 2*.*20) completed the experiment in exchange for \$20 (Canadian dollars). They were all right handed, with no history of neurological problems and had normal or corrected-to-normal vision. Participants provided written informed consent to the experimental procedure. This study was approved by the Clinical Research Ethics Board at the University of British Columbia.

## *Stimuli and paradigm*

We recorded EEG and behavioral data while participants performed a time-estimation task (cf. Miltner et al., 1997; Holroyd and Krigolson, 2007). On each trial, participants were required to estimate the duration of one second by pressing a button after an initial auditory cue. The cue was presented at 3000 Hz for 25 ms. Following the participant's estimate, a feedback stimulus was visually presented for 1000 ms at fixation to indicate the accuracy of the guess. After the offset of the feedback stimulus, a blank screen was presented for 400, 500, or 600 ms. Therefore, each trial lasted approximately between 2400 and 2600 ms (i.e., 2500 ms on average). A trial was considered correct if a participant's response occurred within a window of time centered around one second (±100 ms), and was considered incorrect otherwise. In order to maintain a global probability of approximately 0.5 for correct and incorrect feedback stimulus, the size of the response window decreased by 10 ms each time a participant was correct, and increased by 10 ms each time a participant was incorrect.

## *Behavioral measure*

We determined the mean absolute change in response time following correct and error feedback as a function of participants' attentional states. That is, the absolute difference in time estimates between the current and previous trial was calculated in percentages for each participant (cf. Holroyd and Krigolson, 2007), separately for correct and error feedback during on-task and mind wandering states. This measure allows us to examine participants' sensitivity to their own behavioral performance as a function of attentional state.

## *Task-related attention*

Attentional reports were recorded at the conclusion of each trial block, and these reports were then used to sort ERP data based on "on-task" vs. "mind wandering" states. The protocol for measuring task-related attention is identical to Experiment 1 with the following exceptions. The block duration itself was randomly varied between 30 and 90 s (i.e., 12–36 trials), and the duration of the task itself was approximately 65 min.

## *Electrophysiological recording and analysis*

During the task, electroencephalograms (EEGs) were recorded from 32 active electrodes using a Biosemi Active-Two amplifier system. All EEG activity was recorded relative to two additional electrodes located over medial-parietal cortex (CMS/DRL), amplified with a gain of 0.5 and digitized on-line at a sampling rate of 256 samples per-second. To ensure proper eye fixation and allow for the correction and/or removal of events associated with eye movement artifacts, vertical and horizontal electrooculograms (EOGs) were also recorded—the vertical EOGs from an electrode inferior to the right eye, and the horizontal EOGs from two electrodes on the right and left outer canthus. Offline, computerized artifact rejection was used to eliminate trials during which detectable eye movements and blinks occurred. These eye artifacts were detected by identifying the minimum and maximum voltage values on all recorded EOG channels from −50 to 600 ms post visual feedback stimulus for each event epoch, and then removing the trial from subsequent signal averaging if that value exceeded 150µV, a value calibrated to capture all blinks and eye movements exceeding approximately 1◦ of visual angle. For each participant, ERPs for each condition of interest were averaged into 3000 ms epochs, beginning 1500 ms before visual feedback stimulus onset. Subsequently, all ERPs were algebraically re-referenced to the average of the left and right mastoid signals, and filtered with a low-pass Gaussian filter (25.6 Hz halfamplitude cut-off) to eliminate any residual high-frequency artifacts in the waveforms. The resulting ERPs were used to generate grand-averaged waveforms.

## *Statistical analysis*

Statistical quantification of ERP data was based on minimum peak and mean amplitude measures relative to a −200 to 0 ms pre-stimulus baseline. In particular, we derived "difference waves" for the on-task and mind wandering conditions by subtracting the correct feedback averaged waveforms from the incorrect feedback averaged waveforms for each attentional state and participant from electrode site FCz, where the fERN is typically maximal (e.g., Holroyd and Krigolson, 2007; Krigolson et al., 2009), as it was in our data. The fERN was then subsequently identified by an automated computer algorithm as the maximal negative voltage between 250 and 350 ms on the difference waveforms following feedback stimulus onset (see Holroyd and Krigolson, 2007) for more on this peak-picking methodology).

Here we compared both behavioral and ERP responses in the last 15 s prior to the subjective report of attentional state prompted by the probes. That is, the ERP waveforms for each condition of interest (correct vs. error) were based on averaging together the EEG epochs for the six trials preceding each of the two attentional states (on-task vs. mind wandering) report. We extended the analysis period to 15 s prior to each attentional report as an attempt to maximize the number of events to include in each waveform average while not extending the window back so far in time as to consistently capture the preceding attentional state or transition period between states.

## **RESULTS**

## *Behavioral performance*

Similar to Experiment 1, participants completed an average of 63 blocks of trials, of which 44% were reported as "on-task" and 56% as "mind wandering" (Smallwood et al., 2008; Kam et al., 2011). To examine how mind wandering affected behavioral performance, we conducted an omnibus ANOVA that had attentional state (on-task vs. mind wandering) and feedback valence (correct vs. error) as within-subject factors. The overall absolute change in time estimates and the variance of these time estimates during mind wandering periods appeared to be much greater than on-task periods, as shown in **Figure 2**. This data pattern was confirmed via a significant main effect of attentional state [*F(*1*,* <sup>14</sup>*)* = 39*.*51, *p <* 0*.*001]. The main effect of feedback valence was not significant [*F(*1*,* <sup>14</sup>*)* = 1*.*03, *p* = 0*.*328]. However, there was a significant attentional state X feedback valence interaction [*F(*1*,* <sup>14</sup>*)* = 8*.*95, *p* = 0*.*010]. Follow-up analyses revealed that the absolute change in time estimates following error feedback was significantly greater than that following correct feedback during on-task states [*t(*1*,* <sup>14</sup>*)* = −2*.*35, *p* = 0*.*034], but not during mind wandering states [*t(*1*,* <sup>14</sup>*)* = 1*.*93, *p* = 0*.*074]. While the adjustment in time estimates during mind wandering appears to be insensitive to feedback valence, this difference was nonetheless marginally significant. Along with the relatively small behavioral effect during on-task states, this set of finding makes it difficult to draw conclusions about the attentional effect on behavioral performance on this task.

## *Electrophysiology*

Although the behavioral results showed evidence of decreased sensitivity to feedback during mind wandering, we wanted to first confirm normative mind wandering effects in our ERP findings, prior to assessing the fERN. In particular, the P3 elicited by target stimuli has been shown to reliably attenuate in amplitude immediately preceding reports of mind wandering relative

to on-task (e.g., Smallwood et al., 2008; Kam et al., 2011). To confirm the reliability of our subjective reports, we thus wanted to determine that there was in fact a general attenuation of the P3 amplitude elicited by feedback signals immediately preceding mind wandering vs. on-task reports.

Thus, we first conducted repeated-measures ANOVA on P3 with factors of attentional state (on-task vs. mind wandering), feedback valence (correct vs. incorrect), and electrodes (Cz and Pz) to establish the reliability of subjective reports of attentional state. For brevity, we only report effects associated with attentional state and feedback valence. The P3 elicited by the correct and error feedback as a function of attentional state is shown in **Figure 3**. This ERP component was measured at different time points between the two feedback stimulus types because it peaked at different time points for correct vs. error feedback, as can be seen in the figure. Mean amplitude measures were therefore taken across a 290–410 ms time window for correct feedback, and 330–450 ms time window for error feedback. We examined electrode sites Cz and Pz, where the P3 is typically maximal (e.g., Polich, 2007). There was a significant main effect of attentional state [*F(*1*,* <sup>14</sup>*)* = 12*.*06, *p* = 0*.*004] such that regardless of feedback valence, the P3 amplitude elicited by feedback signals was significantly greater immediately preceding on-task vs. mind wandering attentional reports. There was no main effect of feedback valence, nor an interaction between attentional state and feedback valence (*p >* 0*.*829). Importantly, this main effect of attentional state on P3 amplitude was consistent with the normative pattern for mind wandering (Smallwood et al., 2008; O'Connell et al., 2009; Kam et al., 2011).

We then examined the impact of mind wandering on feedback processing, as measured via the fERN on the difference waveforms shown in **Figure 4**. The waveforms elicited by correct and error feedback stimulus as a function of attentional state are shown in **Figure 5**. As can be seen in **Figure 4**, the fERN appeared to be attenuated during mind wandering periods relative to on-task periods. To confirm this, two single-sample *t*-tests first confirmed the presence of a fERN in both the ontask [*t(*14*)* = −5*.*43, *p <* 0*.*001, *d* = −2*.*90] and mind wandering [*t(*14*)* = −3*.*75, *p* = 0*.*002, *d* = −2*.*00] conditions. Next, a comparison of the difference waveforms between on-task and mind wandering conditions revealed that the amplitude of the fERN was significantly reduced during mind wandering [*t(*14*)* = 2*.*22, *p* = 0*.*04, *d* = 0*.*61].

While definitive conclusions about the fERN can only be made with difference waveforms, we wanted to determine whether this fERN attenuation during mind wandering may be driven by a differential attentional modulation of the processing of correct and error feedback. As such, we compared the ERP waveforms of both correct and error feedback at FCz between on-task and mind wandering states, using the same individually-specified time windows as were used to identify the fERN in each individual's difference waveforms. In particular, we conducted repeatedmeasures ANOVAs with factors of attentional state (on-task vs. mind wandering), and feedback valence (correct vs. error). We found a significant interaction between attentional state and feedback valence [*F(*1*,* <sup>14</sup>*)* = 4*.*907, *p* = 0*.*044], suggesting that mind wandering was specifically attenuating fERN-related activity for correct feedback signals. This interpretation was confirmed via separate paired-samples *t*-tests, which revealed a significant main effect of attentional state in response to correct feedback [*t(*14*)* = 2*.*691, *p* = 0*.*018], but not error feedback [*t(*14*)* = 0*.*158, *p* = 0*.*877]. Specifically, while the processing of the correct feedback was significantly attenuated immediately preceding mind wandering (*M* = 6.16, SEM = 1.21) vs. on-task (*M* = 8.63, SEM = 1.31) attentional reports, the processing of error feedback did not significantly differ between mind wandering (*M* = 2.85, SEM = 1.09) and on-task (*M* = 3.04, SEM = 1.55) attentional states.

## **DISCUSSION**

Using both behavioral and electrophysiological measures, Experiment 2 examined the question of whether mind wandering impacts the monitoring and adjustment of behavioral performance. We found decreased behavioral sensitivity accompanied by a reduced P3 to feedback stimulus during periods of mind wandering. Our data also revealed a reduced fERN during mind wandering compared to on-task attentional states. Consistent with the finding that correct trials appear to modulate the fERN amplitude (Holroyd et al., 2008), the reduced fERN was specifically driven by a significant mind wandering effect on correct, but not error, feedback.

## **GENERAL DISCUSSION**

The purpose of this study was to examine the effects of mind wandering on the online adjustment of behavior. Using a visuomotor tracking task in Experiment 1, we observed greater errors in tracking performance during periods of mind wandering. Using a time-estimation task in Experiment 2, we found reduced behavioral and neural sensitivity to performance feedback during mind wandering states, suggesting that the disruption in behavioral control could not be attributed to sensory attenuation *per se*. Extending previous research showing that mind wandering states decouple our attention from incoming sensory and cognitive stimuli (Smallwood et al., 2008; O'Connell et al.,

2009; Kam et al., 2011), these results suggest mind wandering also disengages us from both monitoring and adjustment of our behavior.

That mind wandering was associated with increased error in a continuous tracking task is not surprising given mind wandering has been implicated in performance failures in vigilance tasks (Robertson et al., 1997; Smallwood et al., 2004) and response selection tasks (Schooler et al., 2004; Franklin et al., 2011). Interestingly, Boyd and Linsdell (2009) have implemented the motor tracking task over four practice sessions to induce motor sequence learning, and found that tracking performance at retention did improve as indexed by RMSE (Boyd and Linsdell, 2009). Given this finding, if mind wandering increases tracking error as we have shown in our study, this would not only lead to disruption in task performance and accordingly the learning of the sequence in the current testing session but it may also have a disruptive long term effect on the learning of motor skills over time.

If mind wandering is impacting behavioral feedback processing as measured via the fERN, how does this actually affect behavioral outputs? The fERN is time-locked to external signals of response accuracy, and is generated by a high-level error evaluation system that is tasked with performance optimization (Holroyd and Coles, 2002). As such, the fERN not only involves detecting the relative accuracy vs. inaccuracy of a response, but also reflects the extent to which we use that information for the modification of behavior (Krigolson et al., 2009). Given that mind wandering leads to transient reductions in the extent to which we process behavioral feedback signals, this suggests the functional consequences are two-fold. On the one hand, as our data confirm, the transient phases of mind wandering lead to direct disruption on the moment-to-moment adjustments in motor behavior. However, given that the cortical processes indexed by the fERN are associated with reinforcement learning (Holroyd and Coles, 2002), this would imply over time, mind wandering may also directly affect the trajectory or efficacy of motor learning itself. Together, findings from both experiments would suggest that the more we mind wander, the slower and less efficient motor learning may become.

Our report of a mind wandering effect on feedback processing manifest in the fERN raises the question to what extent might our findings be driven by these sensory and/or more general cognitive effects of mind wandering? In terms of possible visual sensory confounds, prior studies have found visual sensory attenuation for visual stimuli in the upper visual hemifield (Kam et al., 2011) but not for visual stimuli at fixation (Smallwood et al., 2008). As the visual feedback stimuli used in our study were at fixation, this

suggests sensory attenuation is an unlikely explanation for our fERN results. Likewise, when we examined the P3 component in our study, we found attenuation in amplitude during mind wandering that was insensitive to the valence of feedback. In contrast, in the fERN, we found that the attenuation in amplitude during mind wandering was associated with a selective effect of mind wandering for correct feedback signals. This functional dissociation between the P3 and fERN findings thus suggests that the effect of mind wandering on the latter can not simply be ascribed to its effect on the former. Rather, it would appear that mind wandering can have a direct, independent influence on behavioral feedback processes in cortex.

Finally, given our results, it's also important to consider what our data are not showing. In particular, the fERN reflects an evaluation of one's preceding trial performance, based on delayed external feedback signaling whether or not behavior needs to be modified for improved performance. While the external feedback is typically presented in the form of a visual stimulus, the nature of this feedback and its implications in behavioral performance makes it qualitatively distinct from task-relevant sensory stimulus. In contrast, the response ERN is another error-related component that reflects the implicit aspect of response monitoring, whereby the internal evaluation of performance is based on the response itself (Gehring et al., 1993). While our findings suggest that mind wandering impacts the continuous adjustment of motor behavior in the absence of feedback as well as behavioral control associated with external feedback, whether it also affects the implicit evaluation of on-line performance as captured by the response ERN elicited by correct vs. incorrect responses remains to be directly investigated. If so, this would provide further support of the notion that mind wandering promotes response-independent thought.

Given our findings, an important issue concerns how if at all this relates to the attentional lapse literature. While mind wandering and attentional lapses capture a similar neurocognitive phenomenon, they do occur at very different temporal levels.

**FIGURE 5 | Conditional waveforms of on-task and mind wandering attentional states in response to correct and error feedback.** The difference between correct and error feedback appears to be greater during on-task states relative to mind wandering states.

In particular, mind wandering is a phenomenon that spans an extended period of time (i.e., fluctuations of 10–15 s) exceeding a given single event, whereas attentional lapses tend to occur during a much narrower time window capturing the lapse at a single event level. Several recent theoretical and empirical papers have supported and validated these two related models of attention (Dosenbach et al., 2008; Esterman et al., 2012). Specifically, at a theoretical level, Dosenbach and colleagues (2008) have suggested there are multiple controlling systems operating at multiple scales of time. Further, in terms of empirical evidence, the findings of Esterman and colleagues (2012) suggested the occurrence of two attentional states—one tied to the default mode network (reflective of mind wandering) that is more stable and less error prone in terms of behavioral measures, and a second one tied to the dorsal attention network (reflective of attentional lapses) that requires more effortful processing. That the effects of mind wandering appear to parallel effects of attentional lapses actually lends support to the notion that task-related attention (or mind wandering) and selective attention (or attentional lapses) may exert similar forms of top–down attentional control on other neurocognitive processes. In the case of attentional control of sensory response, it has been suggested that there are at least two distinct control systems operating in parallel—one associated with rapid shifts of selective visual attention (e.g., Mangun

## **REFERENCES**


S. E. (2008). A dual-networks architecture of top–down control. *Trends Cogn. Sci.* 12, 99–105.


and Hillyard, 1991; Woldorff et al., 1997) and another one associated with slower fluctuations in task-related attention (O'Connell et al., 2009; Kam et al., 2011). In the case of behavioral control, that Weissman and colleagues have demonstrated that attentional lapses impair goal-directed behavior and are associated with reduced pre-stimulus activation in the anterior cingulate cortex (Weissman et al., 2006) and that we found impaired adjustment of behavioral control are consistent with the idea that varying attentional control systems appear to have similar impact on various neurocognitive processes. Taken together, mind wandering and attentional lapses do appear to be related conceptually, but future work needs to be done to disentangle the overlaying attentional influences linked to dissociable neural systems.

## **ACKNOWLEDGMENTS**

We would like to express our gratitude to Jonathan Smallwood for his comments on earlier drafts. We also thank all our participants for their time and effort. JWYK and TCH are supported by the Natural Sciences and Engineering Research Council. LAB is supported by the Canada Research Chairs and the Michael Smith Foundation for Health Research. OEK is supported by Natural Sciences and Engineering Research Council, as well as the Canadian Foundation for Innovation Leaders Opportunity Fund.


an executive-control task. *J. Exp. Psychol. Learn.* 35, 196–204.


S., Zamarripa, F., et al. (1997). Retinopic organization of early visual spatial attention effects as revealed by PET and ERPs. *Hum. Brain Mapp.* 5, 280–286.

Wulf, G., and Schmidt's, R. A. (1997). Variability of practice and implicit motor learning. *J. Exp. Psychol. Learn.* 23, 987–1006.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 25 September 2012; accepted: 27 November 2012; published online: 14 December 2012.*

*Citation: Kam JWY, Dao E, Blinn P, Krigolson OE, Boyd LA and Handy TC (2012) Mind wandering and motor control: off-task thinking disrupts the online adjustment of behavior. Front. Hum. Neurosci. 6:329. doi: 10.3389/fnhum. 2012.00329*

*Copyright © 2012 Kam, Dao, Blinn, Krigolson, Boyd and Handy. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any thirdparty graphics etc.*

## Towards mastery of complex visuo-motor transformations

## *Herbert Heuer\* and Sandra Sülzenbrück*

*IfADo - Leibniz Research Centre for Working Environment and Human Factors, Dortmund, Germany*

#### *Edited by:*

*Rachael D. Seidler, University of Michigan, USA*

#### *Reviewed by:*

*Marika Berchicci, University of Rome "Foro Italico", Italy Jin Bo, Eastern Michigan University, USA*

#### *\*Correspondence:*

*Herbert Heuer, IfADo - Leibniz Research Centre for Working Environment and Human Factors, Ardeystr. 67, Dortmund 44139, Germany. e-mail: heuer@ifado.de*

In this paper we review and integrate a set of findings on learning the transformation of a sliding first-order lever, a type of tool with a prominent role in minimal access surgery. Its kinematic transformation is characterized by the so-called fulcrum effect, the inversion of the movement direction of the tip of the lever relative to that of the hand for rotations. A second characteristic is gain anisotropy, which results in curved paths of the tip of the lever for straight paths of the hand and vice versa. An internal model of the kinematic transformation is acquired during practice, the accuracy of which can be assessed in visual open-loop test trials. The accuracy of the acquired internal model is enhanced when visual closed-loop control during practice is impeded, and the accuracy of the internal model is reduced when closed-loop control during practice is facilitated. The internal model consists of a rapidly acquired line-symmetric approximation to the transformation of the sliding lever and a slowly acquired fine tuning. The fine tuning is local, that is, it is specific for the region of the workspace encountered during practice. The internal model is transferred to other regions of the workspace, but not adjusted to the fine tuning appropriate for these regions. Whereas the symmetry approximation is most likely explicit, the fine tuning seems to be represented implicitly. Findings on the straightness of the paths of the tip of the lever and the hand suggest that the internal model of the transformation is confined to initial and final positions of aimed movements, whereas their path is not strictly controlled, but affected by the dynamic transformation of the tool. Only when visual closed-loop control is possible, the path of the effective part of the tool is straightened. These characteristics of the internal model of the sliding first-order lever and its acquisition may be partly specific to sufficiently complex extrinsic transformations that arise from mechanical or electronic extensions of the body.

#### **Keywords: motor learning, internal model, transformation, tool use, explicit learning, implicit learning**

Movement execution involves a series of transformations (cf. Heuer and Massen, 2013). For example, efferent commands are transformed into muscular forces, muscular forces are transformed into joint torques, joint torques are transformed into joint rotations, joint rotations are transformed into movements of an end effector such as the hand. Planning and control of a voluntary movement of the end effector requires an internal model of the series of transformations, more precisely, an inverse model that allows to determine the input needed for a certain intended output (Heuer, 1983, p. 15; Wolpert and Kawato, 1998; Kawato, 1999). Such a model has to be plastic because the transformations are subject to change on different time scales (e.g., Körding et al., 2007). They change slowly in the course of bodily growth and involution, they change rapidly in the course of fatiguing exercises. Plasticity becomes possible because the brain has access not only to the neural input of the neuro-mechanical series of transformations, but also to the mechanical output and to intermediate mechanical variables by means of vision and proprioception.

The series of transformations, which is intrinsic to the body, is extended by extrinsic transformations when a tool is used. From an observer perspective, the difference between intrinsic and extrinsic transformations is fairly obvious, but from the perspective of the user of the tool the difference may be rather graded (cf. Heuer and Sülzenbrück, 2013a). On the one hand, movements with and without a tool may lead to similar perceptual experiences. For example, a classic observation, dating back at least to Descartes ([1637]1958), is the projection of tactile sensations into the outside world when we touch objects with a stick. Thus, for the haptic perception of the location of an object it does not make much of a difference whether we touch it with a finger or with a hand-held stick. Such phenomenological observations are complemented by physiological data. For example, Iriki et al. (1996) observed changing receptive fields of certain parietal neurons when a tool was used. On the other hand, a clear difference between ones own limbs and their extension by tools remains. The hand is not just replaced by a tool. It is evident that extrinsic transformations can change more rapidly and radically than intrinsic transformations. It is also likely that successful tool use can invoke higher cognitive processes such as mechanical reasoning in addition to basic processes of sensori-motor adaptation (Johnson-Frey, 2003).

In this paper we review and integrate a set of findings on learning a complex extrinsic transformation as it is inherent to a sliding first-order lever. At first glance this may appear as a rather esoteric kind of tool to study. However, this type of tool has a prominent role in minimal access surgery, which represents one of the currently greatest professional challenges of human sensori-motor skills (cf. Villegas et al., 2003). We start with a description of the transformation of the sliding lever. Following this, we present some findings which suggest a trade-off between visual closed-loop control during practice and the acquisition of an internal model of the extrinsic transformation. The main body of the paper will then deal with the characteristics of the internal model.

## **THE TRANSFORMATION OF THE SLIDING FIRST-ORDER LEVER**

**Figure 1A** shows the basic set-up of several experiments with the sliding first-order lever. The ball bearings of the lever were almost frictionless and allowed horizontal rotations around the fulcrum as well as translations, that is, forward and backward movements. Participants grasped a pen that was attached to the near end of the lever. Its position was recorded by means of a digitizer. The position of the tip of the lever was computed and presented on the monitor as the position of a cursor. The direct view of the hand and the lever was blocked by an opaque shield.

A tool such as the sliding lever implements both a kinematic and a dynamic transformation (see Heuer and Sülzenbrück, 2009, for a detailed description). The input of the kinematic transformation is the position of the hand, and its output is the position of the tip of the lever. The input of the dynamic transformation is the force exerted by the hand, and its output is the acceleration

of the tip of the lever (and the cursor) and appropriately curved paths of the

movement to target 2 (gray line).

of the hand and thus the near end of the lever. The experimental setup allowed varying both transformations independently. The position of the cursor could indicate both the position of the tip of the lever (kinematic transformation of the sliding lever present) or of the pen (kinematic transformation absent). The pen could be attached to the physical lever (dynamic transformation of the sliding lever present) or it could be detached (dynamic transformation absent). When only the kinematic transformation is present, but not the dynamic one, we refer to the tool as a "virtual lever."

The kinematic transformation of the sliding first-order lever can be described in different ways. It is mathematically quite simple when a Cartesian coordinate system with its origin in the fulcrum is chosen, as in **Figures 1B,C**:

$$\underline{\underline{c}} = -\frac{l - \left| \underline{h} \right|}{\left| \underline{h} \right|} \times \underline{h}$$

with *c* as the position of the tip of the lever or, equivalently, of the cursor, *(xc, yc)*, and *h* as the position of the hand, *(xh, yh)*. The length of the lever is l, and - *h* - is the length of the effort arm. In terms of the movements produced, the kinematic transformation has two important characteristics. The more conspicuous one is the reversal of the direction of hand movements at the tip of the lever when the lever is rotated. This reversal is also known as the fulcrum effect (Gallagher et al., 1998). The less conspicuous characteristic is gain anisotropy, that is, the dependence of the visuo-motor gain on movement direction. For translations of the lever the gain is 1, that is, the amplitude of the tip of the lever is the same as that of the hand. For rotations, however, the gain varies. When the effort arm is longer than the load arm, the gain is less than 1, that is, the amplitude of the tip of the lever is smaller than that of the hand. When the effort arm is shorter than the load arm, the gain is larger than 1. When translations and rotations are combined to produce movements in various directions, the gain varies across directions. As a consequence of this gain anisotropy, straight movements of the hand will generally result in curved movements of the tip of the lever, and straight movements of the tip of the lever require particularly curved hand movements—a fact that makes certain surgical tasks quite difficult (e.g., Heuer et al., 2012).

The consequences of the kinematic transformation of the sliding lever for hand movements, which serve to produce straight movements of the tip of the lever, are illustrated in **Figures 1B,C** for two target configurations as they were used in our experiments. In both configurations there was a central start position. In the one configuration (**Figure 1B**) the targets were located on a circle around the start position (radius: 5 cm or similar) with angular separations of 45◦. In the other configuration (**Figure 1C**) the targets were at the intersections of three concentric circles around the fulcrum with radii such as 12.75, 17.75, and 22.75 cm and three radial lines emanating from the fulcrum with angular separations such as 30◦. From **Figures 1B,C** it is apparent that the kinematic transformation is quite complex when it is described in terms of directions and amplitudes, which are relevant parameters of motor control according to the vector-coding model (e.g., Vindras and Viviani, 1998). In fact, in particular with the target configuration of **Figure 1C** some participants tend to produce translations and rotations of the lever in sequence rather than concurrently (see right panel of **Figure 5** in Heuer and Sülzenbrück, 2009), a strategy which simplifies the task in that for each translation or rotation the gain remains constant.

The dynamic transformation of the sliding lever plays only a minor role for the characteristics of movements with this tool. Acquisition of the internal model of the kinematic transformation is essentially unaffected by the presence or absence of the dynamic transformation (Sülzenbrück and Heuer, 2009a, 2010). In the absence of the kinematic transformation (when cursor movements represent movements of the pen at the proximal end of the lever), the dynamic transformation has almost no effects on movement characteristics (Heuer and Sülzenbrück, 2012a). Nevertheless, under certain conditions the dynamic transformation affects the curvature of the hand paths, as will be detailed below. This effect results from the inertial anisotropy of the sliding lever. For translations, the inertial resistance is constant, but for rotations it depends on the relative lengths of the effort arm and the load arm. A general consequence of the inertial anisotropy is a deviation of the direction of movement from the direction of force. If this is not taken into account during movement production, the paths of the hand will be curved. With the lever in our experiments, this curvature of hand paths was generally suited to reduce the curvature of the paths of the tip of the lever that results from the kinematic transformation. Thus, with respect to the curvature of the trajectories of the tip of the lever, the dynamic transformation tended to compensate the effects of the kinematic transformation.

## **CLOSED-LOOP CONTROL AND THE ACQUISITION OF AN INTERNAL MODEL**

Mastery of an extrinsic visuo-motor transformation requires its inversion, so that the hand movements that are appropriate for an intended movement of the effective part of the tool can be determined. In principle, the inversion can be achieved by open-loop control or by closed-loop control (Jordan, 1996). While in the former case a sufficiently accurate internal model of the transformation is required, in the latter case minimal or no adjustments of the parameters of the controller are sufficient.

In the control of limb movements, open-loop control and closed-loop control generally operate in parallel and combine their respective advantages (Cruse et al., 1990; Heuer and Massen, 2013). Thus, one could expect a trade-off between both mechanisms that invert the transformation. Obviously, when the internal model of the transformation is accurate, little is left for closed-loop control, but when the internal model is almost absent because of variable transformations or a transformation that is too complex to be learned, the load on closed-loop control is high. Perhaps less obviously, one might also expect a reverse influence during practice. Depending on the quality of closed-loop control, the performance benefits that accrue from the acquisition of an internal model vary. When closed-loop control is impeded, accurate performance depends on the acquisition of a sufficiently accurate internal representation. Therefore, performance benefits of learning such a representation are high, and a more accurate internal model of the transformation should be acquired. In contrast, when closed-loop control is facilitated, performance benefits of learning the representation are low, and the acquired internal model should be less accurate.

The evidence obtained with the sliding lever is consistent with the trade-off hypothesis. Sülzenbrück and Heuer (2011) compared the acquisition of the internal model of the kinematic transformation across three different practice conditions. In the first condition, visual feedback was presented concurrently during each movement. In the other two conditions visual feedback was terminal, that is, it was presented after the end of each movement only and could not be used for on-line corrections. The one kind of terminal feedback was knowledge of results—only the final position of the cursor was shown after the end of the movement. The other kind of terminal feedback was knowledge of performance—in this case the path of the cursor was shown in addition to its final position.

**Figure 2A** presents movement time in the practice blocks of trials. With concurrent visual feedback, movement time was long and declined in the course of practice, whereas with terminal visual feedback movement time was much faster and essentially constant across practice blocks. The error of movement direction, shown in **Figure 2B**, exhibits the reverse pattern. It was large and declined in the course of practice with terminal visual feedback, but with concurrent visual feedback it was essentially 0 throughout practice. Thus, in terms of accuracy, there was no performance benefit of acquiring an internal model in the presence of concurrent visual feedback, but only in its absence. In terms of movement duration, there may have been some performance benefits; however, it is not clear to what extent the decline of movement time results from the acquisition of an internal model (and the thereby reduced load on closed-loop control) or from the optimization of the closed-loop controller.

The accuracy of internal models of extrinsic transformations can be assessed in visual open-loop tests in which the accuracy of performance critically depends on the accuracy of the model (Heuer, 1983, p. 46; Davidson et al., 2000). In **Figure 2C** the mean hand paths in such open-loop tests after practice with the different kinds of visual feedback are shown. Under these open-loop conditions movements were fairly inaccurate in all three groups, but after practice with concurrent visual feedback errors of direction were even stronger than after practice with terminal visual feedback. The same was true for the variable errors. Thus, by interfering with visual closed-loop control during practice one can facilitate the acquisition of the internal model of the visuo-motor transformation of the sliding first-order lever.

In addition to the prevention of visual closed-loop control, its facilitation does also produce the expected effects, which in this case is a reduced accuracy of the acquired internal model of the extrinsic transformation. Wentink et al. (2002) observed faster performance in a simulated minimal access surgery task

positions (after Sülzenbrück and Heuer, 2011).

**movement was shown together with the target); KP, knowledge of**

when the shaft of the laparoscopic instrument was presented on the monitor. With the visible shaft, the mechanical transparency of the tool is enhanced as compared to the task variant where only a cursor is visible (Heuer and Hegele, 2010; Sülzenbrück and Heuer, 2012, cf. **Figure 3A**). **Figures 3B–D** shows the effects of the visible shaft on visual closed-loop performance. In addition to the group who saw only the cursor (*cursor*−) and the group who saw the shaft of the instrument in addition (*shaft*), there was a group *cursor*+. In this third group only the cursor was visible as for group *cursor*−. However, group *cursor*+ received an initial explanation of the kinematic transformation of the sliding lever in the same way as group *shaft*, while this information was not given to participants of group *cursor*−.

From **Figure 3B** it is apparent that movement time was considerably faster with the visible shaft than when only the cursor was presented on the monitor. This difference was even larger in older adults than in young ones (Heuer and Hegele, 2010) as they participated in the study of Sülzenbrück and Heuer (2012). In addition the path of the cursor was straighter when the shaft was visible (**Figure 3D**), and movement latency—the time from presentation of the target to the start of the movement—was faster (**Figure 3C**). The difference in movement latency suggests that preparatory processes took less time when the shaft was visible, perhaps because of less involvement of open-loop control and the internal model of the visuo-motor transformation.

In a visual open-loop test, which followed the practice period, adaptive errors of direction were largest after practice with the visible shaft (28.0◦) and smaller in groups *cursor*+ (12.6◦) and *cursor*− (18.7◦). Adaptive errors of direction are the direction errors of the hand in a visual open-loop test for which the presence of the transformation is instructed. From these errors those in a pre-test are subtracted in which the hand targets are presented and a 1:1 mapping of hand positions on cursor positions is instructed.

The findings reported thus far are consistent with the hypothesis of a trade-off between the quality of visual closed-loop control during practice and the accuracy of the acquired internal model of the visuo-motor transformation. From a practical perspective, the hypothesis suggests to impede closed-loop control during practice, e.g., by using terminal visual feedback, and to facilitate closed-loop control only after a fairly accurate internal model has been acquired. However, the situation becomes more complicated when additional findings are taken into account. In fact, overall the pattern of results obtained with different types of visual feedback during practice is fairly opaque.

For example, Heuer and Hegele (2010) used a virtual rather than a physical lever and observed benefits of the visible shaft for closed-loop control, but no clear effect on the accuracy of the acquired internal model. When one broadens the range of the visuo-motor transformations beyond that of the sliding lever, comparisons of the effects of concurrent and terminal feedback have sometimes found advantages of terminal feedback (e.g., Bernier et al., 2005; Heuer and Hegele, 2008a), but sometimes advantages of concurrent feedback (e.g., Peled and Karniel, 2012), and sometimes essentially no difference between conditions (Heuer and Hegele, 2008b). To this can be added observations on prism adaptation where terminal and concurrent visual feedback have been shown to result in different types of adaptive changes (e.g., Uhlarik and Canon, 1971). Similarly, Hinder et al. (2008, 2010) observed that concurrent and terminal visual feedback resulted in automatic recalibration and a cognitive strategy, respectively. On the other hand, Heuer and Hegele (2008a) could not find such a difference in tests of automatic after-effects and explicit knowledge. Thus, overall the differences between these different practice conditions are far from being clear. They are more a challenge for future research than a guideline for training schedules of minimal access surgery. The underlying mechanisms are not yet understood, but they are certainly more complex than suggested by the tradeoff hypothesis which accounts for only a subset of the available findings (for a review of this line of research, see Sülzenbrück, 2012).

## **INTERNAL MODELS OF COMPLEX VISUO-MOTOR TRANSFORMATIONS**

#### **APPROXIMATIONS AND FINE TUNING**

In studies of adaptation to extrinsic visuo-motor transformations, certain types of transformation are used preferably, namely visuo-motor rotations and—less frequently—gain changes. These transformations relate to the vector-coding hypothesis (Vindras and Viviani, 1998) according to which movement planning involves independent specifications of movement direction and amplitude. This notion has received considerable support both from behavioral data in humans (e.g., Favilla et al., 1989) and single-cell recordings in behaving monkeys (e.g., Georgopouplos et al., 1986). In addition, vector-coding allows a simple translation of a visually perceived target vector in the one plane, which points from start location to target location, into a movement vector in a different plane, which points from the current position of the hand to its target.

Adaptation to visuo-motor transformations can be conceived in terms of rotations and of length changes of the target vector to obtain the appropriately transformed movement vector. In fact, adaptation to rotations and gain changes differs both in behavioral characteristics and in neural substrates. Adaptation to changes of the visuo-motor gain is fairly rapid and generalizes across directions and amplitudes (Bock, 1992; Bock and Burghoff, 1997; Krakauer et al., 2000; Vindras and Viviani, 2002). In contrast, adaptation to visuo-motor rotations is slower and limited to a range of directions around the practiced one (Pine et al., 1996; Krakauer et al., 2000). Gain adaptation involves mainly subcortical structures (Krakauer et al., 2004), whereas rotation adaptation is accompanied by enhanced activity of cortical regions and the cerebellum (Ghilardi et al., 2000; Imamizu et al., 2000).

Turning to the visuo-motor transformation of the sliding firstorder lever, it can be described in terms of rotations and gain changes, but this is a quite complex description which includes direction-dependent rotations and gain changes (cf. Heuer and Hegele, 2009). Even though adaptation to direction-dependent rotations and gain changes is possible (cf. Heuer and Hegele, 2008b; Hegele and Heuer, 2010a), these are not the ingredients of the internal model of the kinematic transformation of the sliding lever. The detailed analysis of the errors in visual open-loop tests after practice with the transformation of the sliding lever strongly suggests that the internal model captures the characteristics of the transformation in a different format.

**Figure 4A** shows averaged trajectories of the cursor and the hand in an open-loop test after the end of practice, as reported by Sülzenbrück and Heuer (2009a). Movements do not end at their targets. However, the errors are highly systematic. Rather than at the targets, the movements end close to positions which are marked by outline squares. These are the correct final positions according to a line-symmetric approximation. Basically, to transform the target vector into an appropriate movement vector, it is reflected at a horizontal axis in the sagittal plane which runs through the start position of the hand (or a vertical axis through the start position of the cursor). In the experiment of Sülzenbrück and Heuer (2009a) the deviations from the line-symmetric approximation were only small. In a subsequent experiment (Sülzenbrück and Heuer, 2010) we used the target configuration of **Figure 1C** rather than the configuration of **Figure 1B** and terminal rather than concurrent visual feedback. Under these conditions the final positions of hand movements deviated more from the line-symmetric approximation and were gradually shifted toward the correct positions. **Figure 4B** shows the mean ratios of the observed direction errors divided by the directional deviations of the symmetry approximation. These ratios are 1 if the movements end exactly at the positions according to the approximation, and they are 0 if they end exactly at the correct target positions.

The observed errors in open-loop tests strongly suggest that the internal model of the kinematic transformation of a sliding first-order lever does not consist of direction-dependent rotations and gain changes, but of a rapidly acquired line-symmetric transformation and a slowly acquired fine tuning. To what extent the fine tuning is acquired at all depends on practice conditions. For

open-loop test after practice with the sliding first-order lever (after Sülzenbrück and Heuer, 2009a). **(B)** Mean ratios of the observed direction errors divided by the directional deviations of the symmetry corresponds to the symmetry approximation, and of 0 if they correspond exactly to the correct target position (after Sülzenbrück and Heuer, 2010).

reasons described above, a more precise fine tuning is acquired with terminal than with concurrent visual feedback, and perhaps also with target configurations for which the symmetry approximation results in larger errors than for target configurations for which the symmetry approximation is more accurate.

According to Werner and Bock (2010), an internal model of a line-symmetric transformation (either horizontal or vertical inversion) is acquired with an initial point-symmetric approximation (combined horizontal and vertical inversion), for which the hand movement is in the direction opposite to the target. Thus, acquisition of an internal model of the transformation of the sliding lever might also start with a point-symmetric approximation which precedes the line-symmetric one. The notion of a sequence of approximations to the internal model of the transformation of the sliding lever suggests that a line-symmetric transformation is acquired more rapidly and—to the extent that the fine tuning is incomplete—more accurately than the lever transformation. An internal model of a point-symmetric transformation should be acquired even faster. This is what Heuer and Sülzenbrück (2012c) observed, as shown in **Figure 5**. In fact, with the point-symmetric transformation accuracy of movements with terminal visual feedback was best from the very start and did not improve during practice. The analysis of movement endpoints during practice with the lever transformation showed the typical line-symmetric approximation, but only a very short-lived—if at all—point-symmetric approximation.

Thus far the symmetry approximation has been observed only with the lever transformation. It is not clear whether this

approximation is also involved in the acquisition of internal models of other types of transformation. In addition, there are a number of questions that are not yet answered. A central question concerns the symmetry axis. In all experiments reported thus far, the start position of the hand, the tip of the lever, and the cursor were roughly aligned in the sagittal plane. The symmetry axis was also in this plane. What happens, however, when initially the lever is rotated, so that, for example, the initial hand position is to the left of the sagittal plane and the initial position of the tip of the lever (and of the cursor) to the right? Will the sagittal plane or the lever serve as the symmetry axis in such situations?

Our tentative answer to this question is: the lever. This answer is tentative because we have not yet run a dedicated experiment to identify the symmetry axes for a broader range of start-target configurations. Therefore, the answer is based on a re-analysis of the movements in the initial warm-up blocks of Experiment 1 of Heuer and Sülzenbrück (2009). In that experiment practice was with terminal visual feedback. The target configuration was of the type shown in **Figure 1C**, but in addition the configurations were rotated around the fulcrum so that in the start positions the lever was rotated clockwise or counter-clockwise relative to the sagittal plane. For the final positions of the hand in the left part of the workspace of the lever the mean Euclidean deviations from the correct positions were 24.6 mm, from the positions according to the line-symmetric approximation around the initial orientation of the lever 28.8 mm, and from the positions according to the line-symmetric approximation around a horizontal axis parallel to the sagittal plane 49.9 mm. For the final positions in the right part of the workspace the corresponding deviations were 31.4, 27.9, and 46.7 mm, respectively. Thus, the movements of the hand ended closer to the positions appropriate for a line-symmetric approximation around the axis defined by the initial orientation of the lever than to the positions appropriate for a line-symmetric approximation around an axis parallel to the sagittal plane.

## **LOCAL AND GLOBAL CHARACTERISTICS**

The kinematic transformation of the sliding first-order lever is defined for its whole workspace. Thus, a rule that is acquired in some region of the workspace could be generalized to other regions. In contrast to studies of generalization, e.g., of adaptation to visuo-motor rotations (cf. Krakauer et al., 2000; Wang and Sainburg, 2005; Heuer and Hegele, 2011), generalization of the rule would imply different hand movements for same target vectors in different regions of the workspace. However, if indeed the internal model consists of the line-symmetric approximation and a fine tuning, generalization could take different formats. First, generalization could be restricted to the symmetry approximation. In this case the same hand movements would go along with same target vectors in different regions of the workspace. Second, fine tuning could generalize in addition. If the acquired fine tuning were local, it would remain invariant across different regions of the workspace. Again, same hand movements would go along with same target vectors. Third, if a general rule were learned for the fine tuning, generalization would be roughly appropriate for the particular fine tuning required in each region of the workspace. Only in this case same target vectors would be associated with different hand movements in different regions of the tool's workspace.

We studied the global vs. local characteristics of the internal model of the kinematic transformation of the sliding lever in a straightforward transfer experiment (Heuer and Sülzenbrück, 2013b). Participants practiced with target configurations of the type shown in **Figure 1B**. In three groups of participants, in the start positions the effort and the load arm were equally long, the load arm was longer, or the effort arm was longer. Thus, different groups of participants practiced in three different regions of the workspace of the lever. After practice, all participants were tested under visual open-loop conditions in the three regions, of which they had encountered only one during practice. **Figure 6** shows the mean end positions of the hand together with the correct ones and the ones appropriate for the symmetry approximation for two groups of participants and two regions of the workspace in which transfer was tested.

From **Figure 6** it is apparent that the required fine tuning, in particular that of movement directions, depends on the region of the workspace. For movements toward the participant (or downward on the monitor) the symmetry approximation is almost correct for the short effort arm, but for the long effort arm there are strong deviations; for movements away from the participant (or upward on the monitor) the symmetry approximation is almost correct for the long effort arm, and for the shorter effort arms the deviations become larger. The final positions of the hand movements of the participants deviate from the symmetry approximation toward the correct end positions. But these deviations are specific for the practice conditions and not for the tests, that is, the deviations from the symmetry approximation acquired during practice with a certain length of the effort arm are transferred to the tests with different lengths of the effort arm without taking the length of the effort arm into account. Accordingly the patterns of mean final positions are different across the two rows of graphs of **Figure 6** (different regions of the workspace during practice), but not across the two columns (different regions of the workspace in transfer tests).

According to these findings, the fine tuning is represented locally, and the locally acquired fine tuning is generalized across the workspace of the lever together with the symmetry approximation. However, no general rule is acquired for the fine tuning that allows adjusting it to the different regions of the workspace. Such a rule, for example, could map the visuo-motor gain of rotations of the lever to the length of the effort arm which varied both within and between the target sets.

## **EXPLICIT AND IMPLICIT REPRESENTATIONS**

In general the execution of a movement involves early processes, such as an intention to reach for a certain object, which are subject to conscious awareness. In contrast, later processes, such as the changes of muscle lengths, remain outside conscious awareness. Somewhat intermediate processes, such as the adjustments of movements to extrinsic visuo-motor transformations, can be both (cf. Heuer et al., 2011; Heuer and Sülzenbrück, 2012b). In the present paper we have used the term "internal model" indiscriminately for implicit and explicit knowledge of the transformation, but it may be useful to more clearly distinguish the internal model which represents implicit knowledge and results in adaptation proper from explicit knowledge which is used for strategic corrections (cf. Heuer and Sülzenbrück, 2013a). For example, implicit knowledge of a visuo-motor transformation has to be

acquired during physical practice, whereas explicit knowledge can also be instructed. Adjustments based on explicit and implicit knowledge are largely additive (Mazzoni and Krakauer, 2006; Sülzenbrück and Heuer, 2009b; Taylor and Ivry, 2011). However, interactions regarding the acquisition and use of the different types of knowledge can result when improved strategic corrections reduce the need to acquire implicit knowledge or when stronger implicit adaptive changes reduce the need for strategic corrections.

Implicit and explicit adjustments to visuo-motor transformations have a number of different characteristics in addition to the differences with respect to conscious awareness and the different ways of acquisition. Implicit adjustments to visuomotor rotations are restricted to a range of target directions around the practiced ones, whereas explicit adjustments generalize across all target directions (Krakauer et al., 2000; Heuer and Hegele, 2008c); implicit adjustments are stable across the adult age range, whereas explicit adjustments decline (Bock, 2005; Heuer and Hegele, 2008c); implicit adjustments depend on intact cerebellar functions, whereas explicit adjustments do not (Taylor et al., 2010); different implicit adjustments cannot be acquired concurrently when the start position of the hand is the same, but different explicit adjustments can (Hegele and Heuer, 2010b); implicit adjustments to visuo-motor rotations are specific for a certain region of the workspace of the hand, whereas explicit adjustments generalize across a large range of the workspace (Heuer and Hegele, 2011). Most likely implicit and explicit adjustments are based on different types of error information (cf. Taylor and Ivry, 2012).

Explicit knowledge of the visuo-motor transformation of the sliding lever is clearly present. For its assessment we used a checkerboard pattern on the opaque shield that prevented direct sight of the hand and the lever. On this pattern the location of the start position was marked. In each trial the start position together with a target was presented on the monitor. Participants had to indicate the location of the near end of the lever appropriate for its distal end and thus the cursor to reach the target by reading the letter-number combination written in the square of the checkerboard that was just above that location. These verbal responses were transformed into errors of direction and amplitude that were then analyzed in the same way as movement errors. However, the results obtained were rather noisy, and the conclusions are somewhat tentative.

By and large the line-symmetric approximation seems to be represented explicitly, but the fine tuning implicitly. For example, the variations of visual feedback during practice, which affected the accuracy of fine tuning and thus the errors in open-loop tests, had no reliable effects on the errors of explicit judgements. Sülzenbrück and Heuer (2011) observed a somewhat larger error of explicit judgements after practice with concurrent visual feedback than after practice with terminal feedback; the difference, however, did not approach statistical significance. In contrast, the error observed in the visual open-loop test was reliably larger after practice with concurrent visual feedback than after practice with terminal feedback. Similarly, Sülzenbrück and Heuer (2012) observed a somewhat larger error of explicit judgements after practice with the visible shaft of the lever than after practice with only the cursor being visible, but again the difference failed to reach statistical significance—in contrast to the significant difference between movement errors after the different practice conditions.

In the study of generalization across the workspace, explicit judgements did only marginally deviate from the symmetry approximation (Heuer and Sülzenbrück, 2013b). The fine tuning was essentially absent. This suggests that it was represented implicitly rather than explicitly. However, there was essentially no evidence of fine tuning in the after-effects, which are often used as a measure of implicit adjustments. Nevertheless, the fine tuning was clearly present in open-loop tests in which the presence of the transformation of the sliding lever was instructed. Thus, there is little doubt that the symmetry approximation is explicitly represented, but for the fine tuning the issue is somewhat unsettled. Perhaps it is implicitly represented, but the absence of the lever in the after-effect test served as an effective cue not to invoke the internal model of the tool any more (cf. Kluzik et al., 2008).

## **POSITIONS AND MOVEMENT PATHS**

There is some indication that end positions and other characteristics of movements aimed at a target are controlled separately (DiZio and Lackner, 1995; Sainburg and Wang, 2002; Brown et al., 2003). Thus, in principle the internal model of a visuo-motor transformation could map start and target positions on corresponding positions of the hand. Alternatively, of course, it could map desired trajectories of the tip of the lever (or of the cursor). In the first case, only target positions would be transformed, and trajectories would remain those normally found with the particular start-target combinations for the hand. This kind of observation has been reported by Verwey and Heuer (2007) for rapid movements with a non-linear amplitude transformation. Whereas the target positions of the hand were transformed, the velocity profile of the hand was essentially the same as when hand movements to the same targets were produced in the absence of the non-linear amplitude transformation.

In the experiments with the sliding first-order lever we focussed on curvature. Hand movements from a start position to a target have almost straight paths (Morasso, 1981; Abend et al., 1982; Atkeson and Hollerbach, 1985; Kaminski and Gentile, 1986). With the sliding lever, straight paths of the hand result in curved paths of the tip of the lever, as is evident from **Figure 1**. Almost straight paths of the hand would be expected if only visual target positions were transformed into target positions for the hand. Alternatively, if straight paths of the tip of the lever (and the cursor) were planned and transformed into paths of the hand, these should be appropriately curved.

Transverse movements are known to have a slight concave curvature in general (Wolpert et al., 1994; Haggard and Richardson, 1996; Van Thiel et al., 1998). In two experiments with the sliding first-order lever (Sülzenbrück and Heuer, in preparation) concave curvature of hand movements was observed only in a particular condition. In this condition the kinematic transformation of the lever was present, targets were defined for the tip of the lever, visual feedback was terminal or absent, and the pen was detached from the lever so that there was a constant inertial resistance of the tool rather than the inertial anisotropy of the sliding lever (the dynamic transformation of the lever was absent). When the dynamic transformation of the lever was present, that is, when the pen was attached to the lever, concave curvature of hand movements turned into convex curvature. Whereas concave curvature of transverse hand movements increases the curvature of the cursor paths, convex curvature of the hand movements results in a straightening of the cursor paths. Such an effect of the dynamic transformation has also been observed by Sülzenbrück and Heuer (2010), and it can likely result in straighter paths of the tip of the lever (and the cursor) than of the hand (cf. Heuer and Sülzenbrück, 2009).

When terminal visual feedback is replaced by concurrent visual feedback, curvature of hand paths becomes convex both in the presence and in the absence of the dynamic transformation of the lever. Convex curvature of hand movements is associated with a straightening of the paths of the cursor. This finding on the effects of concurrent visual feedback corresponds to observations made with other types of kinematic transformations when the cursor was visible (Flanagan and Rao, 1995; Wolpert et al., 1995). Thus, processing of visual feedback seems to be critical for straight paths of the cursor. Straight paths of the cursor are thus not based on the internal model of the visuo-motor transformation, but they are a characteristic of visual closed-loop control.

Hand paths change from convex or concave curvature toward straightness when targets are presented for the hand rather than for the tip of the lever, that is, when the kinematic transformation of the sliding lever is absent. This was the case both in the presence (cf. Heuer and Sülzenbrück, 2012a) and in the absence of the dynamic transformation, even when there was no concurrent visual feedback. Nevertheless, the straightening of hand paths under those conditions is likely a consequence of closed-loop control based on proprioceptive rather than visual feedback signals.

Movement execution involves a series of transformations, including the extrinsic transformations implemented by tools. Thus, when a more proximal variable such as the movement of the hand is controlled, the more distal variables such as movements of the effective part of the tool are secondary and result from the transformations. However, control can also refer to a more distal variable such as the trajectory of the effective part of the tool. In this case the more proximal variables are secondary and result from the operations that invert the transformation—the inverse internal model of open-loop control and the closed-loop control based on the sensory registration of the controlled variable. Regarding the question whether control is concerned primarily with more proximal or more distal variables, the distal-control hypothesis has gained weight during the last couple of years as a major ingredient of broader conceptions of action control (Prinz, 1992, 1997; Hommel et al., 2001; Kunde et al., 2004; Kunde, 2006).

The present findings with the sliding lever do not fit the simple distinction between proximal and distal control. Control is distal with respect to movement targets. This must be the case as long as movements serve their purpose, provided that targets are distally defined, that is, for the tip of the lever. But when targets are defined for more proximal variables such as the position of the hand, distal variables can be neglected. Depending on the more distal or more proximal variable for which targets were defined, we found (almost) straight paths of the tip of the lever or of the hand (and correspondingly curved paths of the hand and the tip of the lever, respectively). However, this was true only when concurrent feedback on the path of the tip of the lever or of the hand was available. For the tip of the lever, the only source of concurrent feedback is vision, but for the hand there is proprioception in addition. When targets were defined for the tip of the lever and visual information was no longer available, curvature of the path of the tip of the lever was affected by the dynamic transformation of the tool. Thus, straightness of the path of the tip of the lever seems to result from visual closed-loop control, but not from open-loop control. Consistent with the conclusion of Verwey and Heuer (2007), which was based on findings with a quite different paradigm, the internal model seems to translate only visual targets into hand targets. Thereafter the path of the hand is not a controlled, but an emergent property as long as no closed-loop control of the effective part of the tool is possible. This is different when the targets for the hand are defined directly. In this case proprioceptive feedback is used for closed-loop control.

## **CONCLUDING REMARKS**

In this paper we have integrated a number of observations on the mastery of the rather complex transformation of a certain tool, a sliding first-order lever. The study of this tool was motivated both by theoretical and applied considerations. In this section we shall briefly touch upon some open issues from both perspectives.

Learning to operate a sliding first-order lever involves both basic sensori-motor processes and cognitive strategies, likely based on mechanical reasoning to some degree. Thus, there may be differences to less complex extrinsic transformations and to intrinsic transformations. Perhaps these differences are captured by the distinction between cognitive and perceptual learning (Bedford, 1993) or between motor skill acquisition and recalibration (Clower and Boussaoud, 2000). Perhaps a more continuous conception of differences between adjustments to different types of transformations is more appropriate. In any case a theoretical clarification—based on solid experimental data—would be highly desirable. Thus far only fragments of such a theory exist.

In a discussion of differences between learning of intrinsic and extrinsic transformations, Heuer (1983, p. 36–38) noted two major contrasts. The first one is in terms of the timescales of changes of the transformation (cf. Körding et al., 2007). The other one is in terms of the identity of the object to which discrepant visual and proprioceptive position information refer (cf. Bedford, 1995). As a marker of the type of internal model acquired, aftereffects were envisaged which can be observed when the novel transformation is no longer present. After-effects can be conceived as signature of a learned intrinsic transformation, and the absence of after-effects as the signature of a learned extrinsic transformation (cf. Kluzik et al., 2008). A change of the internal model of intrinsic transformations as a result of practice is conceptually similar or even identical to a change of the body schema, a change that also has been inferred from the observation of after-effects (Cardinali et al., 2009).

The distinction between the two kinds of transformation is fuzzy, at least for the learner. He or she is faced with the credit assignment problem whether changes of intrinsic or extrinsic transformations are responsible for the changes of sensori-motor performance. The principles by which the problem is solved are not yet fully clear. There is evidence from prism-adaptation studies for the role of repeated changes between transformations, in the course of which after-effects of the optical displacement disappear (Kravitz, 1972; Welch et al., 1993), and for the role of experienced object identity (Welch, 1972). More recent findings by Kluzik et al. (2008) show reduced after-effects also with the abrupt rather than gradual introduction of a force field, similar to previous observations on extrinsic visuo-motor rotations (Kagerer et al., 1997). Thus, there is likely a gradual transition between characteristics of acquired internal models of extrinsic and intrinsic transformations. Learning of the complex extrinsic transformation of the sliding lever may differ even more from adaptation to intrinsic transformations than learning to use simple tools such as levers and rakes because of the role of mechanical reasoning (Johnson-Frey, 2003).

A valid theoretical framework for adjustments to different types of transformations could also be helpful to structure apparently contradictory results. A particularly conspicuous set of conflicting and opaque findings are those on the effects of concurrent and terminal visual feedback during practice. Even though our own results are largely in line with the trade-off hypothesis according to which better conditions for closedloop control during practice result in poorer acquisition of an internal model of the transformation, findings from other laboratories strongly suggest the existence of not yet identified conditions that critically modulate the effect of practice conditions (cf. Sülzenbrück, 2012).

The theoretical framework at stake would certainly have to build on a distinction of different processes involved in mastering complex visuo-motor transformations. In this paper we have not only distinguished between implicit and explicit adjustments, but also between a discrete approximation, that is rapidly acquired, and a slowly acquired graded fine tuning. Similar distinctions between discrete approximations and graded fine tuning have been suggested by Abeele and Bock (2001) and Werner and Bock (2010). In addition, formal models with two or more concurrent processes operating at different rates have been proposed to account for a large set of findings (Smith et al., 2006; Lee and Schweighofer, 2009). At present the relations between the different two- or multi-process models are not clear.

Turning to the applied perspective, the sliding first-order lever shares fundamental mechanical characteristics with the tools used in minimal access surgery. To the extent that surgicalskills training becomes separated from the operating theater and physical or virtual simulators are added to the traditional apprenticeship model of surgical training, principles of motor learning and performance gain relevance for the design of training devices and procedures (e.g., Wulf et al., 2010). Of course, the generalization of basic-research findings to the design

## of training procedures needs specific validations. Nevertheless, the findings reported in this paper suggest a few practical considerations.

According to the trade-off hypothesis of closed-loop control during practice and the acquisition of an internal model, visual feedback during (simulator) practice should be poor so that a more accurate internal model of the transformation of the tool can be developed. In contrast, when performance rather than learning is critical, conditions for visual feedback should be optimized, e.g., by using a large visual field to the extent that this is possible. Even with an optimized internal model, performance—in particular with respect to accuracy will continue to depend critically on visual closed-loop control. Finally, training should take the specificity of the internal model for certain regions of the workspace of the tool into account. Therefore, it should cover the whole workspace and not only parts of it.

## **ACKNOWLEDGMENTS**

This paper is based on research funded by the Deutsche Forschungsgemeinschaft, grants HE 1187/16-1 and SU 693/1-1. We thank Miya Kato Rand for helpful comments on a previous version of the manuscript.

## **REFERENCES**


in targeted force impulses. VI. Independent specification of response amplitude and direction. *Exp. Brain Res.* 75, 280–294.


dual adaptation to visuomotor rotations. *Conscious. Cogn.* 19, 906–917. Heuer, H. (1983). *Bewegungslernen.* Stuttgart: Kohlhammer.


visuo-motor transformations and their age-related changes. *Hum. Mov. Sci.* 30, 916–930.


impedes open-loop control of a complex tool. *Exp. Brain Res.* 218, 283–294.


Werner, S., and Bock, O. (2010). Mechanisms for visuomotor adaptation to left-right reversed vision. *Hum. Mov. Sci.* 29, 172–178.

Wolpert, D. M., Ghahramani, Z., and Jordan, M. I. (1994). Perceptual distortion contributes to the curvature of human reaching movements. *Exp. Brain Res.* 98, 153–156.

Wolpert, D. M., Ghahramani, Z., and Jordan, M. I. (1995). Are arm trajectories planned in kinematic or dynamic coordinates? An adaptation study. *Exp. Brain Res.* 103, 460–470.

Wolpert, D. M., and Kawato, M. (1998). Multiple paired forward and inverse models for motor control. *Neural Netw.* 11, 1317–1329.

Wulf, G., Shea, C., and Lewthwaite, R. (2010). Motor skill learning and performance: a review of influential factors. *Med. Educ.* 44, 75–84.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 21 September 2012; accepted: 26 January 2013; published online: 13 February 2013.*

*Citation: Heuer H and Sülzenbrück S (2013) Towards mastery of complex* *visuo-motor transformations. Front. Hum. Neurosci. 7:32. doi: 10.3389/ fnhum.2013.00032*

*Copyright © 2013 Heuer and Sülzenbrück. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*

## A supplementary circuit rule-set for the neuronal wiring

## *Kunjumon I. Vadakkan\**

*Division of Neurology, Department of Internal Medicine, Faculty of Medicine, University of Manitoba, Winnipeg, MB, Canada*

#### *Edited by:*

*Rachael D. Seidler, University of Michigan, USA*

#### *Reviewed by:*

*Florent Lebon, Université de Bourgogne, France Kristine L. Snyder, University of Michigan, USA*

#### *\*Correspondence:*

*Kunjumon I. Vadakkan, Division of Neurology, Department of Internal Medicine, Faculty of Medicine, University of Manitoba, GF532-820 Sherbrook Street, Winnipeg, MB R3A1R9, Canada. e-mail: umvadakk@cc.umanitoba.ca* Limitations of known anatomical circuit rules necessitate the identification of supplementary rules. This is essential for explaining how associative sensory stimuli induce nervous system changes that generate internal sensations of memory, concurrent with triggering specific motor activities in response to specific cue stimuli. A candidate mechanism is rapidly reversible, yet stabilizable membrane hemi-fusion formed between the closely apposed postsynaptic membranes of different neurons at locations of convergence of sensory inputs during associative learning. The lateral entry of activity from the cue stimulus-activated postsynapse re-activates the opposite postsynapse through the hemi-fused area and induces the basic units of internal sensation (namely, semblions) as a systems property. Working, short-term and long-term memories can be viewed as functions of the number of re-activatible hemi-fusions present at the time of memory retrieval. Blocking membrane hemi-fusion either by the insertion of the herpes simplex virus (HSV) glycoproteins or by the deposition of insoluble intermediates of amyloid protein in the inter-postsynaptic extracellular matrix (ECM) space leads to cognitive impairments, supporting this mechanism. The introduction of membrane fusion blockers into the postsynaptic cell cytoplasm that attenuates long-term potentiation (LTP), a correlate of behavioral motor activities in response to memory retrieval, provides further support. The lateral spread of activity through the inter-postsynaptic membrane is capable of contributing to oscillating neuronal activity at certain neuronal orders. At the resting state these oscillations provide sub-threshold activation to many neurons at higher orders, including motor neurons maintaining them at a low initiation threshold for motor activity.

**Keywords: circuit rules, motor learning, internal sensation, connectome, membrane hemi-fusion, long-term potentiation (LTP), wiring rules**

## **INTRODUCTION**

Neuronal wiring patterns have been examined using simple behavioral paradigms (Asakawa et al., 2008; Bronson et al., 2008; Cardona et al., 2010; Yu et al., 2010), microscopic examinations (Briggman and Denk, 2006; Hell, 2007), and genetic dissections (Luo et al., 2008; Bernard et al., 2009; Arenkiel, 2011) of neuronal circuits (Kohl and Jefferis, 2011). In addition, viral tracing methods, heterologous receptor expression systems, and optogenetic technologies have been used to examine changes in the neural circuitry of adult-born new neurons (Arenkiel, 2011). Even after using these methods, it was not possible to formulate the functional attributes of neuronal circuitry. Blood-oxygenation level-dependent (BOLD) signal sequences in functional magnetic resonance imaging (fMRI) studies (Logothetis, 2008; Rossier, 2009; Dosenbach et al., 2010) require a supplementary mechanism for the delay-corrected voxel-signals to explain the formation of higher brain functions. Even though the locations of corresponding neurons and their local network were studied by using *in vivo* two-photon calcium imaging followed by electron microscopical examination (Bock et al., 2011), the results are insufficient to explain their functional roles. This has left a huge gap in our understanding about the relationship between neuronal activity and higher brain functions. In addition, different network connectivity analyses have found that similar networks become activated during different tasks (Dosenbach et al., 2007, 2008; Seeley et al., 2007; Stevens et al., 2007; Demirci et al., 2009), requiring an explanation for the overlap. What additional wiring rules should be operating in unison with the known anatomical wiring that enable the formation of internal sensations of higher brain functions along with behavioral motor outputs?

A large body of experimental evidence demonstrates the firing of specific sets of neurons by one of the stimuli that took part in associative learning. Both experimental (Gelbard-Sagiv et al., 2008; Tye et al., 2008) and computational (Kepecs et al., 2008; Lavigne and Darmon, 2008) studies have shown activity from new sets of neurons during memory retrieval, leading to the understanding that this specific set of neurons represents memories. The current difficulties in explaining how neuronal firing creates higher brain functions have suggested the need to explore mechanisms that can explain cognitive functions (Abbott, 2008; Yuste, 2008) and to discover suitable wiring principles (Abbott, 2008; Yuste, 2008) that may explain what constitutes the internal

**Abbreviations:** AMPA, 2-amino-3-(5-methyl-3-oxo-1,2-oxazol-4-yl) propanoic acid; BOLD, Blood oxygenation level dependent; CA1, Cornu Ammonis region 1; DE, Dendritic excrescence; ECM, Extracellular matrix; EFA, Essential fatty acid; EPSP, Excitatory postsynaptic potential; fMRI, functional magnetic resonance imaging; GN, Granule neuron; LINK, link (capital letters are used to highlight its importance); LTP, Long-term potentiation; mEPSP, miniature excitatory postsynaptic potential; NMDA, N-methyl D-Aspartic acid; Postsynapse, Postsynaptic terminal (dendritic spine); Presynapse, Presynaptic terminal; SNAP, Synaptosomalassociated protein.

representations in the brain (Sullivan, 2010). Decoding the internal sensations of higher brain functions requires examining the circuit properties capable of encoding new information and later producing internal sensations along with motor neuron activations. Even though motor functions have been used in assessing memory retrieval in experiments, it is clear that the nervous system creates internal sensations even when all the motor actions are restricted. This makes the formation of internal sensations an obligatory property of the nervous system.

An alternative to the conventional connectome studies (Jarrell et al., 2012) is to examine possible basic building units of the nervous system similar to DNA sequences (Zador et al., 2012). These units are expected to have a supplementary mechanism operating along with the known anatomical circuitry, creating internal sensations concurrent with motor neuron activation that execute motor activities. On a functional level, these operations should facilitate beneficial interactions of the system with the environment. We expect the simultaneous arrival of multiple sensory inputs from a nearby item to create specific re-activatible changes in the nervous system. This is expected to facilitate the creation of the semblance of the remaining sensations from the item at the moment when the fastest travelling sensory stimulus reaches the animal when the animal moves away from the item. In the same way, if the animal is close to the item, the arrival of one of the associatively learned stimuli should evoke semblances of the remaining properties of the item. Re-activatible changes taking place at the time of associative learning are likely to occur at locations where different sensory pathways converge after a certain number of orders of neurons; for example, the hippocampus. These re-activatible changes should be able to concurrently activate motor neurons and create effective behavioral motor responses.

## **CIRCUIT PROPERTIES FOR EVOKING INTERNAL SENSATIONS**

The artificial stimulation of an intermediate order of neurons produces various sensory hallucinations (Selimbeyoglu and Parvizi, 2010), the complexities of which gradually increase as the location of these stimulations moves toward the higher neuronal orders. This lateral entry-induced hallucination of receiving sensory input can be considered an intrinsic property of the system. From this property, we can infer that the naturally present operational mechanism that gets activated during associative learning can be re-activated by the cue stimulus for evoking the internal sensation of the sensory properties of the previous associatively learned item. It is reasonable to assume that the perception in hallucinations and the internal sensation of retrieved memories belong to a spectrum of internal sensations depending on the strength of their formation. Since such changes are expected to occur at the locations of convergence of sensory inputs, (for example, the hippocampus), we examined changes occurring at such locations. The lateral entry of activity from the cue stimulus is hypothesized to induce the internal sensations of the remaining sensory qualities of the item. The most suitable cellular location for normal lateral entry gates should be a location where activity does not flow in a retrograde direction after crossing the synapses. This makes the postsynapses (postsynaptic membranes or dendritic spines or spines) (**Figures 1A**,**B**) ideal locations.

Since simultaneously-activated adjacent postsynaptic membranes are often apposed to each other at locations of convergence of sensory inputs (for example, the hippocampus), with negligible extracellular matrix (ECM) between them (Harris and Stevens, 1989), we examined the interaction between the postsynaptic membranes. It has been observed that the average inter-spine (inter-dendritic spine) distance is greater than the average spine head circumference (Konur et al., 2003), and adjacent neurons share only a small percentage of their inputs (Ecker et al., 2010). This increases the probability of the dendritic spines of different neurons being apposed to each other. This, in turn, increases the feasibility of certain interactions between specific dendritic spines (postsynapses) (between postsynapses B and D in **Figure 1B**) during associative learning and is referred to as a functional LINK (capital letters are used to highlight its importance) formation (Vadakkan, 2011b). Additional associative learning will result in more postsynapses becoming functionally LINKed. In a cross-sectional view through the inter-LINKed postsynaptic membranes, they can be viewed as islets of functional LINKs (between postsynapses B-D-F-H-J-L in **Figure 2**, bottom panel).

After associative learning, when the cue stimulus passes through different neuronal orders, it re-activates the interpostsynaptic functional LINKs (**Figure 1**) and instantaneously induces the semblance of sensory inputs arriving at the latter. The basic units of semblances are called semblions (**Figure 2A**) (Vadakkan, 2011b). The natural integration of semblions occurring at physiological time-scales results in the internal sensation of memories. Depending on the specificity of the cue stimulus,

*(Continued)*

set of lower order neurons {X}. The set of neurons {X} in turn are activated by a further large set of its lower order neurons {W}. Continuing this extrapolation

CA1 neuron receives a baseline summated EPSP short of one EPSP to

#### **FIGURE 2 | Continued**

trigger an action potential, then the additional EPSP arriving from the postsynapse D can add to sub-threshold EPSP, inducing an action potential in the CA1 neuron, resulting in its concurrent activation during memory retrieval; this CA1 neuron will not otherwise be activated in the absence of prior associative learning. This can explain place cell (CA1neuron) firing occurring concurrently with spatial memory retrieval. Bottom Panel: Cross-section through the postsynapses showing a newly formed functionally LINKed postsynapses B and D during associative learning. Three other islets are also shown. **(B)** Stimulation of the Schaffer collateral induces LTP by inducing postsynaptic membrane hemi-fusion between postsynapses that belong to islets of postsynapses B-D and F-H-J-L forming a mega-islet

a specific set of inter-postsynaptic functional LINKs gets reactivated and induces specific semblances, enabling the retrieval of specific memories. If integration of the semblances from different locations produces an excessive net semblance, it will allow memories to form even if some of the locations of their formation are damaged. This offers an explanation for the circuit property of transfer of memories from the hippocampus to the cortex, namely, consolidation (Vadakkan, 2011a). Since the cue stimulus re-activates the functional LINKs at sparsely distributed individual synapses at various brain locations, the combined effect of the net internal sensations induced during memory retrieval is expected to produce only a virtual internal sensation. In comparison, the internal sensations of hallucinations occurring during artificial stimulation of intermediate orders of neurons (Selimbeyoglu and Parvizi, 2010) should induce denser net semblances, producing a compelling sense of reality. Similarly, perception can be viewed as semblances formed based on previous associative learning. From **Figure 2A**, it can be seen that neither the physical presence of the neuron marked CA3, its lower orders of neurons {Y}, {X}, {W} nor the corresponding sensory receptors are required to evoke the cellular hallucination (semblance) at postsynapse D. This can explain how the internal sensation of phantom limb is formed.

Reversible as well as stabilizable properties of the interpostsynaptic functional LINKs make it feasible to view different types of memories as a continuum of the same process occurring at different time-scales, depending on the number of reactivatible units present at the time of memory retrieval. The involvement of previously-formed re-activatible basic operational units explains the ease of related learning. In the case of repetition of a specific associative learning event, related learning or learning between items that activate the same sensory receptor subsets, the newly formed inter-postsynaptic functional LINKs will be maintained long-term through stabilization by certain factors. Retrieval of memories will also maintain inter-postsynaptic functional LINKs. This will enable the maintenance of memories for a long period of time. If the functional LINKs are not re-activated or the stabilizing factors are lost, it will lead to the reversal of inter-postsynaptic functional LINKs, causing memory loss. When the re-activated set of inter-postsynaptic functional LINKs are distributed sparsely at higher neuronal orders, then the net semblances induced from these locations can provide an internal sensation for the specific key features of the item whose memories are retrieved.

B-D-F-H-J-L. A regular stimulus at the stimulating electrode has now an increased probability of reaching the recording electrode through the large number of hemi-fused postsynaptic membranes within the large mega-islet, showing a potentiated effect when recorded from the CA1 neuron. Neuronal orders from 1 to 6 are numbered from the sensory receptors. Bottom Panel: Cross-section of an area containing the newly formed mega-islet of functionally LINKed postsynapses B-D-F-H-J-L formed during LTP induction. Two other islets are also shown. {SR}, Set of sensory receptors; {sr}, subset of sensory receptors. If LTP-induced mega-islets include postsynapses B and D, it reduces the specificity of retrieved memories in retrieving memories since spread of activity through different non-specific postsynapses of the islet induces non-specific semblances [Modified from Vadakkan (2011b)].

## **MOTOR ACTIVITIES CONCURRENT WITH SEMBLANCE FORMATION**

The lateral spread of activity through inter-postsynaptic functional LINKs can contribute to the horizontal component responsible for the neuronal oscillations at certain neuronal orders (Vadakkan, 2012a). Continuous baseline activity of these neurons causes certain neurons at higher orders to receive sub-threshold summated excitatory postsynaptic potentials (EPSPs), short of eliciting an action potential. As the cue stimulus activity moves toward these higher neuronal orders, additional EPSPs through the re-activated functional LINKs are added to the net EPSPs, allowing it to cross the threshold for eliciting an action potential. Experiments that continuously recorded extracellularly from the CA1 neuronal layer in moving animals have shown that certain CA1 neurons specifically fire (elicit action potential) when the animal reaches specific locations within the field. These cells are called place cells (O'Keefe and Dostrovsky, 1971). Continuous oscillatory neuronal activity at lower neuronal orders provides sub-threshold activation (just a few EPSPs short of an action potential) to some of the CA1 neurons. Therefore, the addition of a few EPSPs arriving from specific cue stimuli from the environment (spatial cue) will be sufficient to fire an action potential in these CA1 neurons when animals reach specific locations within the field.

The same mechanism explains the activation of specific neurons in different regions of the brain during memory retrieval (Gelbard-Sagiv et al., 2008); the current thought is that this activation encodes specific memories. Some of these neurons that are fired concurrent with the arrival of the cue stimulus are motor neurons responsible for motor outputs. Even though all the memory studies have been carried out by measuring the behavioral motor outputs, it is reasonable to assume that the internal sensation of memories of specific items are correlated to the behavioral motor activity resulting from the activation of those motor neurons.

The firing neurons that contribute to the oscillating neuronal activities re-activate a large non-specific set of previously-formed normal functional LINKs (that represent the sensory properties of the items and events from the environment) at higher neuronal orders. The integral of the resulting non-specific set of semblances was hypothesized to provide a framework for consciousness (Vadakkan, 2010a), a baseline requirement for nervous system functions. Blocking synaptic transmission or inducing changes in the oscillatory waveforms during sleep (Massimini et al., 2005) or anesthesia (Llinas and Steriade, 2006) prevents the formation of internal sensations and concurrent motor activities.

## **REVERSIBLE WIRING FOR INTER-POSTSYNAPTIC FUNCTIONAL LINKs**

Inter-postsynaptic functional LINKs can operate as a universal mechanism, provided they can be quickly reversed back to independent membranes (explaining working memory as the net semblances formed from the re-activation of the transient interpostsynaptic functional LINKs before they reverse) or stabilized as hemi-fused inter-postsynaptic membranes for a long period of time (explaining long-term memories). What ideal properties can allow the inter-postsynaptic functional LINKs to operate between two postsynaptic membranes? Even though the spread of neurotransmitters to neighboring synapses (Coggan et al., 2005; Fernandes et al., 2010), 2-amino-3-(5-methyl-3-oxo-1,2-oxazol-4-yl) propanoic acid (AMPA) receptor trafficking (Makino and Malinow, 2009) and ephaptic coupling (Anastassiou et al., 2011) can be regarded as candidate mechanisms occurring between the postsynapses, they lack either the specificity or the timescales of formation and re-activation or the time-dependent reversibility that are required features for the inter-postsynaptic functional LINK.

At this juncture, we examined disease states that produce symptoms of loss of function, from which the structure-function aspect of the operational mechanism of inter-postsynaptic functional LINKs can be derived. From factors that can disturb memories, it should be possible to understand the nature of the normal operation of the functional LINKs. Once identified, the mechanism can be theoretically tested for its suitability to explain most of the previous experimental findings in all the related fields. Furthermore, if non-specific inter-postsynaptic functional LINKs occur at certain neuronal orders, they are expected to cause cognitive deficits via the formation of non-specific semblions, the activation of a new set of neurons, hallucinations resulting from semblances connecting features of different previously associatively learned items and changes in oscillatory neuronal activities resulting in changes in consciousness. Since all these features are seen in schizophrenia, we examined this disease in detail (Vadakkan, 2012b). Since a large number of previous studies show lipid membrane composition changes in schizophrenia, many of which were explained by chromosomal deletions involving proteins in lipid metabolic pathways, possible changes at the postsynaptic lipid membranes were examined. Significant prevention of the progression of the prodromal stage to schizophrenia through dietary essential fatty acid (EFA) supplementation in a doubleblinded randomized control trial (Amminger et al., 2010) and other similar studies indicate that pathological inter-postsynaptic functional LINKs become reversible at the prodromal stage. The non-reversibility of the fully manifested disease state with EFA indicates that the reversible mechanism becomes non-reversible over time. We found that time-dependent irreversibility of the phenomenon is possible when there is a physical interaction between the postsynaptic membranes.

Rapidly reversible membrane hemi-fusion has been observed extensively in biological systems (Melikyan and Chernomordik, 1997; Kozlov et al., 2010). Since membrane hemi-fusion is dependent on lipid composition, particularly the exchangeable sn2 and sn3 positions of the fatty acid structure, it is reasonable to argue that the derivatives of EFA become incorporated into the membranes, prevent them from forming non-specific interpostsynaptic membrane hemi-fusions and stop the conversion of the prodrome state (Cannon et al., 2008) to schizophrenia (Amminger et al., 2010). Membrane hemi-fusion that can be temporarily and permanently stabilized through the insertion of trans-membrane proteins (**Figure 3**) can function as re-activatible gates, meeting the requirements of the functional LINKs. The progression of the prodromal state to the disease state where it becomes non-reversible with EFA supplementation can be explained by the insertion of trans-membrane proteins across the hemi-fused inter-postsynaptic membrane segments (**Figure 3**) (Vadakkan, 2012b). EPSP can spread through the hemi-fused inter-postsynaptic membrane segment to the functionally LINKed postsynaptic membrane (**Figure 4**) both to

associative learning and LTP induction. A and C are presynapses and B and D are their corresponding postsynapses. **(B)** The boxed region in Panel **(A)** is expanded to show adjacently located postsynaptic cell membranes. **(C)** Simultaneous activation of the postsynapses both during learning and LTP induction can result in instantaneous hemi-fusion between the postsynaptic

**(D**) Repeated formation of the same hemi-fused inter-postsynaptic membrane segment will lead to its stabilization as a homeostatic cellular process. A trans-membrane protein makes the hemi-fused area temporarily stable, depending on the life-span of the protein or its ability to shift away from the hemi-fused area by lateral displacement [Modified from Vadakkan (2012b)].

induce semblance formation as a system property and to simultaneously allow this EPSP to spread to its neuronal soma (Note: hereafter, inter-postsynaptic functional LINKs and hemi-fused postsynaptic membranes are used interchangeably).

## **OBSTRUCTIONS TO HEMI-FUSION LEAD TO MEMORY DEFECTS**

Some of the herpes simplex virus-1 (HSV-1) glycoproteins can induce the formation of membrane hemi-fusion by getting inserted into the host membranes (Subramanian and Geraghty, 2007). Since these pathological membrane hemi-fusions are expected to form very non-specifically, neurons infected with HSV-1 can lead to severe cognitive defects as seen in herpes simplex encephalitis. This provides a feasible mechanism for the role of reversible membrane hemi-fusion in the operations for cognitive functions. Similarly, the accumulation of insoluble biochemical intermediates in the ECM space between the postsynaptic membranes (for example, deposition of amyloid proteins in Alzheimer's disease) can explain a mechanism that prevents membrane hemi-fusion at specific inter-postsynaptic membrane locations. This explains the patho-physiology of the cognitive defects in these disorders.

## **LTP AND SEMBLANCE FORMATION**

Studies of the patient H. M. (Scoville and Milner, 1957) revealed that the patient was unable to make any motor expression indicative of experiencing the internal sensations of retrieved memories of associatively learned items or events during a certain period of time prior to the surgical removal of H.M's hippocampi. This case study led to electro-physiological experimentations using isolated rodent hippocampi. The application of an initial brief repetitive stimulation at the axonal regions of the CA3 layer of neurons (Schaffer collaterals) in the hippocampal slices induced a potentiated effect at the CA3-CA1 synapses in response to a regular stimulus applied at the same location at a later time. This was observed by recordings from the CA1 region and is called long-term potentiation (LTP) (Bliss and Lomo, 1973). Following this finding, a large number of studies have shown correlations between behavioral motor outputs indicative of memory retrieval and LTP (Morris et al., 1986, 2003; Whitlock et al., 2006). Such a correlation is possible if similar changes can take place at a location between the site of stimulation (a group of Shaffer collaterals) and collection of responses (one CA1 neuron) during both associative learning and LTP induction (**Figure 2B**). LTP induction activates bundles of axonal fibers of the CA3 neurons (Schaffer collaterals) and can cause hemi-fusion between large numbers of postsynaptic membranes (dendritic spines) of the CA1 neurons. A normal stimulus at the same stimulating location can then travel through these hemi-fused postsynapses and arrive through a large number of dendrites of a given CA1 neuron, resulting in the recording of a potentiated effect from the latter's soma (**Figure 5**). The formation of inter-postsynaptic functional LINKs by membrane hemi-fusion both during associative learning and LTP induction provides a feasible explanation for the observed correlation. The reversal of the membrane hemifusion can explain the waning of recorded LTP over time and provides a comparable cellular explanation for the loss of memory over time.

**hemi-fusion in LTP.** The stimulation of a large number of Shaffer collateral fibers during LTP induction results in hemi-fusion between two postsynapses that belong to two large islets of functionally LINKed postsynapses (in green and red; Note that the double-walled circles are cross sections through the postsynapses as demonstrated in **Figure 2**). The critical location of hemi-fusion is shown by an asterisk. This results in the merger of these islets to form mega-islets. Following the LTP induction, a regular stimulus at the stimulating electrode induces EPSPs at the postsynapses belonging to both the islets. The recording electrode that records potentials arriving through the blue-colored islet of LINKed postsynapses will start recording the cumulative EPSPs arriving from both the red and blue colored postsynapses of the newly formed mega-islet [Modified from Vadakkan (2010b)].

Inputs from different sensations reach thalamic projection neurons whose axonal terminals (presynapses) synapse with postsynapses (dendritic spines) of the neurons in the lateral amygdala. We anticipate functional LINK formation between the postsynapses of different lateral amygdala neurons. In patch-clamp experiments following fear conditioning, an example of associative learning, when thalamic afferents are stimulated to measure the EPSPs at the pyramidal neurons in the lateral amygdala, an increase in the amplitude of the AMPA current is observed (Tye et al., 2008). This can be explained as the result of the arrival of additional AMPA currents through the functionally LINKed postsynapses induced during learning (**Figure 5**). Following fear conditioning, recordings from slices of the amygdala show an increase in miniature EPSP (mEPSP) amplitude (Tye et al., 2008). It is generally interpreted that an increase in mEPSP amplitude corresponds to an increase in the number or function of AMPA receptors (Malenka and Nicoll, 1999), one of the glutamate receptor subtypes. Based on the present work, the increase in mEPSP amplitude can be explained as a function of the additionally measured AMPA channel currents from the functionally LINKed postsynapses (formed during fear conditioning) reaching the patch-clamped neuron (Tye et al., 2008). The formation of functional LINKs between the postsynapses (dendritic spines) of the recording and other neurons may provide the route for the spread of mEPSPs.

N-methyl D-aspartate (NMDA) receptors of the excitatory neurotransmitter glutamate have been shown to be necessary for behavioral motor activities indicative of memory retrieval (Morris et al., 1986), the induction of LTP (Collingridge et al., 1983), and the activation of specific neurons that fire when the animal reaches a specific place in the field (place cell firing) (Kentros et al., 1998). This can be explained by the requirement for cue-induced synaptic activation (synapse A–B in **Figure 2**) that will then re-activate the functional LINK that induces both the formation of the internal sensation of memory and provides the additional EPSPs required for the activation of sub-thresholdactivated neurons. Severe defects in memory and consciousness occur when the NMDA receptors are blocked by auto-antibodies in NMDA receptor antibody encephalitis (Dalmau et al., 2008), demonstrating that cue-induced activation of postsynapses followed by the re-activation of the inter-postsynaptic functional LINKs are essential steps. Further support comes from the previous report that synaptosomal-associated protein (SNAP) inhibitors block membrane fusion and attenuate LTP (Lledo et al., 1998). The effective target of this inhibition is likely taking place at the level of inter-postsynaptic membrane hemi-fusion. Additional evidence is the observation of the possible structural changes from hemi-fusion between the adjacent postsynaptic membranes in the electron microscopic pictures [Figures 2B and 4D in Burette et al. (2012) and Figure 2 in Harris and Stevens (1989), He et al. (1998), Sirvanci et al. (2005)], even though the resolution of the images is limited.

## **NECESSARY CONDITIONS FOR SEMBLANCE FORMATION**

The formation of semblances is viewed as a property of a system in which the lateral entry of activity through the inter-postsynaptic functional LINKs enables its formation at the opposite postsynapse while simultaneously providing the horizontal component responsible for the oscillating neuronal activity. The necessary condition for evoking the semblance of activity from the presynaptic terminal C when postsynapse D (in **Figure 2A**) is activated by the lateral entry of activity through the inter-postsynaptic functional LINK is that postsynapse D should otherwise be normally activated by its presynapse C (in **Figure 2A**) in a continuous manner. Continuous quantal release from the presynaptic synaptic vesicles even during periods of rest provides regular arrival of miniature potentials at the postsynapses, which is recorded as mEPSPs or "minis" (**Figure 4**). The fact that it is very difficult to block mEPSPs "even in experimental conditions" indicates that it is a highly conserved default operation of the system. Another necessary condition is the maintenance of oscillatory neuronal activity. The finding that electrical stimulation of the visual cortex produces a visual percept (phosphene) only when high-frequency gamma oscillations are induced in the temporo-parietal junction (Beauchamp et al., 2012) emphasizes the role of oscillating neuronal activity as a system requirement for semblance formation for creating internal sensations.

## **POSSIBLE ROLE OF DENDRITIC EXCRESCENCES AND RECURRENT COLLATERALS AT THE CA3 NEURONAL ORDER**

If associative learning between sensory stimuli that pertains to the physical properties of items from the environment constantly arrives at the hippocampus, where sensory inputs converge, it is reasonable to anticipate that evolution must have tried to conserve those functional LINKs most probably in the form of structural LINKs. Dendritic excrescences formed by the fusion of postsynaptic membranes at the closely located dendrites of individual CA3 neurons of the hippocampus (Chicurel and Harris, 1992; Gonzales et al., 2001; Murakawa and Kosaka, 2001) likely to represent evolutionarily maintained inter-postsynaptic functional LINKs resulting from obligatory associative learning between the sensory stimuli from fixed physical properties of the items in the animal's environment and the relationship between various items based on their physical properties (**Figure 6**). Therefore, a set of functional LINKs for a given nervous system is likely unique to the animal's physical environment. Based on the present work, activity arriving from any of the many presynaptic terminals of an excrescence will induce the semblance of activity at the

neighboring postsynaptic membrane segments of the excrescence depending on the spread of activity. Moreover, when the extrapolation of semblances from a higher-order postsynapse reaches the CA3 neuron excrescence (see **Figure 2A**) it must include all the semblances from all the postsynapses at the excrescence. This includes all the related (previously associatively learned) sensory inputs from the environment, depending on the physical properties of the items in the environment. The induction of semblances at the excrescence also depends on the relationship with oscillations in the neighboring neuronal orders. An additional feature of CA3 neurons is the presence of recurrent collaterals from their axonal terminals that synapse on to themselves, facilitating the re-entry of their own activity (**Figure 6**). Recurrent collaterals synapsing to the excrescences can provide continuous semblances for the sensory inputs related to the physical properties of the environment, the nature, and effects of which need to be explored.

## **EFFECT OF INCORPORATION OF NEW NEURONS IN AN INTERMEDIATE NEURONAL ORDER**

The continuous integration of new neurons in the circuitry at the granule neuron layer of the hippocampus introduces new locations of functional LINK formation at higher neuronal orders during the repetition of associative learning. This leads to the formation of more semblions at the time of memory retrieval. Even though the specific features of semblions formed at the hippocampus and cortex are likely to be different, their cumulative effect is expected to strengthen the net semblance. In humans, the continuation of this process for nearly ten years will allow the formation of sufficient semblances from the cortex (locations of secondary and higher levels of convergence of sensory inputs) such that the nervous system becomes capable of retrieving similar memories even when the hippocampi are removed. This explains the process of consolidation of memories (Vadakkan, 2011a) (**Figure 7**). However, the incorporation of new neurons without the repetition of learning or the activation of the same set of sensory receptors used in previous associative learning events can lead to a reduction in the net semblance (**Figure 7**). We have previously seen that at the time of learning a very large number of reversible inter-postsynaptic membrane hemi-fusions are formed that can contribute to the large net semblance for working memory. As they reverse over time, the net semblance for memory reduces. The incorporation of new neurons can further reduce memories through the addition of non-specific semblances if there are no repetitions of learning. On the beneficial side, continued incorporation of new neurons along with repetitions of learning or related learning or the simultaneous activation of sensory receptor pairs result in the widely distributed locations of semblance formation with an excess net semblance during memory retrieval (**Figure 6**).

## **NEURONAL OSCILLATIONS MAINTAIN LOW INITIATION THRESHOLD NEURONS FOR MOTOR ACTIVITIES**

Regions in the brain where the functional LINKs are densely located (for example, the hippocampus) show slow oscillations (Sirota and Buzsaki, 2005; Beauchamp et al., 2012). These regions are expected to have both horizontal and vertical vector components driving these oscillations. The synaptic transmission can

**FIGURE 7 | Schematic diagram showing the effect of incorporation of new neurons within the neuronal circuitry.** The incorporation of thousands of granule neurons on a daily basis can result in the introduction of new connections between the neurons in the entorhinal cortex and the CA3 neurons. Following learning, the establishment of new neuronal connections can evoke unrelated semblances in response to a specific cue stimulus since the new neuronal connections necessitate the inclusion of the semblances evoked from the unrelated pathways to which it gets connected. This can reduce net memories. However, repetition of the associative learning, related learning, or the simultaneous activation of receptor pairs originally activated during the associative learning can lead to the formation of new inter-postsynaptic membrane hemi-fusions at higher neuronal orders, increasing net semblances for memories. The following are examples of some of the conditions that influence net semblance for memory. Note that the diagrammatic expression of the extrapolation of the semblance (see **Figure 2**) is simplified here. **(A1,A2)** Learning and memory retrieval in a circuitry before the incorporation of new neuron. **(A1)** Learning before the incorporation of new neurons in the hippocampal granule layer. L, Item to be learned. An inter-postsynaptic functional LINK is formed at location **R** when activity from the cue stimulus (one of the associatively learned stimuli) and the item to be learned reach the apposed postsynapses at this location. **(A2)** Memory retrieval before the incorporation of new neurons in the hippocampal granule layer. Here, a semblance is formed from the re-activation of an inter-postsynaptic functional LINK within the hippocampus. **(B1–B4)** Reduction in memory due to dilution of semblances following new neuron connections in the circuitry in the absence of repetition of learning or related learning or simultaneous activation of the sensory receptor pair involved in the associative learning. **(B1)** Associative learning before the addition of a new neuron. Note the presence of an extra-hippocampal inter-postsynaptic functional LINK at location **U** formed from associative learning between items Lp and Lq. **(B2)** Memory retrieval by the cue stimulus induces specific semblances at location **T** immediately following learning (before the incorporation of the new neuron in the circuitry). **(B3)** A new granule neuron is incorporated into the circuitry in the absence of repetition of learning or related learning or simultaneous activation of the receptor pairs. The labels are given in light colors to denote that there are no active stimuli at this time point. **(B4)** Memory retrieval following the incorporation of a new neuron by the cue stimulus induces unrelated semblances through the inter-postsynaptic functional LINK at location **U** that reduces memory. **(C1–C3)** Learning and memory retrieval after the incorporation of a new neuron. **(C1)** Associative learning after the incorporation of new neurons in the hippocampal granule layer. Note the formation of an inter-postsynaptic functional LINK within the hippocampus at location **V**. Also note that while the input pathway from the item to be learned passes through the hippocampal new granule neuron to reach the higher neuronal orders, the inputs from the cue stimulus bypasses the hippocampus to reach the higher neuronal orders and forms an inter-postsynaptic functional LINK at the location **W**. **(C2)** During memory retrieval, after the incorporation of new neurons in the circuitry, the net semblance is stronger than before the introduction of the new neuron. **(C3)** Memory retrieval after the removal of the hippocampus. Since semblance formation does not require the physical presence of a connection toward the postsynapses at which it is formed, semblance from the location **W** is evoked as **S1**. This is similar to the formation of the phantom limb phenomenon. Even though the number of inter-postsynaptic functional LINKs increases at higher neuronal orders, the nature of the semblances that are formed will be less specific when the hippocampus is removed. **(D1–D3)** Loss of memories following the removal of the hippocampus. **(D1)** Associative learning between the cue stimulus and the item to be learned. Neurons at the locations of convergence receive a different number of unrelated sensory inputs; for example input from **P**. **(D2)** Backward extrapolation from the postsynapse at which semblance is induced should include all the synaptic inputs through which activity had arrived via the neuron of its presynaptic terminal (see **Figure 2**), making the pathway from **P** a possibility for semblance formation. Semblance induced at location **Y** by the cue stimulus after the removal of hippocampus induces non-specific semblance **S2,** reducing memory. **(D3)** As the net non-specific semblances **(S2)** induced after the removal of hippocampus become more than the net specific semblances **(S1)**, the net semblance required for specific memory retrieval is reduced [Modified from Vadakkan (2011a)].

provide the vertical component and the spread of activity through the inter-postsynaptic functional LINKs can provide the horizontal component responsible for the oscillatory pattern of neuronal activations (**Figure 4**). As a result of these oscillations, a large number of neurons and their connected pathways remain activated during rest and during the operation of visual, sensory, motor, language, and cognitive functions explaining the findings in different imaging studies (Cordes et al., 2000; Beckmann et al., 2005; Fransson, 2005; Dosenbach et al., 2007; Seeley et al., 2007). Since the activation of these neurons during baseline oscillations doesn't spontaneously evoke motor activity, it suggests that they are under strong inhibitory or modulatory control. In this context, it is important to note that experiments to electrically stimulate the visual cortex for inducing visual percepts (pressure phosphene) succeeded only when artificial stimulation had evoked high-frequency gamma oscillations in the temporoparietal junction (Beauchamp et al., 2012). This implicates that the frequency of oscillations determines the intrinsic property of internal sensation induced by the system.

Oscillatory neuronal activity results in the sub-threshold summation of EPSPs at the axon hillocks of a very large number of neurons at the higher neuronal orders. Maintenance of these subthreshold activated neurons serves an important physiological role by providing "ready-to-fire" neurons. For example, a subthreshold-activated neuron just short of one EPSP to elicit an action potential is expected to become activated with the arrival of a single EPSP at one of its dendritic spines (postsynaptic terminals) in the dendritic tree through the re-activation of a functional LINK by activity arriving from the cue stimulus. Similarly, subthreshold motor neurons maintained by continuous oscillatory neuronal activity can have a significant role in central pattern generator functions at different locations of the nervous system and in the initiation and maintenance of locomotion.

## **WIRING DIAGRAM SUPPORTING INTERNAL SENSATIONS AND CONCURRENT MOTOR ACTIVITIES**

The formation of internal sensations depends on the nature of the semblions formed in response to the specificity of the cue stimulus. The lateral spread of activity through the inter-postsynaptic functional LINKs induces physiological oscillatory neuronal activity, which maintains large numbers of sub-threshold activated neurons at the higher orders. The latter are activated by the arrival of one or a few EPSPs and determine the neurons that are activated by the arrival of the cue stimulus (Kentros et al., 1998; Gelbard-Sagiv et al., 2008). Motor activity triggered by these neurons can occur concurrently with the formation of semblances during memory retrieval (**Figure 8**). By introducing inhibitory control over these neurons, their activation can be utilized for efficient physiological purposes. The nervous system that commands motor actions in response to a cue stimulus immediately receives feedback sensory inputs from the resulting motor activity. These arrive through visual inputs, the activation of vestibular labyrinths, and somato-sensory afferents (superficial sensations and proprioceptors from the joint capsules, tendon, and muscle position sense). The feedback inputs fine-regulate the operations of the system. Thus, the system is getting updated regarding each step of the motor action until the end of its execution.

**FIGURE 8 | Comparison between the known anatomical circuitry and the inter-postsynaptic functional LINK-mediated wiring. Left panel:** Synaptically connected conventional neuronal circuit diagram. There is one synaptic connection between neurons N1 and N2. The activation of neuron N1 induces an EPSP at postsynaptic membrane B. Provided neuron N2 is simultaneously receiving EPSPs from other neurons, the sum of which is just one EPSP short for spatial summation to trigger an action potential, then the EPSP arriving at postsynapse B from the activation of neuron N1 will lead to the firing of neuron N2. The contribution of the EPSP from the activation of Neuron N1 toward the temporal summation of EPSPs to elicit an action potential in neuron N2 should also be considered. Otherwise, a single EPSP or a train of few EPSPs reaching at postsynapse B alone may not induce an action potential of neuron N2. **Right panel:** Wiring diagram based on the present work. The activation of neuron N1 activates the inter-postsynaptic functional LINKs between the postsynapses in the islet of functional LINKs (**Figure 2**, **bottom panel**). The re-activation of postsynapse B that belongs to neuron N2 can provide EPSP and enable neuron N2 to fire an action potential similar to the threshold conditions explained for neuron N2 of the conventional wiring diagram (in the **left panel**). In addition, EPSPs spread to other hemi-fused postsynapses D, F, H, J, and L (depending on the extent of the spread through the islet) that can reach toward their neuronal somata. According to the supplementary rules, a total of six postsynapses are re-activated here, in comparison to only one by the canonical synaptic transmission (**left panel**). This increases the probability for firing of sub-threshold activated neurons in the next order by bringing them toward the threshold for activation. For example, neuron N6 continuously receives (*n* − 1) EPSPs, just short of one EPSP toward either spatial or temporal summation to elicit an action potential. Arrival of the *n*th EPSP from the islet of functionally LINKed postsynapses enables neuron N6 to cross the threshold to elicit an action potential (shown in red). If neuron N6 is a motor neuron, it can evoke motor activity concurrent with the re-activation of the functionally LINKed postsynapses B, D, F, H, J, and L. Activity through these LINKed postsynapses will also evoke semblions for the formation of internal sensations provided these are located at regions of oscillatory neuronal activity. All the neurons in red receive sufficient summated EPSPs and fire action potentials.

Since the nervous system has only a finite number of synapses at which inter-postsynaptic functional LINKs can be formed, continuous associative learning using an infinite number of sensory stimuli from the environment results in the sharing of a large number of functional LINKs. Considering that the functional LINKs are a part of the new wiring that occurs during the acquisition of information from the environment, its effects need to be incorporated into the circuitry. Even though it appears to occur only for the duration of time that these functional LINKs exist, a thorough examination can show that their effects on the circuitry at higher neuronal orders may last longer than their own existence.

The stability of the newly formed functional LINK-induced circuitry depends on the repetition of the associative learning that (a) maintains the required molecular changes and may eventually convert them to near-structural LINKs, and (b) incorporates more new neurons in the circuitry to expand the number of functional LINKs from which a large number of combinatorial semblances can be induced for retrieving different memories. Exposure to rare combinations of sensory stimuli will lead to the formation of specific new sets of functional LINKs at higher neuronal orders. The cognitive abilities that depend on the capacity to associatively learn specific patterns of physical properties of various items in the environment are likely to depend on the available unique combinations of postsynapses that can be functionally LINKed. Many functional LINKs are expected to be formed by simultaneous inputs from the environment that depends on the fixed physical properties of the items. It is possible that the functional LINKs get evolutionarily preserved as structural LINKs and are maintained through genetic mechanisms. In a novice nervous system, synaptic neurotransmission, and the spread of activity through innate structural LINKs between postsynapses will be responsible for innate behavioral responses (movement toward the source of food, sucking, and swallowing etc.) required for basic survival needs.

It is anticipated that successful stable memories for an item will have excess of net semblances beyond what is required, so that the system can afford to lose some of the functional LINKs without losing the required minimum net semblances for a specific memory (Vadakkan, 2010b). The brain circuitry is expected to quickly equilibrate with the changes including the effect of functional LINK re-activation, the non-linear integration of semblances, and the cellular changes that maintain stability. Given the constant formation of transient functional LINKs, their reversibility, and the activation of new sets of neurons, the nervous system circuitry is very dynamic at all times. At the instance of the arrival of new combinations of sensory (cue) stimuli, new combinations of internal sensations are created. The extent and complexity of the latter can

## **REFERENCES**


(2010). Long-chain omega-3 fatty acids for indicated prevention of psychotic disorders: a randomized, placebo-controlled trial. *Arch. Gen. Psychiatry* 67, 146–154.

Anastassiou, C. A., Perin, R., Markram, H., and Koch, C. (2011). Ephaptic possibly create complex higher brain functions; for example, emotions.

## **CONCLUSION**

The gold standard requirement for the operational mechanism of a complex system is an interrelated framework that can explain almost all its functions. The inter-postsynaptic functional LINKs can provide these requirements, which include the retrieval of memories at physiological time-scales, the consolidation of memories (Vadakkan, 2011a); the ability to support a framework of consciousness (Vadakkan, 2010a), the ease of learning related items; working memory resulting from semblances from all the functional LINKs immediately after learning; deterioration of the strength of memories immediately following learning as the hemi-fused membranes reverse back to their low-energy state independent membranes; the repetition of learning maintaining the hemi-fusions for long periods of time until certain transmembrane proteins are inserted across them enabling long-term memories; the role of new neurons in losing, improving, and expanding the locations of formation of memories; and the sharing of the mechanism of inter-postsynaptic membrane hemifusion in memory with LTP. Since the explanation for the large number of nervous system functions is possible from unitary functional units, the additional rules discussed here should be considered a testable biological mechanism of nervous system functions.

We have presented a supplementary circuit rule-set that can operate in unison with existing circuit rules and provides interconnected frameworks to explain various nervous system functions. It was imperative to make reasonable assumptions to view the formation of semblances as an emergent property of a system having oscillatory neuronal activity at certain neuronal orders. In such systems, the lateral entry of activity re-activating the inter-postsynaptic functional LINKs provides the horizontal component responsible for the neuronal oscillations along with the formation of basic units of internal sensations; namely, semblions. The concurrent formation of semblances and behavioral motor activity that depends on the frequency of neuronal oscillations provides a finely-regulated system. The present work highlights the importance of developing technologies to measure the summated EPSPs from the soma of the neurons, both at rest and during a cognitive operation, as an initial step followed by developing methods to trace the synapses from where they arrive. Verifying the wiring rules by examining the basic structural mechanisms of operations will help us understand additional information regarding the first-person perspective of different higher brain functions.

## **ACKNOWLEDGMENTS**

This work was supported by the Neurosearch Center, Toronto.

coupling of cortical neurons. *Nat. Neurosci.* 14, 217–223.

Arenkiel, B. R. (2011). Genetic approaches to reveal the connectivity of adult-born neurons. *Front. Neurosci.* 5:48. doi: 10.3389/fnins.2011.00048

Asakawa, K., Suster, M. L., Mizusawa, K., Nagayoshi, S., Kotani, T., Urasaki, A., et al. (2008). Genetic dissection of neural circuits by Tol2 transposon-mediated Gal4 gene and enhancer trapping in zebrafish. *Proc. Natl. Acad. Sci. U.S.A.* 105, 1255–1260.


and their synaptic relationships with mossy fiber boutons in the rat hippocampus. *J. Comp. Neurol.* 325, 169–182.


fluctuations: an fMRI investigation of the resting-state default mode of brain function hypothesis. *Hum. Brain Mapp.* 26, 15–29.


fusion and long-term potentiation. *Science* 279, 399–403.


intrinsic connectivity networks for salience processing and executive control. *J. Neurosci.* 27, 2349–2356.


virus type 1 mediates fusion through a hemifusion intermediate by sequential activity of glycoproteins D, H, L, and B. *Proc. Natl. Acad. Sci. U.S.A.* 104, 2903–2908.


postsynaptic functional LINKs, presumed biological parallels of K-lines proposed for building artificial intelligence. *Front. Neuroeng.* 4:8. doi: 10.3389/ fneng.2011.00008


Zador, A. M., Dubnau, J., Oyibo, H. K., Zhan, H., Cao, G., and Peikon, I. D. (2012). Sequencing the connectome. *PLoS Biol.* 10:e1001411. doi: 10.1371/journal. pbio.1001411

**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 01 November 2012; accepted: 16 April 2013; published online: 01 May 2013.*

*Citation: Vadakkan KI (2013) A supplementary circuit rule-set for the neuronal wiring. Front. Hum. Neurosci. 7:170. doi: 10.3389/fnhum.2013.00170*

*Copyright © 2013 Vadakkan. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.*