Probabilistic Motor Sequence Yields Greater Offline and Less Online Learning than Fixed Sequence

Introduction

In the laboratory, studies employing the serial reaction time (SRT) task (Nissen and Bullemer, 1987) have demonstrated that adults can learn a motor sequence quickly within a single training session (i.e., in 4 to 8 practice blocks; Nissen and Bullemer, 1987; Willingham et al., 1989; Robertson, 2007). This initial stage of motor sequence learning is referred to as fast learning that leads to the initial acquisition of sequences (Honda et al., 1998; Karni et al., 1998; Walker et al., 2002; Dayan and Cohen, 2011; Censor et al., 2012). Fast learning develops over the course of a single training session, where an individual practices a new motor sequence and demonstrates considerable performance improvement. It has been suggested that such improvement in the performance of motor sequences are driven by online learning (Bornstein and Daw, 2012, 2013; Verstynen et al., 2012), where performance progressively improves as the task is practiced. After the fast learning stage, performance is strengthened without further practice (i.e., offline learning) by an early offline boost (Hotermans et al., 2006; Schmitz et al., 2009) or memory consolidation (Robertson et al., 2004b, 2005; Brown and Robertson, 2007a; Nettersheim et al., 2015). To date, it is unclear whether offline learning drives the acquisition of motor sequence in the fast learning stage. The purpose of this study, therefore, is to examine whether fast learning of a motor sequence arises from offline learning. Furthermore, given that offline learning in the SRT task has been found to be associated with procedural memory (Robertson et al., 2004a; Brown and Robertson, 2007a,b), we further investigate whether a bias towards procedural or declarative memory in the SRT task modulates offline and online sequence learning.

Learning motor sequences in the SRT tasks typically involves both procedural and declarative memory (Willingham et al., 1989; Curran and Keele, 1993; Reber and Squire, 1994; Willingham and Goedert-Eschmann, 1999; Destrebecqz and Cleeremans, 2001; Brown and Robertson, 2007a; Robertson, 2007). In this task, participants press keys on the keyboard to respond to sequential visual stimuli that are presented in a pattern (e.g., a fixed order). Since participants are not informed of the presence of the sequence, learning in the SRT task requires procedural memory. However, participants may recognize the presence of the sequence after they perform the task and thus form a declarative memory of the sequence (Perruchet et al., 1997; Willingham and Goedert-Eschmann, 1999). This entanglement of procedural and declarative learning suggests the infeasibility of eliminating or isolating either of them from the SRT task. Nonetheless, manipulating the sequence type and the preliminary knowledge (PK) of the sequence can modulate procedural or declarative learning. Particularly, it has been shown that learning a probabilistic sequence favors more procedural memory compared to learning a fixed sequence (Jiménez et al., 1996; Song et al., 2007). In contrast, PK of the sequence facilitates declarative learning (Curran and Keele, 1993; Curran, 1997; Destrebecqz, 2004).

In this study, we bias the involvement of procedural/declarative memory by manipulating the sequence type and PK of the sequence in the SRT task to examine whether offline or online learning mediate the acquisition of motor sequences in the fast learning stage. Before the experiment, we informed half of the participants that the visual stimuli followed a specific pattern, but no further information was provided about the sequence. No information about the presence of a sequence was provided to the other participants. The participants were further divided into two groups. In one group, the visual stimuli followed a fixed sequence (i.e., 10 repetitions of a 12-trial sequence) while in the other group; the visual stimuli followed a probabilistic sequence that was generated by a first-order Markov process. We found that a motor sequence is learned quickly through concurrent online and offline learning. However, the involvement of procedural or declarative memory mediated the use of online and offline learning. Particularly, learning of a fixed sequence arose from greater online learning. In contrast, acquisition of a probabilistic sequence resulted from significant offline learning, regardless of PK. These results suggest that the involvement of procedural and declarative memory modulates how a motor sequence is learned in the fast learning stage.

Materials and Methods

This study was carried out in accordance with the recommendations and approval of the Institutional Review Board at the University of Maryland, College Park. All participants signed consent forms prior to their participation. Each participant received $10 after the completion of the experiment.

Participants

Forty-eight right-handed adults (24 males, see Table 1) were randomly assigned to one of four groups: fixed sequence with PK of the sequence (PK_Fixed; mean age: 21.8 ± 1.91), fixed sequence without PK of the sequence (NPK_Fixed; mean age: 21.5 ± 1.41), probabilistic sequence with PK of the sequence (PK_Prob; mean age: 21.2 ± 0.893), and probabilistic sequence without PK of the sequence (NPK_Prob; mean age; 21.3 ± 0.830). All participants completed a health questionnaire to exclude those with any neurological and motor impairments, the Edinburgh Handedness Inventory (Oldfield, 1971) to assess that participants were right-handed, and the Global Physical Activity Questionnaire (Armstrong and Bull, 2006) to insure that groups did not differ in their level of physical activity.

TABLE 1

Table 1. Participant demographic information.

Serial Reaction Time Task

Participants were seated in front of a computer monitor (19″) and keyboard. Participants placed the middle finger of their left hand on the keyboard’s “D” key, the index finger of their left hand on the “F” key, the index finger of their right hand on the “J” key, and the middle finger of their right hand on the “K” key (see Figure 1A). At the beginning of each trial, a mouse appeared in one of four squares on the screen and the participant pressed the key that corresponded to the location of the stimulus. After the participant pressed a key, the next stimulus appeared after an interval of 300 ms. No visual feedback was provided to participants and a wooden board blocked vision of their finger position. Participants were first randomly assigned to either the PK group or no preliminary knowledge (NPK) group and were further randomly assigned to either the fixed or probabilistic sequence. The probabilistic sequence was created based on a Markov chain transitional matrix with probabilities associated with each stimulus (Figures 1C,D). The probabilistic sequence was constrained such that the same stimuli were not repeated one after the other and that each stimulus appeared an equal number of times in each block.

FIGURE 1

Figure 1. (A) Experimental setup. At the beginning of each trial, a stimulus appeared in one of four squares on the screen and the participant pressed the key that corresponded to the location of the stimulus. Participants placed the middle finger of their left hand on the keyboard’s “D” key, the index finger of their left hand on the “F” key, the index finger of their right hand on the “J” key, and the middle finger of their right hand on the “K” key. (B) Experimental paradigm. Participants performed the learning blocks (blocks 1–4) with either the fixed or probabilistic sequence, followed by randomly ordered stimuli in block 5, and ended with the same sequence in block 6. All blocks consisted of 120 trials. Participants were given a mandatory 2 min break between each block. (C) Sequence Types. Participants were randomly assigned to the fixed sequence group or the probabilistic sequence group. The probabilistic sequence was created using the probabilities defined in the transitional matrix, T. (D) Example of how the probabilistic sequence was created using matrix T. If the current stimulus is 2, there is a probability of 0.6 that the next stimulus will be 1, a 0.3 probability that the next stimulus will be 3, and a 0.1 probability that the next stimulus will be 4.

There were a total of six blocks for all groups (see Figure 1B), each consisting of 120 trials. Prior to the first block, participants practiced a random sequence. These initial trials were included to ensure that participants were able to accurately associate each finger with a corresponding key before the experimental practice blocks commenced. That is, we observed that participants did not produce reaction times (RTs: amount of time taken to press the corresponding button after the stimulus was presented) that were slower than 2000 m because of incorrect key pressing. After the practice block, participants in the PK groups were informed that a sequence would be present in the subsequent blocks and that they should look for the sequence. No other information about the nature of the sequence was provided. The first four blocks (Blocks 1–4) were the learning blocks consisting of the 120-trial probabilistic sequence or the fixed sequence in which the sequence was repeated 10 times in each block. Block 5 consisted of 120 trials of stimuli occurring in a random order and block 6 consisted of the assigned probabilistic or fixed sequence (Figures 1C,D). Participants were given a 2 min mandatory break between each block. The participants’ RT was recorded for each trial.

All participants completed a posttest after the completion of the six blocks to determine the amount of declarative knowledge of the sequence. Participants were first asked to recall the sequence and attempted to write down the 12 items of the sequence and rated how confident they were that the sequence they wrote was correct. Participants were then asked to complete a recognition task. They were given eight chunks (i.e., four three-element and four four-element chunks where two of each were correct) and were asked to choose the chunks they thought were included in the sequence.

Data Analysis

The RTs were trimmed according to the individual participant’s mean and standard deviation. Within each block for an individual participant, any RT greater or less than 2.5 standard deviations was excluded from the analysis (Ratcliff, 1993; Whelan, 2008). Mean RTs were calculated for each block and were averaged across participants in each group. Learning was measured through a decrease in RT from block 1 to 4 (stimuli in assigned sequence) and an increase in RT from block 4 (stimuli in assigned sequence) to block 5 (stimuli in random order). Online learning was defined as the amount of learning within a block and was determined by performing a linear regression on the 120 RTs within a block. Offline learning was computed as the RT change after a short break without performing the task. Given that the fixed sequence consisted of 10 repetitions of a 12-item long sequence, the difference between mean RT of the last 12 taps in one block and that of the first 12 taps in the succeeding block was used to quantify offline learning. In addition, since participants typically acquire the sequence transitions of higher probabilities in probabilistic sequence learning (Hunt and Aslin, 2001; Howard et al., 2004; Bornstein and Daw, 2012), we expect that participants in the two probabilistic sequence groups would only learn sequential stimuli that were associated with transitional probabilities of 0.3 and 0.6 and fail to learn those associated with transitional probability of 0.1. Thus, we computed mean RT, offline- and online-learning of stimuli with transitional probabilities of 0.3 and 0.6 in the two probabilistic sequence groups.

A controversy regarding offline improvement in RT is whether the improvement results from reactive inhibition/fatigue (Rickard et al., 2008; Brawn et al., 2010) or it is driven by active learning mechanisms (i.e., offline learning; Eysenck and Frith, 1977; Robertson et al., 2004a). According to Eysenck and Frith (1977), in the case of reactive inhibition/fatigue-induced offline improvement, post-rest performance should return to the starting performance level before the rest or so called pre-rest performance, but without improvement over that level. In contrast, post-rest performance is superior to the pre-rest performance if offline improvement arises from offline learning. Given that RT increased (i.e., became slower) within blocks in some participants so that the mean RT of the last 12 taps may not reflect the pre-rest performance, we calculated corrected offline learning. Specifically, if RT increased (i.e., became slower) within the previous block, corrected offline learning was calculated by subtracting the amount of RT deterioration (i.e., negative online learning) within the previous block from the amount of offline learning so that the corrected offline learning reflects the difference between the pre-rest and post-rest performance. If RT improved (i.e., became faster) within the previous block, indicating no RT deterioration, corrected offline learning was the same as offline learning, computed as the difference between mean RT of the last 12 taps in the block and that of the first 12 taps in the succeeding block. We expect that all groups should exhibit the same amount of corrected offline learning (none), if offline improvement in RT observed in this study were caused by reactive inhibition or fatigue.

To measure the amount of declarative knowledge of the sequence, we calculated the recognition score as the number of correct chunks that participants chose in the recognition task. The recognition score was normalized by four as there were four correct chunks. To compare the recall score among participants, we calculated the number of three-element chunks that participants could recall. Given there were 12 three-element chunks in the fixed sequence, the number that a participant recalled was normalized by 12 to compute a percentage. To make the amount of declarative knowledge between probabilistic and fixed sequences comparable, the number of three-element chunks that participants could recall was also used in the two groups who performed the probabilistic sequence. Since participants only learned the stimulus transition with transitional probabilities of 0.3 and 0.6 (for details, see “Results” Section), there were 16 three-element chunks in the probabilistic sequence. Thus, the percent of recalled chunks was normalized by 16 in the two probabilistic sequence groups. Importantly, the chance level for guessing differed between the fixed and probabilistic sequence. Specifically, the chance level for a three-element chunk in the fixed sequence was 18.75% (i.e., given the first element, 75% chance for the second element and 25% chance for the third element) while it was 25% for a three-element chunk in the probabilistic sequence (i.e., given the first element, 50% chance for the second and third elements), we corrected the percentage of recalled chunks by the chance level specific to each sequence group.

Statistical Analysis

A three-way (block × knowledge × sequence) repeated measures analysis of variance (ANOVA) was used to compare differences in RT between the blocks and groups. Separate pairwise comparisons were conducted on the priori contrasts of interest (block 1 vs. block 4 and block 4 vs. block 5) to determine any significant differences between the sequenced blocks and the random block. A three-way (block × knowledge × probability) ANOVA was used to compare differences in RT of stimuli with different probabilities in the two probabilistic groups. All repeated measures ANOVAs were performed in SAS with the MIXED procedure. Thus, the co-variance matrix structures were determined by the Akaike information criterion (AIC). A two-way (knowledge × sequence) ANOVA was employed to examine the effects of PK and sequence type on online, offline learning, and corrected offline learning. A two-way (knowledge × sequence) ANOVA was employed to examine the effects of PK and sequence type on the recall score. Given the violation of the normality assumption, the effects of PK and sequence type on the recognition score was examined by the Scheirer-Ray-Hare test. Tukey-Kramer post hoc tests were used to decompose any significant effects. Student’s t-tests/Wilcoxon tests were used to examine whether recall/recognition scores were different from the corresponding chance level for each group. The statistical significance level was set as α = 0.05.

Results

Figure 2A shows the mean RT across the six blocks. The repeated measures ANOVA reveals a significant interaction between PK, sequence type, and block (F_(5,44) = 2.79, p < 0.05). Post hoc analyses with the Tukey-Kramer correction found that all four groups produced comparable RTs in all blocks (all p > 0.2). However, RT in two groups who performed the fixed sequences (i.e., PK_Fixed and NPK_Fixed) improved from blocks 1 to 4 and 6 (all p < 0.0001). In contrast, RT remained the same from block 1 to 4 in the other two probability sequence groups (i.e., PK_Prob and NPK_Prob, Figure 2B; all p > 0.1). Nevertheless, RT was faster in block 6 compared to block 1 in the NPK_Prob group (p < 0.01) and this improvement approached significance in the PK_Prob group (p = 0.09). In addition, when a random sequence was introduced in block 5, RT in the PK_Fixed and NPK_Fixed groups deteriorated (both p < 0.0001) while it remained the same between blocks 4 and 5 in the PK_Prob and NPK_Prob groups (both p = 1; Figure 2C).

FIGURE 2

Figure 2. (A) Mean RT and SE bars across the six blocks for all four groups. (B) Difference between the RT in block 1 and block 4 to assess whether sequence learning occurred. (C) Difference between block 5 and block 4 to assess whether RT increases in block 5 when a random sequence is presented. (D) Mean RT and SE bars across the six blocks for only the probabilistic sequence in which the three transitional probabilities (Pro 0.1, Pro 0.3, and Pro 0.6) have been extracted and plotted separately. (E) Difference between RT in block 1 and block 4. (F) In block 5 and block 4 separated for the three transitional probabilities in the probabilistic sequence. PK, preliminary knowledge; NPK, no preliminary knowledge; RT, reaction time; SE, standard error.

The inferior learning in the probabilistic sequence (as expressed in no change in RT from block 1 to 4 and between blocks 4 and 5) is consistent with the hypothesis that probabilistic sequences are harder to learn compared to fixed sequences (Schvaneveldt and Gomez, 1998). However, given our hypothesis that participants typically acquire the sequence transitions of higher probabilities (Hunt and Aslin, 2001; Howard et al., 2004; Bornstein and Daw, 2012), the marginal learning effect on the probabilistic sequence likely resulted from the difference in RT among stimuli with different transitional probabilities (Figure 1C). Thus, we compared RTs between these stimuli (Figure 2D) in the probabilistic sequence. A three-way (block × knowledge × probability) repeated measures ANOVA found that PK does not significantly affect RT and there was a significant interaction between block and probability (F_(10,220) = 17.07, p < 0.0001). Post hoc analyses with the Tukey-Kramer correction revealed that RTs of stimuli with a transitional probability of 0.1 were comparable to that of stimuli with transitional probability of 0.3, while RTs of stimuli with transitional probability of 0.3 were slower than that of probability of 0.6 (p < 0.01). However, as learning progressed, RTs of stimuli with a transitional probability of 0.1 remained the same. In contrast, RTs improved from blocks 1 to 4 in stimuli with higher transitional probabilities 0.3 (p < 0.01) and 0.6 (p < 0.0001), suggesting learning of these higher transitional probabilities (Figure 2E). Additionally, introduction of a random sequence in block 5 did not impair RT of stimuli with transitional probabilities of 0.1 and 0.3, but RTs of stimuli with a transitional probability of 0.6 deteriorated in block 5 (p < 0.0001; Figure 2F). These results confirm that the participants learned stimulus transitions with higher probabilities, specifically 0.6 and perhaps 0.3.

Since participants only learned higher transitional probabilities when stimuli followed a probabilistic pattern, we re-compared the learning effects among groups by using RT for stimuli with transitional probabilities 0.3 and 0.6 in PK_Prob and NPK_Prob groups. A repeated measures ANOVA revealed a significant interaction among the effects of block, PK, and sequence (F_(5,44) = 3.1, p < 0.05). Tukey-Kramer-corrected post hoc analyses suggest that all groups had comparable mean RTs across all blocks (Figure 3A). In addition, all groups demonstrated improved mean RT from block 1 to 4 (all p < 0.0001) and deteriorated mean RT from block 4 to 5 (all p < 0.005). However, contrast analyses showed that the PK_Fixed group had the greatest change in RT from block 1 to 4 compared to the NPK_Fixed (p < 0.05), PK_Prob (p < 0.0005), and NPK_Prob groups (p < 0.0005; Figure 3B), while the latter three groups exhibited the same change in RT. Similarly, the RT change from block 4 to 5 was greater in the PK_Fixed group compared to the other three groups (all p < 0.01) who had the same RT change (Figure 3C). These results suggest that the PK_Fixed group learned better than the other three groups.

FIGURE 3

Figure 3. Mean RT and SE bars to assess learning. Only the RT of stimuli with transitional probabilities of 0.3 and 0.6 were extracted and are shown for the probabilistic sequences. (A) Mean RT across the six blocks. (B) Difference between the RT in block 1 and block 4 to assess whether sequence learning occurred. (C) Difference between block 5 and block 4 to assess whether RT increases in block 5 when a random sequence is presented. PK, preliminary knowledge; NPK, no preliminary knowledge; RT, reaction time; SE, standard error.

Although participants learned either fixed or probabilistic sequences with or without PK of the sequence, learning across trials exhibited different patterns (Figure 4A). Specifically, learning of a fixed sequence exhibits decreased RT within blocks while learning of a probabilistic sequence exhibits reduced RT after rest without practice. A two-way (knowledge × sequence) ANOVA found a significant effect of sequence on offline learning (F_(1,44) = 8.84, p < 0.005). Particularly, the acquisition of the probabilistic sequence arises from greater offline learning compared to the acquisition of the fixed sequence (Figure 4B). Although sequence type was also found to significantly affect online learning (F_(1,44) = 18.72, p < 0.0001), it was shown that greater online learning was produced when a fixed sequence was learned (Figure 4B). Interestingly, when learning a probabilistic sequence, participants did not exhibit online learning. Instead, RT became slower within blocks. We further compared whether online or offline learning contributed more to the acquisition of a motor sequence. A two-way (knowledge × sequence) ANOVA on the RT difference between offline and online learning revealed a significant effect of sequence type (F_(1,44) = 15.27, p < 0.0005). Student’s t-tests found equal online and offline learning when a fixed sequence is performed (p = 0.59), while greater offline compared to online learning was found when a probabilistic sequence was performed (p < 0.0001).

FIGURE 4

Figure 4. (A) Mean RT of each 12 taps to reflect online and offline learning. (B) Comparison of online and offline learning between the groups. (C) Comparison of online and corrected offline learning between the groups. Error bars represent standard errors. PK, preliminary knowledge; NPK, no preliminary knowledge; RT, reaction time.

We also analyzed the corrected offline learning. The same results were found compared to the original offline learning data (Figure 4C). A two-way (knowledge × sequence) ANOVA found a significant effect of sequence (F_(1,44) = 4.99, p < 0.05). Specifically, there was greater corrected offline learning in PK_Prob and NPK_Prob groups compared to PK_Fixed and NPK_Fixed groups. These results suggest that offline learning rather than reactive inhibition/fatigue underlies the offline improvement in RT.

In the posttest, we found that the recognition score did not differ from chance (i.e., 50%) in all four groups and there were no effects of sequence type and PK on the scores. Figure 5A shows the percentage of recalled three-element chunks. It is clear that participants in the fixed sequence groups had higher than chance recall, while recall was at chance in the two probabilistic sequence groups. The corrected percentage according to the chance level was shown in Figure 5B. A two-way ANOVA found a significant effect of sequence type (F_(1,44) = 6.75, p < 0.01). Specifically, recall of the fixed sequence was superior compared to that of probabilistic sequence. In addition, using Student’s t-tests with an adjusted p level of $\frac{\alpha }{4}$ = 0.0125 to control the familywise error rate for the four simultaneous t-tests, the recall in the PK_Fixed was significantly higher than chance level (p < 0.0001) The recall in the NPK_Fixed did not differ from chance (but approached significance, p = 0.0146), In contrast, recall in the two groups that performed the probabilistic sequence (i.e., PK_Prob and NPK_Prob) was not significantly different from the chance level (both p > 0.2).

FIGURE 5

Figure 5. (A) Percentage of three-element chunks recalled in the posttest. (B) Corrected percentage according to chance level. Error bars represent standard errors. PK, preliminary knowledge; NPK, no preliminary knowledge.

Discussion

In this study, we demonstrated that both fixed and probabilistic motor sequences can be learned quickly (i.e., in one training session). Further, this initial acquisition of a fixed sequence in the fast learning stage arises from both online and offline learning, while acquisition of a probabilistic sequence is driven predominantly by offline learning. Given that learning a probabilistic or fixed sequence requires greater procedural or declarative memory, respectively, our results suggest that a bias toward procedural or declarative memory modulates how a motor sequence is learned in the fast learning stage.

Offline learning, as a salient feature underlying motor sequence learning (Robertson et al., 2004a), can boost the memory of a newly acquired sequence 5–30 min after the initial acquisition (Albouy et al., 2006; Hotermans et al., 2006, 2008; Schmitz et al., 2009; Nettersheim et al., 2015) or consolidate the memory a few hours later without sleep (Robertson et al., 2004a; Brown and Robertson, 2007a,b) or after sleep (Walker et al., 2002; Robertson et al., 2004b; Censor et al., 2012; Nettersheim et al., 2015). Thus, offline learning has been widely considered to occur only after the initial acquisition of sequences that develops over the course of a single training session, referred to as fast learning (Honda et al., 1998; Karni et al., 1998; Walker et al., 2002; Dayan and Cohen, 2011; Censor et al., 2012). Unlike the widely-found offline learning that occurs following the fast learning stage, we observed offline learning that drives the fast acquisition of sequences within a first single training session. This result suggests that in addition to online learning (Cleeremans and McClelland, 1991; Bornstein and Daw, 2012), offline learning also contributes to rapid improvements in performance that allow sequences to be learned quickly in a single training session.

The concurrent effect of online and offline learning could be modulated by the involvement of declarative and procedural memory. It is widely accepted that both memory systems cooperate and compete during motor sequence learning (Meulemans et al., 1998; Brown and Robertson, 2007b). Remarkably, the presence of declarative memory inhibits offline learning of procedural memory and thus disruption of declarative memory induces offline improvement in procedural skills 4 h after the initial acquisition (Brown and Robertson, 2007a). In our study, similar effects of declarative and procedural memory were observed on offline learning in the fast learning stage. The recognition and recall tests were used to measure the engagement of declarative and procedural memory in the SRT task. Although the recognition test shows no differences in the amount of declarative knowledge acquired by participants regardless of the sequence type and PK (see details below), the recall scores reveal that, participants acquired less declarative knowledge of the probabilistic sequence. Notably, participants exhibited greater offline learning when performing probabilistic sequences, suggesting that the offline learning in the fast learning stage was strengthened when greater procedural memory and less declarative memory were required to learn the motor sequences. On the other hand, when greater declarative memory was involved in learning fixed sequences, as indicated by higher recall scores, reduced offline and greater online learning were observed. This inverse relationship between online and offline learning confirms the inhibition effect of declarative memory on offline learning. More importantly, our finding extends our understanding of the competition between multiple memory systems. That is, unlike previous studies that demonstrated this competition after skills are acquired (Poldrack et al., 2001; Foerde et al., 2006; Brown and Robertson, 2007b), we demonstrated that the competition begins as soon as learning starts and that declarative and procedural memory may be identified by their distinct behavioral expressions.

The offline learning observed within a single training session (i.e., the fast learning stage) is associated with procedural memory as is offline learning that takes place hours after the initial acquisition and is responsible for memory consolidation. However, it remains unclear whether this offline learning that allows fast initial acquisition of a motor sequence is related to offline learning that consolidates the memory of a newly acquired sequence. It is possible that offline learning that drives the fast acquisition is a precursor of the later occurring memory consolidation, or they may be the same process. To elucidate their relationship, further systematic investigations are needed.

A debate within the offline learning literature is whether offline improvement in performance after rest, referred to as reminiscence (Eysenck and Frith, 1977), results from fatigue or reactive inhibition (Rickard et al., 2008; Brawn et al., 2010) or an active learning mechanism (Eysenck and Frith, 1977; Robertson et al., 2004a). It has been suggested that offline learning and reactive inhibition/fatigue are usually combined to lead to reminiscence (Eysenck, 1965), thus making it difficult to determine if reactive inhibition/fatigue is a potential cause of reminiscence. However, observations from our data favor offline learning to reactive inhibition/fatigue as the primary mechanism underlying offline improvement in RT or reminiscence observed in the SRT task. Specifically, with the same amount of practice, only participants who performed the probabilistic sequence slowed down their RT, while such “fatigue” was not observed when participants performed a fixed sequence. In addition, if fatigue appeared as soon as participants in the probabilistic sequence groups started to perform the task, it would be unlikely that their learning would arise quickly (i.e., over four learning blocks) and to a comparable level as the participants in the fixed sequence groups who did not exhibit fatigue. Moreover, according to Eysenck and Frith (1977), reminiscence is task-specific. For example, reminiscence that results from reactive inhibition or fatigue usually occurs in a task that does not involve learning, where performance on the task is already perfect when an individual starts to perform the task. In contrast, reminiscence that arises from offline learning usually takes place in a learning task. Obviously, the SRT task involves sequence learning and our data demonstrated that participants learned the sequence. Further evidence supporting offline learning rather than reactive inhibition or fatigue comes from the observation on corrected offline learning. In the probabilistic sequence groups, performance after the short break is superior to the best performance level before the break. Therefore, without fully excluding the effect of reactive inhibition/fatigue, our results favor the statement that the offline improvement in RT is driven by offline learning rather than reactive inhibition or fatigue. Meanwhile, we suggest that it is necessary to systematically examine the reactive inhibition or fatigue effects in future sequence learning studies.

Although it appears that offline learning rather than reactive inhibition or fatigue is the primary mechanism underlying the offline improvement in RT, the cause of increased RT when learning a probabilistic sequence is unclear. One likely reason is the interference of stimuli transitions with a probability of 0.1. It has been found that adults learned a sequence by iteratively updating the internal model of the motor sequence (Cleeremans and McClelland, 1991; Bornstein and Daw, 2012, 2013; Verstynen et al., 2012) and our data provide consistent evidence that participants acquired the stimulus transitions with probabilities of 0.3 and 0.6. However, the introduction of stimulus transition governed by a probability of 0.1 may mislead the updating of the internal model (i.e., transitional probability matrix) and thus impair RT when the probabilistic sequence was performed.

In addition to the primary findings on online and offline learning, our results provide insights into the learning of probabilistic sequences. Sequence structure plays a critical role in motor sequence learning (Curran and Keele, 1993; Jiménez et al., 1996; Bennett et al., 2007; Song et al., 2007). To date, a variety of probabilistic sequences have been used in the SRT task, but only a few studies have employed probabilistic sequences that represent the stochastically related events of daily life, such as sequences produced by a finite state grammar (Jiménez et al., 1996) or a Markov chain. We found that participants acquired stimulus transitions with higher probabilities of 0.3 and 0.6 and the learning of these higher stimulus transitions was comparable to that of the fixed sequence. Moreover, the facilitating effect of PK of a sequence depends on the sequence structure, which is consistent with previous studies (Jiménez et al., 1996; Stefaniak et al., 2008). Specifically, PK only facilitates the learning of a simple sequence, such a fixed sequence (Curran and Keele, 1993; Frensch and Miner, 1994; Curran, 1997; Destrebecqz, 2004; Stefaniak et al., 2008) and not a sequence with a complex structure.

Finally, one caveat worthy of further study is the measurement of the amount of declarative knowledge. Both recognition and recall tests are most widely used to examine procedural learning in the SRT task (Shanks and Johnstone, 1999; Wilkinson and Shanks, 2004; Destrebecqz and Peigneux, 2005). In particular, these tests examine whether participants can explicitly recollect the acquired sequence knowledge. However, results from the recognition tests are equivocal in the literature (Perruchet and Amorim, 1992; Willingham et al., 1993; Reed and Johnson, 1994; Shanks and Johnstone, 1999). Similarly in our study, unlike the recall tests demonstrating the common finding that probabilistic sequence learning favors more procedural memory (Jiménez et al., 1996; Song et al., 2007), the recognition tests reveals no difference in the amount of acquired declarative knowledge despite the sequence type and PK. In addition, the recognition scores in all four groups were not greater than chance. Given that in the recognition test, participants were presented with sequence segments and were asked to determine whether these segments are from the sequence they learned or a new sequence they did not see in the SRT task, it is hard to know whether the chance-level score was due to the participant’s inability to explicitly recollect sequence knowledge or that the participant did not learn some segments of the sequence. These two possibilities that may simultaneously account for the chance-level recognition must be addressed by other tests in future studies. Moreover, in our study, only four correct sequence segments were given to participants, while there were more than 10 segments within the learned sequence, the chance-level recognition score was caused possibly because some participants may learn segments other than the four displayed in the recognition test.

In summary, we found that concurrent online and offline learning allows motor sequences to be acquired quickly in the fast learning stage and can be identified by their manifestations in the progressive changes in RT. Remarkably, online and offline learning can be mediated by the declarative and procedural memory that are required to learn motor sequences. In addition, the modulation of online and offline learning may reflect the competition between both memory systems during motor sequence learning that begins in the fast learning stage. How the offline learning that drives the initial acquisition of sequences is related to the offline learning that is responsible for memory consolidation occurring hours after the initial acquisition remains to be investigated.

Author Contributions

YD, SP, IS, and JEC designed the experiment. IS performed the experiment. YD and SP analyzed the data and prepared all tables and figures. YD, SP, and JEC wrote the manuscript. All authors reviewed the manuscript.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Albouy, G., Ruby, P., Phillips, C., Luxen, A., Peigneux, P., and Maquet, P. (2006). Implicit oculomotor sequence learning in humans: time course of offline processing. Brain Res. 1090, 163–171. doi: 10.1016/j.brainres.2006.03.076

PubMed Abstract | CrossRef Full Text | Google Scholar

Armstrong, T., and Bull, F. (2006). Development of the world health organization global physical activity questionnaire (GPAQ). J. Public Health 14, 66–70. doi: 10.1007/s10389-006-0024-x

CrossRef Full Text | Google Scholar

Bennett, I. J., Howard, J. H., and Howard, D. V. (2007). Age-related differences in implicit learning of subtle third-order sequential structure. J. Gerontol. B Psychol. Sci. Soc. Sci. 62, P98–P103. doi: 10.1093/geronb/62.2.p98

PubMed Abstract | CrossRef Full Text | Google Scholar

Bornstein, A. M., and Daw, N. D. (2012). Dissociating hippocampal and striatal contributions to sequential prediction learning. Eur. J. Neurosci. 35, 1011–1023. doi: 10.1111/j.1460-9568.2011.07920.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Bornstein, A. M., and Daw, N. D. (2013). Cortical and hippocampal correlates of deliberation during model-based decisions for rewards in humans. PLoS Comput. Biol. 9:e1003387. doi: 10.1371/journal.pcbi.1003387

PubMed Abstract | CrossRef Full Text | Google Scholar

Brawn, T. P., Fenn, K. M., Nusbaum, H. C., and Margoliash, D. (2010). Consolidating the effects of waking and sleep on motor-sequence learning. J. Neurosci. 30, 13977–13982. doi: 10.1523/jneurosci.3295-10.2010

PubMed Abstract | CrossRef Full Text | Google Scholar

Brown, R. M., and Robertson, E. M. (2007a). Inducing motor skill improvements with a declarative task. Nat. Neurosci. 10, 148–149. doi: 10.1038/nn1836

PubMed Abstract | CrossRef Full Text | Google Scholar

Brown, R. M., and Robertson, E. M. (2007b). Off-line processing: reciprocal interactions between declarative and procedural memories. J. Neurosci. 27, 10468–10475. doi: 10.1523/jneurosci.2799-07.2007

PubMed Abstract | CrossRef Full Text | Google Scholar

Censor, N., Sagi, D., and Cohen, L. G. (2012). Common mechanisms of human perceptual and motor learning. Nat. Rev. Neurosci. 13, 658–664. doi: 10.1038/nrn3315

PubMed Abstract | CrossRef Full Text | Google Scholar

Cleeremans, A., and McClelland, J. L. (1991). Learning the structure of event sequences. J. Exp. Psychol. Gen. 120, 235–253. doi: 10.1037/0096-3445.120.3.235

PubMed Abstract | CrossRef Full Text | Google Scholar

Curran, T. (1997). Effects of aging on implicit sequence learning: accounting for sequence structure and explicit knowledge. Psychol. Res. 60, 24–41. doi: 10.1007/bf00419678

PubMed Abstract | CrossRef Full Text | Google Scholar

Curran, T., and Keele, S. W. (1993). Attentional and nonattentional forms of sequence learning. J. Exp. Psychol. Learn. Mem. Cogn. 19, 189–202. doi: 10.1037/0278-7393.19.1.189

CrossRef Full Text | Google Scholar

Dayan, E., and Cohen, L. G. (2011). Neuroplasticity subserving motor skill learning. Neuron 72, 443–454. doi: 10.1016/j.neuron.2011.10.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Destrebecqz, A. (2004). The effect of explicit knowledge on sequence learning: a graded account. Psychol. Belg. 44, 217–247. doi: 10.5334/pb-44-4-217

CrossRef Full Text | Google Scholar

Destrebecqz, A., and Cleeremans, A. (2001). Can sequence learning be implicit? New evidence with the process dissociation procedure. Psychon. Bull. Rev. 8, 343–350. doi: 10.3758/bf03196171

PubMed Abstract | CrossRef Full Text | Google Scholar

Destrebecqz, A., and Peigneux, P. (2005). Methods for studying unconscious learning. Prog. Brain Res. 150, 69–80. doi: 10.1016/s0079-6123(05)50006-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Eysenck, H. J. (1965). A three-factor theory of reminiscence. Br. J. Psychol. 56, 163–182. doi: 10.1111/j.2044-8295.1965.tb00956.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Eysenck, H. J., and Frith, C. D. (1977). Reminiscence, Motivation, and Personality. London: Plenum.

Google Scholar

Foerde, K., Knowlton, B. J., and Poldrack, R. A. (2006). Modulation of competing memory systems by distraction. Proc. Natl. Acad. Sci. U S A 103, 11778–11783. doi: 10.1073/pnas.0602659103

PubMed Abstract | CrossRef Full Text | Google Scholar

Frensch, P. A., and Miner, C. S. (1994). Effects of presentation rate and individual differences in short-term memory capacity on an indirect measure of serial learning. Mem. Cognit. 22, 95–110. doi: 10.3758/BF03202765

PubMed Abstract | CrossRef Full Text | Google Scholar

Honda, M., Deiber, M. P., Ibáñez, V., Pascual-Leone, A., Zhuang, P., and Hallett, M. (1998). Dynamic cortical involvement in implicit and explicit motor sequence learning. A PET study. Brain 121, 2159–2173. doi: 10.1093/brain/121.11.2159

PubMed Abstract | CrossRef Full Text | Google Scholar

Hotermans, C., Peigneux, P., de Noordhout, A. M., Moonen, G., and Maquet, P. (2008). Repetitive transcranial magnetic stimulation over the primary motor cortex disrupts early boost but not delayed gains in performance in motor sequence learning. Eur. J. Neurosci. 28, 1216–1221. doi: 10.1111/j.1460-9568.2008.06421.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Hotermans, C., Peigneux, P., Maertens de Noordhout, A., Moonen, G., and Maquet, P. (2006). Early boost and slow consolidation in motor skill learning. Learn. Mem. 13, 580–583. doi: 10.1101/lm.239406

PubMed Abstract | CrossRef Full Text | Google Scholar

Howard, J. H., Howard, D. V., Dennis, N. A., Yankovich, H., and Vaidya, C. J. (2004). Implicit spatial contextual learning in healthy aging. Neuropsychology 18, 124–134. doi: 10.1037/0894-4105.18.1.124

PubMed Abstract | CrossRef Full Text | Google Scholar

Hunt, R. H., and Aslin, R. N. (2001). Statistical learning in a serial reaction time task: access to separable statistical cues by individual learners. J. Exp. Psychol. Gen. 130, 658–680. doi: 10.1037/0096-3445.130.4.658

PubMed Abstract | CrossRef Full Text | Google Scholar

Jiménez, L., Méndez, C., and Cleeremans, A. (1996). Comparing direct and indirect measures of sequence learning. J. Exp. Psychol. Learn. Mem. Cogn. 22, 948–969. doi: 10.1037/0278-7393.22.4.948

CrossRef Full Text | Google Scholar

Karni, A., Meyer, G., Rey-Hipolito, C., Jezzard, P., Adams, M. M., Turner, R., et al. (1998). The acquisition of skilled motor performance: fast and slow experience-driven changes in primary motor cortex. Proc. Natl. Acad. Sci. U S A 95, 861–868. doi: 10.1073/pnas.95.3.861

PubMed Abstract | CrossRef Full Text | Google Scholar

Meulemans, T., Van der Linden, M., and Perruchet, P. (1998). Implicit sequence learning in children. J. Exp. Child Psychol. 69, 199–221. doi: 10.1006/jecp.1998.2442

PubMed Abstract | CrossRef Full Text | Google Scholar

Nettersheim, A., Hallschmid, M., Born, J., and Diekelmann, S. (2015). The role of sleep in motor sequence consolidation: stabilization rather than enhancement. J. Neurosci. 35, 6696–6702. doi: 10.1523/JNEUROSCI.1236-14.2015

PubMed Abstract | CrossRef Full Text | Google Scholar

Nissen, M. J., and Bullemer, P. (1987). Attentional requirements of learning: evidence from performance measures. Cogn. Psychol. 19, 1–32. doi: 10.1016/0010-0285(87)90002-8

CrossRef Full Text | Google Scholar

Oldfield, R. C. (1971). The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9, 97–113. doi: 10.1016/0028-3932(71)90067-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Perruchet, P., and Amorim, M. A. (1992). Conscious knowledge and changes in performance in sequence learning–evidence against dissociation. J. Exp. Psychol. Learn. Mem. Cogn. 18, 785–800. doi: 10.1037/0278-7393.18.4.785

PubMed Abstract | CrossRef Full Text | Google Scholar

Perruchet, P., Bigand, E., and BenoitGouin, F. (1997). The emergence of explicit knowledge during the early phase of learning in sequential reaction time tasks. Psychol. Res. 60, 4–13. doi: 10.1007/bf00419676

CrossRef Full Text | Google Scholar

Poldrack, R. A., Clark, J., Paré-Blagoev, E. J., Shohamy, D., Moyano, J. C., Myers, C., et al. (2001). Interactive memory systems in the human brain. Nature 414, 546–550. doi: 10.1038/35107080

PubMed Abstract | CrossRef Full Text | Google Scholar

Ratcliff, R. (1993). Methods for dealing with reaction time outliers. Psychol. Bull. 114, 510–532. doi: 10.1037/0033-2909.114.3.510

PubMed Abstract | CrossRef Full Text | Google Scholar

Reber, P. J., and Squire, L. R. (1994). Parallel brain systems for learning with and without awareness. Learn. Mem. 1, 217–229.

PubMed Abstract | Google Scholar

Reed, J., and Johnson, P. (1994). Assessing implicit learning with indirect tests: determining what is learned about sequence structure. J. Exp. Psychol. Learn. Mem. Cogn. 20, 585–594. doi: 10.1037/0278-7393.20.3.585

CrossRef Full Text | Google Scholar

Rickard, T. C., Cai, D. J., Rieth, C. A., Jones, J., and Ard, M. C. (2008). Sleep does not enhance motor sequence learning. J. Exp. Psychol. Learn. Mem. Cogn. 34, 834–842. doi: 10.1037/0278-7393.34.4.834

PubMed Abstract | CrossRef Full Text | Google Scholar

Robertson, E. M. (2007). The serial reaction time task: implicit motor skill learning? J. Neurosci. 27, 10073–10075. doi: 10.1523/jneurosci.2747-07.2007

PubMed Abstract | CrossRef Full Text | Google Scholar

Robertson, E. M., Pascual-Leone, A., and Miall, R. C. (2004a). Current concepts in procedural consolidation. Nat. Rev. Neurosci. 5, 576–582. doi: 10.1038/nrn1426

PubMed Abstract | CrossRef Full Text | Google Scholar

Robertson, E. M., Pascual-Leone, A., and Press, D. Z. (2004b). Awareness modifies the skill-learning benefits of sleep. Curr. Biol. 14, 208–212. doi: 10.1016/j.cub.2004.01.027

PubMed Abstract | CrossRef Full Text | Google Scholar

Robertson, E. M., Press, D. Z., and Pascual-Leone, A. (2005). Off-line learning and the primary motor cortex. J. Neurosci. 25, 6372–6378. doi: 10.1523/jneurosci.1851-05.2005

PubMed Abstract | CrossRef Full Text | Google Scholar

Schmitz, R., Schabus, M., Perrin, F., Luxen, A., Maquet, P., and Peigneux, P. (2009). Recurrent boosting effects of short inactivity delays on performance: an ERPs study. BMC Res. Notes 2:170. doi: 10.1186/1756-0500-2-170

PubMed Abstract | CrossRef Full Text | Google Scholar

Schvaneveldt, R. W., and Gomez, R. L. (1998). Attention and probabilistic sequence learning. Psychol. Res. 61, 175–190. doi: 10.1007/s004260050023

CrossRef Full Text | Google Scholar

Shanks, D. R., and Johnstone, T. (1999). Evaluating the relationship between explicit and implicit knowledge in a sequential reaction time task. J. Exp. Psychol. Learn. Mem. Cogn. 25, 1435–1451. doi: 10.1037/0278-7393.25.6.1435

PubMed Abstract | CrossRef Full Text | Google Scholar

Song, S. B., Howard, J. H., and Howard, D. V. (2007). Implicit probabilistic sequence learning is independent of explicit awareness. Learn. Mem. 14, 167–176. doi: 10.1101/lm.437407

PubMed Abstract | CrossRef Full Text | Google Scholar

Stefaniak, N., Willems, S., Adam, S., and Meulemans, T. (2008). What is the impact of the explicit knowledge of sequence regularities on both deterministic and probabilistic serial reaction time task performance? Mem. Cognit. 36, 1283–1298. doi: 10.3758/MC.36.7.1283

PubMed Abstract | CrossRef Full Text | Google Scholar

Verstynen, T., Phillips, J., Braun, E., Workman, B., Schunn, C., and Schneider, W. (2012). Dynamic sensorimotor planning during long-term sequence learning: the role of variability, response chunking and planning errors. PLoS One 7:e47336. doi: 10.1371/journal.pone.0047336

PubMed Abstract | CrossRef Full Text | Google Scholar

Walker, M. P., Brakefield, T., Morgan, A., Hobson, J. A., and Stickgold, R. (2002). Practice with sleep makes perfect: sleep-dependent motor skill learning. Neuron 35, 205–211. doi: 10.1016/S0896-6273(02)00746-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Whelan, R. (2008). Effective analysis of reaction time data. Psychol. Rec. 58, 475–482.

Google Scholar

Wilkinson, L., and Shanks, D. R. (2004). Intentional control and implicit sequence learning. J. Exp. Psychol. Learn. Mem. Cogn. 30, 354–369. doi: 10.1037/0278-7393.30.2.354

PubMed Abstract | CrossRef Full Text | Google Scholar

Willingham, D. B., and Goedert-Eschmann, K. (1999). The relation between implicit and explicit learning: evidence for parallel development. Psychol. Sci. 10, 531–534. doi: 10.1111/1467-9280.00201

CrossRef Full Text | Google Scholar

Willingham, D. B., Greeley, T., and Bardone, A. M. (1993). Dissociation in a serial response time task using a recognition measure: comment on Perruchet and Amorim (1992). J. Exp. Psychol. Learn. Mem. Cogn. 19, 1424–1430. doi: 10.1037/0278-7393.19.6.1424

CrossRef Full Text | Google Scholar

Willingham, D. B., Nissen, M. J., and Bullemer, P. (1989). On the development of procedural knowledge. J. Exp. Psychol. Learn. Mem. Cogn. 15, 1047–1060. doi: 10.1037/0278-7393.15.6.1047

PubMed Abstract | CrossRef Full Text | Google Scholar