LearningRx Cognitive Training for Children and Adolescents Ages 5–18: Effects on Academic Skills, Behavior, and Cognition

Introduction

Among American students with learning disabilities in 2014, nearly 70% had failed at least one course and 50% had received a disciplinary suspension or expulsion from school (Cortiella and Horowitz, 2014). According to the National Assessment of Educational Progress (U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, 2015), 67% of fourth grade clients with learning disabilities and 63% of eighth grade clients with learning disabilities scored below basic in reading. To facilitate learning for academically challenged clients, many school districts are seeking research-based interventions to use in the classroom or on an individual basis (Fuchs et al., 2008). In addition to traditional academic intervention methods such as tutoring and supplemental services offered as part of a response-to-intervention program, current research also suggests that cognitive skills training may be an effective way to help improve academic performance (Holmes and Gathercole, 2014).

Cognitive training is a rapidly growing intervention for improving cognitive function for many populations including the elderly, survivors of traumatic brain injury, and clients with learning disabilities. However, the results of extant research on cognitive training for children are inconsistent at best. There is also a glaring gap in the literature on how cognitive training transfers to real-life functioning, and evidence of parent-reported changes in behavior following cognitive training is sparse. In response to the question of whether outcomes following cognitive training include functional benefits, the current study’s primary aim was to examine the real-life benefits of clinician-delivered cognitive training on academic skills and oppositional behavior among school-age clients. A secondary aim of the study was to examine objectively measured cognitive outcomes for children who completed cognitive training.

Cognitive Training Effects on Cognition

In the last decade, a revolution in learning theory and educational psychology has brought dramatic changes in our current understanding of cognitive development processes. This new direction for learning theory and cognitive education was influenced in part by Feuerstein’s mediated learning experience approach to education (Kozulin and Presseisen, 1995). Feuerstein’s theory of cognitive structural modifiability concentrates on the experience of mediated learning, which is the intentional process of focusing a learner’s interaction with the world (Feuerstein et al., 2010). Mediated learning experiences are critical to the development of the unique human conditions of modifiability, or the capacity to benefit from exposure to stimuli in a more generalized way (Feuerstein, 1990).

Research also indicates that brain training can change cognitive functioning based on concepts of neural plasticity and environmental stimulation. For example, Willis et al. (2006) cites evidence that sustained engagement in cognitively stimulating activities impacts neural structure and given appropriate practice, humans consequently improve on essentially every task performed. Nouchi et al. (2013) suggest that adaptive cognitive training techniques in particular reveal far transfer effects. Furthermore, new research on brain training programs has been shown to manifest both near transfer and far transfer of training-specified tasks as well as other non-trained cognitive functions (Smith et al., 2015). Encouraging outcomes following cognitive training include significant post-training improvement in working memory (Dahlin, 2013), processing speed (Gibson et al., 2015), and general intelligence (Jausovec and Jausovec, 2012; Carpenter et al., 2016). Evidence of improvements in math and reading skills were found in a study using BrainWare Safari with children with learning disabilities (Avtzon, 2012). Neural activity changes following cognitive training have also been documented (Westerberg and Klingberg, 2007). However, there are many studies that do not support the use of cognitive training for remediating cognitive deficits. For example, a meta-analysis of 13 cognitive training programs indicated consistently low-effect sizes for the training effect on executive functions (Karch et al., 2013). Further, Redick et al. (2013) found no evidence of intelligence improvement following working memory training—a finding consistent with the oft-cited analysis of Melby-Lervag and Hulme (2016) whose review of working memory training studies produced “no convincing evidence” that training leads to cognitive benefits.

Cognitive Training Effects on Academic and Learning Skills

Although prior cognitive training studies lend insight on neural plasticity, further research is needed to investigate the transfer of cognitive change to academic success. Despite the controversy surrounding the efficacy of cognitive training, brain training interventions have gained appeal in the past decade for both educators and clinicians as an alternative and supplement to traditional skills-based approaches for learning (Kearns and Fuchs, 2013). Fiorello and Primerano (2005) state that underlying cognitive abilities are associated with academic achievement in school, and contend the way a client processes, stores, retrieves, and analyzes information influences how that client will perform in school. Thus, certain specific abilities may be important for understanding the development of particular skills above and beyond the understanding gained from general cognitive and achievement clusters. Several studies on cognitive training support this notion. For example, Shiran and Breznitz (2011) reported improvements in decoding, reading rate, and comprehension for both dyslexic and skilled readers following working memory training—suggesting a relationship between working memory capacity and reading ability. Dunning and Holmes (2014) also found improvements in 6-year-olds’ reading and math grades following 23 sessions of computerized executive function training. Titz and Karbach (2014) assert that interventional training in working memory and executive function has been shown to amplify academic achievement.

However, there is contrasting evidence as well. In a randomized controlled trial of Fast ForWord—a reading intervention that targets auditory and visual discrimination, attention, and working memory—researchers found little differences among the three intervention groups and the control group (Gillam et al., 2008). A study using the Cognitive Training for Children inductive reasoning intervention also failed to show significant differences between treatment and controls on measures of academic achievement (Barkl et al., 2012). Similarly, an adaptive working memory training program also failed to affect treatment and controls differently on measures of single-word reading, spelling skills, or mathematics tests (Henry et al., 2014). Indeed, the results are mixed.

Cognitive Training Effects on Behavior

Although there is some evidence that cognitive skills training improves standardized test scores and academic achievement, behavioral changes are important to address as well. Rabipour and Raz (2012) suggest that cognitive training can produce changes measured at the behavioral as well as the neuroanatomical and functional levels. Early research in this area reveals that computer-based cognitive rehabilitation for patients with addiction-related neurological deficits improved both cognitive functioning and socially appropriate behaviors (Fals-Stewart and Lucente, 1994). In other words, behavioral improvements coincided with increases in neurological functioning. More recently, a meta-analysis of cognitive training studies (Karch et al., 2013) revealed favorable effects on secondary measures of outcomes such as behavior, regardless of academic gains. In addition, Farias et al. (2017) reported a reduction in both internalizing and externalizing behavior problems following cognitive training with Captain’s Log for children with ADHD. Yet, there is extant research with opposing findings. Three studies using the Cogmed working memory training program failed to find parent-reported improvements in behavior (Gray et al., 2012; Egeland et al., 2013; Chacko et al., 2014). Although other studies have shown improvements in parent-reported behavior (Green et al., 2012) and teacher ratings of oppositional behavior (Beck et al., 2010), the ratings did not differ significantly from the control groups in either study.

The aim of the current study was to examine parent-reported academic and behavioral benefits—as well as cognitive improvements—from clinician-delivered LearningRx cognitive training interventions. Prior research on LearningRx has revealed significant gains in cognitive skills following cognitive training. In a randomized controlled trial with children ages 8–14 with learning difficulties, there were statistically significant differences between intervention and controls on measures of working memory, long-term memory, auditory processing, visual processing, logic and reasoning, processing speed, and IQ score (Carpenter et al., 2016). These results were similar to an earlier controlled study (Gibson et al., 2015) and corroborate observational study findings on nearly 18,000 clients between 2010 and 2015 (Moore and Wainer, 2016). In a randomized controlled study with high-school students, magnetic resonance imaging revealed increased global connectivity in the LearningRx group as well as connections that correlated with gains on cognitive test scores (Ledbetter et al., 2016). However, a controlled study on transfer effects of LearningRx cognitive training has not been conducted. By examining parent ratings of intervention and control groups, the current two-part study addresses that gap in the literature. The study also examined changes in cognitive test scores for the intervention groups.

The questions answered by the first part of the study were:

(a) Does the completion of the ThinkRx or ReadRx cognitive training program reduce symptoms of academic difficulty as reported by parents?

(b) Does the completion of the ThinkRx or ReadRx cognitive training program reduce symptoms of oppositional behavior as reported by parents?

(c) Is there a statistically significant difference in parent-reported ratings of academic difficulty and oppositional behavior between clients who complete the ThinkRx or ReadRx cognitive training and clients who do not?

For the first part of the study, we hypothesized that:

H1. Participants who completed cognitive training with ThinkRx or ReadRx would have reduced symptoms of academic difficulty as reported by parents compared with the control group.

H2. Participants who completed cognitive training with ThinkRx or ReadRx would have reduced symptoms of oppositional behavior as reported by parents compared with the control group.

For the second part of the study, we examined the change from pre-test to post-test in cognitive function as measured by the Woodcock Johnson III (WJ III)—Tests of Cognitive Abilities, for both intervention groups. The questions answered by the study were:

(a) Did participants who completed the ThinkRx or ReadRx cognitive training program achieve statistically significant changes in cognitive test scores?

(b) Were there statistically significant differences between the two intervention groups on changes in cognitive test scores?

For the second part of the study, we hypothesized that:

H3. Participants who completed cognitive training with ThinkRx or ReadRx would have statistically significant changes in cognitive skills as measured by the WJ III.

H4. There would be no difference between intervention groups on changes in cognitive skills as measured by the WJ III.

Materials and Methods

Participants

Prior to recruiting participants, the author obtained ethics approval by Capella University’s Institutional Review Board for his doctoral dissertation research. Participants were recruited via an email invitation sent by LearningRx headquarters to 6,000 parents of clients who had completed a pre-assessment consultation or a training program at any of the 88 LearningRx cognitive training centers in the United States. Parents replied to the email if they were interested in participating. A link to the full description of the study was then sent to those interested participants where they gave online informed consent and completed the Learning Skills Rating Scale (LSRS) via a web-based survey. LearningRx headquarters then provided the author with the cognitive testing and pre-test LSRS survey results previously completed by the same participants when their children had completed testing at a LearningRx center. Because only parent survey and archived test data were collected, assent from children was not required.

The sample for the current study included 178 clients between the ages of 5 and 18 (M = 12.1), including 78 females (44%) and 100 males (56%). Participants who completed the LSRS were placed in one of three groups: those who had completed a 60-h cognitive training program (ThinkRx), those who had completed a 120-h cognitive training plus reading program (ReadRx), and those who had completed an initial assessment but had not enrolled in a cognitive training program (Control).

The ThinkRx group (n = 67) included 29 females and 38 males with parent-reported disabilities of ADHD (n = 22), autism (n = 3), dyslexia (n = 4), and speech delay (n = 4). The ReadRx group (n = 53) included 27 females and 26 males with parent-reported disabilities of ADHD (n = 18), autism (n = 3), dyslexia (n = 12), and speech delay (n = 9). The control group (n = 58) included 22 females and 36 males with parent-reported disabilities of ADHD (n = 17), autism (n = 4), dyslexia (n = 6), and speech delay (n = 7). In the current study, the clients in the ThinkRx intervention group had completed 4–5 h of training each week for an average of 12 weeks. Clients in the ReadRx group had attended 4–5 h of training per week for an average of 24 weeks.

Procedures

For the current study, we used a quasi-experimental design to examine the pre-training to post-training differences in academic difficulty and oppositional behavior symptoms of participants who had and had not completed a LearningRx cognitive training program; and the archived cognitive test data from participants who had completed the ThinkRx or ReadRx intervention.

Intervention

The LearningRx cognitive training programs are delivered one-on-one by a cognitive trainer during 60–90-min training sessions 3–4 days per week. The programs target seven primary cognitive skills as well as multiple sub-skills through intense and repeated engagement in mental tasks. Using a synergistic “drill for skill” approach to mastering each task, the cognitive trainers use adaptive levels of intensity, hierarchical sequencing, task loading, and dynamic feedback to move clients through the curriculum. Trainers keep every training session focused and demanding to push clients beyond their current skill levels, including adding deliberate distractions to tax the client’s focus and attention skills. The use of a metronome adds an element of multi-tasking while also increasing the intensity of the training procedures. Client workbooks outline a detailed progression of mastery over each training procedure to ensure fidelity of intervention implementation across clients.

All LearningRx clients complete ThinkRx (Gibson et al., 2003b), the 60-h basic proprietary cognitive training intervention that uses 23 different mental training procedures with more than 1,000 adaptive difficulty levels. The program is detailed in Carpenter et al. (2016), but Figure 1 illustrates an example of a training task. Memory Match is a working memory training task that also targets visual discrimination, processing speed, visual span, and sustained attention. The client and trainer sit across from each other with a workboard between them. The trainer creates a pattern of cards on his side of the board and the client recreates the pattern from memory while counting on beat to the metronome. There are nine difficulty levels and 34 variations of this training task.

FIGURE 1

Figure 1. Example of a working memory training task.

At LearningRx cognitive training centers, a reading program is an optional follow-on to the ThinkRx program for clients with reading difficulties. The ReadRx (Gibson et al., 2003a) proprietary training program is an additional 60 h of an intensive sound-to-code reading and spelling intervention delivered through the cognitive training methodology. Clients receive training on auditory processing, basic code, and complex coding skills to target improved reading rate, accuracy, fluency, comprehension, spelling, and writing. The ReadRx program includes seven lessons on basic reading code such as three letter clusters and/aw/versus/au/, followed by 15 lessons on complex reading code such as the schwa, word parts, and code overlap for vowels. Auditory processing drills focus on tasks to master phoneme manipulation skills such as blending, segmenting, switching, and dropping.

Measures

Academic Skills and Behavior Rating Scale

The pre- and post-training parent survey instrument used in the current study is the LSRS (Gibson, 2007). Completed by parents as part of the intake assessment battery at LearningRx centers, the LSRS is a 64-item survey of functioning in six academic skill areas (paying attention, timeliness of work, basic reading skills, remembering instructions and facts, visualization, and problem-solving skills) as well as ratings of oppositional behavior. Ratings are based on a Likert scale from 0 to 4, representing less often to significantly more, in response to the prompt, “Compared with others of the same age and gender, this behavior occurs….” The LSRS has been validated for use with clients ages 5–18 on a sample of 624 same-age peers. Coefficient alphas for each index range from 0.81 to 0.89 with the exception of the sensory motor skills index (0.74). Sensory motor skills, however, are not included as part of the current study and are no longer measured in the revised LSRS instrument (Gibson, 2014). Split-half reliability indices range from 0.82 to 0.92. Concurrent validity coefficients with the WJ III Tests of Cognitive Abilities range from 0.27 to 0.47. The range of scores for each skill is 0–24. The greater the score, the more significant the symptoms. Lower scores indicate less difficulty with that skill.

Academic Skills Ratings

Ratings are based on eight survey items per skill. Ratings of paying attention include items like: is distracted from the task at hand, is impulsive, has difficulty maintaining attention, has difficulty organizing activities, avoids prolonged mental effort, and has difficulty remembering things just heard. Items on the index for timeliness of work include: completes math assignments slowly, is often one of the last to complete tasks, reading is slow, takes a long time to complete tasks, and writing assignments take too long. Items on the index for basic reading skills include: poor reading comprehension, difficulty sounding out unknown words, avoids reading, oral reading is slow or choppy, and has difficulty finding words for oral expression. Items on the index for remembering instructions and facts include: often asks to have things repeated, has difficulty remembering phone numbers, has difficulty following verbal directions, has difficulty recalling stories or jokes, and needs to reread or restudy materials. Items on the index for problem-solving skills include: has difficulty planning steps to solve problems, takes a while to catch on to new things, has difficulty understanding stories, and avoids games like chess or checkers. Items on the index for visualization skills include: poor sense of direction, misreads similar words, has difficulty with puzzles, and has difficulty creating mental pictures.

Behavioral Skills Rating

Oppositional behavior is assessed using eight survey items. Example items include: is angry or resentful, refuses requests or disobeys rules, loses temper, argues with authority figures, holds grudges or seeks revenge, and blames others for mistakes.

Cognitive Assessment

WJ III–Tests of Cognitive Abilities (WJ III–COG)

Participants in both intervention groups were tested by a master’s-level clinician before and after completing a training program using seven subtests from the WJ III–COG, including Visual-Auditory Learning, Spatial Relations, Sound Blending, Concept Formation, Visual Matching, Numbers Reversed, and Pair Cancelation. The WJ III–COG has been extensively researched and found to have strong psychometric properties, including reliability coefficients of 0.80 and above and concurrent validity correlations of 0.67–0.76 (McGrew et al., 2007). The Visual-Auditory Learning test requires encoding and retrieval of auditory and visual associations, and measures long-term retrieval and semantic memory. The client is taught a rebus and is then asked to recall the meaning of each picture. The Spatial Relations test requires the client to match puzzle pieces to a completed shape to assess visual processing skills. The Sound Blending test measures auditory processing ability by requiring the client to listen to a series of sounds and then blend them to form a word. The Concept Formation test assesses fluid reasoning and inductive logic by requiring the client to apply rules to a set of shapes that share similarities and differences. The Visual Matching test measures perceptual processing speed by requiring the client to identify sets of similar numbers within a time limit. The Numbers Reversed test requires the client to repeat a series of numbers in reverse order and measures short-term and working memory. The Pair Cancellation test measures broad attention and concentration by requiring the client to identify pairs of matching pictures within a set amount of time while maintaining sustained attention and vigilance on the task.

Statistical Analyses

Preliminary Analyses

An analysis of variance indicated no significant pre-test differences between the three groups on disability (ps > 0.05) or gender (p = 0.39). After Bonferroni correction for multiple comparisons and adjusting the alpha to p < 0.007, differences between groups were not significant on pre-test ratings of oppositional behavior (p = 0.21), paying attention (p = 0.08), speed of work (p = 0.05), or problem-solving skills (p = 0.01). There were, however, pre-test differences between groups on remembering (p = 0.003), visualization (p = 0.002), and basic reading skills (p = 0.00).

Main Analyses

To examine difference between all three groups on the changes in symptom ratings from pre-training to post-training, data were analyzed using multivariate analysis of covariance (MANCOVA) on post-training symptom ratings as the dependent variables with the pre-training ratings as covariates. A Bonferroni correction was applied in SPSS due to the number of comparisons. Effect sizes were estimated using partial eta squared. Linear regression was conducted to determine if age or gender were predictors of differences in outcomes. To examine changes in cognitive test scores for the intervention groups, paired samples t-tests were conducted on pre- and post-scores to determine within group significant pre-test to post-test change, and a reliability-corrected analysis of covariance (MANCOVA) on change scores using corrected pre-test values as the covariate to examine differences between intervention groups. Then, the same analyses were repeated with two subgroups: ages 5–12 (n = 135) and ages 13–18 (n = 43).

Results

Visual inspection of the data revealed that symptom ratings of overall academic difficulty and behavior decreased for both the ReadRx and ThinkRx intervention groups, and difficulty ratings increased for the control group. The omnibus MANCOVA indicated significant differences between the groups on the combined dependent variables when controlling for pre-training symptom ratings, F (2, 174) = 2.3, p = 0.004, Wilks’ Λ = 0.82, partial η² = 0.09. The results of the regression analysis indicated that neither age (p = 0.71) nor gender (p = 0.66) were significant predictors of change in academic difficulty ratings reported by parents. Results of parent-reported ratings on the individual academic skills and oppositional behavior for all three groups are detailed in the following sections, followed by results of cognitive testing for the intervention groups which show statistically significant changes from pre-test to post-test with large to very large effect sizes across all cognitive skills.

Intervention Effects on Parent-Reported Change in Academic Skills

Bonferroni-corrected post hoc univariate tests and pairwise comparisons on the difficulty ratings of individual academic skills revealed a similar trend. Figure 2 illustrates pre-training and post-training parent ratings for each individual academic skill.

FIGURE 2

Figure 2. Pre-training and post-training parent ratings in academic skills.

Difficulty Paying Attention

There was an overall significant difference between groups on post-training parent ratings of difficulty paying attention,F (2, 168) = 10.4, p < 0.001, partial η² = 0.11, with significant differences between the control group and the ReadRx group (p < 0.001) and between the control group and the ThinkRx group (p = 0.001), but no significant difference between the two intervention groups. In the subsample of participants ages 5–12, there was an overall significant difference between groups, F (2, 125) = 5.6, p = 0.005, partial η² = 0.08, with significant differences between the control group and the ReadRx group (p = 0.005) but no significant differences between the control group and the ThinkRx group (p = 0.09) or between the two intervention groups (p = 0.73). In the subsample of participants ages 13–18, there was an overall significant difference between groups, F (2, 33) = 3.3, p = 0.04, partial η² = 0.17, with significant differences between the control group and the ThinkRx group (p = 0.04) but no significant differences between the control group and the ReadRx group (p = 0.53) or between the two intervention groups (p = 1.0).

Speed of Work

There was an overall significant difference between groups on post-training parent ratings of speed of work, F (2, 168) = 13.0, p < 0.001, partial η² = 0.13, with significant differences between the control group and the ReadRx group (p < 0.001) and between the control group and the ThinkRx group (p < 0.001), but no significant difference between the two intervention groups. In the subsample of participants ages 5–12, there was an overall significant difference between groups, F (2, 125) = 8.1, p < 0.001, partial η² = 0.12, with significant differences between the control group and the ReadRx group (p < 0.001) and between the control group and the ThinkRx group (p = 0.03), but no significant difference between the two intervention groups (p = 0.47). In the subsample of participants ages 13–18, there was an overall significant difference between groups, F (2, 33) = 3.9, p = 0.03, partial η² = 0.20, with a significant difference between the control group and the ThinkRx group (p = 0.02) but no significant differences between the control group and the ReadRx group (p = 0.64) or between the two intervention groups (p = 0.82).

Basic Reading Skills

There was an overall significant difference between groups on post-training parent ratings of basic reading skills, F (2, 168) = 11.7, p < 0.001, partial η² = 0.12, with significant differences between the control group and the ReadRx group (p < 0.001) and between the control group and the ThinkRx group (p < 0.001), but no significant difference between the two intervention groups. In the subsample of participants ages 5–12, there was an overall significant difference between groups, F (2, 125) = 6.2, p = 0.003, partial η² = 0.09, with significant differences between the control group and the ReadRx group (p = 0.003) and between the control group and the ThinkRx group (p = 0.04), but no significant difference between the two intervention groups (p = 0.88). In the subsample of participants ages 13–18, there was an overall significant difference between groups, F (2, 33) = 3.4, p = 0.04, partial η² = 0.17, with a significant difference between the control group and the ThinkRx group (p = 0.03) but no significant differences between the control group and the ReadRx group (p = 0.50) or between the two intervention groups (p = 1.0).

Remembering Instructions and Facts

There was an overall significant difference between groups on post-training parent ratings of remembering instructions and facts, F (2, 168) = 11.8, p < 0.001, partial η² = 0.12, with significant differences between the control group and the ReadRx group (p < 0.001) and between the control group and the ThinkRx group (p < 0.001), but no significant difference between the two intervention groups. In the subsample of participants ages 5–12, there was an overall significant difference between groups, F (2, 125) = 5.4, p = 0.006, partial η² = 0.08, with significant differences between the control group and the ReadRx group (p = 0.005) and no significant difference between the control group and the ThinkRx group (p = 0.12), or between the two intervention groups (p = 0.66). In the subsample of participants ages 13–18, there was an overall significant difference between groups, F (2, 33) = 5.9, p = 0.006, partial η² = 0.27, with a significant difference between the control group and the ThinkRx group (p = 0.005) but no significant differences between the control group and the ReadRx group (p = 0.48) or between the two intervention groups (p = 0.43).

Problem-Solving

There was an overall significant difference between groups on post-training parent ratings of problem-solving [F (2, 168) = 8.9, p < 0.001, partial η² = 0.10], with significant differences between the control group and the ReadRx group (p = 0.005) and the ThinkRx group (p < 0.001), but no significant difference between the two intervention groups. In the subsample of participants ages 5–12, there was an overall significant difference between groups, F (2, 125) = 3.7, p = 0.03, partial η² = 0.06, with significant differences between the control group and the ReadRx group (p = 0.04) and no significant difference between the control group and the ThinkRx group (p = 0.12) or between the two intervention groups (p = 1.0). In the subsample of participants ages 13–18, there was an overall significant difference between groups, F (2, 33) = 5.4, p = 0.009, partial η² = 0.25, with a significant difference between the control group and the ThinkRx group (p = 0.008) but no significant differences between the control group and the ReadRx group (p = 0.72) or between the two intervention groups (p = 0.37).

Visualization

There was an overall significant difference between groups on post-training parent ratings of visualization, F (2, 168) = 6.8, p = 0.001, partial η² = 0.08, with significant differences between the control group and the ThinkRx group (p = 0.002) and the ReadRx group (p = 0.04) but no significant difference between the two intervention groups. In the subsample of participants ages 5–12, there was an overall significant difference between groups, F (2, 125) = 4.8, p = 0.01, partial η² = 0.07, with significant differences between the control group and the ReadRx group (p = 0.01) and a nearly significant difference between the control group and the ThinkRx group (p = 0.05) but no significant difference between the two intervention groups (p = 1.0). In the subsample of participants ages 13–18, there was not an overall significant difference between groups, F (2, 33) = 2.3, p = 0.10, partial η² = 0.13 on parent ratings of visualization skills.

Intervention Effects on Parent-Reported Change in Oppositional Behavior

Parent ratings of oppositional behavior decreased for both the intervention groups, and ratings increased for the control group. Figure 3 illustrates the pre-training and post-training parent ratings in oppositional behavior by group. The trend for the control group indicates a slight increase in oppositional behavior ratings by parents while the trends for both intervention groups reveal a downward trend in oppositional behavior ratings reported by parents. Bonferroni-corrected post hoc univariate tests and pairwise comparisons on the ratings of oppositional behavior revealed a similar trend but were not significant in the subsamples.

FIGURE 3

Figure 3. Changes in parent ratings of oppositional behavior.

There was an overall significant difference between groups on post-test ratings of oppositional behavior [F (2, 168) = 3.4, p = 0.04, partial η² = 0.04], with a significant difference between the control group and the ThinkRx group (p = 0.03) but no significant difference between the control group and the ReadRx group (p = 0.31) or between the two intervention groups. In the subsample of participants ages 5–12, there was not an overall significant difference between groups, F (2, 125) = 1.2, p = 0.29, partial η² = 0.02. In the subsample of participants ages 13–18, there was not an overall significant difference between groups, F (2, 33) = 2.1, p = 0.13, partial η² = 0.11 on parent ratings of oppositional behavior.

Finally, to see if the results varied by age or gender, we ran linear regression analyses with age and gender as predictors of post-training changes. The results of the analysis indicated that age was not a significant predictor of change in oppositional behavior symptoms (p = 0.23). The results of the analysis indicated that gender was also not a significant predictor of change in oppositional behavior symptoms (p = 0.82).

Overall Intervention Effect on Objective Cognitive Test Scores in Both Intervention Groups

A paired samples t-test was conducted on pre-test and post-test WJ III–COG scores for all clients who had completed a LearningRx intervention. After Bonferroni correction for seven comparisons, the results indicated statistically significant changes from pre-test to post-test with large to very large effect sizes across all cognitive skills: long-term memory [t (116) = −11.0, p < 0.001, 95% CI (−15.7, −10.9), d = 1.02], visual processing [t (116) = −8.2, p < 0.001, 95% CI (−8.6, −5.3), d = 0.76], auditory processing [t (74) = −8.2, p < 0.001, 95% CI (−12.9, −7.9), d = 0.95], reasoning [t (115) = −11.0, p < 0.001, 95% CI (−12.8, −8.9), d = 1.02], processing speed [t (87) = −5.6, p < 0.001, 95% CI (−11.4, −5.4), d = 0.60], working memory [t (113) = −7.06, p < 0.001, 95% CI (−12.6, −7.1), d = 0.66], and attention [t (116) = −13.1, p < 0.001, 95% CI (−15.3, −11.3), d = 1.21]. Figure 4 illustrates the pre-training and post-training cognitive test scores for both intervention groups.

FIGURE 4

Figure 4. Cognitive test scores for both intervention groups.

For the subsample of participants ages 5–12, all changes from pre-test to post-test were statistically significant at p < 0.001. For the subsample of participants ages 13–18, all changes from pre-test to post-test were statistically significant at p < 0.001 except for logic and reasoning which was significant at p = 0.002.

To see if there were overall treatment effect differences between the two intervention groups on cognitive test scores gains, we ran a reliability-corrected multivariate analysis of variance (MANCOVA) on change scores using corrected pre-test values as the covariate. This method is preferred when comparing outcomes in non-equivalent group designs (Trochim, 2006). Because the groups were non-equivalent on six of the seven pre-test scores, we first corrected for pre-test measurement error using the following formula: $X_{\text{adj}}=\overline{X}\text{}+r(X-\overline{X})$ where X_adj is the adjusted pre-test value, $\overline{X}$is the original mean pre-test value, X is the individual pre-test score, and r is Cronbach’s Alpha reliability coefficient for each Woodcock Johnson test. Then we ran the MANCOVA using the adjusted pre-test values. There were no significant differences in cognitive test score gains between the two treatment groups (all ps > 0.05). Multiple regression on number of training hours as a predictor of cognitive test gains was not significant (p = 0.053).

For the subsample of intervention participants ages 5–12, we ran a reliability-corrected multivariate analysis of variance (MANCOVA) on change scores using corrected pre-test values as the covariate. There was no significant difference between intervention groups on any of the cognitive test measures (ps > 0.05). Multiple regression on number of training hours as a predictor of cognitive test gains was not significant (p = 0.14). For the subsample of intervention participants ages 13–18, we ran a reliability-corrected multivariate analysis of variance (MANCOVA) on change scores using corrected pre-test values as the covariate. There was no significant difference between intervention groups on any of the cognitive test measures (ps > 0.05). Multiple regression on number of training hours as a predictor of cognitive test gains was not significant (p = 0.56).

Discussion and Conclusion

The current study examined academic, behavioral, and cognitive changes following completion of a LearningRx cognitive training program. First, changes in parent-reported academic difficulty and oppositional behavior ratings were compared across three groups: 60-h ThinkRx, 120-h ReadRx, and controls. Three of the four hypotheses were supported. First, we hypothesized that clients who completed cognitive training would have reduced symptoms of academic difficulty as reported by parents. As hypothesized, completion of a LearningRx cognitive training program was associated with reduced symptoms of academic difficulty, and there was a statistically significant difference between the intervention groups and the control group overall on all measures of academic difficulty following cognitive training. That is, the intervention groups overall realized a significant reduction in academic difficulty ratings following training including paying attention, remembering, basic reading, visualization, problem-solving, and speed of completing schoolwork which supported our initial hypothesis.

The analyses of subsamples by age group revealed some interesting trends, though. For example, among the 13–18-year old participants, only the ThinkRx intervention group saw significantly greater reductions in academic difficulty ratings compared with the control group. There are several possible explanations for this finding. First, the ThinkRx group of teenagers began the program with higher cognitive skills than teenagers in the ReadRx group as illustrated in Figure 4. Teens in the ThinkRx group may have more cognitive flexibility and were able to more rapidly transfer their improved thinking skills to learning. Alternatively, teens who need an intensive sound-to-code reading intervention like ReadRx may have greater classroom struggles than teens who already have strong reading skills. This suggests to us that the ReadRx group may take longer to apply new learning skills or may need additional training before the training effects will transfer to new academic habits. It indeed illustrates an example of the achievement gap in adolescent literacy. For school-age children, the ReadRx training procedures are timely and should enable young children to participate in the ongoing reading instruction in the classroom at school. In contrast, teenagers may need tutoring following the completion of the ReadRx intervention in order to catch up to their peers in content-based coursework where their deficient reading skills may have impeded knowledge acquisition.

Another interesting trend in the age group analyses was that only the ReadRx intervention participants in the younger age group saw significantly greater reductions in academic difficulty ratings compared with the control group on measures of four learning skills: paying attention, remembering instructions and facts, problem-solving, and visualization. Perhaps children in the 5–12 age range require intensive reading instruction in additional to basic cognitive training in order to see transfer of training to observable learning skills. The ReadRx program is not delivered in isolation but through cognitive training exercises, so it is plausible that the school-age participants needed a full 120 h of training to realize transfer of gains in those four areas.

Our second hypothesis reading change in oppositional behavior ratings was not supported. Although the differences in oppositional behavioral ratings between the intervention and control groups were not statistically significant, this finding is important in parsing out the specific effects of the training. That is, it is a differential finding. The program does not target oppositional behavior, so it should not have been expected that the program would change oppositional behavior ratings. Instead, the program targets cognitive skills. Any transfer effects should relate to those targeted areas. For example, transfer effects of the training to reduction in parent ratings on difficulty paying attention can be traced to the intense focus of the intervention on building attention skills. A key component of LearningRx cognitive training is the use of deliberate distractions rather than the elimination of environmental stimuli. Deliberate distractions are built in to the training sessions to tax the client’s capacity for sorting and evaluating the importance of incoming information. Multiple training tasks target sustained, divided, and selective attention skills.

Transfer to the ratings of speed of work can also be attributed to the focus of the training tasks. Twelve of the 24 training procedures in the ThinkRx program and additional 12 training procedures in the ReadRx program all target the remediation of processing speed. Nearly, all training procedures include the use of a stopwatch to ensure progressively faster response or completion times for each task. Tasks that require the use of the metronome are also sequenced from 60 to 160 beats per minute to increase speed of processing. For example, a participant is given 20 s to add or subtract a fixed number to every number in a given column. A version of the same training tasks includes responding with the correct sum or difference on beat to the metronome. Transfer of training to ratings of remembering instructions and facts is also attributable to the training procedures, including 12 tasks that target working memory, 5 tasks that target short-term memory, and 4 that target long-term memory. For example, clients are taught humorous and connected visual representations of all of the United States presidents and are trained to recite them in ascending and descending order within 50 s. Other memory training tasks utilize computation, dual-taskings, and visual and auditory prompts that target associations as well as rote memory. The transfer of training effects to decreased difficulty ratings on problem-solving and visualization are also plausible given the focus of ThinkRx training tasks, including 16 training tasks that target visual processing skills such as visual discrimination, visual manipulation, visualization, and visual span with a variety of pictures and manipulatives. Another five tasks directly target reasoning skills with stories, concept cards, and manipulatives.

The transfer of training effects to ratings on basic reading skills is supported by the immense focus on auditory processing skills targeted by both ThinkRx and ReadRx programs. Six of the ThinkRx procedures and all 16 of the ReadRx training procedures aim to increase skill in auditory analysis, blending, discrimination, and segmenting, as well as fluency and reading comprehension through intense drills and systematic instruction in phonics. Therefore, transfer of training effects to learnings skills is plausible versus transfer to oppositional behavior which is not addressed in either intervention.

In the second part of the study, we analyzed the objective cognitive testing outcomes for the two intervention groups. As hypothesized, students who completed a LearningRx cognitive training program realized significant improvements across all cognitive skills. Because the ReadRx intervention includes all of the ThinkRx training procedures as well, we anticipated and indeed saw no differences between the two intervention groups in cognitive test results. Therefore, both hypotheses regarding cognitive testing outcomes were supported. Had we collected comprehensive measures of reading before and after training, we may have noted some differences in outcomes between the intervention groups. That is certainly an area worthy of future study. Regardless, the gains across all cognitive skill measures suggest that both intervention programs were associated with increased cognitive skills in both age groups studied.

The results of this study indicate some transfer of training effects to functional benefits. By targeting the underlying cognitive skills necessary for learning, individual learning skills improved in clients who completed a LearningRx training program. A reduction in parent-reported academic difficulty ratings was found across all areas overall. This suggests broad generalization of benefits from targeted cognitive training to the academic skills. Aligned with Feuerstein’s (Feuerstein, 1990) theory of structural cognitive modifiability, the intensive one-on-one intervention used in this current study was associated with multiple academic skill improvements as reported by parents. This finding suggests that the ThinkRx and ReadRx curricula may be contenders for use in a response-to-intervention environment for clients with overall learning struggles.

The results are encouraging but there are several limitations that should be addressed. First, part of the data collected was based on parent-reported symptoms. Although the tool was previously validated, there is a risk of response bias with any self-report questionnaire (Breakwell et al., 2002). However, the use of parent-reports is supported in the existing research including language assessment (Boudreau, 2005), social-emotional competencies (Merrell et al., 2011), executive functions (Parrish et al., 2007), child health and academic achievement (Bevans et al., 2012), and cognitive functioning (Limbers et al., 2009). Although not blind to the intervention condition and, therefore, not completely free of the risk of bias, we find much value in the ecological validity of the parent ratings. This was a clinical sample—actual clients who were trained in learning centers around the country. The parent ratings are representative of what we see day to day rather than the results of laboratory-controlled research. This is an example of strong ecological validity because it so closely approximates the real-life phenomenon we are investigating (Nestor and Schutt, 2014)—parent views on changes in their children with or without cognitive training. In addition, the gains shown on the standardized tests of participants in the intervention groups serve to mitigate the risk of drawing conclusions based solely on parent-reports.

Another limitation is the non-random nature of the study sample. The non-equivalent group design presents a potential selection threat to internal validity. However, the inclusion of pre-test scores as a covariate provides a statistical control for that threat by accounting for group differences at pre-test. Another potential limitation is the difference in pre-training parent ratings on three measures: remembering, visualization, and basic reading skills. The results of changes in those variables should be interpreted with caution. Additionally, cognitive test scores were only available for the intervention groups. However, other studies on LearningRx programs reported statistically significant differences following the intervention on standardized measures of intelligence and cognitive functioning compared with controls (Gibson et al., 2015; Carpenter et al., 2016; Hill et al., 2016). The current findings simply supplement the parent-reported improvements in the intervention groups by providing an additional dataset to consider.

A final limitation of the study is the inability to track dose–response outcomes using archived data. This would be an important area for future research in order to determine the minimum number of training hours needed to create change. The intervention is lengthy and expensive, so it would be valuable to evaluate the dose–response relationship in order to make the program as cost effective as possible to avail the program to the widest number of people who might benefit from the intervention.

Finding research-based interventions to remediate academic skills for clients with learning disabilities is a priority among educators worldwide, particularly in response to the Nation’s Report Card description of percentages of below basic readers among clients with learning disabilities (U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, 2015). Further, the 70% failure rate and high percentage of disciplinary actions associated with learning disabilities (Cortiella and Horowitz, 2014) necessitates action to find effective interventions for both academic skills and behavior. In addition to traditional academic intervention methods such as tutoring and supplemental services offered as part of a response-to-intervention program, current research also suggests that cognitive skills training may be an effective way to improve academic performance (Holmes and Gathercole, 2014). The current study suggests these prior suggestions may have some merit.

The completion of the 120 h, clinician-delivered ReadRx cognitive training program was associated with parent-reported improvements in ratings of academic difficulty across all skills for school-age participants with learning struggles, including statistically significant differences in paying attention, remembering instructions and facts, basic reading, visualization, problem-solving, and speed of completing schoolwork compared with controls. School-age participants in the ReadRx program also achieved statistically significant changes across all cognitive skills as measured by the WJ III Tests of Cognitive Abilities. Further, parents of school-age participants in the ThinkRx group reported statistically significant improvements in speed of work and basic reading skills compared with controls. School-age ThinkRx participants also achieved statistically significant changes across all cognitive skills as measured by the WJ III. The completion of the 60 h, clinician-delivered ThinkRx cognitive training program was associated with parent-reported improvements in ratings of academic difficulty across five skills for teenage participants with learning struggles, including statistically significant differences in paying attention, remembering instructions and facts, basic reading, problem-solving, and speed of completing schoolwork compared with controls. Teenage ThinkRx and ReadRx participants also achieved statistically significant changes across all cognitive skills as measured by the WJ III.

Although these results cannot support a cause-and-effect conclusion about efficacy, the implication of these findings is that a one-on-one cognitive training program with and without a reading component may be a useful adjunct to current intervention protocols for children with learning struggles. This is the first study to examine functional or academic skill changes following a LearningRx cognitive training program. In future research, transfer to actual academic outcomes such as grades should be explored.

Author Contributions

The current study was conducted by a single author, EJ, who designed the study, collected and analyzed the data, and crafted all parts of the manuscript.

Conflict of Interest Statement

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The current study was conducted as part of the author’s doctoral dissertation while at Capella University. The author would like to acknowledge John Malpass, PhD who served as dissertation mentor and dissertation committee chairperson.

References

Avtzon, S. A. (2012). Effect of neuroscience-based cognitive skill training on growth of cognitive deficits associated with learning disabilities in children grades 2-4. Learn. Disabil. 18, 111–122.

Google Scholar

Barkl, S., Porter, A., and Ginns, P. (2012). Cognitive training for children: effects on inductive reasoning, deductive reasoning, and mathematical achievement in an Australian school setting. Psychol. Sch. 49, 828–842. doi: 10.1002/pits.21638

CrossRef Full Text | Google Scholar

Beck, S. J., Hanson, C. A., Puffenberger, S. S., Benninger, K. L., and Benninger, W. B. (2010). A controlled trial of working memory training for children and adolescent with ADHD. J. Clin. Child Adolesc. Psychol. 39, 825–836. doi:10.1080/15374416.2010.517162

CrossRef Full Text | Google Scholar

Bevans, K. B., Riley, A. W., and Forrest, C. B. (2012). Development of the healthy pathways parent-report scales. Qual. Life Res. 21, 1755–1770. doi:10.1007/s11136-012-0111-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Boudreau, D. (2005). Use of a parent questionnaire in emergent and early literacy assessment of preschool children. Lang. Speech Hear. Serv. Sch. 36, 33–47. doi:10.1044/0161-1461(2005/004)

PubMed Abstract | CrossRef Full Text | Google Scholar

Breakwell, G. M., Hammond, S., and Fife-Schaw, C. (2002). Research Methods in Psychology, 2nd Edn. London, England: SAGE.

Google Scholar

Carpenter, D., Ledbetter, C., and Moore, A. L. (2016). LearningRx cognitive training effects in children ages 8-14: a randomized controlled study. Appl. Cognitive Psychol. 30, 815–826. doi:10.1002/acp.3257

CrossRef Full Text | Google Scholar

Chacko, A., Bedard, A. C., Marks, D. J., Feirsen, N., Uderman, J. Z., Chimiklis, A., et al. (2014). A randomized clinical trial of Cogmed Working Memory Training in school-age children with ADHD: a replication in a diverse sample using a control condition. J. Child Psychol. Psychiatry 55, 247–255. doi:10.1111/jcpp.12146

PubMed Abstract | CrossRef Full Text | Google Scholar

Cortiella, C., and Horowitz, S. H. (2014). The State of Learning Disabilities: Facts, Trends and Emerging Issues. National Center for Learning Disabilities. Available from: https://www.ncld.org/wp-content/uploads/2014/11/2014-State-of-LD.pdf

Google Scholar

Dahlin, K. I. E. (2013). Working memory training and the effect on mathematical achievement in children with attention deficits and special needs. J. Educ. Learn. 2, 118–133. doi:10.5539/jel.v2nlpll8

CrossRef Full Text | Google Scholar

Dunning, D. L., and Holmes, J. (2014). Does working memory training promote the use of strategies on untrained working memory tasks? Mem. Cognit. 42, 854–862. doi:10.3758/s13421-014-0410-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Egeland, J., Aarlien, A. K., and Saunes, B.-K. (2013). Few effects of far transfer of working memory training in ADHD: a randomized controlled trial. PLoS ONE 8:e75660. doi:10.1371/journal.pone.0075660

PubMed Abstract | CrossRef Full Text | Google Scholar

Fals-Stewart, W., and Lucente, S. (1994). The effect of cognitive rehabilitation on the neuropsychological status of patients in drug abuse intervention who display neurocognitive impairment. Rehabil. Psychol. 39, 75–94. doi:10.1037/h0080316

CrossRef Full Text | Google Scholar

Farias, A. C., Cordeiro, M. L., Felden, A. P. G., Bara, T. S., Benko, C. R., Coutinho, D., et al. (2017). Attention-memory training yields behavioral and academic improvement sin children diagnosed with attention deficit hyperactivity disorder comorbid with a learning disorder. Neuropsychiatr. Dis. Treat. 13, 1761–1769. doi:10.2147/NDT.S136663

CrossRef Full Text | Google Scholar

Feuerstein, R. (1990). “The theory of structural modifiability,” in Learning and Thinking Styles: Classroom Interaction, eds B. Presseisen, R. Sternberg, K. Fischer, C. Knight, and R. Feuerstein (Washington, DC: National Education Association), 68–134.

Google Scholar

Feuerstein, R., Feuerstein, R. S., and Falik, L. H. (2010). Beyond Smarter: Mediated Learning and the Brain’s Capacity for Change. New York: Teacher’s College Press.

Google Scholar

Fiorello, C. A., and Primerano, D. (2005). Research into practice: Cattell-Horn-Carroll cognitive assessment in practice: eligibility and program development issues. Psychol. Sch. 42, 525–536. doi:10.1002/pits.20089

CrossRef Full Text | Google Scholar

Fuchs, D., Compton, D. L., Fuchs, L. S., and Bryant, J. (2008). Making “secondary intervention” work in a three-tier responsiveness-to-intervention model: findings from the first-grade longitudinal reading study at the National Research Center on Learning Disabilities. Read. Writ. 21, 413–436. doi:10.1007/s11145-007-9083-9

CrossRef Full Text | Google Scholar

Gibson, K. (2007). Unlock the Einstein Inside: Applying New Brain Science to Wake Up the Smart in Your Child. Colorado Springs, CO: LearningRx.

Google Scholar

Gibson, K. (2014). Learning Skills Rating Scale—Revised. Colorado Springs, CO: LearningRx.

Google Scholar

Gibson, K., Carpenter, D. M., Moore, A. L., and Mitchell, T. (2015). Training the brain to learn: beyond vision therapy. Vision Dev. Rehabil. 1, 120–129.

Google Scholar

Gibson, K., Hanson, K., Mitchell, T., and Tenpas, D. (2003a). ReadRx: Sound to Code Reading and Spelling Program. Colorado Springs: LearningRx.

Google Scholar

Gibson, K., Mitchell, T., and Tenpas, D. (2003b). ThinkRx: Cognitive Training Procedures Workbook. Colorado Springs: LearningRx.

Google Scholar

Gillam, R. B., Loeb, D. F., Hoffman, L. M., Bohman, T., Champlin, C. A., Thibodeau, L., et al. (2008). The efficacy of Fast ForWord language intervention in school-age children with language impairment: a randomized controlled trial. J. Speech Lang. Hear. Res. 51, 97–119. doi:10.1044/1092-4388(2008/007)

PubMed Abstract | CrossRef Full Text | Google Scholar

Gray, S. A., Chaban, P., Martinussen, R., Goldberg, R., Gotlieb, H., Kronitz, R., et al. (2012). Effects of a computerized working memory training program on working memory, attention, and academics in adolescents with severe LD and comorbid ADHD: a randomized controlled trial. J. Child Psychol. Psychiatry 53, 1277–1284. doi:10.1111/j.1469-7610.2012.02592.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Green, C. T., Long, D. L., Green, D., Iosif, A., Dixon, J. F., Miller, M. R., et al. (2012). Will working memory training generalize to improve off-task behavior in children with attention-deficit/hyperactivity disorder? Neurotherapeutics 9, 639–648. doi:10.1007/s13311-012-0124-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Henry, L. A., Messer, D. J., and Nash, G. (2014). Testing for near and far transfer effects with a short, face-to-face adaptive working memory training intervention in typical children. Infant Child Dev. 23, 84–103. doi:10.1002/icd.1816

CrossRef Full Text | Google Scholar

Hill, O. W., Zewelanji, S., and Faison, O. (2016). The efficacy of the LearningRx cognitive training program: modality and transfer effects. J. Exp. Educ. Learn. Instruct. Cognit. 84, 600–620. doi:10.1080/00220973.2015.1065218

CrossRef Full Text | Google Scholar

Holmes, J., and Gathercole, S. E. (2014). Taking working memory training from the laboratory into schools. Educ. Psychol. 34, 440–450. doi:10.1080/01443410.2013.797338

PubMed Abstract | CrossRef Full Text | Google Scholar

Jausovec, N., and Jausovec, K. (2012). Working memory training: improving intelligence—changing brain activity. Brain Cogn. 79, 96–106. doi:10.1016/j.bandc.2012.02.007

CrossRef Full Text | Google Scholar

Karch, D., Albers, L., Renner, G., Lichtenauer, N., and von Kries, R. (2013). The efficacy of cognitive training programs in children and adolescents. Deutsches Aerzteblatt Int. 110, 643–652. doi:10.3238/arztebl.2013.0643

CrossRef Full Text | Google Scholar

Kearns, D. M., and Fuchs, D. (2013). Does cognitively focused instruction improve the academic performance of low-achieving clients? Except. Child. 79, 263–290.

Google Scholar

Kozulin, A., and Presseisen, B. (1995). Mediated learning experience and psychological tools: Vygotsky’s and Feurerstein’s perspectives in a study of client learning. Educ. Psychol. 30, 67–75. doi:10.1207/s15326985ep3002_3

CrossRef Full Text | Google Scholar

Ledbetter, C., Faison, M. O., and Patterson, J. (2016). “Correlation of cognitive training gains and resting state functional connectivity,” in Society for Neuroscience (San Diego, CA).

Google Scholar

Limbers, C. A., Heffer, R. W., and Varni, J. W. (2009). Health-related quality of life and cognitive functioning from the perspective of parents of school-aged children with asperger’s syndrome utilizing the PedsQL(TM). J. Autism Dev. Disord. 39, 1529–1541. doi:10.1007/s10803-009-0777-5

CrossRef Full Text | Google Scholar

McGrew, K. S., Schrank, F. A., and Woodcock, R. W. (2007). Woodcock-Johnson III Normative Update. Rolling Meadows: Riverside Publishing.

Google Scholar

Melby-Lervag, M., and Hulme, C. (2016). There is no convincing evidence that working memory training is effective: a reply to Au et al. (2014) and Karbach and Verhaeghen (2014). Psychon. Bull. Rev. 23, 324–330. doi:10.3758/s13423-015-0862-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Merrell, K. W., Felver-gant, J., and Tom, K. M. (2011). Development and validation of a parent report measure for assessing social-emotional competencies of children and adolescents. J. Child Fam. Stud. 20, 529–540. doi:10.1007/s10826-010-9425-0

CrossRef Full Text | Google Scholar

Moore, A. L., and Wainer, H. (2016). LearningRx Client Outcomes and Research Results. Colorado Springs, CO: Gibson Institute of Cognitive Research.

Google Scholar

Nestor, P. G., and Schutt, R. K. (2014). Research Methods in Psychology: Investigating Human Behavior, 2nd Edn. Thousand Oaks, CA: SAGE.

Google Scholar

Nouchi, R., Taki, Y., Takeuchi, H., Hashizume, H., Nozawa, T., Kambara, T., et al. (2013). Brain training game boosts executive functions, working memory and processing speed in the young adults: a randomized controlled trial. PLoS ONE 8:e55518. doi:10.1371/journal.pone.0055518

CrossRef Full Text | Google Scholar

Parrish, J., Geary, E., Jones, J., Raj, S., Hermann, B., and Seidenberg, M. (2007). Executive functioning in childhood epilepsy: parent-report and cognitive assessment. Dev. Med. Child Neurol. 49, 412–416. doi:10.1111/j.1469-8749.2007.00412.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Rabipour, S., and Raz, A. (2012). Training the brain: fact and fad in cognitive and behavioral remediation. Brain Cogn. 79, 159–179. doi:10.1016/j.bandc.2012.02.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Redick, T. S., Shipstead, Z. A., Harrison, T. L., Hicks, K. L., Fried, D. E., Hambrick, D. Z., et al. (2013). No evidence if intelligence improvement after working memory training: a randomized placebo-controlled study. J. Exp. Psychol. 142, 359–379. doi:10.1037/a0029082

CrossRef Full Text | Google Scholar

Shiran, A., and Breznitz, Z. (2011). The effect of cognitive training on recall range and speed of information processing in the working memory of dyslexic and skilled readers. J. Neurolinguistics 24, 524–537. doi:10.1016/j.neuroling.2010.12.001

CrossRef Full Text | Google Scholar

Smith, G. N. L., Conway, C. M., Bauernschmidt, A., and Pisoni, D. B. (2015). Can we improve structured sequence processing? Exploring the direct and indirect effects of computerized training using a mediational model. PLoS ONE 10:e0127148. doi:10.1371/journal.pone.0127148

PubMed Abstract | CrossRef Full Text | Google Scholar

Titz, C., and Karbach, J. (2014). Working memory and executive functions: effects of training on academic achievement. Psychol. Res. 78, 852–868. doi:10.1007/s00426-013-0537-1

CrossRef Full Text | Google Scholar

Trochim, W. (2006). “The nonequivalent groups design,” in The Research Methods Knowledge Base, 2nd Edn, ed. W. M. K. Trochim. Available from: https://www.socialresearchmethods.net/kb/quasnegd.php

Google Scholar

U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. (2015). National Assessment of Educational Progress (NAEP): 2015 Mathematics and Reading Assessments. Available from http://www.nationsreportcard.gov/reading_math_2015/#reading/acl?grade=4

Google Scholar

Westerberg, H., and Klingberg, T. (2007). Changes in cortical activity after training of working memory—a single-subject analysis. Physiol. Behav. 9, 186–192. doi:10.1016/j.physbeh.2007.05.041

CrossRef Full Text | Google Scholar

Willis, S. L., Tennstedt, S. L., Marsiske, M., Ball, K., Elias, J., Koepke, K. M., et al. (2006). Longterm effects of cognitive training on everyday functional outcomes in older adults. JAMA 296, 2805–2813. doi:10.1001/jama.296.23.2805

CrossRef Full Text | Google Scholar