Edited by: Ann Dowker, University of Oxford, UK
Reviewed by: Andrew Keith Dunn, Nottingham Trent University, UK; Koen Luwel, KU Leuven - Campus Brussels, Belgium; Annemie Desoete, Ghent University, Belgium
*Correspondence: Larissa Rauscher
This article was submitted to Developmental Psychology, a section of the journal Frontiers in Psychology
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Calcularis is a computer-based training program which focuses on basic numerical skills, spatial representation of numbers and arithmetic operations. The program includes a user model allowing flexible adaptation to the child's individual knowledge and learning profile. The study design to evaluate the training comprises three conditions (Calcularis group, waiting control group, spelling training group). One hundred and thirty-eight children from second to fifth grade participated in the study. Training duration comprised a minimum of 24 training sessions of 20 min within a time period of 6–8 weeks. Compared to the group without training (waiting control group) and the group with an alternative training (spelling training group), the children of the Calcularis group demonstrated a higher benefit in subtraction and number line estimation with medium to large effect sizes. Therefore, Calcularis can be used effectively to support children in arithmetic performance and spatial number representation.
Already at an early stage of development there are considerable differences between children regarding number processing and calculation (Dowker,
In the past years, different models of number processing and calculation have been proposed (e.g., McCloskey et al.,
The triple-code model constitutes the end state of numerical development and calculation abilities. Several theoretical models address the question how numerical cognition develops over childhood (for example, von Aster,
Designing a program to enhance number processing and arithmetic skills involves the consideration of a series of challenging aspects. Children differ with respect to learning speed (Brown et al.,
A computer-based training adapting to the child's individual learning profile and development level can contribute to these requirements. It allows an optimal level of difficulty and learning speed by an individually customized task selection. Another key advantage is the possibility of immediate feedback about the correctness of a solved task. Direct chronological proximity is crucial for knowledge acquisition (Krajewski and Ennemoser,
Furthermore, the computer represents an attractive learning medium (Kulik and Kulik,
Particularly for children experiencing difficulties in math, a computerized training provides the possibility of a learning environment detached from performance pressure and peer comparisons in school context and offers therefore a less stressful and risk-free setting to explore mathematics (Käser and von Aster,
Different meta-analyses examined the effects of computer-based math instruction, revealing positive effects. For example, Kulik (Kulik and Kulik,
Although a number of different computerized learning programs for mathematics exist, the majority of the programs lack empirically based analyses of their effectiveness (Butterworth and Laurillard,
In the following, a selection of empirically evaluated computerized trainings is presented. “Rechenspiele mit Elfe und Mathis I” (Lenhard and Lenhard,
The program “Number Race” (Wilson et al.,
The objective of the program “Rescue Calcularis” (Kucian et al.,
Calcularis uses core elements of “Rescue Calcularis” (Kucian et al.,
The present article represents the evaluation of Calcularis in a large study sample (
The main objective of the present study is to evaluate the efficacy of the computer-based training program Calcularis by combining two different approaches. First, we aimed to determine the general immediate efficacy by comparing the Calcularis training group with an untrained control group. The implementation of an untrained waiting control group allows controlling for developmental and schooling effects as well as arithmetic development under regular conditions. Second, we compared the performance of the Calcularis training group with a group that received a computerized spelling training to examine the domain specificity of the training effects. Thus, the efficacy of the training can be determined by taking novelty and Hawthorne effects as well as unspecific training effects on domain-general functions into consideration. We hypothesized that the Calcularis group will demonstrate an increased arithmetic performance (addition, subtraction) and spatial number representation (in the number range 0–10 and 0–100) compared to both groups with small (computerized spelling training) to medium (untrained control group) effect sizes.
Calcularis is an adaptive computer-based training program. The program‘s theoretical foundation of numerical cognition and development is based on the triple-code model (Dehaene,
The key components (number representation, arithmetic operations) are trained by main and support games. While main games require a combination of abilities, support games train specific skills that serve as prerequisites for the main games. A consistent number notation that accentuates the properties of numbers is used throughout the training program. The notation is encoded by color, form and topology. It is assumed, that this design of the numerical stimuli enhances the different number modalities and strengthens the link between them. A more detailed description of the training including a selection of examples of the games can be found in Käser et al. (
Calcularis includes a user model allowing flexible adaptation on the basis of the internally mapped learning and knowledge profile of the individual child. All children start the training with the same game. After each completed item, the program estimates the actual knowledge state of the child and displays a new task adjusted to this state. The mathematical structure of Calcularis represents a model of the cognitive processes of mathematical development by a dynamic Bayes net. The Bayes net comprises a directed acyclic graph which represents various mathematical skills and their relationships. The user model is based on the student model and control algorithm presented in Käser et al. (
The BUEGA (Esser et al.,
Two subtests (similarities, block design) of the HAWIK-IV (Hamburg-Wechsler intelligence test for children; Petermann and Petermann,
Arithmetic performance was assessed on the basis of the two subscales “addition” and “subtraction” of the HRT (Haffner et al.,
The arithmetic performance test (Kucian et al.,
As a measure of spatial representation of numbers, children indicated the location of 20 verbally and visually presented numbers on a number line from 0 to 100. The items of the number line test were evenly distributed across the number range from 0 to 100 as two numbers of every teen were selected. The percent absolute estimation error (PAE) for target number and the indicated location (estimated number) on the number line was calculated (PAE = |estimated number – target number|/scale of estimates, cf. Siegler and Booth,
A computer-based mathematical test (Käser et al.,
The subtest for
We used three different tests (HRT, arithmetic test and computer-based mathematical test) examining the performance in addition and subtraction tasks. These instruments were implemented since they focus on specific aspects (e.g., computational fluency, degree of mastery) of arithmetic in different number ranges. Furthermore, they are used to compare the results to those of previous studies examining the efficacy of computer based trainings on spatial number representation and calculation (Kucian et al.,
Children and their parents completed a training evaluation questionnaire at the end of the study. The questionnaire includes 4 items concerning the self-evaluation of the enjoyment of the training, improvement of self-perceived arithmetic skills and changes regarding self-confidence. For example, children were presented with the statement “I enjoyed the training” and responded on a 4-point Likert scale ranging from “disagree” to “strongly agree” (0–3). The internal consistency coefficients determined for the children that trained with Calcularis is satisfactory (α = 0.83). Additionally, children judged the difficulty level of the training on a 5-point Likert scale ranging from “far too simple” to “much too difficult.” Furthermore, parents were asked to rate how much their child liked the training (“My child enjoyed the training”).
The study design comprised three groups (Calcularis group, waiting group, spelling training group). Children were randomly assigned to one of three groups. The Calcularis group completed a 6–8 weeks training whilst the waiting group started the training with a 6 week rest period. The spelling training group served as a second control group receiving a computer-based spelling training (Dybuster; Kast et al.,
Children trained with the program 5 times per week with daily training sessions of 20 min in their own home environment. Initial diagnostic (
Children were recruited via flyers sent to elementary schools and psychotherapeutic outpatient clinics. All children attended regular primary schools. Inclusion criteria comprised at least average IQ-scores (min: 16th percentile, T-score ≥ 40) (BUEGA, HAWIK-IV) and age 7;0 to 10;11 years.
One hundred and fifty-five German-speaking children were included in the study. Children were randomly assigned to the groups (Calcularis group:
Group differences were analyzed by means of Analyses of Variance (ANOVA) and Chi-square tests. A series of repeated measures general linear model (GLM) analyses were conducted to evaluate training effects with assessment time point (
The final study sample consisted of 138 children at the age of 7;0 to 10;11. The mean age was 8.46 (
Age (years) | 8.48 (0.86) | 8.54 (0.84) | 8.34 (0.66) | 0.82 |
0.444 |
Gender (f/m) | 31/12 | 31/18 | 33/13 | 1.10 |
0.576 |
BUEGA verbal intelligence |
50.02 (8.49) | 49.14 (9.90) | 48.02 (9.03) | 0.53 |
0.589 |
BUEGA nonverbal intelligence |
53.40 (8.61) | 51.24 (8.68) | 53.70 (9.13) | 1.09 |
0.338 |
BUEGA reading |
49.98 (11.10) | 49.16 (12.15) | 46.91 (10.01) | 0.92 |
0.403 |
BUEGA writing |
44.93 (12.40) | 45.12 (12.26) | 44.22 (12.15) | 0.07 |
0.934 |
HAWIK IV block design |
49.38 (7.84) | 46.74 (7.00) | 47.75 (7.83) | 1.54 |
0.218 |
HAWIK IV similarities |
51.78 (8.49) | 49.30 (6.65) | 51.23 (6.97) | 1.06 |
0.351 |
Mean intelligence (HAWIK, BUEGA) | 51.15 (5.86) | 49.18 (4.94) | 50.18 (5.33) | 1.54 |
0.219 |
BUEGA math word problems |
44.23 (12.24) | 44.73 (12.11) | 45.22 (12.50) | 0.07 |
0.931 |
Mean arithmetic performance |
41.28 (10.96) | 42.16 (10.03) | 42.22 (9.88) | 0.12 |
0.891 |
Using the highest level of education of either parent as an index of socioeconomic status (SES), the results demonstrated that 44% finished university (
The repeated-measures GLM for the HRT addition task demonstrated a significant main effect of time [
For HRT subtraction, results demonstrated a significant main effect of time [
The GLM revealed a significant main effect of time [
For subtraction, results demonstrated a significant main effect of time [
The analysis for the number line test yielded no main effects of time [
The initial analyses examined whether the spatial representation is better explained by a linear or logarithmic function. The regressions to the estimates of children for each of the 20 numbers that were presented were calculated for each child. A paired-sample
The analysis regarding individual
HRT (addition) |
CAL | 43 | 39.91 (11.03) | 43.70 (11.18) |
WG | 49 | 40.69 (10.73) | 43.73 (11.74) | |
ST | 46 | 40.78 (9.58) | 42.17 (9.56) | |
HRT (subtraction) |
CAL | 43 | 39.70 (11.22) | 44.70 (11.37) |
WG | 49 | 41.04 (10.96) | 42.24 (12.23) | |
ST | 46 | 40.65 (10.30) | 41.59 (10.42) | |
Arithmetic performance test (addition) |
CAL | 43 | 15.67 (4.61) | 17.00 (3.94) |
WG | 49 | 15.37 (5.60) | 15.65 (4.80) | |
ST | 46 | 14.93 (4.73) | 15.28 (4.85) | |
Arithmetic performance test (subtraction) |
CAL | 43 | 13.12 (4.81) | 14.79 (4.02) |
WG | 49 | 13.45 (5.12) | 13.61 (5.61) | |
ST | 46 | 13.02 (5.13) | 13.57 (5.30) | |
Number line test, PAE |
CAL | 43 | 9.13 (4.26) | 7.90 (5.02) |
WG | 49 | 8.27 (3.47) | 8.03 (4.24) | |
ST | 46 | 8.94 (4.53) | 9.51 (4.94) | |
Number line test, linearity |
CAL | 43 | 0.86 (0.16) | 0.87 (0.20) |
WG | 49 | 0.88 (0.13) | 0.88 (0.15) | |
ST | 46 | 0.85 (0.19) | 0.82 (0.21) | |
Computer-test (addition) |
CAL | 33 | 28.03 (7.94) | 29.88 (12.55) |
WG | 35 | 33.69 (15.51) | 32.29 (14.38) | |
ST | 35 | 27.51 (11.41) | 28.83 (12.36) | |
Computer-test (subtraction) |
CAL | 33 | 21.58 (9.48) | 25.46 (11.84) |
WG | 35 | 27.17 (12.76) | 25.09 (12.05) | |
ST | 35 | 22.60 (11.20) | 23.83 (11.78) | |
Computer-test (number line 0-10), PAE |
CAL | 29 | 1.40 (0.97) | 0.81 (0.86) |
WG | 32 | 0.80 (0.51) | 1.01 (0.89) | |
ST | 29 | 1.29 (0.95) | 1.37 (0.99) | |
Computer-test (number line 0-10), linearity |
CAL | 29 | 0.82 (0.20) | 0.93 (0.08) |
WG | 32 | 0.90 (0.12) | 0.86 (0.21) | |
ST | 29 | 0.86 (0.17) | 0.79 (0.24) |
The analyses for the addition task did not indicate significant main effects of time [
For subtraction, no significant main effects of time [
There were no significant main effects of time [
The R2 lin was determined for each child individually. The GLM indicated that there was no main effect of time [
Although there were no significant differences between the groups for age, we have to consider the large variation in age. Therefore, we re-analyzed the data using age as a covariate in the GLM. However, the results demonstrated that there were no substantial changes in the results of any group × time interaction.
Descriptive analyses of the feedback questionnaire demonstrated that the training was well received (
Several studies demonstrate that a notable proportion of children show insufficient basic knowledge of mathematics, which is predictive for further difficulties in learning mathematics (Jordan et al.,
The results are promising and showed significant improvements in half of the analyzed measures. This is in line with the results of a pilot study with a smaller sample (
The results demonstrated no effects with regard to arithmetic performance measures for addition. To explain this finding, the hierarchical structure of Calcularis has to be considered. The training of addition/subtraction is carried out in ascending number ranges starting with the low number range 0–10. The next higher number range (0–10, 0–20, 0–100 etc.) is not unblocked before the child demonstrate arithmetic competencies (addition/subtraction) to a specific probability. Since the pre-test raw scores demonstrated that children performed better in addition than in subtraction, that the program provided in its adaptive design more training in subtraction leading to larger effects in subtraction than in addition.
Regarding spatial number processing two number line tasks with different number ranges (0–10, 0–100) were assessed to get more differentiating information of the improvements of spatial number representation since the program starts the training of mental number line tasks within the number range 0–10 and proceeds then to the number range 0–100. The Calcularis group showed stronger improvements in PAE and
This result is especially relevant as the formation of a mental number line constitutes a vital step in the numerical development (von Aster and Shalev,
Since studies that evaluate computerized training programs to enhance arithmetic performance or spatial number representation differ highly with regard to study samples (e.g., at risk learners or dyscalculic children) and targeted outcome measures, only a very limited amount of studies is available for adequate comparisons. Training studies demonstrating a high degree of comparability to our study, such as Lenhard et al. (
Compared to the spelling training group, the Calcularis group demonstrated stronger improvements in subtraction. In contrast to the findings of the Calcularis group compared to the waiting group, effect sizes are smaller. With regard to number line representation, children of the Calcularis group demonstrated improvements within the number range 0–10 (PAE,
Fuchs et al. (
We expected smaller effect sizes for the comparison of the Calcularis with the spelling training group than for the comparison of the Calcularis with the waiting group. We assume that different cognitive as well as affective factors might be influenced leading to an improvement in the arithmetical outcome measures in both training groups (Calcularis and spelling training). However, the results indicated that children of the spelling training group did not increase more than the waiting group in almost all tasks, with the exception of the computer test subtraction. Nevertheless, we assume some relevant factors influencing training outcomes. First, the daily computerized training might have an effect on attention or working memory capacities. Therefore, the program influences superordinate cognitive functions that are crucial for information processing (e.g., learning, fact retrieval, problem solving), which might also be reflected in improved arithmetic performance. Second, the daily practice with the program might have an effect on affective variables (e.g., attitude, anxiety, self-concept), which might also contribute to the children's enhanced performance. There is extensive research demonstrating the relevance of attitude toward school subjects and the related academic achievement in the respective subjects (Ma and Kishor,
In summary, to obtain a more complete picture of the single mechanisms of actions underlying an efficient computerized training, domain-specific and domain-general aspects of number processing and arithmetic as well as affective variables need to be taken into consideration (Kaufmann and von Aster,
Furthermore, when interpreting the results it has to be considered that there could be additional factors influencing the observed progression in the three groups that were not within the scope of this study. Conceivable are developmental factors that promote for instance a non-linear progression of development. These developmental changes may be not found immediately after the training, but later on.
The feedback questionnaire of the children and their parents revealed positive results, which correspond to the experiences and impressions of the study team during the training supervision sessions. This is especially significant as Calcularis is a learning system that is not embedded in a story game and has no reward system implemented. The majority of the children rated the general difficulty level of the training as appropriate. This serves as an indicator for the quality of the adaptation to the individual performance level and profile, the learning speed and the maintenance of the children's zone of proximal development (Vygotsky,
Several important limitations of this study should be noted. First, Calcularis combines the training of basic numerical skills, spatial number representations and arithmetic operations. The evaluative data revealed good effects for subtraction and spatial representation of numbers. However, basic numerical skills (e.g., number/quantity comparisons, subitizing) were not assessed. Therefore, training effects with regard to basic numerical skills could not be determined. In addition, there was no assessment of the children's performance in multiplication and division which would have served as a measures for training transfer. Second, the discriminate training transfer could not be evaluated since we did not assess non-numerical measures (e.g., reading, spelling) after the training. Third, the reported results respond to immediate efficacy. Therefore, the present results do not allow for conclusions about increased mathematical competencies or transfer to other arithmetic performances in the long-term. Fourth, in the present study more than two thirds of the study population were girls, but gender ratio deviated not significantly over the groups. Participants were recruited through advertisements distributed to professionals and schools. Therefore, the offer of a computerized training program for enhancing numerical cognition showed to be especially attractive to girls. Future research with larger samples and equal gender ratios is needed for more differentiating analyses of possible effects of gender. Following this first evaluation stage of Calcularis, a further evaluative study is currently conducted that includes the implementation of a control group receiving conventional integrative learning therapy for children with developmental dyscalculia. Thus, we aim to understand the underlying processes and mechanisms of action of a computerized training program compared to non-computerized learning approaches.
This study demonstrates that the adaptive training program Calcularis can be used effectively to support children in their numerical development and to enhance subtraction and spatial number representation. While other computerized training programs revealed good training effects for arithmetic performance (e.g., Fischer et al.,
GE, KK, MV, and JK conceived the study. GE, KK, MV, JK, UM, VM, and TK designed the study. JK, LR, VM, KK, TK, and UM performed research. JK, LR, and MV analyzed the data. JK and LR wrote the manuscript.
This work was supported by research grant 01GJ1011 from the German Federal Ministry of Education and Research (BMBF).
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The authors wish to thank the children and their parents for participating. We also would like to thank the student assistants that helped to collect the study data.