Skip to main content

ORIGINAL RESEARCH article

Front. Psychol., 24 February 2017
Sec. Quantitative Psychology and Measurement

Evaluation of Mother-Child Agreement and Factorial Structures of the SCARED Questionnaire in an Italian Clinical Sample

\r\nSimona Scaini,*Simona Scaini1,2*Anna Ogliari,Anna Ogliari2,3Ludovica De CarolisLudovica De Carolis4Laura Bellodi,Laura Bellodi3,4Clelia Di Serio,Clelia Di Serio4,5Chiara Brombin,Chiara Brombin4,5
  • 1Faculty of Psychology, Sigmund Freud University, Milan, Italy
  • 2Developmental Psychopathology Unit, Vita-Salute San Raffaele University, Milan, Italy
  • 3Department of Clinical Neuroscience, San Raffaele Scientific Institute, Milan, Italy
  • 4Faculty of Psychology, Vita-Salute San Raffaele University, Milan, Italy
  • 5University Centre of Statistics in the Biomedical Sciences, Vita-Salute San Raffaele University, Milan, Italy

Background: A great part of the literature has confirmed the importance of both child and parents reports as source of factual information, especially for childhood emotional syndromes. In our study we aimed at: (i) calculating mother-child agreement and (ii) evaluating factorial structure of the Screen for Child Anxiety Related Emotional Disorders (SCARED) questionnaire in an Italian clinical sample. The novelty of this contribution is two-fold: first, from a clinical point of view, we investigated the parent-child agreement level and examined separately the factorial structures of both parent and child versions of the SCARED for the first time in an Italian clinical sample. Second, unlike previous studies, we used statistical approaches specifically suited to account for the ordinal nature of the collected variables.

Method: In a clinical sample of 171 children and adolescents aged 8–18 and their mothers we evaluated inter-rater agreement using weighted kappa indices to assess agreement for each item belonging to a certain SCARED subscale. Exploratory factor analysis for ordinal data was then performed on the polychoric correlation matrix calculated on SCARED items. Differences in the numbers of symptoms reported by children and parents were evaluated as well.

Results and Conclusions: Our results reveal moderate to strong mother-child agreement. A significant age effect is present. Two different factorial solutions emerged for parent and child SCARED versions (a 5 factor structure for parents and a 6 factor solution in the child version, including a new factor “Worry about Parents”). This study confirmed the importance of evaluating both child and parent reports in assessment protocols for anxiety disorders. Our findings could help clinicians to determine which information, and from which rater, must be accounted for in evaluating treatment decisions. Moreover, we find that patients characteristics, such as gender and age, should be taken into account when assessing agreement.

Introduction

Anxiety disorders are the most common psychiatric illnesses in both adulthood and childhood (Beesdo-Baum and Knappe, 2012; Kessler et al., 2012). Anxiety symptoms tend to begin early in life and are likely to become chronic and persistent (Beesdo et al., 2009). Thus, early recognition and treatment are desirable (Beidel and Turner, 2007; Davis et al., 2011; Scaini et al., 2016). However, since anxious children do not manifest behavioral difficulties as clearly as children with externalizing disorders, they often go unrecognized and underdiagnosed (Costello et al., 2005; Beesdo et al., 2009).

In light of these considerations, it is important to find new adequate and validated instruments to screen the severity of anxiety symptoms in childhood. In addition to structured psychiatric interviews, which are specific and detailed, questionnaires may be used as flexible, easy to administer, less costly, and less time consuming first screening instruments (Verhulst and Van der Ende, 2006; Garcia-Lopez et al., 2015). Several anxiety self-report questionnaires have been developed to assess anxiety symptoms in both clinical and general population. In a review on anxiety rating scales, Myers and Winters (2002) reported that the Screen for Child Anxiety Related Emotional Disorders (SCARED) is one of the best self-report instrument to evaluate anxiety symptoms. Moreover, a meta-analysis by Hale et al. (2011) suggested that the SCARED presents good psychometric properties and can be used as a cross-cultural screening instrument. The SCARED offers both self- and a parent-report version to assess childhood anxiety disorders by including subscales on General Anxiety, Separation Anxiety, Somatic/Panic, Social Anxiety, and School Phobia. Several reports have already investigated psychometric properties in community and clinical populations reporting good internal consistency, test-retest reliability, and discriminant validity (Birmaher et al., 1997, 1999; Ogliari et al., 2006; Crocetti et al., 2009; Hale et al., 2013).

A great part of the literature has confirmed the importance of both child and parent report as sources of factual information, especially for childhood emotional syndromes. Indeed, when using rating scales, it is crucial that clinicians collect information from different sources. The information provided by several raters, which see children in different contexts, is important to obtain a comprehensive picture of the child's functioning (Verhulst and Van der Ende, 2006). As reported by Verhulst and Van der Ende (2006) parents are usually familiar with their child's functioning in many situations and throughout time. However, children and adolescents' self-reports are also indispensable, especially to have a better overview of internalizing problems that are more difficult to detect (Costello et al., 2005; Beesdo et al., 2009).

Previous studies using different methods and measures indicated low levels of agreement among informants, showing higher scores for anxiety, and depressive symptoms in child report than in parent report (De Los Reyes and Kazdin, 2005; Salbach-Andrae et al., 2009).

Besides focusing on low agreement levels as a nuisance source (Verhulst and Van der Ende, 2006), in existing works it has been reported the importance of multiple informants as a potentially valid source of information (Kraemer et al., 2003). Self-reports filled in by parents and children are now regarded as being of equal weight in the diagnostic process (Angold, 2002). Several reports showed that parents perception of anxiety may be influenced by cultural factors (Gaines et al., 1997; Murphy et al., 2003; Wren et al., 2007).

Until now, the agreement of the Italian child and parent versions of SCARED has not been examined. This lack calls for a systematic analysis of parent-child agreement in the Italian culture. Moreover, the possible effect of sex and age on agreement was investigated. Better knowledge of the impact of these factors could help in shaping more effective assessment protocols and treatments. For instance, information on sex and age could encourage clinicians in considering also these variables in their clinical decisions (Rutter et al., 2008).

To the best of our knowledge, no previous studies have analyzed parent-child agreement using indices suited for ordinal variables such as κ index. Moreover, the level of agreement (taking into account age and sex effect) for each item within each SCARED subscale has not been addressed in the previous literature so far.

Indeed, all recent studies that have evaluated the psychometric properties of the SCARED questionnaire (e.g., Wren et al., 2007; Su et al., 2008; Jastrowski Mano et al., 2012) in exploratory fashion, applied a principal component analysis without specifying on which correlation matrix (Pearson, Spearman, or polychoric correlation matrix) was the analysis carried out. In the common practice, ordinal variables with at least five categories are treated as continuous variables and analyzed with a traditional factor analysis procedure, thus ignoring the ordinal nature of the outcomes. Treating nominal and ordinal data as interval or ratio data, may lead to unreliable parameter estimates as well as to poor goodness of fit, especially in presence of highly skewed non Gaussian data or polarized/bimodal distributions (Muthén and Kaplan, 1985).

As emphasized in Holgado-Tello et al. (2010) in presence of ordinal data, factor analysis should be carried out on the matrix of polychoric correlations rather than on Pearson correlation matrix since analyses based on the former produce a more accurate reproduction of the original correlation structure.

Motivated by the outlined importance of accounting for different sources of information in screening for anxiety disorders, we investigate the parent-child agreement level and examined separately the factorial structures of both parent and child versions of the SCARED. For the first time in the literature we have carried out the analysis in an Italian clinical sample.

In particular, we analyze the cross-informant agreement levels for each questionnaire item and, considering age and sex effect, for each factor-based scales. We, then inspect the differences in the number of symptoms reported by children and parents. Lastly, we examine separately the factor structures of parent- and self-report versions of SCARED questionnaire.

Materials and Methods

Participants and Procedure

Two hundred and ten children and adolescents (54% boys and 46% girls) who were referred to an outpatient treatment center of Development and Psychopathology Unit at San Raffaele Hospital filled in the SCARED. Mean age of the sample was 12.0 years (SD 2.9, range 7–17 years). Most children (>90%) were Caucasian. Children and mothers completed the SCARED in the lab after having received their first neuropsychiatric visit. During the questionnaire compilation, a clinical psychologist remains available to answer questions. Then, the Kiddie-Sads-Present and Lifetime Version (K-SADS-PL, Kaufman et al., 1997) was administered by clinical psychologists to diagnose psychiatric disorder. None of the children involved in this study met the full criteria for a current or past diagnosis of psychiatric or neurological disorder. However, clinical assessment revealed the presence of subclinical psychiatric symptoms in the sample (27% of the children manifested symptoms of Affective Disorders, 15% of Anxiety Disorders, 16% of Attention Deficit/Hyperactivity Disorder, 14% of Oppositional Defiant Disorder, and 16% of Conduct Disorder).

The study was conducted according to the Declaration of Helsinki. Data were collected during clinical assessment and parents gave their informed consent to use, anonymously, data for research purposes. According to the hospital's ethics committee guidelines, this type of study did not require an approval by the institutional review board. The head physician of the Development and Psychopathology Unit at San Raffaele Hospital and the dean of the Faculty of Psychology at San Raffaele University examined and approved the study.

Measures

Screen for Child Anxiety Related Emotional Disorders (SCARED)

Children and their mothers filled in the Italian version of the 41-item SCARED questionnaire (Ogliari et al., 2006). The SCARED questionnaire was originally devised to screen Anxiety Disorders (AD) in clinical samples (Birmaher et al., 1997, 1999; Ogliari et al., 2006; Crocetti et al., 2009), but it has been also employed as valuable screening tool in community samples (e.g., Boyd et al., 2003; Crocetti et al., 2009). Children were asked to rate the frequency with which they experienced each symptom on a 3-point likert scale (0 = “almost never,” 1 = “sometimes,” 2 = “often”). The questionnaire was completed also by their mothers who were asked to rate the frequency with which their children experienced each symptom on the same scale. Via principal component factor analysis, Birmaher et al. (1997), identified five subscales, i.e., Panic/Somatic Anxiety (PD), General Anxiety (GAD), Separation Anxiety (SAD), Social Phobia (SOC), and School Phobia (SCH). With reference to the psychometric properties, the tool showed good internal consistency, test-rest reliability, discriminative validity. The authors reported moderate parent-child agreement (ρ = 0.20–0.47).

When a cut-off point of 25 was applied to the total score endorsed by subjects across these five factors, data showed good sensitivity (70%) and good ability to discriminate between children with AD versus those without AD (specificity: 67%), and between children with AD versus those with depression, or disruptive disorders: 61 and 71%, respectively; (Birmaher et al., 1999; Monga et al., 2000). Finally, in a meta-analytic study, Hale et al. (2011) found that internal consistency for the total score was good (α = 0.89–0.91) although substantial variation was found in the internal consistency of the subscales (α = 0.43–0.93).

Statistical Analyses

We used a weighted kappa index with “squared” weights (Cohen, 1960; Fleiss et al., 2013) to evaluate chance-corrected mother-child agreement for each item belonging to a certain subscale. The weighted version of this index has been introduced by Cohen (1968) as a method to deal with ordinal data.

While traditional kappa does not distinguish between disagreements, treating them on an equal footing (Banerjee et al., 1999), there are circumstances (e.g., when making diagnosis) where some disagreements have more severe implications than others. Hence, a weighted version should be preferred in such cases. By choosing the “squared” weight option, disagreements are weighted according to their squared distance from perfect agreement.

Spearman correlation coefficients were calculated to examine mother-child agreement on the five SCARED subscales.

Exploratory factor analysis for ordinal data was performed on polychoric correlation matrices calculated on the SCARED items (Gilley and Uhlig, 1993; Jöreskog and Moustaki, 2001). Ordinary Least Squares (OLS) procedure that minimizes the residual (off-diagonal) correlation matrix (minres algorithm; Harman and Jones, 1966) was applied to extract factors.

Parent and child versions were analyzed separately to uncover potential differences among emerging factorial structures.

In order to select the number of factors to retain in the factorial model, several statistical criteria have been considered. In particular, we focused on Horn's Parallel Analysis (Horn, 1965), Velicer's Minimum Average Partial criterion (MAP; Velicer, 1976), the Very Simple Structure criterion (VSS; Revelle and Rocklin, 1979) as well as the empirically derived Bayesian Information Criterion (eBIC).

Goodness of model fit has been examined through the Root Mean Square of the Residuals (RMSR) that evaluates the difference between the observed correlation matrix and the estimated one as reproduced by the factorial model. The choice of the proposed factorial solution was led by considerations on the total amount of variance explained, the number of variables that load on each factor and parsimony of the representation.

Analyses were performed using R Statistical Software (R Development Core Team, 2014), version 3.3.1. In particular, psych (Revelle, 2014) and irr (Gamer et al., 2012) packages were used to implement factor analyses on polychoric correlation matrices and to calculate coefficients of inter-rater agreement, respectively.

Results

Evaluation of Inter-Rater Agreement

Mother-child agreement on SCARED subscales was assessed by computing Spearman bivariate correlations between subscale scores (Table 1). Correlation strength has been interpreted according to Bartz (1999).

TABLE 1
www.frontiersin.org

Table 1. Mean scores for each SCARED subscale and for each perspective along with mother-child agreement assessed through Spearman correlations.

Overall (i.e., considering all subjects regardless of children age, after pairwise deletion, in a sample of n = 133) all correlations were from moderate to strong (r ranges from 0.48 to 0.67, PD scale r = 0.56, GAD scale r = 0.48, SAD scale r = 0.67, SOC scale r = 0.66, SCH scale r = 0.63, all statistically significant).

In order to assess agreement invariance across ages, four age groups have been defined (8–10 yrs, n= 66; 11–13 yrs, n = 62; 14–15 yrs, n = 40; 16–18 yrs, n = 41, 1 missing data) representing different school grades and stages of development. All correlations related to the first group (8–10 yrs, n = 40 after pairwise deletion), ranged from moderate to strong (r ranges from 0.41 to 0.71, PD scale r = 0.48, GAD scale r = 0.49, SAD scale r = 0.61, SOC scale r = 0.41, SCH scale r = 0.71, all significant).

In the second group (11–13 yrs, n = 40 after pairwise deletion), the mother-child agreement r ranges from 0.41 to 0.79 (PD scale r = 0.55, GAD scale r = 0.41, SAD scale r = 0.62, SOC scale r = 0.79, SCH scale r = 0.55, all significant).

The third group of students, aged between 14 and 15 years (n = 28 after pairwise deletion), shows correlations varying from moderate to strong range (r ranges from 0.51 to 0.71, PD scale r = 0.52, GAD scale r = 0.61, SAD scale r = 0.51, SOC scale r = 0.71, SCH scale r = 0.58, all significant).

Similar correlations were found in the last group, including adolescents aged between 16 and 18 years (n = 25 after pairwise deletion) with r ranging from 0.57 to 0.75 (PD scale r = 0.60, GAD scale r = 0.57, SAD scale r = 0.75, SOC scale r = 0.69, SCH scale r = 0.67, all significant).

A weighted kappa index with squared weights was calculated to evaluate mother-child agreement for each item belonging to a certain subscale (see Figure 1). κ values for the items of the PD scale range between 0.11 and 0.61 (from slight to substantial), values for items in the GAD scale range between 0.21 and 0.45 (from slight to moderate), for items included in the SAD scale agreement indices range between 0.38 and 0.72 (from fair to substantial), for items in the SOC scale κ-values range between 0.38 and 0.57 (from fair to moderate), and finally for the SCH scale they range between 0.38 and 0.62 (from fair to substantial). It seems that there is more agreement in the separation anxiety disorder scale.

FIGURE 1
www.frontiersin.org

Figure 1. Graphical representation of Cohen's κ for mother-child agreement within each item and for each SCARED subscale.

Comparison of Factor Structure Analyses

We perform a factor analysis on polychoric correlation matrices of both parent and child version of SCARED questionnaire (Table 2).

TABLE 2
www.frontiersin.org

Table 2. SCARED items description both in English and Italian.

In the current literature, only Wren et al. (2007) evaluated differences between parent and child questionnaire structures, using separate principal component analyses and reporting a four factor solution for both versions largely replicating the original factor structure found in Birmaher et al. (1997, 1999), where the school subscale did not emerge.

In our data, parallel analysis on SCARED parent version, completed by the mothers (n = 126, complete observations), suggested a 5 factor solution (Figure 2). Velicer's MAP criterion and empirical Bayesian Information Criterion (eBIC) confirmed this solution, while VSS criteria (complexity 1, complexity 2) were more conservative, suggesting 1 or 2 factors. However, caution is adviced when interpreting VSS criterion since it is not always optimal and simulation results have shown that this criterion is not appropriate when data shows a complex factor structure (Revelle, 2014).

FIGURE 2
www.frontiersin.org

Figure 2. Diagrams representing factor structure for mother (left) and child (right) SCARED questionnaires obtained through exploratory factor analysis for categorical data.

The Root Mean Square of the Residuals (RMSR) for the 5 factor solution was 0.07. Five factors explained 62% of the variance.

For the sake of interpretability, factor loadings were orthogonally rotated using the Varimax method. Oblique rotation was not appropriate in our case since within-factors correlations were very low.

Based on the items that load on each factor, the first factor was named “Panic/Somatic Anxiety” (it includes 9 out of the 13 items that make up the PD scale), the second one was named “Social Phobia” (all the items that make up the SOC scale originally proposed in Birmaher et al., 1997), the third “Separation Anxiety” (7 out of the 8 items that construct the SAD scale), the fourth “General Anxiety” (8 out of the 9 items that construct the GAD scale) and the last “School Phobia” (all the items that make up the school phobia scale).

Parallel analysis on SCARED child version, completed by n = 147 children in total (considering complete records), suggested a 4 factor solution. Velicer's MAP criterion suggested a 3 factor solution, eBIC a 6 factor solution, while VSS criteria (complexity 1, complexity 2) suggested 1 and 4 factors, respectively.

We explored a range of solutions going from 4 to 6 factors. Actually we choose a 6 factor solution as best, explaining 60% of the total variance (Figure 2). The RMSR was 0.08. Again, orthogonal (Varimax) rotation was used.

Based on the items that load on each factor, the first factor was named “Panic/Somatic Anxiety” (it includes 12 out of the 13 items that make up the PD scale in the solution proposed by Birmaher et al., 1997), the second “Separation Anxiety” (5 out of the 8 items included in the original SAD scale proposed in Birmaher et al., 1997), the third was named “Social Phobia” (all the items that make up the SOC scale originally proposed), the fourth “General Anxiety” (5 out of the 9 items that construct the GAD scale), the fifth including the 2 items of “Separation Anxiety” subscale associated with a possible danger for parents was named “Worry about Parents” and the last was called “School Phobia” (3 out of the 4 items originally that make up the school phobia scale).

Discussion and Conclusion

Although structured interviews still represent the gold standard to determine the presence and degree of severity of a clinical disorder, the use of reliable and valid self-reported questionnaires may support clinicians in the diagnosis and may be used as screening tools for children (Alfano et al., 2002; Viana et al., 2008).

The present study represents the first contribution that evaluates cross-informant agreement of the Italian child and parent versions of SCARED. Moreover, statistical approaches specifically suited to ordinal data have been applied.

Our results confirmed hypotheses that children report significant information about anxiety symptoms and thus contribute with a different perspective to the assessment of the disorder.

Our analysis shows a moderate to strong mother-child agreement. When accounting for age, we found that the level of cross-informant agreement for the total SCARED score increased with age. This can be interpreted in the following way: as children grow up, they become more able to communicate and explain their symptomatology to parents who in turn become more aware of the child's anxiety. Our results suggested that, independently on age, the subscale with the higher rate of agreement is SAD. From a clinical perspective, this reflects the fact that SAD symptoms emerge clearly in the family since the child would often ask for help and assurances. In addition, SAD symptoms compromise family life and increase parental stress limiting the activities of siblings and parents (Fischer et al., 1999). In fact, SAD's essential feature is an excessive anxiety concerning separation of the child from its home or parents (American Psychiatric Association, 2013). Differently from other anxiety disorders, the fear of separation causes great distress to the child and may interfere with both the child's and the family's normal activities (Fischer et al., 1999). A child suffering from SAD may often show avoidance and oppositional behavioral patterns in an attempt to prevent separation, or may refuse to participate in activities requiring separation from the attachment figure (Fischer et al., 1999).

Moreover, current studies (e.g., Scaini et al., 2012) suggest that separation anxiety develops from multiple and interrelated factors that include genetic vulnerabilities and environmental factors, such as negative family experiences (Chorpita, 2001; Kearney, 2008).

Our results appear consistent with recent research (Wren et al., 2004; Su et al., 2008; Weitkamp et al., 2010; Jastrowski Mano et al., 2012) that reports agreement values higher than the ones previously reported by Birmaher et al. (1997, 1999).

Despite evidence of good cross-informant agreement in the present study, strong differences in number of reported symptoms and their severity emerged from the analyses.

When accounting for the age effect, we observe that children reported on average more symptoms of Separation Anxiety at 8–10 years and of General Anxiety at 16–18 years. Moreover, mean scores in General Anxiety scale increase with age.

The bulk of excessive symptoms reported by children was for Somatic/Panic Anxiety and General Anxiety and came from girls. These results are consistent with other reports investigating parent-child agreement (Wren et al., 2004) and with epidemiological studies that reported higher rates of anxiety symptoms for girls (Kessler et al., 2005; Ogliari et al., 2006; Beesdo et al., 2009). However, we found similar mother-child agreement when stratifying by gender. This suggests that mother-son and mother-daughter accord is similar.

Following the approach of Birmaher et al. (1997) we separately analyzed parent- and self-report versions of the SCARED.

Differently from all the recent studies that have evaluated psychometric properties of the SCARED, an exploratory factor analysis for ordinal data was carried out on polychoric correlation matrices calculated on SCARED items. This approach has been recommended in the literature to properly analyze Likert type ordinal items (Muthén and Kaplan, 1985; Gilley and Uhlig, 1993; Jöreskog and Moustaki, 2001) with factor analysis procedures. Simulation studies showed that the solutions obtained using polychoric correlations provide a more accurate reproduction of the measurement model used to generate the data (Holgado-Tello et al., 2010).

Two different factorial solutions emerged for parent and child SCARED versions. Analyses on parent version revealed a 5 factor structure that approximately reproduces the original structure proposed by Birmaher et al. (1997, 1999) and by other authors (e.g., Jastrowski Mano et al., 2012) for parent version. Based on our findings, a 6 factor solution was chosen as best in the child version, with the first 5 factors reproducing “Birmaher” factors, whereas the 6th include the two items of Separation Anxiety subscale associated with a possible danger for parents (16. I have nightmares about something bad happening to my parents/31. I worry that something bad might happen to my parents). This result suggests that, in children, anxiety concerning of “being alone” or “a possible danger for children” emerged separately from anxiety about a possible danger for parents.

A limitation of the present study concern sample size. Indeed, for factor analysis, we considered SCARED questionnaire completed by 126 mothers and 147 children. However, when analyzing age effects in the evaluation of mother-child agreement, sample size within each age category is unbalanced and some categories are not well-represented. In particular, descriptive statistics for the SCARED scales are actually based on the information provided by 49 children in the age-class category 8–10 yrs, 53 children in the age-class category 11–13 yrs, 34 children in the age-class category 14–15 yrs and 35 in the last category 16–18.

In future work, the current analysis could be enhanced by addressing the following issues.

Data on parent reports were gathered from mother alone: analyses integrating father ratings may yield more informative results thus providing greater generalizability.

Future research should focus on longitudinal study designs, thus allowing test-retest reliability evaluations.

Hence, to conclude, our findings could help clinicians to determine which information, and from which rater, must be accounted for in evaluating treatment decisions. Moreover, due to the emerging differences in reported symptoms and cross-informant agreement depending on respondent's age, clinicians should consider potential age effect in interpreting SCARED results during assessment process.

Author Contributions

SS and CB conceived of the study, participated in its design and coordination and drafted the manuscript; CB performed the statistical analysis; AO, CD, and LB participated in the design, coordination of the study and helped to draft the manuscript; LD contributed to literature collection and helped with data analysis. All authors read and approved the final manuscript.

Funding

This research has been funded by the FIRB Project RBFR12VHR7 entitled “Interpreting emotions: a computational tool integrating facial expressions and biosignals based on shape analysis and Bayesian networks.”

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Alfano, C. A., Beidel, D. C., and Turner, S. M. (2002). Cognition in childhood anxiety: conceptual, methodological, and developmental issues. Clin. Psychol. Rev. 22, 1209–1238. doi: 10.1016/S0272-7358(02)00205-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Angold, A. (2002). “Diagnostic interviews with parents and children,” in Child and Adolescent Psychiatry, 4th Edn eds M. Rutter and E. Taylor (Oxford: Blackwell Publishing), 32–51.

American Psychiatric Association (2013). “Diagnostic and statistical manual for mental disorders (DSM-5),” 5th Edn. (Arlington, VA: American Psychiatric Publishing).

Banerjee, M., Capozzoli, M., McSweeney, L., and Sinha, D. (1999). Beyond kappa: a review of interrater agreement measures. Can.j. Stat. 27, 3–23. doi: 10.2307/3315487

CrossRef Full Text | Google Scholar

Bartz, A. E. (1999). Basic Statistical Concepts, 4th Edn, Upper Saddle River, NJ: Prentice Hall.

Beesdo, K., Knappe, S., and Pine, D. S. (2009). Anxiety and anxiety disorders in children and adolescents: developmental issues and implications for DSM-V. Psychiatr. Clin. North Am. 32, 483–524. doi: 10.1016/j.psc.2009.06.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Beesdo-Baum, K., and Knappe, S. (2012). Developmental epidemiology of anxiety disorders. Child Adolesc. Psychiatr. Clin. N. Am. 21, 457–478. doi: 10.1016/j.chc.2012.05.001

PubMed Abstract | CrossRef Full Text

Beidel, D. C., and Turner, S. M. (2007). Shy Children, Phobic Adults: Nature and Treatment of Social Phobia, 2nd Edn, Washington, DC: American Psychological Association.

Google Scholar

Birmaher, B., Brent, D. A., Chiappetta, L., Bridge, J., Monga, S., and Baugher, M. (1999). Psychometric properties of the screen for child anxiety related emotional disorders (SCARED): a replication study. J. Am. Acad. Child Adolesc. Psychiatry 38, 1230–1236. doi: 10.1097/00004583-199910000-00011

PubMed Abstract | CrossRef Full Text | Google Scholar

Birmaher, B., Khetarpal, S., Brent, D., Cully, M., Balach, L., Kaufman, J., et al. (1997). The screen for child anxiety related emotional disorders (SCARED): scale construction and psychometric characteristics. J. Am. Acad. Child Adolesc. Psychiatry 36, 545–553. doi: 10.1097/00004583-199704000-00018

PubMed Abstract | CrossRef Full Text | Google Scholar

Boyd, R. C., Ginsburg, G. S., Lambert, S. F., Cooley, M. R., and Campbell, K. D. (2003). Screen for child anxiety related emotional disorders (SCARED): psychometric properties in an African-American parochial high school sample. J. Am. Acad. Child Adolesc. Psychiatry, 42, 1188–1196. doi: 10.1097/00004583-200310000-00009

PubMed Abstract | CrossRef Full Text | Google Scholar

Chorpita, B. F. (2001). “Control and the development of negative emotion,” in The Developmental Psychopathology of Anxiety, eds M. W. Vasey and M. R. Dadds (Oxford: Oxford University Press), 112–142. doi: 10.1093/med:psych/9780195123630.003.0006

CrossRef Full Text

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educ. Psychosoc. Meas. 20, 37–46. doi: 10.1177/001316446002000104

CrossRef Full Text | Google Scholar

Cohen, J. (1968). Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychol. Bull. 70:213.

Google Scholar

Costello, E. J., Egger, H. L., and Angold, A. (2005). The developmental epidemiology of anxiety disorders: phenomenology, prevalence, and comorbidity. Child Adolesc. Psychiatr. Clin. North Am. 14, 631–648. doi: 10.1016/j.chc.2005.06.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Crocetti, E., Hale, W. W. III., Fermani, A., Raaijmakers, Q., and Meeus, W. (2009). Psychometric properties of the screen for child anxiety related emotional disorders (SCARED) in the general Italian adolescent population: a validation and a comparison between Italy and the Netherlands. J. Anxiety Disord. 23, 824–829. doi: 10.1016/j.janxdis.2009.04.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Davis, T. E. III., May, A., and Whiting, S. E. (2011). Evidence-based treatment of anxiety and phobia in children and adolescents: current status and effects on the emotional response. Clin. Psychol. Rev. 31, 592–602. doi: 10.1016/j.cpr.2011.01.001

PubMed Abstract | CrossRef Full Text | Google Scholar

De Los Reyes, A., and Kazdin, A. E. (2005). Informant discrepancies in the assessment of childhood psychopathology: a critical review, theoretical framework, and recommendations for further study. Psychol. Bull. 131, 483–509. doi: 10.1037/0033-2909.131.4.483

PubMed Abstract | CrossRef Full Text | Google Scholar

Fischer, D. J., Himle, J. A., and Thyer, B. A. (1999). “Separation anxiety disorder,” in Handbook of Prescriptive Treatments for Children and Adolescents, 2nd Edn, eds R. T. Ammerman, M. M. Hersen, and C. Last (Boston, MA: Allyn & Bacon), 141–154.

Fleiss, J. L., Levin, B., and Paik, M. C. (2013). Statistical Methods for Rates and Proportions. New York, NY: John Wiley & Sons.

Google Scholar

Gaines, S. O. Jr., Marelich, W. D., Bledsoe, K. L., Steers, W. N., Henderson, M. C., et al. (1997). Links between race/ethnicity and cultural values as mediated by racial/ethnic identity and moderated by gender. J. Pers. Soc. Psychol. 72, 1460–1476. doi: 10.1037/0022-3514.72.6.1460

PubMed Abstract | CrossRef Full Text | Google Scholar

Gamer, M., Lemon, J., Fellows, I., and Singh, P. (2012) irr: Various Coefficients of Interrater Reliability Agreement. R package version 0.84. Available online at: http://CRAN.R-project.org/package=irr

Garcia-Lopez, L. J., Saez-Castillo, A. J., Beidel, D., and La Greca, A. M. (2015). Brief measures to screen for social anxiety in adolescents. J. Dev. Behav. Pediatr. 36, 562–568. doi: 10.1097/DBP.0000000000000213

PubMed Abstract | CrossRef Full Text | Google Scholar

Gilley, W. F., and Uhlig, G. E. (1993). Factor analysis and ordinal data. Education 114, 258–265.

Google Scholar

Hale, W. W. III., Crocetti, E., Raaijmakers, Q. A., and Meeus, W. H. (2011). A meta-analysis of the cross-cultural psychometric properties of the screen for child anxiety related emotional disorders (SCARED). J. Child Psychol. Psychiatry 52, 80–90. doi: 10.1111/j.1469-7610.2010.02285.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Hale, W. W., Raaijmakers, Q. A., García-López, L. J., Espinosa-Fernández, L., Muela, J. A., and del Mar Díaz-Castela, M. (2013). Psychometric properties of the screen for child anxiety related emotional disorders for socially anxious and healthy Spanish adolescents. Span. J. Psychol. 16:E25. doi: 10.1017/sjp.2013.36

PubMed Abstract | CrossRef Full Text | Google Scholar

Harman, H. H., and Jones, W. H. (1966). Factor analysis by minimizing residuals (minres). Psychometrika 31, 351–368. doi: 10.1007/BF02289468

PubMed Abstract | CrossRef Full Text | Google Scholar

Holgado-Tello, F. P., Chacón-Moscoso, S., Barbero-García, I., and Vila-Abad, E. (2010). Polychoric versus pearson correlations in exploratory and confirmatory factor analysis of ordinal variables. Qual. Quant. 44, 153–166. doi: 10.1007/s11135-008-9190-y

CrossRef Full Text | Google Scholar

Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika 30, 179–185. doi: 10.1007/BF02289447

PubMed Abstract | CrossRef Full Text | Google Scholar

Jastrowski Mano, K. E., Evans, J. R., Tran, S. T., Anderson Khan, K., Weisman, S. J., and Hainsworth, K. R. (2012). The psychometric properties of the screen for child anxiety related emotional disorders in pediatric chronic pain. J. Pediatr. Psychol. 37, 999–1011. doi: 10.1093/jpepsy/jss069

PubMed Abstract | CrossRef Full Text | Google Scholar

Jöreskog, K. G., and Moustaki, I. (2001). Factor analysis of ordinal variables: a comparison of three approaches. Multivariate Behav. Res. 36, 347–387. doi: 10.1207/S15327906347-387

PubMed Abstract | CrossRef Full Text | Google Scholar

Kaufman, J., Birmaher, B., Brent, D., Rao, U., Flynn, C., Moreci, P., et al. (1997). Schedule for affective disorders and schizophrenia for school-age children-present and lifetime version (K-SADS-PL): initial reliability and validity data. J. Am. Acad. Child Adolesc. Psychiatry 36, 980–988. doi: 10.1097/00004583-199707000-00021

PubMed Abstract | CrossRef Full Text | Google Scholar

Kearney, C. A. (2008). An interdisciplinary model of school absenteeism in youth to inform professional practice and public policy. Educ. Psychol. Rev. 20, 257–282. doi: 10.1007/s10648-008-9078-3

CrossRef Full Text | Google Scholar

Kessler, R. C., Avenevoli, S., Costello, E. J., Georgiades, K., Green, J. G., Gruber, M. J., et al. (2012). Prevalence, persistence, and sociodemographic correlates of DSM-IV disorders in the national comorbidity survey replication adolescent supplement. Arch. Gen. Psychiatry 69, 372–380. doi: 10.1001/archgenpsychiatry.2011.160

PubMed Abstract | CrossRef Full Text | Google Scholar

Kessler, R. C., Berglund, P., Demler, O., Jin, R., Merikangas, K. R., and Walters, E. E. (2005). Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the national comorbidity survey replication. Arch. Gen. Psychiatry 62, 593–602. doi: 10.1001/archpsyc.62.6.593

PubMed Abstract | CrossRef Full Text | Google Scholar

Kraemer, H. C., Measelle, J. R., Ablow, J. C., Essex, M. J., Boyce, W. T., and Kupfer, D. J. (2003). A new approach to integrating data from multiple informants in psychiatric assessment and research: mixing and matching contexts and perspectives. Am. J. Psychiatry 160, 1566–1577. doi: 10.1176/appi.ajp.160.9.1566

PubMed Abstract | CrossRef Full Text | Google Scholar

Monga, S., Birmaher, B., Chiappetta, L., Brent, D., Kaufman, J., Bridge, J., et al. (2000). Screen for child anxiety-related emotional disorders (SCARED): convergent and divergent validity. Depress. Anxiety 12, 85–91. doi: 10.1002/1520-6394(2000)12:2<85::AID-DA4>3.0.CO;2-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Murphy, F. C., Nimmo-Smith, I., and Lawrence, A. D. (2003). Functional neuroanatomy of emotions: a meta-analysis. Cogn. Affect. Behav. Neurosci. 3, 207–233. doi: 10.3758/CABN.3.3.207

PubMed Abstract | CrossRef Full Text | Google Scholar

Muthén, B., and Kaplan, D. (1985). A comparison of some methodologies for the factor analysis of non-normal likert variables. Br. J. Math. Stat. Psychol. 38, 171–189. doi: 10.1111/j.2044-8317.1985.tb00832.x

CrossRef Full Text | Google Scholar

Myers, K., and Winters, N. C. (2002). Ten-year review of rating scales. II: scales for internalizing disorders. J. Am. Acad. Child Adolesc. Psychiatry 41, 634–659. doi: 10.1097/00004583-200206000-00004

PubMed Abstract | CrossRef Full Text | Google Scholar

Ogliari, A., Citterio, A., Zanoni, A., Fagnani, C., Patriarca, V., Cirrincione, R., et al. (2006). Genetic and environmental influences on anxiety dimensions in Italian twins evaluated with the SCARED questionnaire. J. Anxiety Disord. 20, 760–777. doi: 10.1016/j.janxdis.2005.11.004

PubMed Abstract | CrossRef Full Text | Google Scholar

R Development Core Team (2014) R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing. Available online at: http://www.R-project.org/

Revelle, W., and Rocklin, T. (1979). Very simple structure: an alternative procedure for estimating the optimal number of interpretable factors. Multivariate Behav. Res. 14, 403–414. doi: 10.1207/s15327906mbr1404_2

PubMed Abstract | CrossRef Full Text | Google Scholar

Revelle, W. (2014). psych: Procedures for Psychological, Psychometric, and Personality Research. Evanston, IL: Northwestern University, 165.

Google Scholar

Rutter, M., Bishop, D., Pine, D., Scott, S., Stevenson, J., Taylor, E., et al. (2008). Rutter's Child and Adolescent Psychiatry, 5th Edn, Oxford, UK: Blackwell.

Salbach-Andrae, H., Lenz, K., and Lehmkuhl, U. (2009). Patterns of agreement among parent, teacher and youth ratings in a referred sample. Eur. Psychiatry 24, 345–351. doi: 10.1016/j.eurpsy.2008.07.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Scaini, S., Belotti, R., Ogliari, A., and Battaglia, M. (2016). A comprehensive meta-analysis of cognitive-behavioral interventions for social anxiety disorder in children and adolescents. J. Anxiety Disord. 42, 105–112. doi: 10.1016/j.janxdis.2016.05.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Scaini, S., Ogliari, A., Eley, T. C., Zavos, H. M., and Battaglia, M. (2012). Genetic and environmental contributions to separation anxiety: a meta-analytic approach to twin data. Depress. Anxiety 29, 754–761. doi: 10.1002/da.21941

PubMed Abstract | CrossRef Full Text | Google Scholar

Su, L., Wang, K., Fan, F., Su, Y., and Gao, X. (2008). Reliability and validity of the screen for child anxiety related emotional disorders (SCARED) in Chinese children. J. Anxiety Disord. 22, 612–621. doi: 10.1016/j.janxdis.2007.05.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Velicer, W. F. (1976). Determining the number of components from the matrix of partial correlations. Psychometrika 41, 321–327. doi: 10.1007/BF02293557

CrossRef Full Text | Google Scholar

Verhulst, F. C., and Van der Ende, J. (2006). Assessment Scales in Child and Adolescent Psychiatry. Oxon, UK: Informa Healthcare.

Google Scholar

Viana, A. G., Rabian, B., and Beidel, D. C. (2008). Self-report measures in the study of comorbidity in children and adolescents with social phobia: research and clinical utility. J. Anxiety Disord. 22, 781–792. doi: 10.1016/j.janxdis.2007.08.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Weitkamp, K., Romer, G., Rosenthal, S., Wiegand-Grefe, S., and Daniels, J. (2010). German screen for child anxiety related emotional disorders (SCARED): reliability, validity, and cross-informant agreement in a clinical sample. Child Adolesc. Psychiatry Ment. Health 4:19. doi: 10.1186/1753-2000-4-19

PubMed Abstract | CrossRef Full Text | Google Scholar

Wren, F. J., Berg, E. A., Heiden, L. A., Kinnamon, C. J., Ohlson, L. A., Bridge, J. A., et al. (2007). Childhood anxiety in a diverse primary care population: parent-child reports, ethnicity and SCARED factor structure. J. Am. Acad. Child Adolesc. Psychiatry 46, 332–340. doi: 10.1097/chi.0b013e31802f1267

PubMed Abstract | CrossRef Full Text | Google Scholar

Wren, F. J., Bridge, J. A., and Birmaher, B. (2004). Screening for childhood anxiety symptoms in primary care: integrating child and parent reports. J. Am. Acad. Child Adolesc. Psychiatry 43, 1364–1371. doi: 10.1097/01.chi.0000138350.60487.d3

PubMed Abstract | CrossRef Full Text | Google Scholar

Keywords: anxiety, assessment, factor analysis, internalizing disorders, rating scales

Citation: Scaini S, Ogliari A, De Carolis L, Bellodi L, Di Serio C and Brombin C (2017) Evaluation of Mother-Child Agreement and Factorial Structures of the SCARED Questionnaire in an Italian Clinical Sample. Front. Psychol. 8:242. doi: 10.3389/fpsyg.2017.00242

Received: 19 October 2016; Accepted: 07 February 2017;
Published: 24 February 2017.

Edited by:

Pietro Cipresso, IRCCS Istituto Auxologico Italiano, Italy

Reviewed by:

Rajshekhar Bipeta, Gandhi Medical College and Hospital, India
Agnes Von Wyl, Zurich University of Applied Sciences, Switzerland

Copyright © 2017 Scaini, Ogliari, De Carolis, Bellodi, Di Serio and Brombin. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Simona Scaini, s.scaini@milano-sfu.it

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.