Using bifactor exploratory structural equation modeling to examine global and specific factors in measures of sports coaches' interpersonal styles

Stenling, Andreas; Ivarsson, Andreas; Hassmén, Peter; Lindwall, Magnus

doi:10.3389/fpsyg.2015.01303

ORIGINAL RESEARCH article

Front. Psychol., 01 September 2015

Sec. Quantitative Psychology and Measurement

Volume 6 - 2015 | https://doi.org/10.3389/fpsyg.2015.01303

Using bifactor exploratory structural equation modeling to examine global and specific factors in measures of sports coaches' interpersonal styles

$\r\nAndreas Stenling*$ Andreas Stenling¹^*

Andreas Ivarsson²

Peter Hassmén^1,3

Magnus Lindwall^4,5

¹Department of Psychology, Umeå University, Umeå, Sweden
²Center of Research on Welfare, Health and Sport (CVHI), Halmstad University, Halmstad, Sweden
³Faculty of Health, Research Institute for Sport and Exercise, University of Canberra, Canberra, ACT, Australia
⁴Department of Psychology, University of Gothenburg, Gothenburg, Sweden
⁵Department of Food and Nutrition, and Sport Science, University of Gothenburg, Gothenburg, Sweden

In the present work we investigated distinct sources of construct-relevant psychometric multidimensionality in two sport-specific measures of coaches' need-supportive (ISS-C) and controlling interpersonal (CCBS) styles. A recently proposed bifactor exploratory structural equation modeling (ESEM) framework was employed to achieve this aim. In Study 1, using a sample of floorball players, the results indicated that the ISS-C can be considered as a unidimensional measure, with one global factor explaining most of the variance in the items. In Study 2, using a sample of male ice hockey players, the results indicated that the items in the CCBS are represented by both a general factor and specific factors, but the subscales differ with regard to the amount of variance in the items accounted for by the general and specific factors. These results add further insight into the psychometric properties of these two measures and the dimensionality of these two constructs.

Introduction

Coaches' interpersonal styles strongly influence athletes' need satisfaction and motivation in competitive sports (Mageau and Vallerand, 2003). Whereas a need-supportive interpersonal style generally has a positive influence on athletes' motivation, well-being, and performance (Hagger and Chatzisarantis, 2007), a controlling interpersonal style has instead been related to maladaptive outcomes, such as burnout, depression, and disordered eating (Bartholomew et al., 2011). These interpersonal styles are multidimensional constructs, each consisting of theoretically distinguishable subdimensions. Specifically, a need-supportive interpersonal style reflects a global construct as well as three specific subdimensions: autonomy support, structure, and involvement. Similarly, a controlling interpersonal style reflects a global construct and four specific subdimensions: controlling use of rewards, negative conditional regard, intimidation, and excessive personal control. Hence, two distinct sources of psychometric multidimensionality exist: a global factor and specific subdimensions. A comprehensive test of the multidimensional structure thereby requires consideration of both sources (Morin et al., 2015).

The subdimensions of need-support measures (e.g., related to parents, coaches, or exercise practitioners) are suggested to be highly interrelated, and these measures are often treated as unidimensional (e.g., Ryan, 1991; Niemiec et al., 2006; Markland and Tobin, 2010). Given the multidimensional nature of a need-supportive interpersonal style (Deci and Ryan, 2000; Mageau and Vallerand, 2003), as well as recent suggestions that more attention should be given to these various need-supportive interpersonal styles within physical activity settings (e.g., Pope and Wilson, 2012; Standage, 2012), an investigation of the multidimensionality in measures of this construct within the sports context seems warranted. The interpersonal supportiveness scale-coach (ISS-C; Wilson et al., 2009) represents one of the few attempts to develop a multidimensional—autonomy support, structure, and involvement—sport-specific measure in line with the tenets of the self-determination theory (SDT; Deci and Ryan, 2000; Mageau and Vallerand, 2003). One concern has been, however, the relatively high degree of overlap (i.e., common variance) between the three dimensions (Wilson et al., 2009). This concern in turn has raised questions related to the instrument's dimensionality and whether global and/or specific factors are captured by the ISS-C.

When examining the factorial structure of the controlling coach behaviors scale (CCBS), which assesses athletes' perceptions of their coach's controlling interpersonal style from the perspective of SDT (Deci and Ryan, 1985, 2000), Bartholomew et al. (2010) concluded “that a controlling interpersonal style is a multidimensional construct represented by a number of separate, but related, controlling coaching strategies” (p. 205). This conclusion was reached after inspecting and comparing model fit between a four-factor first-order independent cluster model (ICM) confirmatory factor analysis (CFA), a second-order ICM-CFA with four first-order factors, and a one-factor first-order model. This approach to some extent examined multidimensionality; however, it did not identify distinct sources of construct-relevant psychometric multidimensionality in terms of simultaneous estimation of global and specific constructs, as proposed by Morin et al. (2015). Although other studies have examined the psychometric properties of CCBS (e.g., Castillo et al., 2014), a comprehensive test of the structure of the multidimensionality, taking both the global construct and specific subdimensions into consideration, has not yet been performed.

The present studies contribute to the existing literature by investing distinct sources of construct-relevant psychometric multidimensionality in these two sport-specific measures of coaches' need-supportive (ISS-C; Wilson et al., 2009) and controlling interpersonal (CCBS; Bartholomew et al., 2010) styles.

Need-supportive Interpersonal Style

Previous research examining coaches' interpersonal styles in competitive sports contexts (and other contexts—see Ng et al., 2012) have primarily focused on the effects of perceived autonomy support (e.g., Adie et al., 2012; Stenling et al., 2015) and the perceived motivational climate (e.g., Sarrazin et al., 2002; Reinboth and Duda, 2006). Mageau and Vallerand (2003), however, argued that, in addition to autonomy support, coaching behaviors that provide structure and involvement are also important determinants for the satisfaction of athletes' psychological needs and their behavioral regulations. Coaches who provide autonomy support try to understand an athlete's perspective, acknowledge the athlete's feelings, encourage exploration, and curiosity, provide a meaningful rationale, and provide opportunities for choice (Mageau and Vallerand, 2003). Structure involves providing clear and understandable guidelines and expectations, instilling a sense of competence in the athletes, and providing relevant feedback to the athletes (Reeve and Su, 2014). Involvement is displayed when coaches show a genuine interest in their athletes and their well-being and spend a considerable amount of time, energy, and resources on them (Grolnick and Ryan, 1989). Need-supportive environments providing autonomy support, structure, and involvement have been examined in physical education settings (Standage et al., 2005; Haerens et al., 2013) and exercise settings (e.g., Markland and Tobin, 2010), but the potential role of perceived structure and involvement from coaches in sports is still largely unexplored (Mageau and Vallerand, 2003; Pope and Wilson, 2012). Ryan (1991) and others (e.g., Niemiec et al., 2006; Markland and Tobin, 2010) have argued the three support dimensions are highly interrelated; they are therefore often combined into a broader category labeled need support.

Controlling Interpersonal Style

Controlling coaches may actively thwart athletes' basic psychological need satisfaction, and this has been linked to ill-being and stress responses (Bartholomew et al., 2011; Taylor et al., 2015). Based on an extensive literature review, Bartholomew et al. (2009, 2010) identified a number of controlling motivational strategies. The most prominent was controlling use of rewards, which refers to the use of extrinsic rewards and praise to ensure athlete compliance, engagement, and persistence in certain behaviors. This controlling strategy is closely related to the undermining effect of rewards on intrinsic motivation (Deci et al., 1999; Vansteenkiste and Deci, 2003), which refers to the negative effect of tangible rewards (task and performance contingent) on intrinsic motivation, particularly when the reward is expected. Similar effects can also be produced by verbal rewards and praise (Henderlong and Lepper, 2002).

When athletes do not display the desired attributes or behaviors, coaches may withhold attention and affection, labeled negative conditional regard (Assor et al., 2004). Conditional regard from coaches is displayed when their attention and acceptance is highly contingent upon athletes showing appropriate thoughts and behaviors, which oftentimes forces the athletes to give up their autonomy to maintain a satisfactory relationship with their coach (Bartholomew et al., 2010).

An abusive power-based controlling motivational strategy is intimidation, used to belittle and humiliate through verbal abuse, threats, and yelling to control athletes' behaviors and promote external regulation (Bartholomew et al., 2010). Such controlling strategies create pressure from the outside, which promotes athletes to engage in certain behaviors to avoid external punishment (Deci and Ryan, 1987).

Finally, excessive personal control is displayed when coaches engage in intrusive monitoring of athletes' free time and impose strict limits (Bartholomew et al., 2010). Examples of such coach behaviors involve restricting athletes' free time (e.g., setting curfews) or engagement in other sports. This type of controlling interpersonal behavior promotes a sense of pressure from the coach to prioritize one's sports involvement over other important aspects of the athlete's life.

The Present Studies

The two instruments under scrutiny here, the ISS-C (Wilson et al., 2009) and the CCBS (Bartholomew et al., 2010), contain three and four subscales, respectively, and can both be described in terms of a broader construct as well as more specific dimensions or subdomains within that broader construct. Hence, both these instruments are suited for a comprehensive examination of these two sources of construct-relevant variance. One particularly useful approach for this aim, which recently has been rediscovered within psychology research, is the bifactor measurement model (Reise, 2012). The bifactor measurement model originates from the early work by Holzinger and Swineford (1937) but has for a long time been overshadowed by Thurstone's (1947) correlated-factor model. The bifactor model has not only been rediscovered but also extended within an ESEM framework (Jennrich and Bentler, 2011, 2012; Myers et al., 2014; Morin et al., 2015).

Theory-based multidimensional scales, such as the ISS-C and CCBS, often correspond to a bifactor structure with a general latent construct alongside several latent subdimensions that are more narrowly defined (Myers et al., 2014). Applications of bifactor measurement models, however, are rare in sport and exercise psychology research, despite the fact that a bifactor model often provides researchers with an opportunity to match the theory behind the instrument development with the model imposed on the data when evaluating multidimensional scales. This match between theory and model may be lacking when the commonly used correlated first-order model or a second-order factor model is specified because neither of these two models takes the general latent constructs' direct influence on items into account (Myers et al., 2014).

In addition to the rediscovered bifactor measurement model (Reise, 2012), the recently developed ESEM allows researchers to deal with a common problem with ICM-CFA, namely the fallible nature of indicators (Morin et al., 2015). Items incorporate a part of random measurement error, also known as item uniqueness, but items also tend to have some degree of systematic association with other constructs. Such systematic association is typically expressed as cross-loadings in exploratory factor analysis but is constrained to zero in ICM-CFA. In the context of theory-driven multidimensional scales, this assumption of zero cross-loadings might be unrealistically restrictive and, thus, lead to extensive bias in factor correlations and poor model fit (e.g., Asparouhov and Muthén, 2009; Marsh et al., 2014). Multidimensional measures typically include cross-loadings that can be justified by substantive theory or item content (Asparouhov and Muthén, 2009); most items are likely to be imperfect to some degree and have some systematic association with other constructs (Morin et al., 2015).

The recent incorporation of bifactor models and ESEM provides researchers with an opportunity to investigate two sources of “construct-relevant psychometric multidimensionality related to: (a) the hierarchical nature of the constructs being assessed (i.e., the co-existence of global and specific components within the same measurement model) and (b) the fallible nature of indicators which tend to include at least some degree of association with non-target constructs” (Morin et al., 2015, p. 30). According to Morin and colleagues, bifactor models are needed to investigate the first source, whereas the second source calls for ESEM rather than CFA. Furthermore, by estimating ESEM with target rotation, it is possible to specify a priori hypotheses about the factor structure and use ESEM for confirmatory purposes (Asparouhov and Muthén, 2009).

The purpose of the two studies outlined in this report was to apply the recently proposed bifactor ESEM framework (Morin et al., 2015) and investigate distinct sources of construct-relevant psychometric multidimensionality in two sport-specific measures of coaches need-supportive and controlling behaviors: the ISS-C (Wilson et al., 2009) and the CCBS (Bartholomew et al., 2010). Because these two instruments are expected to consist of a general latent factor alongside several narrowly defined subdimensions, we hypothesized that the first-order ESEM would provide a better fit to the data compared to the first-order ICM-CFA and that the bifactor ESEM would provide a better fit to the data compared to first-order ESEM.

Study 1—Interpersonal Supportiveness Scale–Coach

Materials and Methods

Participants

The sample comprised 277 (142 female, 135 male) floorball players competing for clubs in northern Sweden. The athletes' ages ranged from 15 to 22 years (M = 16.8, SD = 1.1) and their competitive levels from regional to international. On average, they practiced floorball 6.8 h (SD = 3.4) per week and had been competing in their sport for 8.4 years (SD = 2.8).

Measures

We used a Swedish version of the Interpersonal Supportiveness Scale–Coach (ISS–C; Wilson et al., 2009) to capture athletes' perceptions of their coach's autonomy support (six items, e.g., “My coach provides me with choices and options”), provision of structure (six items, e.g., “My coach provides clear feedback about my progress”), and involvement (six items, e.g., “My coach puts time and energy into helping me”). Responses to the 18 items were given on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). Internal consistency (omega coefficient; McDonald, 1999) of the three subscales was as follows: autonomy support = 0.860, structure = 0.902, involvement = 0.781.

Statistical Analysis

Data were analyzed with Mplus version 7.3 (Muthén and Muthén, 1998–2012) and the robust maximum likelihood estimator (MLR). MLR provides standard errors and fit indexes that are robust to the Likert nature of the items and non-normality. The items were treated as continuous variables. In a recent simulation study Rhemtulla et al. (2012) showed that the robust maximum likelihood estimator performs equally well or better compared to robust categorical estimators, particularly with seven response categories. A small percentage of missing data was present in the items (< 2.2%). We choose to include all available information and used the full information robust maximum likelihood (FIML) estimation to handle the missing data (Enders, 2010).

We used a model testing procedure proposed by Morin et al. (2015). This procedure allowed us to investigate two sources of construct-relevant psychometric multidimensionality related to the co-existence of global and specific components within the same measurement model and the fallible nature of indicators which tend to include at least some degree of association with non-target constructs. We started by specifying and comparing first-order ICM–CFA with first-order ESEM models to examine the presence of cross-loadings of conceptually related or overlapping constructs. Based on the results in the first step (ICM–CFA vs. ESEM), the second step aimed to identify the presence of construct-relevant multidimensionality due to the presence of hierarchically superior constructs using bifactor models. The ICM–CFA, ESEM, and bifactor ESEM models are graphically depicted in Figure 1¹.

FIGURE 1

Figure 1. Graphical representation of the alternative models tested in these two studies. The top three are the ICM-CFA, first-order ESEM, and bifactor ESEM of the ISS-C. Bottom three are the ICM-CFA, first-order ESEM, and bifactor ESEM of the CCBS. Dotted lines represent non-target loadings. CUR, controlling use of rewards; NCR, negative conditional regard; INT, intimidation; and ECP, excessive personal control.

Conventional fit indices were used to evaluate the model fit in the ICM-CFA and ESEM models, such as the comparative fit index (CFI), the Tucker-Lewis Index (TLI), the standardized root mean residual (SRMR), and the root mean square error of approximation (RMSEA). Traditional cutoff criteria with CFI and TLI values around 0.90 and SRMR and RMSEA values around 0.08 were used to indicate acceptable fit (Marsh, 2007). Target rotation was used in the ESEM models, which allows for the specification of target and non-target factor loadings in a confirmatory manner (Browne, 2001; Asparouhov and Muthén, 2009). All cross-loadings were specified to be close to zero, while all the main loadings were freely estimated (Morin et al., 2015). In line with bifactor assumptions that the specific factors explains item variance not explained by the general factor and that the general factor explains variance that is shared across all items, the general and specific factors were orthogonal in order to ensure interpretability (Chen et al., 2006; Reise, 2012). The first-order ESEM models were estimated with an oblique target rotation (Browne, 2001; Asparouhov and Muthén, 2009).

We followed the guidelines for nested model comparisons in small samples (< 300) by Chen (2007), who suggested that a change in CFI (ΔCFI) ≥ −0.005 accompanied by a change in RMSEA (ΔRMSEA) of ≥ 0.010 would support the simpler model over the more complex model. In addition, Marsh et al. (2010) suggested that some indices (e.g., TLI and RMSEA) penalize for parsimony so that a more parsimonious model can fit the data better than a less parsimonious model. Therefore, a more conservative approach is to consider the more parsimonious model as supported, if the TLI or RMSEA is as good or better compared to the more complex model. We also used several information criteria when comparing alternative models: the Akaike Information Criterion (AIC; Akaike, 1987), the Bayesian Information Criterion (BIC; Schwartz, 1978), and the sample-size adjusted BIC (ABIC; Sclove, 1987). These information criteria do not in themselves describe model fit, but a model with a lower value indicates a better fitting model compared to a model with a higher value when alternative models are compared. Note, however, that all of these guidelines for model fit have been established for CFA, and more work is needed regarding their generalizability outside of the CFA framework. Regardless of the framework these guidelines are applied within (e.g., CFA, ESEM), they should not be taken as “golden rules” but rather as rough guidelines used in combination with parameter estimates, statistical conformity, and theoretical adequacy (Marsh et al., 2004; Morin et al., 2015).

Procedure

The initial contact was made with the head coach of each team, and when permission was granted to approach the athletes, a time, and place were scheduled for an informational meeting with them. During this meeting, a research assistant invited the athletes to participate in the study. Upon agreement to participate, the athletes provided written informed consent and responded to a multi-section questionnaire that took approximately 20 min to complete. Data was collected at midseason to ensure that the athletes had had enough time to establish a perception of their coach's interpersonal style. Prior to the data collection, ethical approval was obtained from the Regional Ethical Review Board at the first author's university.

Results

Item correlations, means, standard deviations, skewness, and kurtosis are displayed in Table 1. As seen in Table 2, the first-order ICM–CFA model displayed an acceptable fit to the data. However, the first-order ESEM model displayed a better representation of the data as indicated by the ΔCFI = 0.049, ΔRMSEA = 0.018, higher TLI and lower RMSEA values, and lower AIC, BIC, and ABIC values. The ICM–CFA model displayed large factor correlations between the three factors (0.956–0.992), but these factor correlations were lower in the first-order ESEM model (0.473–0.708; see Table 3). The first-order ESEM model also revealed several cross-loading items, with relatively high loadings on non-target factors. Taken together, the ESEM model provided a better representation of the data compared to the ICM–CFA and was therefore retained in the second step when estimating the bifactor model. As displayed in Table 2, the bifactor ESEM model provided an excellent fit to the data and had lower AIC and ABIC compared to the first-order ESEM. The factor loading pattern from the bifactor ESEM model is displayed in Table 4, and almost all items had a strong standardized factor loading on the general factor and a weak loading on the specific factor. These results indicate that most of the variance in the items in the ISS-C was accounted for by the general factor.

TABLE 1

Table 1. Correlations and descriptives for the ISS-C items.

TABLE 2

Table 2. Model Fit of the ICM-CFA and ESEM Models.

TABLE 3

Table 3. Latent factor correlations between the subdimensions in the ISS-C.

TABLE 4

Table 4. Standardized Loading Pattern for the Bifactor ESEM of the ISS-C (N = 277).

Discussion

The comparison between the first-order ESEM and the bifactor ESEM indicated that the latter provided a better fit to the data. However, the first-order ESEM indicated a substantial degree of cross-loading items, and the bifactor ESEM displayed that most of the variance in the ISS–C items was accounted for by the general factor. This indicates that the ISS–C items were captured by the general construct need support and not by the specific subdimensions autonomy support, structure, and involvement. These results are similar to previous multidimensional leadership scales, for example, within transformational leadership research (e.g., van Knippenberg and Sitkin, 2013), where the subdimensions often are highly interrelated. A high degree of overlap between subdimensions within multidimensional SDT need-support scales is quite common (e.g., Niemiec et al., 2006; Wilson et al., 2009) and the three need-support dimensions are often suggested to be highly interrelated (Ryan, 1991; Markland and Tobin, 2010). Much SDT research has therefore treated need support as a unidimensional variable (e.g., only using autonomy support), and have not acknowledged the multidimensional nature of need support as consisting of autonomy support, structure, and involvement. The high correlations between the need-support dimensions could be explained by the fact that coaches who are need supportive with regard to one dimension (e.g., autonomy support) also are need supportive with regard to the other dimensions. Such an explanation has previously been suggested, for example, within transformational leadership research, where high factor correlations are commonly observed in research with self-report measures (Barling et al., 2011). Another explanation is related to the use of self-report measures and that the respondents may not be able to distinguish between items intended to capture these various dimensions of leadership. A remedy for this type of common method bias could be to collect other types of data (Podsakoff et al., 2012), such as observational data of coach behaviors in various situations. SDT-based observations (e.g., Haerens et al., 2013; Smith et al., 2015) could be a useful source for data collection to avoid common method bias when combined with other data sources (e.g., self-report).

Having measures that can discriminate between subdimensions in multidimensional constructs is important because this would allow researchers to explore whether these leadership dimensions are additive (i.e., the more, the better), which is assumed when sum scores of need support are used or if their relationships have other forms (van Knippenberg and Sitkin, 2013). For example, instead of assuming an additive effect, we could specify a minimum value on each of these dimensions that determines when someone can be characterized as need supportive. The need-support dimensions could also be interactive, indicating that the engagement in one type of need support would make the other types more effective. This type of moderating effect has been found in educational research (Jang et al., 2010) where autonomy support and structure were two interacting engagement-fostering interpersonal styles among teachers associated with students' classroom engagement. A third possibility could be that one need-support dimension can compensate for the lack of another (van Knippenberg and Sitkin, 2013). The evidence so far suggests that the three need-support dimensions are interactive, in that structure, and involvement will be enhanced when provided in an autonomy-supportive way (Reeve and Su, 2014). In order to investigate these various hypotheses, it is imperative that the three need-support dimensions are distinguishable. The multidimensionality in need-support instruments is an important avenue of future research.

Study 2—Coaches Controlling Behaviors Scale

Materials and Methods

Participants

This sample comprised 233 male ice hockey players competing for clubs in northern Sweden. Their ages ranged from 15 to 20 years (M = 17.1, SD = 1.4) and their competitive levels from regional to international. On average, they had been competing in their sport for 10.6 years (SD = 2.1).

Measures

A Swedish version of the Controlling Coach Behaviors Scale (CCBS; Bartholomew et al., 2010) was used to measure the athletes' perceptions of their coach's controlling interpersonal style. The CCBS consists of four subscales capturing athletes' perceptions of coaches' controlling use of rewards (four items, e.g., My coach tries to motivate me by promising to reward me if I do well), negative conditional regard (four items, e.g., My coach is less supportive of me when I am not training and competing well), intimidation (four items, e.g., My coach intimidates me into doing the things that he/she wants me to do), and excessive personal control (three items, e.g., My coach tries to interfere in aspects of my life outside of my sport). Responses to the 15 items were given on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). Internal consistency (omega coefficient; McDonald, 1999) of the four subscales was as follows: controlling use of rewards = 0.723, negative conditional regard = 0.865, intimidation = 0.719, and excessive personal control = 0.706.

Statistical Analysis

The same statistical analyses as in Study 1 were conducted in Study 2.

Procedure

The same procedure as in Study 1 was used in Study 2.

Results

Item correlations, means, standard deviations, skewness, and kurtosis are displayed in Table 5. The first-order ICM–CFA displayed an acceptable fit to the data (Table 2), but also for the CCBS the first-order ESEM model provided a better representation of the data as indicated by the ΔCFI = 0.033, higher TLI value, and lower AIC and ABIC values. The ΔRMSEA did not reach Chen's (2007) recommendation of 0.010 but was lower for the ESEM model, and taken together, we concluded that the ESEM model provided a better representation of the data. As seen in Table 6, the factor correlations were slightly higher in the ICM–CFA model (0.149–0.597) compared to the first-order ESEM model (0.131–0.476). No substantive cross-loadings were observed in the first-order ESEM, the largest cross-loading was −0.26 (standardized). As seen in Table 2, the bifactor ESEM displayed an excellent fit to the data and had lower AIC and ABIC compared to the first-order ESEM. The factor loading pattern from the bifactor ESEM model is displayed in Table 7. The factor loadings of items in two of the specific factors, negative conditional regard and excessive personal control, indicate that the items in both these factors are explained by the general factor as well as their specific factors. Items on the subscale controlling use of rewards displayed relatively weak loadings on the general factor and strong loadings on the specific factor, whereas items on the subscale intimidation displayed an opposite pattern with stronger loadings in the general factor and weak loadings on the specific factor. These results indicate that the items in the CCBS are represented by a general factor and specific factors but that the subscales differ with regard to the amount of variance accounted for by the general and specific factors.

TABLE 5

Table 5. Correlations and descriptives for the CCBS items.

TABLE 6

Table 6. Latent factor correlations between the subdimensions in the CCBS.

TABLE 7

Table 7. Standardized Loading Pattern for the Bifactor ESEM of the CCBS (N = 233).

Discussion

Also for the CCBS, the bifactor ESEM provided a better fit to the data compared to the first-order ESEM. The bifactor ESEM provided an interesting pattern of multidimensionality, indicating that the CCBS consists of a general factor as well as specific factors but that the subdimensions differed with regard to the amount of variance accounted for by the general and specific factors. Researchers have previously concluded that the CCBS is a multidimensional instrument using first- and second-order ICM–CFAs (e.g., Bartholomew et al., 2010; Castillo et al., 2014). Although these previous studies, to some extent, examined multidimensionality, they did not examine distinct sources of multidimensionality in terms of the simultaneous estimation of global and specific factors. By using a bifactor ESEM approach (Morin et al., 2015), we extended previous research on the factor structure of the CCBS with a statistical method that matches the theory underlying the development of the CCBS with the model imposed on the data (Myers et al., 2014).

The bifactor structure for the CCBS indicated that the negative conditional regard factor and the excessive personal control factor have a bifactor pattern with items loading relatively strong onto both the general and the specific factors. These two subdimensions seem to consist of two sources of construct-relevant multidimensionality (Morin et al., 2015). Items on the factor controlling use of rewards had relatively weak loadings on the general factor compared to the other factors (range 0.181–0.278) and relatively strong loadings on its specific factor (range 0.440–0.830). This pattern was also seen in the first-order ESEM with relatively low factor correlations between the controlling use of rewards factor and the other factors (range 0.131–0.394). These results indicate that the controlling use of rewards factor may represent a slightly different aspect of coaches' controlling behaviors compared to the other three specific factors that to a larger degree seem to have a common core, as represented by the general factor. Research looking at the different CCBS factors' relationship to other variables is scarce, but controlling use of rewards has, for example, shown a weaker relationship to autonomy support compared to the other three factors in the CCBS (Bartholomew et al., 2010). Whether the factors in the CCBS predict outcomes differently is an interesting question for future research that would provide more insight into similarities and differences between the CCBS factors. Finally, the items on the intimidation factor displayed relatively strong loadings onto the general factor and weak loadings onto its specific factor. Hence, this factor is mostly explained by one source of construct-relevant multidimensionality, the general factor.

Furthermore, three items displayed meaningful cross-loadings (factor loadings > 0.20) onto the negative conditional regard factor: CUR_1, My coach tries to motivate me by promising to reward me if I do well, CUR_4, My coach only uses rewards/praise so that I complete all the tasks he/she sets in training, EPC_1, My coach expects my whole life to center on my sport participation. Future research should investigate these items further and explore whether they need to be revised or if this complexity is theoretically meaningful (cf. Myers et al., 2014). It should be noted, however, that pure items are not a requirement of a well-defined factor structure and that one can argue that it is more important to find an accurate set of items rather than a pure set of items (Asparouhov and Muthén, 2009). Hypotheses about pure factor structures with items loading solely onto their intended factors may in many cases be too restrictive due to the fallible nature of indicators, and the ESEM framework provides researchers with a tool that accounts for items' associations with non-target constructs (Asparouhov and Muthén, 2009; Myers et al., 2014; Morin et al., 2015).

General Discussion and Conclusions

Our purpose was to investigate distinct sources of construct-relevant psychometric multidimensionality in two sport-specific measures of coaches' need-supportive and controlling behaviors, the ISS-C (Wilson et al., 2009) and the CCBS (Bartholomew et al., 2010). We expected that these two multidimensional instruments would consist of a general latent factor alongside several narrowly defined subdimensions and therefore adopted a recently proposed bifactor ESEM approach (Morin et al., 2015) suitable for examining such multidimensional structures.

In the first step, we compared a traditional ICM–CFA with ESEM to examine the possibility that the items within these two instruments have systematic associations with non-target constructs that needs to be accounted for (Asparouhov and Muthén, 2009; Morin et al., 2015; Perry et al., 2015). In both cases did the ESEM model provide a better fit to the data compared to the ICM–CFA, and we also noted a decrease in factor correlations because the relations between the items have to a lesser degree been channeled through the factors. Previous research with simulated (e.g., Asparouhov and Muthén, 2009) and real data (e.g., Marsh et al., 2013) have consistently shown that factor correlations in ICM–CFA are likely to be positively biased and that ESEM more accurately estimates factor correlations (Marsh et al., 2014). A pattern of positively biased factor correlations was evident also in this study. When the assumption of zero cross-loadings does not hold, the factor correlations will be positively biased because systematic associations between items and non-target factors are not accounted for (Marsh et al., 2014). Marsh and colleagues also argued that when the assumption of zero cross-loadings does hold, allowing for cross-loadings is unlikely to result in negative bias in factor correlations. Recently there have been suggestions in the methodological literature that ICM–CFA and ESEM should routinely be performed on the same data in order to investigate these potential sources of bias inherent in many ICM–CFA models (Marsh et al., 2013, 2014).

The present study contributes to the growing body of knowledge regarding the factor structure of the ISS–C and the CCBS. However, some limitations exist. The samples in these two studies were in a quite narrow age range, only included team sport athletes, and only male athletes were included in Study 2. Although the CCBS have been found invariant across type of sport and gender (Bartholomew et al., 2010), exploring whether these results can be replicated in other samples of different ages and types of sports would be interesting in future research. Researchers could also examine if perceptions of coaches need supportive and controlling interpersonal styles vary with athletes' age and competitive level. Future research should also explore the predictive ability of the general and specific factors in these scales on various outcomes, for example, basic need thwarting and satisfaction (see Gunnell and Gaudreau, 2015, for such an approach). The results from Study 1, however, suggests that the ISS-C might best be represented as a unidimensional structure and that almost all of the variance in the items was accounted for by the general factor. The unidimensional structure of the ISS-C indicates that it might not be necessary to use the general factor from a bifactor ESEM model as an outcome or predictor; a single factor can be adequate for such purposes².

Taken together, the present study illustrates a useful approach for examining dimensionality in multidimensional self-report scales. The results related to the ISS–C, combined with findings from previous research, indicate that the multidimensionality of the need support construct can be questioned. Future research should thoroughly examine whether this lack of multidimensionality in need-support scales can be attributed to theoretical, methodological, and/or empirical aspects. The results related to the CCBS showed an interesting pattern of psychometric multidimensionality that should be cross-validated in future research. The integration of bifactor analysis and ESEM (Myers et al., 2014; Morin et al., 2015) provide researchers with a framework uniquely suited for examinations of multidimensional constructs.

Author Contributions

AS came up with the initial idea and design, performed the analyses and interpreted the data, and was in charge of drafting the manuscript. AI made substantial contribution to the analyses and interpretation of the data, and revised the manuscript critically for important intellectual content. PH helped out in the interpretation of data and revised the manuscript critically for important intellectual content. ML made substantial contribution to the analyses and interpretation of the data, and revised the manuscript critically for important intellectual content. All four authors (AS, AI, PH, ML) provided final approval of the version to be published, and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

Grants from Umeå School of Sport Sciences and the Swedish National Centre for Research in Sports (CIF) supported this research (Grant P2014-0043). We express our sincere appreciation to John Andersson, Kristian Fagerström, Fredrik Hansson, and Pierre Hedin for helping out with the data collection.

Footnotes

1 ^We did not include second-order models in our model testing sequence because we wanted to examine the direct relation between the global and specific factors and the items. In a second-order model the relation between the general factor and the items is represented as an indirect effect through the first-order factors. In addition, second-order models rely on far more stringent assumptions than bifactor models as well as stringent proportionality constraints not likely to hold in real-world settings with complex instrument (Morin et al., 2015; Reise, 2012).

2 ^To further examine this issue with the ISS-C we correlated factor scores from a single-factor CFA model and factor scores from the general factor in the bifactor ESEM model. The correlation was 0.996 indicating very little impact of ignoring multidimensionality and suggests that using the ISS-C as a single factor might be adequate.

References

Adie, J. W., Duda, J. L., and Ntoumanis, N. (2012). Perceived coach-autonomy support, basic need satisfaction and the well-and ill-being of elite youth soccer players: a longitudinal investigation. Psychol. Sport Exerc. 13, 51–59. doi: 10.1016/j.psychsport.2011.07.008