A methodology for introducing competitive anxiety and pressure in VR sports training

1. Introduction

Motor skill training is an active research field in virtual reality (Miles et al., 2012). Virtual reality (VR) platforms allow the design of specific and complex training protocols, which are not feasible in real life. Nevertheless, VR training programs dealing with pressure or anxiety management have barely been addressed. Only a few works have recently started to address this issue (Wellner et al., 2010; Stinson and Bowman, 2014). External factors such as the audience, athletes’ expectations, or their psychological state, can increase pressure and anxiety (Martens et al., 1990; Schmidt and Wrisberg, 2004), resulting in decreased performance (Ehrlenspiel, 2006). Although small levels of competitive anxiety can be beneficial for the athlete, increasing their attention and focus, higher levels of anxiety can result in decreased concentration and attention, leading to unforced errors and decreased performance (Wilson et al., 2009). Furthermore, competitive anxiety has also been correlated with the risk of sport injuries (Kellmann, 2010).

In order to deal with competitive anxiety, traditional approaches focus on mental rehearsal skills (Cumming and Hall, 2002). They have proved to be useful in order to overcome stressful situations that the athlete has experienced before and can increase the athletes’ performances (Short et al., 2006). Training routines can also be used to reproduce anxiety conditions (Oudejans and Pijpers, 2009), but environmental factors are hard to reproduce, especially, high pressure situations like the last putt in golf or the decisive throw in a basket match. In contrast, virtual reality studies have proven the feasibility of the simulation of potentially stressful situations, which are able to modify how people behave and perceive the environment [e.g., public speech (Pertaub et al., 2002), the virtual pit (Meehan et al., 2002), and phobia treatment (Wiederhold et al., 1998; Powers and Emmelkamp, 2008)].

In this work, we propose a new methodology for the design of experiments focusing on competitive anxiety training, considering the social-evaluative threat theory (Rohleder et al., 2007) and Game Flow (Sinclair et al., 2007). An experiment re-creating a 10 m air pistol shooting illustrates the proposed methodology. The goal is to quantify the effect of environmental factors (stressors) in terms of users’ performances, behaviors, subjective measures, and physiological responses. The task, although simple in nature, required participants to maintain a high level of concentration and execute repetitive aiming motions (motor skill task). The results showed significant differences in the user behavior and in their subjective impressions of the task.

The remainder of the paper is structured as follows: Section 2 reviews literature in competitive stress and virtual reality systems. Section 3 presents the proposed methodological framework. Section 4 details the experimental design and the protocol of the experiment, followed by the analysis and discussion of the results in Section 5. The paper ends with the concluding remarks and future work.

2. Related Work

2.1. Competition Anxiety and Pressure

Mental rehearsal skills are one of the main traditional approaches of sport psychology to mitigate the effects of state anxiety (Cumming and Hall, 2002). Imagery interventions strive in familiarizing the athlete with a specific task, through mentally revisiting stress-evoking situations. Many protocols focus on re-creating stress-eliciting situations in the athletes’ minds and draw attention to the sensation of stress in the body. The athlete is believed to get the impressions of successful actions and reduce competitive anxiety.

In addition to competition anxiety, pressure can also decrease the athlete’s performance. Pressure appears in situations in which the athlete has situational incentives for optical, maximal, or superior performance (Baumeister and Showers, 1986). The impact of pressure can vary according to moderator factors such as individual differences and the complexity of the task. Regarding person characteristics, the factor more explored is trait anxiety, which has been positively correlated to decreased performance. However, this has been mostly explored for cognitive tasks (Ehrlenspiel, 2006). Regarding task complexity, although it seems to play an important role, the fact that the definition of what is a simple or complex task is not obvious limits its implications. In contrast, the nature of the task (quantitative vs. qualitative) has been found to be more consistent. While quantitative tasks benefit from pressure (e.g., task completion time), qualitative tasks are hindered (accuracy, number of errors) (Strauss, 2002). Finally, pressure is one of the main factors for choking (Ehrlenspiel, 2006). Choking refers to a dramatic loss of the athlete’s performance at decisive moments and can be defined as the “occurrence of suboptimal performance under pressure conditions” (Baumeister and Showers, 1986).

The performance of athletes is often influenced by environmental conditions (Schmidt and Wrisberg, 2004). In particular, the presence of an audience can lead to increased motivation, which occurs mainly for effort-dominant tasks, or choking, which occurs mainly for skill-dominant tasks (Lewis and Linder, 1997). Audience plays an important role (Baumeister and Showers, 1986), depending on the size of the audience, the athlete’s perception of the audience, its salience and their expectations. The expectation of the audience is the factor providing the strongest effect. For example, if athletes feel that they have to meet the audience’s expectations, they appear more likely to choke. Different studies have explored the effect on performance of motor tasks (coordination tasks) focusing on the audience (Strauss, 2002). In general, the results showed that tasks with demands on coordination have a decreased performance in the presence of an audience. However, there are no clear findings on how the effect of the audience and the participants’ skills can alter performance on motor tasks.

2.2. Measuring Pressure and User Interaction in VR

So far, we have listed the potential factors that might decrease the athlete’s performance. However, in order to quantify the effect of potential stressors, we must be able to measure how they influence the user. Regarding how these factors affect humans, two main theories exist: attention and drive (Ehrlenspiel, 2006). On the one hand, attentional theories consider cognitive processes are altered, leading to changes in performance. Pressure can hinder user attention, either by increasing self-focus attention (turning automatic processes into manual) or causing the athlete to focus on superfluous and irrelevant stimuli. In both cases, the athlete’s attention will be impaired. User performance can be measured objectively according to the results of the task.

On the other hand, drive theories are based on the assumption that task performance is linked with the level of “arousal,” and thus physiological processes. Although there is no absolute method to measure the “arousal” based on physiological monitoring, most studies in the literature focus on the analysis of heart rate and galvanic skin response (Meehan et al., 2002; Slater et al., 2006; Groenegress et al., 2010). For example, physiological monitoring has been used to detect arousal, stress, comparing virtual and real situations, or measuring the human response when interacting with virtual characters (Slater et al., 2006).

The heart and the circulatory system are regulated by several body processes. In the “pit” experiment (Meehan et al., 2002), the evidence strongly suggested that heart rate measured as beats per minute (bpm) increased when the stress was induced. In addition, several studies focus on video games and involve cardiac measurement as an index of valence, arousal, attention, cognitive effort, and stress (Kivikangas et al., 2010). However, in addition to the temporal analysis of the heart rate [e.g., time between beats (RR interval)], frequency analysis has been also considered (Slater et al., 2006). Heart rate variability (HRV) is influenced by the interaction between the sympathetic and parasympathetic branches of the autonomic nervous system, which is a measure of the capacity for regulated emotional response (Applehans and Luecken, 2006). Increases in the participant arousal could generate an increase of HRV and a shift toward the high frequency power band.

In addition, the events experienced by the participants can produce changes in the electrical properties of the skin. Galvanic skin response (GSR) correlates positively with novelty, intensity, or emotional content (Andreassi, 2000). Skin conductance (SC) can be decomposed into tonic and phasic components (Benedek and Kaernbach, 2010). The tonic component is the baseline level of the SC, which varies over time depending on the physiological state and the autonomic regulation. However, it tends to increase over time due to the increase of sweat around the electrodes. Tonic component is also known as the skin conductance level. In contrast, the phasic component is sensitive to instantaneous events, such as discrete environmental stimuli. The phasic component will evoke time related changes, which are also known as skin conductance responses which might last from 10 to 20 s (Prokasy and Raskin, 1973).

Although there exist a wide number of studies that have shown the correlation between physiological responses and the psychological state, the physiological signals tend to respond more to exercise than to real psychological changes (Wellner et al., 2010). Deriving the psychological state from physiological signals when the user is doing aerobic exercise could produce erroneous results.

2.3. VR and Sport Training

The usage of virtual reality technology for sport training is an active area of research (Miles et al., 2012). However, most existing works are focused on the improvement of motor skills. Training routines can be designed to enhance particular motor skills and the training can be done progressively, for example, by increasing the level of difficulty of the task. VR training systems are unique since they enable the simulation of any situation and allow the display additional information for guidance. Considering another field of research, VR has a strong visibility in military training. VR enables the simulation of a wide range of stressful and complex situations, which soldiers might face during real operations. Some examples include psychological training for aircraft pilots (McClernon et al., 2011) or stress resilience (Rizzo et al., 2012).

However, only few works have tackled the benefits of VR training programs on sports psychology. The main exceptions are the studies of Wellner et al. (2010) and Stinson and Bowman (2014). The work of Wellner et al. explored how the behavior of a virtual audience is able to alter the physiological responses of an athlete during a virtual rowing task. Their results were inconclusive and the hypothesis that the virtual audience might influence psycho-physiological processes could not be confirmed. The authors concluded that one of the possible explanations was the required physical activity. Interestingly, there was a strong deviation when asking about participants about the effect of the real audience in their performance (survey with professional rowers). Similarly, Stinson and Bowman (2014) explored how different external factors were able to influence user anxiety (system fidelity, field of regard, and anxiety triggers). In contrast to the work of Wellner et al., Stinson et al. focused their analysis on the data obtained through anxiety questionnaires (STICSA and CSAI-2R). Their analysis showed correlations between different anxiety triggers and the anxiety levels obtained from the questionnaires. Nevertheless, there is no existing methodological work exploring the different factors that have to be considered, nor providing design guidelines when conceiving such training protocols.

3. Methodological Framework

In this section, we describe the theoretical foundations of the proposed methodology for the design of stress sport training systems in immersive virtual environments. The goal is to design an engaging experience in which the user has to face a challenging situation, which might induce changes on behavioral and physiological responses. The methodology is based on the concepts of flow (Csikszentmihalyi, 1990), game flow (Sweetser and Wyeth, 2005), and the social-evaluative threat theory (Rohleder et al., 2007).

3.1. Flow and Game Flow

The flow theory from Csikszentmihalyi (1990) states that maximum performance and immersion is achieved when the level of challenge is matched with the user’s skills. In challenging situations, the user will shift toward an arousal state while if it is insufficiently challenging, the user will be relaxed as he/she will be in control of the situation. In extreme situations, when the mismatch between the user’s skills and the presented challenge is extreme, situations of boredom or anxiety might arise. Some of the characteristics of a flow experience involve full concentration on the task and the distortion of the temporal experience.

The flow theory can be applied for a wide range of user interactions such as music, education, sports, and video games (Sweetser and Wyeth, 2005). Particularly, it has been considered in the design of exergames (Sinclair et al., 2007). In order to create an engaging task, which requires a moderate amount of concentration, our methodology is mostly based on the game flow criteria proposed by Sweetser and Wyeth (2005). Thus, the sport routine should ensure:

1. Concentration and focus: users are required to keep a moderate level of concentration and focus on the task. Concentration should be key in order to achieve the task.

2. Challenge: the system must provide the right amount of challenge according to the user’s skills. In other words, the system should be able to dynamically adjust the difficulty of the task in order to ensure the challenge (the task is neither too easy nor too difficult). Most video games provide mechanisms to adjust the level of difficulty in order to keep the user’s engagement. If the level of difficulty is too low, the user will become disappointed, as it does not provide any challenge. On the other hand, if the difficulty is too high, the user will lose the motivation, as he will feel that there is no way to win.

3. User skills: the system must ensure the development of the user’s mental and/or physical skills, which is a main requirement in all sport training systems.

4. Control: users should feel a sense of control over their actions. The user should be confident about their skills and the system should ensure that the actions performed by the user have an expected result.

5. Merging of action and awareness: the participant should know at any moment the state of his performance and the goals to achieve. The system must provide any means of feedback available to ensure awareness. The most common approach is to provide visual and acoustic feedback.

6. Immersion: as stated by Sweetser and Wyeth (2005): “Users should experience deep but effortless involvement.” This definition of immersion is closely related to engagement and presence. Higher levels of immersion can decrease the users’ awareness of the surroundings or alter the sense of time or decrease the users’ self-awareness. Nevertheless, the achieved effect can be dependent on the level of presence (Witmer and Singer, 1998).

Although, game flow considers an additional component of social interaction, we are not considering it due to the fact that we are targeting scenarios, which do not require the interaction of the user with other users. Nevertheless, future work considering team-play scenarios will have to consider social interaction.

3.2. Social-Evaluative Threat

While the task in itself could generate a certain level of arousal/anxiety when the degree of challenge surpasses the athlete’s skills, the athlete can be also vulnerable to social evaluation. Social evaluation plays an important role on self-esteem, decreasing motivation and performance (Crocker and Knight, 2005). In competitive environments, athletes are continuously evaluated, which contributes to their sense of self. This continuous evaluation creates a self-preservation mechanism, which reacts when a potential threat to the self is perceived. As summarized by Rohleder et al. (2007), perceptions and responses to a social-evaluative threat appear more likely to occur under the following conditions:

A. A central goal is involved.

B. The situation requires the display of an attribute or skill that the athlete values.

C. The attribute or skill is evaluated by others.

D. The goal may be threatened in a situation in which a negative evaluation could lead to a loss of social status.

E. Achieving the goal may be impeded by uncontrollable factors.

Conditions A and B highlight the importance of an individual’s performance in a domain that is significant to their self-identity. In contrast, conditions C and D refer to the importance of social ties for this theory, that is, goals and performances that are not evaluated by others will not evoke a social-evaluative threat response. Lastly, uncontrollable elements (E) may increase the social-evaluative threat when athletes perceive that their performance and its evaluation are not completely under their own control.

The aim of the presented methodology is to create a training scenario in which the user might experience a certain degree of “arousal” or stress. The experimental protocols should stride on creating situations in which the user is in control of the scenario although it might lose it easily if there is an increase of the challenge or there is a loss of concentration. Furthermore, external events might be introduced to decrease the level of control inducing a potential social-evaluative threat.

3.3. Measuring Stress

As discussed in Section 2.2, the effects of potential stressors on humans can be observed in altered cognitive processes (attentional) or physiological processes (drive) (Ehrlenspiel, 2006). A virtual reality training facility enables the recording of athletes actions and can be easily instrumented to provide a wide range of measurements. We propose the monitoring of qualitative and quantitative measures classified into four different groups: performance, behavior, physiology, and subjective measures.

• Performance: including success rate, task completion time, or overall score. Most training routines have an implicit set of intrinsic quantitative measures. Changes in such intrinsic measures, although not providing direct information on why the athlete’s performance is altered, provide proof that there is an effect from the training or experimental conditions.

• Behavior: differences on the training routine might induce behavioral changes (cognitive and motor). Changes on attention might modify the way the athlete performs the task, for example, increased reaction times due to diminished attention or inaccurate movements due to excessive focus on motor actions. Furthermore, according to the task, behavior patterns can be extracted and analyzed.

• Physiology: experimental conditions might induce physiological changes. However, there is a need to isolate the changes due to the experimental conditions and due to the task on itself such as physical activity (Wellner et al., 2010). Nevertheless, they can be an indicator for instantaneous events, such as errors. Heart rate and electrodermal activity are typically the most used ones, although EEG monitoring can be envisioned.

• Subjective: qualitative measures can be directly gathered from users. Questionnaires, informal interviews, or standardized questionnaires are the most common approach. Such subjective measures can provide global effects of the training protocol and the psychological state of the user. For example, standard questionnaires to provide anxiety assessments such as STICSA or CSAI-2R can be used (Stinson and Bowman, 2014).

4. Illustrative Application: Simulating an Olympic Shooting Range

In order to illustrate and validate our methodology, we present a training scenario, which reproduces a 10 m Olympic shooting environment (see Figures 1 and 2). The experiment reproduces two different scenarios, which take place in a virtual shooting range. The first scenario (Training) does not aim to create a social-evaluative thread. In other words, it is designed to minimize elements, which can raise anxiety or pressure. In contrast, the second scenario (Competition) follows the proposed methodology. It integrates several potential stressors that might raise a social-evaluative threat. We decided to integrate all the different stressors in the same condition in order to maximize their effect. If successfully induced, the social-evaluative threat can increase the pressure perceived and the anxiety of participants. For each condition, we analyzed whether changes in the virtual reality simulation are able to modify the performance, behavior, and/or the physiological state of the user.

FIGURE 1

Figure 1. Illustrative application. The proposed methodology is illustrated and evaluated in a virtual Olympic shooting experiment. The experiment was conducted in a wide immersive projection system being able to enclose a ten meter wide shooting range with six virtual opponents and one participant.

FIGURE 2

Figure 2. Virtual shooting range. The shooting range is 10 m wide and has seven lanes, fitting entirely on our wide immersive projection system. Informative panels were placed on the side walls. In addition, each lane had a target monitor showing the impact of the last shot.

The task, 10 m Olympic shooting, requires a high degree of concentration and does not involve intense physical activity. The lack of physical activity ensures less biased physiological recordings. In the design of the experiment, for both conditions, we took special care on designing the different elements present in the environment in order to deliver a realistic simulation and ensure the game flow. The following sections detail the design of the task based on the game flow theory and how the different design choices relate to the creation of a social-evaluative threat.

4.1. Game Flow Design

Due to the fact that we were targeting non-expert shooters, one of our main concerns was to provide an engaging task, ensuring that the users maintain the engagement over the experiment. We followed the guidelines proposed by Sinclair et al. (2007) for the design of exergames, in order to create an engaging task, which requires a moderate amount of concentration. The design of the task followed the game flow principles provided in Section 3.1:

1. Concentration and focus: in order to succeed in the task, participants must keep a moderate level of concentration as the task requires a high degree of accuracy. The target is placed 7.5 m away and its size is 17 cm × 17 cm. For each shot, the score ranged from 0 to 10 points, leading to a maximum score of 300 points (30 shots per condition). Considering the size and the distance toward the target, a deviation of 0.075° in the gun’s orientation is the difference between scoring 10 and 0.

2. Challenge: the system should dynamically adjust the difficulty of the task in order to ensure the challenge (the task should be neither too easy nor too difficult). In the experiment, in addition to the participant’s individual score, the participant had to compete against virtual competitors. A dynamic approach was used to ensure that there were not big differences among the participant and the virtual competitors. Every virtual competitor had a different performance profile, making the ones closer to the participant more accurate. It also ensured that at least one virtual competitor fights for the first place and the participant is never the last. In general, participants were able to win if they had an average score higher than 7.

3. Control: the participant knows at any moment the state of his performance through visual and acoustic feedback. In addition, the shooting range includes score panels showing the accumulated score for all participants.

4. Merging of action and awareness: at any moment, the system provides feedback ensuring that participants know the actions to perform. In order to ensure that the duration of the experiment is constant, participants had to shoot a bullet in time steps of 25 s. First, users had to aim the target for at least 5 s; then they got another 10 s to shoot; and finally there was a resting phase of 10 s between shots. The timing was ensured through the color coding of the target’s support. Green, the user aims the target. White, the user is allowed to shoot. Red, the user has to rest until the next trial begins. If the user does not shoot in the time window, the shot is considered as missed. For each shot, the participant receives instantaneous feedback of his performance. The participant is able to see his score in the score boards, and a virtual monitor placed in front of the participant shows the hit point for each shot (see Figure 2). In addition, acoustic feedback (beep sound) is provided when the user misses the target.

5. Immersion: the experimental protocol was designed to avoid any interaction with the experimenter, minimizing the interferences from the real world. For example, after each condition, instruction panels were shown to avoid breaks. The length of the task, 30 shots in a row, ensured that participants stayed in the virtual environment more than 12 min. Furthermore, the visual stimuli were delivered by a 4-sided wide projection system (see Section 4.4).

4.2. Creating a Social-Evaluative Threat

As stated previously, the experiment is subdivided into two scenarios: training vs. competition. Although the task performed is the same, a run of 30 trials, the environmental conditions differ. The goal of the experiment is to observe whether differences in the provided feedback are able to modify how the user behaves and performs. The changes introduced aims at creating a social-evaluative threat. The following list details the differences between both scenarios (Competition vs. Training) and their link with the different social-evaluative threat conditions listed in Section 3.2 (letters in parenthesis). In the experiment, as we wanted to generate the strongest possible effect, we introduced the maximum amount stressors, still keeping the task realistic.

• Score boards (A, B, C): score boards enable the participant to directly compare his score with the score of the other virtual competitors. During the training condition, the score boards on the walls (see Figure 2, top) only show the accumulated score of the participant.

• Virtual competitors (B): in the competition condition, the user competes against six virtual competitors. Competitors shoot at variable times (between 1 and 4 s) and are animated to express the result of their score (see Figure 3). For example, if an avatar scores 10 points, the avatar will express joy by waving his arms.

• Acoustic feedback (B, C): for both conditions, acoustic feedback is provided when the participant shoots. The sound relates whether the virtual bullet hits the target (impact sound) or misses it (beep sound). Moreover, during the competition, the sound of a virtual audience is introduced (mumbling ambient sound), which also reacts according to the score of contestants. The behavior of the virtual audience was designed to support the virtual characters. If the participant’s score is high, the audience will reprobate him (booing). If the participant scores lower than the virtual competitors, the audience will applause. Finally, if the score of the participant is lower than 3, the audience will laugh.

• Aiming accuracy (E): the tracking data coming from the optical tracking system are not filtered, which introduced an uncontrolled variability on the aiming orientation. In addition, during competition, 10% of the shots are considered as missed even if the target is hit. To ensure a fair score, the 10% error rate is also considered for avatars.

FIGURE 3

Figure 3. Avatar animations. Leftmost, virtual competitors are animated according to their performance. Right, Snapshots of several of the animations played by the avatars after each round. Rightmost, animations are played for higher scores.

In the design of the experiment, we did not consider any factor categorized as (D). This would be only possible if users were experts and a strong rivalry was present, or the real trainer was present during the experiment.

4.3. Experimental Design

The experiment followed a one factor, within-subjects design. The factor was the condition (Training vs. Competition). To avoid ordering effects, the order for each condition was counter-balanced. In order to account for the different categories of potential effects (performance, behavior, physiology, and subjective), the dependent variables considered were:

Score: for each shot, the user can score from 0 to 10 points, resulting in a maximum score of 300 points for each condition. User performance can be influenced by the different stimuli of each condition. On the one hand, the stressors introduced during the competition can decrease their concentration and potentially the score. On the other hand, the increase of motivation due to a clear goal (beat the other avatars) might generate the opposite effect. Nevertheless, considering that participants were not experts, our first hypothesis (H1) is that there will not be significant differences in the overall score.

Shooting time: time spent (in seconds) by participants from when they are allowed to shoot until they actually shoot. Differences on the shooting time can reflect behavioral changes. Participants might use the maximum time available in order to ensure a better shot. In contrast, the opposite effect might appear, users might eager to shoot due to the pressure introduced by other competitors. Our hypothesis (H2) is that time spent to shoot will vary between conditions.

Heart rate: increased arousal or anxiety might alter heart rate measurements. In the current study, we considered as dependent variables the RR (mean time between heartbeats), the heart rate variability (HRV), and the relationship between the low and high frequency analysis of the RR series following the recommendations from the Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology (1996). The task had a low effort profile, minimizing the changes on the user physiology due to physical activity. We hypothesize that the different stimuli presented in each condition will modify (up to some extend) the heart rate variability (HRV). Our third hypothesis (H3) is that the different markers will differ between conditions, being significantly higher for the competition condition.

Skin conductance: changes on the physiological state of the user might increase the levels of skin conductance. Our hypotheses are that the tonic skin conductance will increase over the experiment (H4) while the phasic skin conductance level will be higher during the competition condition (H5).

4.4. Apparatus and Participants

The experiment was conducted in a wide 4-sided immersive projection system (9.6 m × 3 m × 3 m) using active stereo and ARTrack optical tracking. As a pointing device (gun), we used the ART Flystick. We have to notice, that the weight of the Flystick is 400 g, when the weight of a real competition gun is around 1 kg (Figure 4).

FIGURE 4

Figure 4. Participant during the pilot experiment. The stand beside the user was needed due to the wired measurement system. The stand has a minimum interference with the projected virtual environment.

In order to assess the physiological state of participants, heart rate (ECG) and skin conductance (GSR) were recorded using gTec¹ medical grade physiological sensors. For the ECG, four active electrodes were used, which were placed on the torso of the participant. Skin conductance was captured with two sensors placed on two finger tips of the non-dominant hand. All data were sampled at 512 Hz using a gTec g.USBAmp amplifier, which ensured the data synchronization. Additionally, we flagged the data according to the different phases of the experiment (aim, shoot, relax). All the data recording were done using OpenViBE (Renard et al., 2010).

Eighteen unpaid participants, 14 male and 4 female, took part in the experiment (age: $\overline{x}=25.632:\mathrm{\sigma }=0.876$), most of them having previous experience in VR and all of them being able to correctly perceive stereoscopic images. In order to avoid any bias, the information regarding the purpose of the experiment was provided after the experiment finished. Participants were instructed to perform as best as possible.

4.5. Procedure

The experiment had a total duration of 1 h, and was subdivided into the following steps:

Instructions and written consent: participants were asked for their written consent and were informed about: the nature of the experiment, the equipment used, the virtual reality setup, and the data recorded (which was anonymized). In addition, users were asked to fill a questionnaire to gather information about their background (gaming, VR, sports experience).

Equipment setup: the experimenter sets up the different physiological sensors, and the stereoscopic glasses and the Flystick are provided to the participant. During the remainder of the experiment, the system provides written instructions in order to avoid communication with the experimenter. This avoids interactions which might alter the physiological responses of the user and maximizes immersion.

Training: users performed 10 practice shots in order to fully understand the shooting procedure. This session was performed with the environmental configuration of the training condition.

Experiment: participants performed two series of 30 shots, one for each condition. Before each condition, there was a 2 min and a half calibration step to measure the baseline for each physiological signal. In addition, there was a 5 min break between conditions. Regarding the length of the experiment, and although Olympic rules state that one run consists in 60 shots in 105 min for men and 45 shots in 75 min for women, for practical reasons, the experiment length did not vary among gender and the length of the experiment was reduced to 30 shots.

Post-questionnaires: at the end of the experiment, participants were asked to fill a questionnaire regarding their subjective impressions. The questions covered their engagement, the task difficulty, the physical fatigue, the level of immersion and annoyance.

5. Results

5.1. Performance and Behavioral Measures

In order to account for potential ordering effects, the analysis also considers the order as an additional between-subjects factor.

5.1.1. Score

The two-way ANOVA of the score vs. condition and order did not show any main effect for Condition (F_(1,16) = 0.00; p = 0.982) neither on the order (F_(1,16) = 0.05; p = 0.505). We did not observe any trend along the trials (e.g., users did not improve their performance during the experiment). The average user score was $\left(\overline{x}=5.74,\mathrm{\sigma }=2.54\right)$, thus supporting H1.

5.1.2. Shooting time

The two-way ANOVA of the shooting time vs. condition and order, did not show any main effect for condition $\left(F_{\left(1,16\right)}=0.387;p=0.387;{\mathrm{\eta }}_p^2=0.047\right)$ neither for order $\left(F_{\left(1,16\right)}=0.13;p=0.728;{\mathrm{\eta }}_p^2<0.01\right)$. However, we observed a noticeable interaction effect between order and condition $\left(F_{\left(1,16\right)}=3.32;p=0.087;{\mathrm{\eta }}_p^2=0.17\right)$. Although not being significant, we observe that for the competition condition, participants who performed first, the training condition had the tendency to take more time to shoot $\left(\overline{x}=1.875s,\mathrm{\sigma }=0.565s\right)$ than if they started directly with the competition $\left(\overline{x}=1.560s,\mathrm{\sigma }=0.538s\right)$. These results do not support H2.

In addition, we explored potential bias of the avatar behavior and the correlation between shooting time and score. First, we pooled the data from real and virtual competitors to explore the potential bias introduced by the virtual competitors pacing. The one-way ANOVA shooting time vs. competitor (virtual, real) showed a main effect on competitor $\left(F_{\left(1,34\right)}=27.01;p<0.001;{\mathrm{\eta }}_p^2=0.44\right)$. Post hoc tests showed that virtual competitors took significantly more time to shoot $\left(\overline{x}=2.42s,\mathrm{\sigma }=0.0045s\right)$ than real competitors. Second, no correlation was found between score and shooting time (r² = 0.28).

5.2. Physiological Responses

5.2.1. Heart rate

ECG data from both conditions and baseline recordings were analyzed to identify the QRS complex in the ECG waveform. The follow up analysis is based on the time between two R peaks (RR interval). For each participant, a manual check of the RR series was performed to ensure the validity of the data and artifact removal. We removed one participant only from the heart rate analysis due to irregular RR intervals. Frequency analysis was performed using a parametric (model-based) power spectrum estimation. Specifically, we used an autoregressive (AR, order p = 64) approach and we followed the recommendations by Boardman et al. (2002). In order to prepare the data for the frequency domain analysis, we detrended the RR series. This ensures that non-stationarities of the RR series do not influence the analysis. For the detrending, we used the Smoothness Priors method with a λ = 300, representing a cutoff of 0.035 Hz (Tarvainen et al., 2002). The frequency bands considered were (1) very low frequency (VLF, from 0 to 0.03 Hz), (2) low frequency (LF, from 0.03 to 0.18 Hz), and (3) high frequency (HF, from 0.18 to 0.4 Hz). The VLF band was removed from the analysis as their implications are still unclear and might distort the analysis. The ECG analysis was done using Kubios HRV².

The measurements computed were (1) the mean time between two consecutive R peaks in the QRS complex, (2) the SD of the RR intervals, (3) the HRV index, highly correlated with the std RR but insensitive to artifacts and ectopic beats, and (4) low and high frequency power in normalized units. The statistical analysis considered the baseline data after each condition as an additional within-subject factor with two levels (Calibration, Experiment). Table 1 summarizes the statistical analysis for all variables.

TABLE 1

Table 1. Statistical summary for the different measurements considered for the ECG analysis.

All the measurements show that there were no significant differences in all the variables considered among both conditions, thus rejecting H3. However, significant differences appeared between the baseline and the experiment recordings. Bonferroni post hoc tests (all p < 0.05) showed that heart rate was lower (higher RR mean) and that the heart rate variability is also lower (std RR and HRV index) during the baseline recordings. Regarding the frequency analysis, we observed a significant shift from high frequencies to low frequencies during the experiment.

5.2.2. Skin conductance

SC data analysis showed a strong effect on the order. The length of the experiment (40 min) was expected to raise skin conductance (SC) due to increased sweat around the electrodes. Two-way ANOVA of order and condition vs. mean SC only showed a main effect on order (F_(1,17) = 13.0; p < 0.001). Post hoc tests showed that the mean skin conductance value increased with the time (p < 0.05). Follow up analysis considered the two components of the SC: the phasic and the tonic. The decomposition was done using the LedaLAB³ Matlab toolbox, precisely, we used the Continuous Decomposition Analysis (CDA). SC data were downsampled to 4 Hz.

The two-way ANOVA order and condition vs. the mean tonic SC showed a main effect on order (F_(1,17) = 12.96; p < 0.001) but not in condition. These results are consistent with the previous ones; the tonic component accounts for the baseline level of the SC, thus accepting H4. In contrast, no main effect was found for the order when considering the phasic SC levels (F_(1,16) = 0.97; p = 0.34), thus rejecting H5. Similar to the ECG analysis, we also considered the differences between the baseline recordings and the experiment recordings. Here, we observed the same trends for the tonic SC but the phasic levels were significantly higher during the experiment (F_(1,16) = 20.02; p < 0.001). Additional analyses are required considering the instantaneous events of the SC values.

5.3. Subjective Analysis

At the end of the experiment, users filled a questionnaire (7-Likert scale) composed of 23 questions related to the equipment (1–3), the task (4–6), their mental state (7–10), their motivation (11–13), and their perception of the virtual environment during the competition (14–23). The full questionnaire and the statistical analysis are summarized in Table 2. Users had to rank the different questions considering both conditions (Training and Competition) except for questions related to the competition (from 16 to 23).

TABLE 2

Table 2. Summary of the user questionnaire and the statistical analysis of the subjective responses (7-likert scale).

Data were analyzed using the Mann–Whitney non-parametric test, which showed only significant differences (p < 0.05) for the questions related to the difficulty of the task (4), the level of frustration (9), the level of stress (10), and how often they looked to the accumulative score board (14). Our main goal of the experiment was to create a realistic competitive scenario which was able (up to some extend) to influence their perception of the task. The fact that users felt significantly more frustrated (9) and stressed (10) during the competition supports the fact that changes in the virtual environment, although performing the same task, are able to change their perception of the task, and thus influence them. This result relates with the decreased shooting time for the Competition condition.

In addition, from the other results we can highlight that no disrupting effect happened during the experiment [low scores for fatigue (1), dizziness (2) or equipment disturbance (3)]. Regarding the task, it was perceived to be harder during the competition (4), a moderate feeling of improvement over time was perceived (5) and users considered the system not to work properly (6), this last result was mainly due to tracking noise and jitter. Also, participants showed a high level of concentration (7) and motivation (11), which only decreased slightly during the experiment (13). Finally, regarding the appreciation during competition, we highlight that users perceived that the public was against them (16), they showed an eagerness to win (20) and they did not consider that the avatar played a script (21).

Regarding ordering effects, also tested using the Mann–Whitney test, we observed a tendency for physical fatigue (p = 0.07) (1). Overall, fatigue ratings are consistently higher for the second block, which is reasonable considering the length of the experiment. Also, there are significant differences (all with p < 0.05) for skill improvement (5) and task involvement (13, 20). Participants felt higher skill improvement and had a lower task involvement during the second block.

6. Discussion

The analysis of the experimental results showed that the differences in both conditions were able to modify the subjective impressions of the participants. In contrast, performance and physiological measures were constant among the different conditions. Considering that all our participants were non-expert shooters, we hypothesize that the level of potential anxiety and pressure, which can be generated was limited. This is visible in the results for the performance and physiological measures. Furthermore, the inability to generate a situation in which a negative evaluation could lead to a loss of social status could have played an important role.

Statistical analysis on ECG and GSR measures did not show any significant differences, the different stressors were not able to significantly alter the physiological responses of participants. However, changes in the physiological signals between baseline recordings and the experiment were observed. Increased heart rate variability is observed when the user is performing the task. Although the task was not physically demanding, changes in the physiological responses can be due to the physical effort or changes in the breathing pattern of the participants. For example, some of the participants, during the post-interview, said that they stopped breathing while aiming to increase their hand stability. Additional analyses of the data are required and correlations between different physiological responses have to be considered. Furthermore, there was a strong user variability.

On the other hand, subjective data from the questionnaires showed that the competition condition raised the ratings on difficulty, frustration, and stress. The fact that participants had to (1) compete against virtual avatars and (2) deal with an aggressive audience, played an important role. For example, some participants were annoyed by the audience (“It was annoying when the audience was laughing at me.”) while others were annoyed with the behavior of other avatars (“I could perceive my neighbor and I felt annoyed when she took too much time to shoot.”). Furthermore, one user was very irritated by the arbitrary misses. He verbally complained every time they occurred. This makes us conclude that the different stressors introduced were able to change, up to some extent, the user perception of the task. However, they were unable to raise the anxiety and pressure levels enough to be detectable in the physiological and performance data.

7. Conclusion

Virtual reality training systems have shown a great potential for motor skill training. However, very few works have addressed how such systems can simulate and account for competitive anxiety and pressure. In this paper, we have proposed a methodological framework to design sport training scenarios both considering the elements that a training routine must ensure to keep the engagement and how external factors might raise the stress of trainees. The main goal of the methodology is to train athletes to perform under situations of anxiety and pressure similar to the ones they can face in a real competition. A training routine should keep athletes close to the arousal and anxiety zones as defined by the flow theory. This resembles a competition scenario in which the athlete has left his comfort zone and is struggling to achieve his performance objective. In addition to control, the level of difficulty of the task, external anxiety, and pressure triggers can be introduced to further place the athlete outside his comfort zone. During real competition, that is the scenario, which maximizes pressure and anxiety, the athlete is constantly evaluated (e.g., the audience and other competitors). Such evaluation and the uncontrolled factors in the environment can create a social-evaluative thread that is likely to raise the anxiety and pressure levels.

In order to illustrate the proposed methodology, it was applied to the design of a virtual training environment for Olympic shooting, and a pilot experiment with non-expert shooters was conducted. In the experiment, we analyzed the user performance and physiological responses. Two different conditions were evaluated: a scenario minimizing potential anxiety and pressure triggers, and a scenario maximizing them. The competitive virtual training scenario was designed to place participants outside their comfort zone, a task requiring a moderate level of concentration, skilled virtual competitors, and an aggressive audience. The results showed differences in the subjective ratings, where there was an increased feeling of anxiety and pressure during the competitive scenario. In contrast, we only observed trends in performance and physiological recordings. As seen in previous work, physiological data are dependent on different body and mental processes, which makes it difficult to compute a correlation between them. Future work addressing this analysis should also consider an additional baseline condition in which participants perform the same physical task without any feedback. Our results seem consistent with the results from Stinson and Bowman (2014). In both experiments, the physiological measures show differences between baseline and experimental conditions, and the subjective data shows differences among different experimental factors. We believe that such a consistency supports the claim that VR can be considered for anxiety training in sports. Nevertheless, it has to be noted that the virtual environment and the engagement of the trainees play an important role. Future work should focus on the analysis of the influence of different factors both in terms of performance and the effects on the user. We believe that the analysis of the physiological responses of the user is key to better understand the effects of the training program and they provide one of the few unbiased quantitative measurements. Nevertheless, it will require better extraction and analysis methods for physiological measurements.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at http://www.frontiersin.org/article/10.3389/frobt.2015.00010/abstract

Footnotes

1. ^http://www.gtec.at/
2. ^http://kubios.uef.fi
3. ^http://www.ledalab.de

References

Andreassi, J. L. (2000). Psychophysiology Human Behavior and Physiological Response. Mahwah, NJ: Erlbaum.

Google Scholar

Applehans, B. M., and Luecken, L. J. (2006). Attentional processes, anxiety, and the regulation of cortisol reactivity. Anxiety Stress Coping 19, 81–92. doi: 10.3389/fnhum.2012.00306

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Baumeister, R. F., and Showers, C. J. (1986). A review of paradoxical performance effects: choking under pressure in sports and mental tests. Eur. J. Soc. Psychol. 16, 361–383. doi:10.1002/ejsp.2420160405

CrossRef Full Text | Google Scholar

Benedek, M., and Kaernbach, C. (2010). A continuous measure of phasic electrodermal activity. J. Neurosci. Methods 190, 80–91. doi:10.1016/j.jneumeth.2010.04.028

CrossRef Full Text | Google Scholar

Boardman, A., Schlindwein, F. S., Rocha, A. P., and Leite, A. (2002). A study on the optimum order of autoregressive models for heart rate variability. Physiol. Meas. 23, 325–336. doi:10.1088/0967-3334/23/2/308

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Crocker, J., and Knight, K. M. (2005). Contingencies of self-worth. Curr. Dir. in Psychol. Sci. 14, 200–203. doi:10.1111/j.0963-7214.2005.00364.x

CrossRef Full Text | Google Scholar

Csikszentmihalyi, M. (1990). Flow: The Psychology of Optimal Experience. Harper & Perennial.

Google Scholar

Cumming, J., and Hall, C. (2002). Athletes’ use of imagery in the off-season. Sport Psychol. 16, 160–172.

Google Scholar

Ehrlenspiel, F. (2006). Chocking Under Pressure – Attention and Motor Control in Performance Situations. PhD thesis, University of Potsdam, Germany.

Google Scholar

Groenegress, C., Spanlang, B., and Slater, M. (2010). The physiological mirror: a system for unconscious control of a virtual environment through physiological activity. Vis. Comput. 26, 649–657. doi:10.1007/s00371-010-0471-9

CrossRef Full Text | Google Scholar

Kellmann, M. (2010). Preventing overtraining in athletes in high-intensity sports and stress/recovery monitoring. Scand. J. Med. Sci. Sports 20(Suppl. 2), 95–102. doi:10.1111/j.1600-0838.2010.01192.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Kivikangas, J. M., Ekman, I., Chanel, G., Järvelä, S., Cowley, B., Henttonen, P., et al. (2010). “Review on psychophysiological methods in game research,” in Proceedings of 1st Nordic DiGRA. Stockholm, 8.

Google Scholar

Lewis, B. P., and Linder, D. E. (1997). Thinking about choking? Attentional processes and paradoxical performance. Pers. Soc. Psychol. Bull. 23, 937–944. doi:10.1177/0146167297239003

CrossRef Full Text | Google Scholar

Martens, R., Vealey, R. S., and Burton, D. (1990). Competitive Anxiety in Sport. Human Kinetics.

Google Scholar

McClernon, C. K., McCauley, M. E., O’Connor, P. E., and Warm, J. S. (2011). Stress training improves performance during a stressful flight. Hum. Factors 53, 207–218. doi:10.1177/0018720811405317

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Meehan, M., Insko, B., Whitton, M., and Brooks, F. P. (2002). Physiological measures of presence in stressful virtual environments. ACM Trans. Graph. 21, 645–652. doi:10.1145/566654.566630

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Miles, H. C., Pop, S. R., Watt, S. J., Lawrence, G. P., and John, N. W. (2012). A review of virtual environments for training in ball sports. Comput. Graph. 36, 714–726. doi:10.1016/j.cag.2012.04.007

CrossRef Full Text | Google Scholar

Oudejans, R. R. D., and Pijpers, J. R. (2009). Training with anxiety has a positive effect on expert perceptual-motor performance under pressure. Q. J. Exp. Psychol. 62, 1631–1647. doi:10.1080/17470210802557702

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Pertaub, D.-P., Slater, M., and Barker, C. (2002). An experiment on public speaking anxiety in response to three different types of virtual audience. Presence Teleop. Virt. Environ. 11, 68–78. doi:10.1162/105474602317343668

CrossRef Full Text | Google Scholar

Powers, M. B., and Emmelkamp, P. M. G. (2008). Virtual reality exposure therapy for anxiety disorders: a meta-analysis. J. Anxiety Disord. 22, 561–569. doi:10.1016/j.janxdis.2007.04.006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Prokasy, W. F., and Raskin, D. C. (1973). Electrodermal Activity in Psychological Research. Academic Press.

Google Scholar

Renard, Y., Lotte, F., Gibert, G., Congedo, M., Maby, E., Delannoy, V., et al. (2010). OpenViBE: an open-source software platform to design, test, and use brain-computer interfaces in real and virtual environments. Presence Teleop. Virt. Environ. 19, 35–53. doi:10.1162/pres.19.1.35

CrossRef Full Text | Google Scholar

Rizzo, A., Buckwalter, J. G., John, B., Newman, B., Parsons, T., Kenny, P., et al. (2012). STRIVE: stress resilience in virtual environments: a pre-deployment VR system for training emotional coping skills and assessing chronic and acute stress responses. Stud. Health Technol. Inform. 173, 379–385.

Pubmed Abstract | Pubmed Full Text | Google Scholar

Rohleder, N., Beulen, S. E., Chen, E., Wolf, J. M., and Kirschbaum, C. (2007). Stress on the dance floor: the cortisol stress response to social-evaluative threat in competitive ballroom dancers. Pers. Soc. Psychol. Bull. 33, 69–84. doi:10.1177/0146167206293986

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Schmidt, R. A., and Wrisberg, C. A. (2004). Motor Learning and Performance. Human Kinetics.

Google Scholar

Short, S. E., Ross-Stewart, L., and Monsma, E. V. (2006). Onwards with the evolution of imagery research in sport psychology. Athletic Insight 8, 1–15.

Google Scholar

Sinclair, J., Hingston, P., and Masek, M. (2007). “Considerations for the design of exergames,” in Proceedings of the 5th International Conference on Computer Graphics and Interactive Techniques in Australia and Southeast Asia – GRAPHITE ’07, page 289 (Perth: ACM Press).

Google Scholar

Slater, M., Guger, C., Edlinger, G., Leeb, R., Pfurtscheller, G., Antley, A., et al. (2006). Analysis of physiological responses to a social situation in an immersive virtual environment. Presence Teleop. Virt. Environ. 15, 553–569. doi:10.1162/pres.15.5.553

CrossRef Full Text | Google Scholar

Stinson, C., and Bowman, D. A. (2014). Feasibility of training athletes for high-pressure situations using virtual reality. IEEE Trans. Vis. Comput. Graph 20, 606–615. doi:10.1109/TVCG.2014.23

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Strauss, B. (2002). Social facilitation in motor tasks: a review of research and theory. Psychol. Sport Exerc. 3, 237–256. doi:10.1016/S1469-0292(01)00019-X

CrossRef Full Text | Google Scholar

Sweetser, P., and Wyeth, P. (2005). GameFlow: a model for evaluating player enjoyment in games. ACM Comput. Entertain. 3, 1–24. doi:10.1145/1077246.1077253

CrossRef Full Text | Google Scholar

Tarvainen, M. P., Ranta-Aho, P. O., and Karjalainen, P. A. (2002). An advanced detrending method with application to HRV analysis. IEEE Trans. Biomed. Eng. 49, 172–175. doi:10.1109/10.979357

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. (1996). Heart rate variability: standards of measurement, physiological interpretation, and clinical use. Eur. Heart J. 17, 354–381. doi:10.1093/oxfordjournals.eurheartj.a014868

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Wellner, M., Sigrist, R., von Zitzewitz, J., Wolf, P., and Riener, R. (2010). Does a virtual audience influence rowing? Proc. Inst. Mech. Eng. P J. Sports Eng. Technol. 224, 117–128.

Google Scholar

Wiederhold, B. K., Gevirtz, R., and Wiederhold, M. D. (1998). Fear of flying: a case report using virtual reality therapy with physiological monitoring. Cyberpsychol. Behav. 1, 97–103. doi:10.1089/cpb.1998.1.97

CrossRef Full Text | Google Scholar

Wilson, M. R., Vine, S. J., and Wood, G. (2009). The influence of anxiety on visual attentional control in basketball free throw shooting. J. Sport Exerc. Psychol. 31, 152–168.

Pubmed Abstract | Pubmed Full Text | Google Scholar

Witmer, B. G., and Singer, M. J. (1998). Measuring presence in virtual environments: a presence questionnaire. Presence Teleop. Virt. Environ. 7, 225–240. doi:10.1162/105474698565686

CrossRef Full Text | Google Scholar