Manuela Schuetze1,2,3,4*
Christiane S. Rohr1,2,5
Deborah Dewey2,6,7
Adam McCrimmon2,8
Signe Bray1,2,4,5,7- 1Child and Adolescent Imaging Research Program, University of Calgary, Calgary, AB, Canada
- 2Behaviour and the Developing Brain, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, AB, Canada
- 3Department of Neuroscience, University of Calgary, Calgary, AB, Canada
- 4Hotchkiss Brain Institute, Calgary, AB, Canada
- 5Department of Radiology, University of Calgary, Calgary, AB, Canada
- 6Department of Community Health Sciences, University of Calgary, Calgary, AB, Canada
- 7Department of Pediatrics, University of Calgary, Calgary, AB, Canada
- 8Educational Psychology, Werklund School of Education, University of Calgary, Calgary, AB, Canada
Early behavioral interventions are recognized as integral to standard care in autism spectrum disorder (ASD), and often focus on reinforcing desired behaviors (e.g., eye contact) and reducing the presence of atypical behaviors (e.g., echoing others' phrases). However, efficacy of these programs is mixed. Reinforcement learning relies on neurocircuitry that has been reported to be atypical in ASD: prefrontal-sub-cortical circuits, amygdala, brainstem, and cerebellum. Thus, early behavioral interventions rely on neurocircuitry that may function atypically in at least a subset of individuals with ASD. Recent work has investigated physiological, behavioral, and neural responses to reinforcers to uncover differences in motivation and learning in ASD. We will synthesize this work to identify promising avenues for future research that ultimately can be used to enhance the efficacy of early intervention.
Introduction
Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by impairments in communication and social interactions, as well as repetitive and stereotyped behaviors, and restricted interests (American Psychiatric Association, 2013). Although there is no known cure for ASD, early intervention has been shown to improve cognition and adaptive behavior (Dawson et al., 2010; Sullivan et al., 2014) and alter brain responses to social stimuli (Dawson et al., 2012).
Early work showed that operant learning strategies, which involve training a behavior by providing reinforcement, could be used to increase social behaviors such as communication and social interaction (Wolf et al., 1963; Allen et al., 1964; Jensen and Womack, 1967), imitation (Metz, 1965), instruction following (Davison, 1964), and object naming (Martin et al., 1968) in children with ASD. These reinforcement learning (RL) strategies have been refined into a structured and systematic treatment system called Applied Behavioral Analysis (ABA). ABA-based treatment approaches use RL to promote typical social and communication behaviors and to reduce or minimize atypical behaviors (Virués-Ortega, 2010; Dawson and Burner, 2011).
RL-based treatments are supported by evidence and are widely used. Meta-analyses report statistically significant gains with moderate to large effect sizes in treatment groups relative to parent or treatment-as-usual control groups (Remington et al., 2007; Eikeseth, 2009; Dawson et al., 2010; Virués-Ortega, 2010; Reichow et al., 2012). However, it has also been noted that children with ASD vary in the magnitude of their response to ABA-based intervention (Sallows and Graupner, 2005; Sherer and Schreibman, 2005; Schreibman et al., 2009; Perry et al., 2011). Could variable treatment response be related to reduced or inconsistent responses to rewards and/or punishments in ASD, or differences in how the brain forms associations?
Although not a core diagnostic feature in ASD, abnormal responses to rewards and differences in feedback-learning have been documented in several branches of the ASD literature including: social reward (Lin et al., 2012), fear learning (Gaigg and Bowler, 2007), decision-making (Solomon et al., 2015), and perceptual learning (Harris et al., 2015). Although variable response to reinforcement-based therapy is not unique to ASD, this evidence suggests that relying on neurocircuitry associated with operant learning could pose challenges to successful behavior modification in this population. The aim of this non-systematic literature review (Grant and Booth, 2009) is to conduct a survey of the current evidence for atypical RL in ASD in order to address the question of whether abnormal RL, or inter-individual variability, could impact treatment efficacy. The following PubMed search terms were used to retrieve relevant publications: “Autism Spectrum Disorder” AND “Reinforcement Learning,” “Reward,” “Conditioning,” “Intervention,” “Eyeblink Conditioning,” “affective,” “EMG” and a final selection was conducted based on our knowledge from research in the field to include those articles most relevant to the topics under discussion. We also included articles that were cited in these papers covering relevant topics.
We begin by examining whether individuals with ASD respond differently than typically developing (TD) individuals to rewards or punishments Studies on physiological, behavioral, and neural responses to reinforcers in ASD are reviewed. Next, we turn to aversive conditioning studies, which have investigated the formation of passive (Pavlovian) associations between cues and physiological responses. We then review studies of learning and decision making in order to shed light on the learning of stimulus-response-outcome associations. We go on to examine how RL-based treatments change the brain and behavior, and how an individual's response to reinforcers might be consistent with the personal characteristics shown to predict treatment response. Finally, we discuss avenues for future research and implications for improving treatment of ASD symptoms.
Physiological, Behavioral, and Neural Responses to Reinforcers in ASD
Over the course of learning, an association is formed between a cue or action and a reinforcer with some inherent (or previously learned) motivational value. Before considering whether individuals with ASD show abnormalities in learning, a fundamental question that must be considered is whether positive reinforcers (rewards) such as food or social stimuli contain the same motivational value for individuals with ASD as they do for TD individuals. This question is motivated by two theories of ASD. First, the social motivation theory (Chevallier et al., 2012) proposes that individuals with ASD are less motivated to interact with, or orient toward, social stimuli (e.g., faces, eyes, voices), and predicts that typical physiological “reward” responses to social stimuli should be attenuated. Second, the amygdala theory (Baron-Cohen et al., 2000), suggests that impaired function of the amygdala impairs “social intelligence” and understanding of emotional content, and predicts abnormal responses to affective stimuli more generally. Affective stimuli include faces or scenes depicting or invoking specific emotions and may or may not include social content (e.g., both a picture of snakes and a picture of a crying face may be considered negative or unpleasant affective stimuli). Inappropriate physiological responses to either social or affective stimuli could lead to abnormalities in the formation of conditioned associations with these stimuli when they are used as reinforcers. Studies reviewed in this section are detailed in Table 1.
Table 1. Participant characteristics of reviewed ASD studies in section Physiological, Behavioral, and Neural Responses to Reinforcers in ASD.
Physiological Responses
Different tasks have been used to measure responses to social and affective stimuli in ASD, including passive viewing of static pictures (Wilbarger et al., 2009), viewing of video clips (Hubert et al., 2009), overt ratings of emotional features (Hubert et al., 2009), and ratings of non-emotional features such as gender of faces (Hubert et al., 2009). During these tasks, physiological responses can be quantified by measuring muscular contractions, skin conductance response, or pupil dilation, and can provide insight into affective (“emotional”) responses in ASD.
Muscular contractions in response to startle or facial muscle responses (i.e., smiling and frowning) can be measured with electromyography (EMG). One study of adolescents and young adults with ASD found typical facial muscle and eye blink startle responses to negative pictures from the International Affective Picture System (IAPS), but potentiated startle response to positive IAPS pictures, despite typical facial EMG and overt ratings (Wilbarger et al., 2009). Exaggerated eyeblink responses to pleasant IAPS images were also shown in a study by Dichter et al. (2010), as well as exaggerated postauricular muscular responses to unpleasant images. Neither of these studies provided measures of intellectual functioning for their samples. A third study in adults with ASD who showed no cognitive impairment (often referred to as “high functioning ASD”) found no difference in eyeblink or facial EMG measurements to pleasant and unpleasant IAPS pictures (Mathersul et al., 2013a). These authors note that in contrast to previous studies, the images they used were more consistent, containing only social affective scenes, rather than image sets comprised of social and non-social images, which may have led to a more consistent overall response.
Facial EMG can also be used to measure spontaneous mimicry of others' facial expressions (Hess and Blairy, 2001), and studies in ASD have found some evidence for atypical mimicry (Cattaneo et al., 2007; Oberman et al., 2009). One study recorded mouth-opening mylohyoid (MH) muscle activity in participants while they watched an experimenter grasp and eat food or grasp a piece of paper and place it in a container on his shoulder (Cattaneo et al., 2007). TD children showed increased MH activity during the food grasping compared to the paper-grasping action while MH activity in children with ASD did not differ between conditions. Another study looking at involuntary mimicry recorded EMG from five groups of facial muscles while children categorized emotions of 192 facial expressions shown on a screen (Oberman et al., 2009). These authors found a significant delay of peak facial mimicry in children with ASD compared to TD children—even though the groups did not differ in their ability to categorize the presented emotions. These studies suggest that spontaneous mimicry is altered in children with ASD. A study of high-functioning adults with ASD also found attenuated spontaneous mimicry when images of emotional facial expressions were presented briefly (i.e., backwards masked), independent of whether participants judged emotions or gender of the images (Mathersul et al., 2013b). Since spontaneous mimicry is modulated by the reward value of stimuli (Sims et al., 2012) this could be linked to the reduced motivational value of facial expressions in this population. In contrast, another study that recorded EMG looked at two groups with ASD: one group of children with high functioning ASD and one group of children with ASD with severe impairments of social responsiveness (Social Responsiveness Scale values > 75, Constantino and Gruber, 2012). They found no differences in facial mimicry between high functioning and control children, but children with ASD and severe social impairments showed significantly decreased facial mimicry when watching film clips in which different children expressed anger, sadness, happiness, and fear (Deschamps et al., 2015). In general, EMG based markers show some sensitivity to atypical processing of social and affective information and the latter study hints at variability within the autism spectrum as an underlying cause for atypical mimicry. Future studies should collect additional information on symptom severity in order to fully understand differences in ASD subgroups.
Skin conductance response (SCR) is a sympathetic sweat response that can be evoked by emotional stimuli and is sensitive to arousal (“alertness due to stimuli”), rather than valence (“pleasantness of stimuli”) (Greenwald et al., 1989; Lang et al., 1993). Children with ASD have shown typical SCRs to distress cues (e.g., picture of a crying face) and atypical responses to threatening objects (e.g., picture of a gun; Blair, 1999); both of these types of stimuli are associated with high arousal and negative valence. A study on SCRs during passive viewing of pictures of neutral faces reported increased SCR amplitude in children and adolescents with ASD (Joseph et al., 2008), suggesting increased arousal. However, it is unclear whether increased arousal to facial stimuli is due to eye contact or not: while some studies measuring pupil dilation (Bradley et al., 2008) and SCR (Kylliäinen and Hietanen, 2006) showed increased arousal toward direct eye contact, another study using SCR showed that individuals with ASD did not differ in their SCR based on gaze direction (Joseph et al., 2008). In contrast to these studies in children, adults with ASD showed smaller SCRs relative to TD adults when judging facial emotions from a set of three video clips during which facial expressions evolved from neutral to happy or angry (Hubert et al., 2009). This SCR reduction was interpreted as decreased arousal for emotional faces; however, it was notably specific to an emotion judgment condition and was non-significant when participants judged the age of the same faces. Discrepant findings between studies measuring SCR, that is, hyper- vs. hypo-responses, may point to a developmental trajectory wherein hyper-arousal to faces and direct eye gaze in childhood may attenuate toward hypo-arousal in adulthood. However, differences in the paradigms used across studies (static pictures vs. video-clips, passive viewing vs. emotion-rating) make it difficult to draw firm conclusions. Studies using harmonized task paradigms across a wide age range, and longitudinal work, could help to resolve these questions.
Pupil dilation can measure arousal levels that are non-specific as to valence (Bradley et al., 2008): both pleasant and unpleasant stimuli can evoke dilation during passive viewing of face and non-face stimuli (Anderson et al., 2006; Sepeta et al., 2012). In young children (~4y) with ASD, a pupillary contraction was observed when viewing children's faces that was not seen when viewing landscapes, toys, or animal faces (Anderson et al., 2006). In another study of older children and adolescents, happy faces with gaze directed toward the viewer led to increased pupil diameter in TD but not children with ASD, relative to faces with averted gaze (Sepeta et al., 2012).
Taken together, studies using physiological measures do suggest that there are differences in how social and affective stimuli are processed in individuals with ASD. However, there are inconsistencies as to whether individuals with ASD display hypo- (Hubert et al., 2009; Sepeta et al., 2012) or hyper-arousal (Blair, 1999; Kylliäinen and Hietanen, 2006; Joseph et al., 2008), and the lack of valence information available from physiological techniques makes these results challenging to interpret. Furthermore, large studies that take developmental stage and treatment history into account are lacking, e.g., typical facial muscle and eye blink responses in the studies looking at adult samples could be attributed to successful treatment. Studies including subgroups with different treatment backgrounds, or factoring in the years of treatment, as well as comparing child with adult populations can draw a more detailed picture of symptom trajectories. Nonetheless, this work supports the contention that atypical or variable physiological responses to social and affective stimuli could impact the formation of learned associations between these stimuli and behaviors in individuals with ASD.
Behavioral Responses
While physiological markers such as SCR and pupil diameter suggest differences in emotional processing in individuals with ASD, these measures are non-specific to valence (pleasant or unpleasant), making the interpretation of group differences challenging. A valence-specific measure of how much a human (or animal) values an outcome is how much effort they will exert to obtain it (Hayden et al., 2007). In a “work-to-view” task participants saw pictures of 40 female faces and 20 cars for 800 ms and could either extend the viewing time to up to 5 s by pressing an effortful key combination, or they saw a blank screen until 5 s had passed (Ewing et al., 2013). This task design made it possible for the overall length of the task not to increase when participants “worked to view” a picture. Results showed that children with ASD did not exert less effort to view pictures of faces than a group of TD children (Ewing et al., 2013) and that both groups modulated their effort by the attractiveness of the faces. In contrast, when having to choose to view one of two movie clips that required varying amounts of effort (clips had to be unlocked by pressing a key one, two or three times; Dubey et al., 2015), adults with ASD were less willing to work to view social movies that included direct gaze compared to TD individuals. These effort-based paradigms suggest more typical social “wanting” in children with ASD relative to adults, but given the difference in stimuli (e.g., pictures, movies) it is difficult to draw a firm conclusion. Notably, adults with ASD have also been shown to be differently sensitive to effort expenditure vs. reward, that is, they tend to be more willing to expend effort and are less influenced by changing reward contingencies compared to TD individuals (Damiano et al., 2012), further challenging interpretation of studies using effort-based paradigms.
Overt ratings to assess the subjective value of pictures of people or emotional faces have shown mixed results. While some research has not found a difference between TD and ASD young adults in their rating of subjective pleasantness of emotional facial expressions (Lin et al., 2012), an online study using arousal and valence ratings found significantly lower valence in adults with ASD for “social images” (here, a picture of a single child or adult with a happy expression; Sasson et al., 2012), and higher valence for images related to typical circumscribed interests in ASD (e.g., trains).
Behavioral orienting in the form of attention to social stimuli (eyes, faces, voices) is also reduced in individuals with ASD (Klin et al., 2002; Riby and Hancock, 2008; Hanley et al., 2012). A large online study used the Self-Assessment Manikin (SAM; Bradley and Lang, 1994) to collect valence and arousal ratings on 40 social images (single child or adult with happy expression), 40 High Autism Interest images (HAI; taken from semi-structured parent-report and included trains, electronics, vehicles, construction equipment, airplanes, clocks, blocks, and road signs) and 34 Low Autism Interest images (LAI; clothing, outerwear, office supplies, kitchen supplies, furniture, tools, musical instruments, and plants) (Sasson et al., 2012). They found similar arousal ratings between image categories. However, valence ratings were significantly higher for HAI images compared to social images in the ASD group and the opposite (social images > HAI images) in the TD group (Sasson et al., 2012). Another study showed toddlers videos with 28 scenes of either geometric or social images on monitors next to each other and recorded their eye gaze allocation between monitors as a means of assessing preference (Pierce et al., 2011). They found that toddlers with ASD spent significantly more time on geometric scenes compared to social scenes (Pierce et al., 2011). Studies of visual attention in TD individuals have demonstrated a link between stimulus value and attentional capture (see reviews: Awh et al., 2012; Anderson, 2013), strengthening the argument that differences in allocation of attention in ASD may be related to differences in the relative value of social stimuli.
In summary, effort-based behavioral tasks have an advantage over physiological measures in that they provide valence-specific information. However, an underlying assumption of such tasks is that participants with ASD will exert a similar effort to TD controls for a given reward, which may not be appropriate (Damiano et al., 2012). Nonetheless, the demonstration that individuals with ASD are less willing to work to view social movies with direct gaze (Dubey et al., 2015) suggests reduced motivational value of specific social stimuli. This finding is supported by studies showing lower subjective ratings (Sasson et al., 2012) and reduced attention to social stimuli (Pierce et al., 2011), all of which could indicate that individuals with ASD are less likely to learn when social or socially mediated rewards are used.
Neural Responses
A network of brain regions is sensitive to the hedonic value of stimuli (Bartra et al., 2013). Directly measuring engagement of these regions when an individual is presented with a reward can provide insight into sensitivity to reinforcement. However, this approach should ideally be applied in cooperation with physiological or behavioral measures in order to avoid over-interpreting functional neuroimaging data (Poldrack, 2006).
The ventral striatum (VS), including the nucleus accumbens, receives dopaminergic projections from the ventral tegmental area of the midbrain and plays an important role in reward anticipation (Knutson et al., 2000) and processing of error-feedback during learning (O'Doherty et al., 2003), including social learning (Bray and O'Doherty, 2007). Diminished VS activation in an ASD sample (compared with a TD control sample) during social reward anticipation may indicate diminished social “wanting” (Kohls et al., 2012) and lend support to social motivation theories of ASD (Chevallier et al., 2012).
Several functional neuroimaging studies have used incentive delay tasks (Knutson et al., 2000) during which participants are provided with an incentive for making a speeded response to a target. This task allows investigating neural activity during both the anticipation of a reward and the time a reward is given. One study noted reduced activation in the VS in adults with ASD compared to TD individuals during anticipation of social rewards (pictures of faces; Richey et al., 2014), however, diminished social anticipatory response in the VS has not been consistently reported (Dichter et al., 2012b). Other studies have similarly shown reduced activation in the VS during anticipation of monetary rewards in children (Scott-Van Zeeland et al., 2010) and adults (Dichter et al., 2012b) with ASD compared to TD individuals. While several studies have also noted differences in VS response at the time a reward was given, results have differed in whether response was reduced to social rewards only (Scott-Van Zeeland et al., 2010) or to both social and monetary rewards (Kohls et al., 2013) in ASD. Between these two studies (Scott-Van Zeeland et al., 2010; Kohls et al., 2013) the samples differed only in regards to their medication status with some participants taking psychostimulants, atypical antipsychotics or both in the study by Scott-Van Zeeland et al. (2010) (Table 1). These medications are known to affect dopamine transmission (Seeman, 2002) which is also centrally involved in reward learning and could influence study results. Compared to the studies above, another study used the monetary incentive delay task but only provided negative facial expressions and monetary loss and found hypoactivation in both the VS and caudate in individuals with ASD during anticipation of negative faces (Damiano et al., 2015), whereas only the caudate showed hypoactivation in anticipation of monetary loss. The balance of evidence to date suggests abnormal VS engagement in ASD in response to both appetitive and aversive reinforcers that is not specific to social rewards. These abnormalities may represent a neurobiological marker for diminished incentive salience of rewards in ASD (Kohls et al., 2012).
The ventromedial prefrontal cortex (vmPFC) has been implicated in the representation of stimulus value (Bartra et al., 2013) at the time of outcome (O'Doherty et al., 2004), when evaluating options to make a decision (Kable and Glimcher, 2009) and also during reward imagery (Bray et al., 2010). One study modified the monetary incentive delay task by adding one condition that showed objects which were identified as interesting via semi-structured interviews and eye tracking by the ASD group prior to the task (Dichter et al., 2012a). They found enhanced vmPFC activation at the time of outcome for monetary receipt as well as autism-specific objects of interest in adults with ASD relative to TD controls (Dichter et al., 2012a). A study of children with and without ASD used an incentive go/no-go task during which participants had to either press a button (after seeing a downward arrow) or withhold a button press (upward arrow) and were either rewarded (positive facial expression vs. money gain) or punished (neutral facial expression vs. money loss) depending on task performance (Kohls et al., 2013). Similarly, they found enhanced vmPFC activation at the time children with ASD saw a monetary reward outcome (Kohls et al., 2013). Increased vmPFC (Dichter et al., 2012a) and anterior cingulate (Cascio et al., 2014) activation have also been found in response to autism-specific objects of interest in adults with ASD, relative to TD controls. VmPFC responses to social rewards has been mixed: one study found diminished vmPFC activation for both social and monetary reward feedback in individuals with ASD (Kohls et al., 2013) while two other studies (Scott-Van Zeeland et al., 2010; Dichter et al., 2012b) did not observe a difference in vmPFC for social rewards.
The amygdala plays a role in reward and affective salience (Wassum and Izquierdo, 2015). Increased amygdala activation during social reward anticipation has been shown in adults with ASD and this activation correlated positively with social interaction deficits (Dichter et al., 2012b). In contrast, decreased amygdala activation under social reward conditions has been observed in children with ASD (Kohls et al., 2013). These studies used very similar experimental task designs with comparable reward contingencies (Dichter et al., 2012b; Damiano et al., 2015), suggesting that differential findings in children and adults with ASD might be due to an abnormal developmental trajectory in amygdala reactivity to social incentives (Stanfield et al., 2008; Tottenham and Sheridan, 2009; Schumann et al., 2011).
In general, neuroimaging studies have separately examined response to anticipation and receipt of social, monetary, and “circumscribed interest” rewards. This work has shown attenuated neural response during anticipation of both social and monetary rewards (Scott-Van Zeeland et al., 2010; Dichter et al., 2012a; Richey et al., 2014), but typical or enhanced response at the time of outcome (Dichter et al., 2012b; Kohls et al., 2013; Wassum and Izquierdo, 2015), perhaps indicating a disconnection between the desire to obtain a reward and its hedonic effects. The implications of these findings for reinforcement-based treatment could be that although a reward may be liked or preferred by an individual, it may not consistently provoke the expected anticipatory response, which could affect how associations with this “reward” stimulus are learned. Finally, neuroimaging studies should control for medication status in order to understand whether learning abnormalities are due to ASD symptoms or a possible side effect of medication.
Summary
Physiological, behavioral and neuroimaging measures have each been used to determine whether responses to social and affective stimuli are atypical in ASD. Although there are many inconsistencies in this literature, atypical responses to rewards have been reported in each of these domains. One aspect that remains relatively unexplored, however, is variability between individuals. As the motivational value of rewards is critical to the formation of learned associations, there exists a need to better characterize inter-individual variability in response to social and non-social rewards, as well as developmental trajectories. Notably, to be relevant for treatments starting in early childhood, more research in young children is clearly needed.
Learning in the Context of Aversive Conditioning
While teaching with rewards rather than punishment is emphasized in contemporary behavioral interventions (Schreibman et al., 2015), studies of learning in an aversive context may nonetheless provide insight into atypical associative learning mechanisms. These studies also provide insight into the formation of Pavlovian associations (where a neutral stimulus comes to elicit a conditioned response) in ASD. Studies reviewed in this section are detailed in Table 2.
Table 2. Participant characteristics of reviewed ASD studies in section Learning in the Context of Aversive Conditioning.
Fear Conditioning
In fear conditioning paradigms, neutral stimuli come to elicit a physiological response after repeated pairings with aversive stimuli, such as a mild electric shock or loud noise (Ohman, 2009). Both animal and human lesion evidence points to the amygdala and medial temporal lobes as key structures in the acquisition and expression of conditioned fear (LaBar et al., 1995; Morris et al., 1999; LeDoux, 2000; Phelps and LeDoux, 2005).
Learning a simple cue-outcome association leads to a potentiation of the startle response in ASD that is comparable to TD controls (Salmond et al., 2003; Bernier et al., 2005; South et al., 2011) suggesting appropriate fear conditioning in individuals with ASD. A study involving a more complex discriminative learning task with probabilistic feedback found that although fear responses were acquired in both participants with ASD and IQ-matched TD participants, acquired fear responses were weaker and less discriminative in the ASD group (Gaigg and Bowler, 2007).
Eyeblink Conditioning and the Cerebellum
In classical eyeblink conditioning, an air puff directed at the face (or mild electric shock near the eyes) is repeatedly paired with a cue such as a light or auditory tone, eventually provoking an eyeblink in response to the cue (Schade Powers et al., 2010). This learning relies on the cerebellum acting in concert with brainstem, hippocampal, and striatal regions (Steinmetz, 2000; Gerwig et al., 2007; Cheng et al., 2008, 2014; Thürling et al., 2015), many of which have been suggested to function atypically in ASD (Jou et al., 2009; Wang et al., 2014).
Eyeblink conditioning has been demonstrated in human infants as young as 1 month (Reeb-Sutherland et al., 2011), making children's responses in this paradigm interesting as early markers for neurodevelopmental disorders (Reeb-Sutherland and Fox, 2015), particularly given the suggested link between early cerebellar injury and ASD (Bolduc et al., 2012; Limperopoulos et al., 2014). In children and young adults with ASD, the eyeblink response is learned faster than in TD controls (Sears et al., 1994) and blink timing occurs earlier than in TD individuals once the response is learned. These authors also noted a trend wherein learning was specifically faster for younger children with ASD relative to TD controls. Further study has shown that this behavior may be particular to delay conditioning, in which the stimulus and outcome temporally overlap (Oristaglio et al., 2013). Social outcomes appear to be learned faster than non-social outcomes in 1-month-old TD infants, indicating an early predisposition toward the salience of social cues (Reeb-Sutherland et al., 2011). This is of particular relevance to ASD as deficits in social communication and interactions are a core symptom. Faster learning at 1 month also correlated with emerging social abilities at 9 months of age (Reeb-Sutherland et al., 2012). While these findings are interesting, the positive correlation between faster eyeblink conditioning and social skills in TD infants (Reeb-Sutherland et al., 2012) is opposite to the pattern seen in older children where learning is faster in ASD children (Sears et al., 1994). As such, the potential for eyeblink conditioning as a biomarker for ASD requires further study in a developmental context and should take treatment history into account.
Summary
Aversive conditioning paradigms dependent on amygdalar and cerebellar systems have shown that these associations can be learned in individuals with ASD, sometimes more quickly than in TD individuals (Sears et al., 1994). Fear conditioning studies further suggest intact amygdala reactivity, despite reports of aberrant structure and connectivity (Bauman and Kemper, 1985). Yet, as performance was impaired for a more complex fear learning task, this suggests that differences in fear conditioning may be mediated by an impaired fronto-sub-cortical network rather than amygdala dysfunction per se (Jarrell et al., 1987; Morris et al., 1997). Although these aversive conditioning paradigms are less representative of learning in an “early intervention” context, this literature can nonetheless be informative in a translational context. Specifically, this work suggests that while fearful or unpleasant stimuli are appropriately processed, learning is more efficacious in situations where task contingencies are straightforward.
Learning and Decision-Making
ABA-style therapies rely on operant learning, which can be measured in the laboratory with decision-making tasks: if participants' actions reliably predict a valued outcome, the frequency with which a subject chooses to perform that action provides an index of the strength of the learned association. Studies reviewed in this section are detailed in Table 3.
Table 3. Participant characteristics of reviewed ASD studies Learning and Decision-Making.
In decision-making tasks with probabilistic feedback, young adults with ASD differ in their choice behavior from TD control participants. Lin et al. (2012) used an instrumental learning task during which participants had to learn the reward contingencies of three different slot machines (high-probability, low-probability, or neutral). When monetary rewards were used (win, loss, no change), individuals with ASD did not differ in their choice of the slot machine that had a high-probability of wins compared to TD individuals. However, individuals with ASD chose the high-probability slot machine significantly less if it showed social rewards (happy faces vs. angry or neutral faces) compared to TD individuals. Similarly, another study used a probabilistic selection task, in which participants had to choose between two Japanese symbols with different reward contingencies but were given correct/incorrect feedback after their choice (Solomon et al., 2015). They found that the probability of making the correct choice between two stimuli was lower in ASD participants but only when the reward contingencies were clearest (80/20 vs. 70/30 and 60/40). These authors also noted significantly lower “win-stay” choice behavior (continue choosing the same symbol after given “correct” feedback) in individuals with ASD (Solomon et al., 2015), suggesting more exploratory choice behavior. However, a relatively high number of participants with ASD were excluded from analysis due to failure to learn the task contingencies (7/17 in (Lin et al., 2012), 8/30 in Solomon et al., 2015), meaning that it is not clear how well these findings generalize across the ASD population.
The Iowa Gambling Task requires participants to choose between decks of cards to earn money, and in order to be successful, learn to avoid decks with high risk of large losses. Studies using this task have noted a tendency toward slower learning and more exploratory choice behavior in adolescents with ASD (Johnson et al., 2006; Yechiam et al., 2010; Mussey et al., 2015). However, it is notable that a study of young children (6–7 years) found that choice behavior was similar to TD peers (Faja et al., 2013) and one study of children and adolescents (8–16 years) found superior performance in high functioning ASD participants (South et al., 2014). Mixed results make interpretation difficult: while learning performance seems to be similar between children with and without ASD, studies have shown opposite results for adolescents (Johnson et al., 2006; Yechiam et al., 2010; Faja et al., 2013; Mussey et al., 2015). It would be helpful to know treatment background to understand whether superior performance can be credited to successful treatment programs employing reinforcement learning strategies.
Probabilistic reversal learning is used to assess both learning and cognitive flexibility: task contingencies change only after they are learned. Participants with ASD have shown similar initial learning, but they need a larger number of trials to acquire new contingencies after a reversal and make more regressive errors (D'Cruz et al., 2013). These differences were particularly pronounced in younger participants, suggesting a delay in the maturation of flexible behavioral control.
Further evidence for inflexible learning in ASD comes from the field of perceptual learning. It has been shown that adults with ASD show efficient initial learning, but slower learning when a target location is changed (Harris et al., 2015). This over-specificity could be reversed by reducing stimulus repetition, suggesting that protocols can be modified to circumvent inflexible learning.
While comparable brain networks are engaged during learning in both individuals with ASD and TD controls (Schipul et al., 2012), differences have been found in how these networks adapt over the course of learning (Schipul et al., 2012). One study found reduced medial prefrontal recruitment during early stages of learning a probabilistic selection task in individuals with ASD compared to TD controls and greater orbitofrontal recruitment during later stages, possibly signaling reduced transfer to working memory (Solomon et al., 2015).
Summary
A number of interesting findings have emerged from studies of decision-making in ASD, including reduced choices for high-probability social rewards (Lin et al., 2012), more exploratory choice behavior (Solomon et al., 2015) and less flexible behavior in the context of changing contingencies (Johnson et al., 2006; Yechiam et al., 2010; Mussey et al., 2015). As differences in decision-making may be more pronounced under certain reinforcement schedules (Solomon et al., 2015), a thorough investigation of the influence of probabilistic feedback could be informative for the clinical application of RL. Furthermore, even though a more exploratory learning style in ASD could be interpreted as a general learning difficulty, studies also show that participants with ASD do show initial learning in a range of tasks such as eyeblink conditioning (Sears et al., 1994), operant learning (Salmond et al., 2003; Bernier et al., 2005; South et al., 2011), or perceptual learning (Harris et al., 2015). Learning impairments may instead reveal themselves under changing task contingencies (D'Cruz et al., 2013; Harris et al., 2015) which is in line with the fact that perseveration and inflexibility is often seen in ASD. These findings may represent a fundamental difference between fronto-sub-cortical and cerebellar learning systems; a within-subjects comparison of different learning tasks could shed more light on this question.
Studies of decision making with reward and punishment outcomes can offer a laboratory model for real-world learning situations, and when combined with neuroimaging, elucidate differences in neural mechanisms of learning. However, a gap currently exists in the literature as it is unclear whether performance on laboratory-based tasks translates to real-world learning in the context of behavioral training.
Factors Affecting Response to Reinforcement-Based Intervention
On the whole behavioral treatments appear to be beneficial, however, individuals vary in their response to these programs (Sallows and Graupner, 2005; Sherer and Schreibman, 2005; Schreibman et al., 2009; Perry et al., 2011). Several studies have sought to identify profiles or characteristics of individuals that predict improvements in IQ, adaptive behavior and social communication in response to treatment. Some, but not all (e.g., Smith et al., 2000), studies have identified pre-treatment IQ as a predictor of successful outcomes, with more impaired children showing less improvement (Ben-Itzchak and Zachor, 2007; Perry et al., 2011). Children who show greater initial social responsiveness, language skills and approach behaviors toward both adults and toys have also been shown to benefit more (Sallows and Graupner, 2005; Sherer and Schreibman, 2005; Schreibman et al., 2009). Starting intensive treatment at an earlier age has also been suggested to lead to greater gains (Harris and Handleman, 2000).
An important concept in the successful deployment of reinforcement-based intervention is reinforcer preference assessment. That is, since using the individual's own preferred stimuli as reinforcers is presumed to be most effective, systematic procedure to identify such stimuli (Kang et al., 2013), or their preferred magnitudes (Trosclair-Lasserre et al., 2008), have been developed. For example, behavioral procedures that present individuals with an array of stimuli and assess their approach/choice behaviors over multiple trials have been developed and are particularly critical for identifying appropriate reinforcers when working with young and non-verbal children. A study with TD children showed that preferred stimuli may be more effective at eliciting desired behaviors as schedule requirements during learning become more challenging (Penrod et al., 2008), thus making accurate preference assessment important for the success of learning. Interestingly, for children with ASD, the number of socially mediated reinforcers a child enjoys has been shown to predict better outcome, while the number of automatic reinforcers predicts poorer outcome (Klintwall and Eikeseth, 2012). Here, socially mediated reinforcers were defined as reinforcers that a practitioner could provide to a child (e.g., food, toys) while automatic reinforcers are those behaviors that a child can produce themselves (e.g., rocking, hand flapping). Though this was relatively small, retrospective, study, these intriguing results suggest that how an individual responds to the types of reinforcers that practitioners provide can have a bearing on that individual's learning and eventual outcome.
Implications for Treatment and Questions for Future Research
Despite the obvious connection between RL and behavioral therapies, few studies have examined whether response to reinforcement, or learning characteristics, are predictive of treatment outcomes (Klintwall and Eikeseth, 2012). However, such work could pave the way for modifying protocols to improve outcomes for children who are “non-responders.” Indeed, variation in learning and motivation could be used to define subgroups of individuals to whom targeted interventions could be provided (Lai et al., 2013).
Despite some suggestions of aberrant response to social and non-social rewards and differences in learning in ASD, a clear and consistent physiological, behavioral, or neural marker for reduced or atypical motivation is currently lacking. Variation between studies may be due to differences in paradigms, but may also be caused by inter-individual variability, which few studies have explicitly explored. Such a marker could be useful in the context of assessing reinforcer preference (Kang et al., 2013), determining the optimal magnitude of a reinforcer (Trosclair-Lasserre et al., 2008), and predicting the likelihood of response to intervention (Klintwall and Eikeseth, 2012). For example, in those individuals with reduced or inconsistent response to rewards, greater training intensity may be necessary. Thus, this work could have important implications for treatment funding and policy. At the same time, inter-individual variability places a challenge for generalizing research findings to clinical interventions that address a wide range of individuals on the autism spectrum. Specifically, most studies have investigated individuals with high functioning autism (full-scale and verbal IQ scores ≥ 70) while intervention programs often target children with more severe forms of ASD and related cognitive disabilities. Nevertheless, the reviewed research builds a foundation for evidence-based practice and can inform which direction clinical trials have to take to further develop interventions.
Functional imaging results in ASD hint at a disconnection between the desire to obtain a reward and its hedonic effects, as has been suggested in disorders such as addiction (Robinson and Berridge, 1993), depression and schizophrenia (Rømer Thomsen et al., 2015). Indeed, given the potential similarities in abnormal learning processes across psychiatric diagnoses, this could be a useful cross-diagnostic research domain within the Research Domain Criteria (RDoC; Insel et al., 2010). In individuals with major depression, pharmacological manipulation of dopamine has been shown to improve learning (Admon et al., 2016), a strategy that could be tested in treatment-resistant individuals with ASD.
Conclusions
The strategies harnessed by ABA-style behavioral treatment approaches are complex, and despite widespread use, our understanding of how these therapies affect behavioral and cognitive changes in ASD is limited. However, accumulating evidence points to abnormalities in the processing of reinforcers and differences in learning, and flexibly adapting, in ASD. Improvements in therapeutic efficacy may be achieved through a better understanding of functional abnormalities in learning systems in ASD. Biologically informed markers for response to rewards could help individualize treatment protocols in terms of intensity and reinforcer schedules. For individuals where there are clear differences in response to reward that impede treatment progress, pharmacological (Admon et al., 2016), or brain stimulation approaches (Reinhart et al., 2015), in combination with ABA-style therapy, could be investigated as a way of improving treatment outcomes.
Author Contributions
All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.
Funding
MS was funded by the Alberta Children's Hospital Research Institute for Child and Maternal Health. CR was funded by the Eyes High Scholars program at the University of Calgary. SB was funded by the SickKids Foundation, CIHR-IHDCYH and the Sinneave Family Foundation.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Admon, R., Kaiser, R. H., Dillon, D. G., Beltzer, M., Goer, F., Olson, D. P., et al. (2016). Dopaminergic enhancement of striatal response to reward in major depression. Am. J. Psychiatry 174, 378–386. doi: 10.1176/appi.ajp.2016.16010111
Allen, K. E., Hart, B., Buell, J. S., Harris, F. R., and Wolf, M. M. (1964). Effects of social reinforcement on isolate behavior of a nursery school child. Child Dev. 35, 511. doi: 10.2307/1126714
American Psychiatric Association (2013). Diagnostic and Statistical Manual of Mental Disorders: DSM-5. Washington, DC: American Psychiatric Association.
Anderson, B. A. (2013). A value-driven mechanism of attentional selection. J. Vis. 13:7. doi: 10.1167/13.3.7
Anderson, C. J., Colombo, J., and Jill Shaddy, D. (2006). Visual scanning and pupillary responses in young children with Autism Spectrum Disorder. J. Clin. Exp. Neuropsychol. 28, 1238–1256. doi: 10.1080/13803390500376790
Awh, E., Belopolsky, A. V, and Theeuwes, J. (2012). Top-down versus bottom-up attentional control: a failed theoretical dichotomy. Trends Cogn. Sci. 16, 437–443. doi: 10.1016/j.tics.2012.06.010
Baron-Cohen, S., Ring, H. A., Bullmore, E. T., Wheelwright, S., Ashwin, C., and Williams, S. C. (2000). The amygdala theory of autism. Neurosci. Biobehav. Rev. 24, 355–364. doi: 10.1016/S0149-7634(00)00011-7
Bartra, O., McGuire, J. T., and Kable, J. W. (2013). The valuation system: a coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. Neuroimage 76, 412–427. doi: 10.1016/j.neuroimage.2013.02.063
Bauman, M., and Kemper, T. L. (1985). Histoanatomic observations of the brain in early infantile autism. Neurology 35, 866–874. doi: 10.1212/WNL.35.6.866
Ben-Itzchak, E., and Zachor, D. A. (2007). The effects of intellectual functioning and autism severity on outcome of early behavioral intervention for children with autism. Res. Dev. Disabil. 28, 287–303. doi: 10.1016/j.ridd.2006.03.002
Bernier, R., Dawson, G., Panagiotides, H., and Webb, S. (2005). Individuals with autism spectrum disorder show normal responses to a fear potential startle paradigm. J. Autism Dev. Disord. 35, 575–583. doi: 10.1007/s10803-005-0002-0
Blair, R. J. R. (1999). Psychophysiological responsiveness to the distress ofothers in children with autism. Pers. Individ. Dif. 26, 477–485. doi: 10.1016/S0191-8869(98)00154-8
Bolduc, M.-E., du Plessis, A. J., Sullivan, N., Guizard, N., Zhang, X., Robertson, R. L., et al. (2012). Regional cerebellar volumes predict functional outcome in children with cerebellar malformations. Cerebellum 11, 531–542. doi: 10.1007/s12311-011-0312-z
Bradley, M. M., and Lang, P. J. (1994). Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25, 49–59. doi: 10.1016/0005-7916(94)90063-9
Bradley, M. M., Miccoli, L., Escrig, M. A., and Lang, P. J. (2008). The pupil as a measure of emotional arousal and autonomic activation. Psychophysiology 45, 602–607. doi: 10.1111/j.1469-8986.2008.00654.x
Bray, S., and O'Doherty, J. (2007). Neural coding of reward-prediction error signals during classical conditioning with attractive faces. J. Neurophysiol. 97, 3036–3045. doi: 10.1152/jn.01211.2006
Bray, S., Shimojo, S., and O'Doherty, J. (2010). Human medial orbitofrontal cortex is recruited during experience of imagined and real rewards. J. Neurophysiol. 103, 2506–2512. doi: 10.1152/jn.01030.2009
Cascio, C. J., Foss-Feig, J. H., Heacock, J., Schauder, K. B., Loring, W. A., Rogers, B. P., et al. (2014). Affective neural response to restricted interests in autism spectrum disorders. J. Child Psychol. Psychiatry 55, 162–171. doi: 10.1111/jcpp.12147
Cattaneo, L., Fabbri-Destro, M., Boria, S., Pieraccini, C., Monti, A., Cossu, G., et al. (2007). Impairment of actions chains in autism and its possible role in intention understanding. Proc. Natl. Acad. Sci. U.S.A. 104, 17825–17830. doi: 10.1073/pnas.0706273104
Cheng, D. T., Disterhoft, J. F., Power, J. M., Ellis, D. A., and Desmond, J. E. (2008). Neural substrates underlying human delay and trace eyeblink conditioning. Proc. Natl. Acad. Sci. U.S.A. 105, 8108–8113. doi: 10.1073/pnas.0800374105
Cheng, D. T., Meintjes, E. M., Stanton, M. E., Desmond, J. E., Pienaar, M., Dodge, N. C., et al. (2014). Functional MRI of cerebellar activity during eyeblink classical conditioning in children and adults. Hum. Brain Mapp. 35, 1390–1403. doi: 10.1002/hbm.22261
Chevallier, C., Kohls, G., Troiani, V., Brodkin, E. S., and Schultz, R. T. (2012). The social motivation theory of autism. Trends Cogn. Sci. 16, 231–239. doi: 10.1016/j.tics.2012.02.007
Constantino, J. N., and Gruber, C. P. (2012). Social Responsiveness Scale (SRS). Torrance, CA: Western Psychological Services.
Damiano, C. R., Aloi, J., Treadway, M., Bodfish, J. W., and Dichter, G. (2012). Adults with autism spectrum disorders exhibit decreased sensitivity to reward parameters when making effort-based decisions. J. Neurodev. Disord. 4:13. doi: 10.1186/1866-1955-4-13
Damiano, C. R., Cockrell, D. C., Dunlap, K., Hanna, E. K., Miller, S., Bizzell, J., et al. (2015). Neural mechanisms of negative reinforcement in children and adolescents with autism spectrum disorders. J. Neurodev. Disord. 7:12. doi: 10.1186/s11689-015-9107-8
Davison, G. C. (1964). A social learning therapy programme with an autistic child. Behav. Res. Ther. 2, 149–159. doi: 10.1016/0005-7967(64)90009-9
Dawson, G., and Burner, K. (2011). Behavioral interventions in children and adolescents with autism spectrum disorder. Curr. Opin. Pediatr. 23, 616–620. doi: 10.1097/MOP.0b013e32834cf082
Dawson, G., Jones, E. J. H., Merkle, K., Venema, K., Lowy, R., Faja, S., et al. (2012). Early behavioral intervention is associated with normalized brain activity in young children with autism. J. Am. Acad. Child Adolesc. Psychiatry 51, 1150–1159. doi: 10.1016/j.jaac.2012.08.018
Dawson, G., Rogers, S., Munson, J., Smith, M., Winter, J., Greenson, J., et al. (2010). Randomized, controlled trial of an intervention for toddlers with autism: the early start denver model. Pediatrics 125, e17–e23. doi: 10.1542/peds.2009-0958
D'Cruz, A.-M., Ragozzino, M. E., Mosconi, M. W., Shrestha, S., Cook, E. H., and Sweeney, J. A. (2013). Reduced behavioral flexibility in autism spectrum disorders. Neuropsychology 27, 152–160. doi: 10.1037/a0031721
Deschamps, P. K. H., Coppes, L., Kenemans, J. L., Schutter, D. J. L. G., and Matthys, W. (2015). Electromyographic responses to emotional facial expressions in 6–7 year olds with autism spectrum disorders. J. Autism Dev. Disord. 45, 354–362. doi: 10.1007/s10803-013-1890-z
Dichter, G., Felder, J. N., Green, S. R., Rittenberg, A. M., Sasson, N. J., and Bodfish, J. W. (2012a). Reward circuitry function in autism spectrum disorders. Soc. Cogn. Affect. Neurosci. 7, 160–172. doi: 10.1093/scan/nsq095
Dichter, G. S., Benning, S. D., Holtzclaw, T. N., and Bodfish, J. W. (2010). Affective modulation of the startle eyeblink and postauricular reflexes in autism spectrum disorder. J. Autism Dev. Disord. 40, 858–869. doi: 10.1007/s10803-009-0925-y
Dichter, G. S., Richey, J. A., Rittenberg, A. M., Sabatino, A., and Bodfish, J. W. (2012b). Reward circuitry function in autism during face anticipation and outcomes. J. Autism Dev. Disord. 42, 147–160. doi: 10.1007/s10803-011-1221-1
Dubey, I., Ropar, D., and de Hamilton, A. F. C. (2015). Measuring the value of social engagement in adults with and without autism. Mol. Autism 6:35. doi: 10.1186/s13229-015-0031-2
Eikeseth, S. (2009). Outcome of comprehensive psycho-educational interventions for young children with autism. Res. Dev. Disabil. 30, 158–178. doi: 10.1016/j.ridd.2008.02.003
Ewing, L., Pellicano, E., and Rhodes, G. (2013). Using effort to measure reward value of faces in children with autism. PLoS ONE 8:e79493. doi: 10.1371/journal.pone.0079493
Faja, S., Murias, M., Beauchaine, T. P., and Dawson, G. (2013). Reward-based decision making and electrodermal responding by young children with autism spectrum disorders during a gambling task. Autism Res. 6, 494–505. doi: 10.1002/aur.1307
Gaigg, S. B., and Bowler, D. M. (2007). Differential fear conditioning in Asperger's syndrome: implications for an amygdala theory of autism. Neuropsychologia 45, 2125–2134. doi: 10.1016/j.neuropsychologia.2007.01.012
Gerwig, M., Kolb, F. P., and Timmann, D. (2007). The involvement of the human cerebellum in eyeblink conditioning. Cerebellum 6, 38–57. doi: 10.1080/14734220701225904
Grant, M. J., and Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info. Libr. J. 26, 91–108. doi: 10.1111/j.1471-1842.2009.00848.x
Greenwald, M. K., Cook, E. W., and Lang, P. J. (1989). Affective judgment and psychophysiological response: dimensional covariation in the evaluation of pictorial stimuli. J. Psychopathol. 3, 51–64.
Hanley, M., McPhillips, M., Mulhern, G., and Riby, D. M. (2012). Spontaneous attention to faces in Asperger syndrome using ecologically valid static stimuli. Autism 17, 754–761. doi: 10.1177/1362361312456746
Harris, H., Israeli, D., Minshew, N., Bonneh, Y., Heeger, D. J., Behrmann, M., et al. (2015). Perceptual learning in autism: over-specificity and possible remedies. Nat. Neurosci. 18, 1574–1576. doi: 10.1038/nn.4129
Harris, S. L., and Handleman, J. S. (2000). Age and IQ at intake as predictors of placement for young children with autism: a four- to six-year follow-up. J. Autism Dev. Disord. 30, 137–142. doi: 10.1023/A:1005459606120
Hayden, B. Y., Parikh, P. C., Deaner, R. O., and Platt, M. L. (2007). Economic principles motivating social attention in humans. Proc. Biol. Sci. 274, 1751–1756. doi: 10.1098/rspb.2007.0368
Hess, U., and Blairy, S. (2001). Facial mimicry and emotional contagion to dynamic emotional facial expressions and their influence on decoding accuracy. Int. J. Psychophysiol. 40, 129–41. doi: 10.1016/S0167-8760(00)00161-6
Hubert, B. E., Wicker, B., Monfardini, E., and Deruelle, C. (2009). Electrodermal reactivity to emotion processing in adults with autistic spectrum disorders. Autism 13, 9–19. doi: 10.1177/1362361308091649
Insel, T., Cuthbert, B., Garvey, M., Heinssen, R., Pine, D. S., Quinn, K., et al. (2010). Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am. J. Psychiatry 167, 748–751. doi: 10.1176/appi.ajp.2010.09091379
Jarrell, T. W., Gentile, C. G., Romanski, L. M., McCabe, P. M., and Schneiderman, N. (1987). Involvement of cortical and thalamic auditory regions in retention of differential bradycardiac conditioning to acoustic conditioned stimuli in rabbits. Brain Res. 412, 285–294. doi: 10.1016/0006-8993(87)91135-8
Jensen, G. D., and Womack, M. G. (1967). Operant conditioning techniques applied in the treatment of an autistic child. Am. J. Orthopsychiatry 37, 30–34. doi: 10.1111/j.1939-0025.1967.tb01064.x
Johnson, S. A., Yechiam, E., Murphy, R. R., Queller, S., and Stout, J. C. (2006). Motivational processes and autonomic responsivity in Asperger's disorder: evidence from the Iowa Gambling Task. J. Int. Neuropsychol. Soc. 12, 668–676. doi: 10.1017/S1355617706060802
Joseph, R. M., Ehrman, K., McNally, R., and Keehn, B. (2008). Affective response to eye contact and face recognition ability in children with ASD. J. Int. Neuropsychol. Soc. 14, 947–955. doi: 10.1017/S1355617708081344
Jou, R. J., Minshew, N. J., Melhem, N. M., Keshavan, M. S., and Hardan, A. Y. (2009). Brainstem volumetric alterations in children with autism. Psychol. Med. 39, 1347–1354. doi: 10.1017/S0033291708004376
Kable, J. W., and Glimcher, P. W. (2009). The neurobiology of decision: consensus and controversy. Neuron 63, 733–745. doi: 10.1016/j.neuron.2009.09.003
Kang, S., O'Reilly, M., Lancioni, G., Falcomata, T. S., Sigafoos, J., and Xu, Z. (2013). Comparison of the predictive validity and consistency among preference assessment procedures: a review of the literature. Res. Dev. Disabil. 34, 1125–1133. doi: 10.1016/j.ridd.2012.12.021
Klin, A., Jones, W., Schultz, R., Volkmar, F., and Cohen, D. (2002). Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Arch. Gen. Psychiatry 59, 809–16. doi: 10.1001/archpsyc.59.9.809
Klintwall, L., and Eikeseth, S. (2012). Number and controllability of reinforcers as predictors of individual outcome for children with autism receiving early and intensive behavioral intervention: a preliminary study. Res. Autism Spectr. Disord. 6, 493–499. doi: 10.1016/j.rasd.2011.07.009
Knutson, B., Westdorp, A., Kaiser, E., and Hommer, D. (2000). FMRI visualization of brain activity during a monetary incentive delay task. Neuroimage 12, 20–27. doi: 10.1006/nimg.2000.0593
Kohls, G., Chevallier, C., Troiani, V., and Schultz, R. T. (2012). Social “wanting” dysfunction in autism: neurobiological underpinnings and treatment implications. J. Neurodev. Disord. 4:10. doi: 10.1186/1866-1955-4-10
Kohls, G., Schulte-Rüther, M., Nehrkorn, B., Müller, K., Fink, G. R., Kamp-Becker, I., et al. (2013). Reward system dysfunction in autism spectrum disorders. Soc. Cogn. Affect. Neurosci. 8, 565–572. doi: 10.1093/scan/nss033
Kylliäinen, A., and Hietanen, J. K. (2006). Skin conductance responses to another person's gaze in children with autism. J. Autism Dev. Disord. 36, 517–525. doi: 10.1007/s10803-006-0091-4
LaBar, K. S., LeDoux, J. E., Spencer, D. D., and Phelps, E. A. (1995). Impaired fear conditioning following unilateral temporal lobectomy in humans. J. Neurosci. 15, 6846–6855.
Lai, M.-C., Lombardo, M. V, Chakrabarti, B., and Baron-Cohen, S. (2013). Subgrouping the autism “spectrum”: reflections on DSM-5. PLoS Biol. 11:e1001544. doi: 10.1371/journal.pbio.1001544
Lang, P. J., Greenwald, M. K., Bradley, M. M., and Hamm, A. O. (1993). Looking at pictures: affective, facial, visceral, and behavioral reactions. Psychophysiology 30, 261–273. doi: 10.1111/j.1469-8986.1993.tb03352.x
LeDoux, J. E. (2000). Emotion circuits in the brain. Annu. Rev. Neurosci. 23, 155–184. doi: 10.1146/annurev.neuro.23.1.155
Limperopoulos, C., Chilingaryan, G., Sullivan, N., Guizard, N., Robertson, R. L., and du Plessis, A. J. (2014). Injury to the premature cerebellum: outcome is related to remote cortical development. Cereb. Cortex 24, 728–736. doi: 10.1093/cercor/bhs354
Lin, A., Rangel, A., and Adolphs, R. (2012). Impaired learning of social compared to monetary rewards in autism. Front. Neurosci. 6:143. doi: 10.3389/fnins.2012.00143
Martin, G. L., England, G., Kaprowy, E., Kilgour, K., and Pilek, V. (1968). Operant conditioning of kindergarten-class behavior in autistic children*. Behav. Res. Ther. 6, 281–294. doi: 10.1016/0005-7967(68)90062-4
Mathersul, D., McDonald, S., and Rushby, J. A. (2013a). Automatic facial responses to affective stimuli in high-functioning adults with autism spectrum disorder. Physiol. Behav. 109, 14–22. doi: 10.1016/j.physbeh.2012.10.008
Mathersul, D., McDonald, S., and Rushby, J. A. (2013b). Automatic facial responses to briefly presented emotional stimuli in autism spectrum disorder. Biol. Psychol. 94, 397–407. doi: 10.1016/j.biopsycho.2013.08.004
Metz, J. R. (1965). Conditioning generalized imitation in autistic children. J. Exp. Child Psychol. 2, 389–399. doi: 10.1016/0022-0965(65)90023-8
Morris, J. S., Friston, K. J., and Dolan, R. J. (1997). Neural responses to salient visual stimuli. Proc. Biol. Sci. 264, 769–775. doi: 10.1098/rspb.1997.0109
Morris, J. S., Ohman, A., and Dolan, R. J. (1999). A subcortical pathway to the right amygdala mediating “unseen” fear. Proc. Natl. Acad. Sci. U.S.A. 96, 1680–1685. doi: 10.1073/pnas.96.4.1680
Mussey, J. L., Travers, B. G., Klinger, L. G., and Klinger, M. R. (2015). Decision-making skills in ASD: performance on the iowa gambling task. Autism Res. 8, 105–114. doi: 10.1002/aur.1429
Oberman, L. M., Winkielman, P., and Ramachandran, V. S. (2009). Slow echo: facial EMG evidence for the delay of spontaneous, but not voluntary, emotional mimicry in children with autism spectrum disorders. Dev. Sci. 12, 510–520. doi: 10.1111/j.1467-7687.2008.00796.x
O'Doherty, J., Dayan, P., Friston, K., Critchley, H., and Dolan, R. J. (2003). Temporal difference models and reward-related learning in the human brain. Neuron 38, 329–337. doi: 10.1016/S0896-6273(03)00169-7
O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., and Dolan, R. J. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454. doi: 10.1126/science.1094285
Ohman, A. (2009). Of snakes and faces: an evolutionary perspective on the psychology of fear. Scand. J. Psychol. 50, 543–552. doi: 10.1111/j.1467-9450.2009.00784.x
Oristaglio, J., Hyman West, S., Ghaffari, M., Lech, M. S., Verma, B. R., Harvey, J. A., et al. (2013). Children with autism spectrum disorders show abnormal conditioned response timing on delay, but not trace, eyeblink conditioning. Neuroscience 248, 708–718. doi: 10.1016/j.neuroscience.2013.06.007
Penrod, B., Wallace, M. D., and Dyer, E. J. (2008). Assessing potency of high- and low-preference reinforcers with respect to response rate and response patterns. J. Appl. Behav. Anal. 41, 177–188. doi: 10.1901/jaba.2008.41-177
Perry, A., Cummings, A., Geier, J. D., Freeman, N. L., Hughes, S., Managhan, T., et al. (2011). Predictors of outcome for children receiving intensive behavioral intervention in a large, community-based program. Res. Autism Spectr. Disord. 5, 592–603. doi: 10.1016/j.rasd.2010.07.003
Phelps, E. A., and LeDoux, J. E. (2005). Contributions of the amygdala to emotion processing: from animal models to human behavior. Neuron 48, 175–187. doi: 10.1016/j.neuron.2005.09.025
Pierce, K., Conant, D., Hazin, R., Stoner, R., and Desmond, J. (2011). Preference for geometric patterns early in life as a risk factor for autism. Arch. Gen. Psychiatry 68, 101–109. doi: 10.1001/archgenpsychiatry.2010.113
Poldrack, R. A. (2006). Can cognitive processes be inferred from neuroimaging data? Trends Cogn. Sci. 10, 59–63. doi: 10.1016/j.tics.2005.12.004
Reeb-Sutherland, B. C., Fifer, W. P., Byrd, D. L., Hammock, E. A. D., Levitt, P., and Fox, N. A. (2011). One-month-old human infants learn about the social world while they sleep. Dev. Sci. 14, 1134–1141. doi: 10.1111/j.1467-7687.2011.01062.x
Reeb-Sutherland, B. C., and Fox, N. A. (2015). Eyeblink conditioning: a non-invasive biomarker for neurodevelopmental disorders. J. Autism Dev. Disord. 45, 376–394. doi: 10.1007/s10803-013-1905-9
Reeb-Sutherland, B. C., Levitt, P., and Fox, N. A. (2012). The predictive nature of individual differences in early associative learning and emerging social behavior. PLoS ONE 7:e30511. doi: 10.1371/journal.pone.0030511
Reichow, B., Barton, E. E., Boyd, B. A., and Hume, K. (2012). Early intensive behavioral intervention (EIBI) for young children with autism spectrum disorders (ASD). Cochrane Database Syst. Rev. 10:CD009260. doi: 10.1002/14651858.CD009260.pub2
Reinhart, R. M. G., Zhu, J., Park, S., and Woodman, G. F. (2015). Medial-frontal stimulation enhances learning in schizophrenia by restoring prediction error signaling. J. Neurosci. 35, 12232–12240. doi: 10.1523/JNEUROSCI.1717-15.2015
Remington, B., Hastings, R. P., Kovshoff, H., degli Espinosa, F., Jahr, E., Brown, T., et al. (2007). Early intensive behavioral intervention: outcomes for children with autism and their parents after two years. Am. J. Ment. Retard. 112, 418–438. doi: 10.1352/0895-8017(2007)112[418:EIBIOF]2.0.CO;2
Riby, D. M., and Hancock, P. J. B. (2008). Viewing it differently: social scene perception in Williams syndrome and autism. Neuropsychologia 46, 2855–2860. doi: 10.1016/j.neuropsychologia.2008.05.003
Richey, J. A., Rittenberg, A., Hughes, L., Damiano, C. R., Sabatino, A., Miller, S., et al. (2014). Common and distinct neural features of social and non-social reward processing in autism and social anxiety disorder. Soc. Cogn. Affect. Neurosci. 9, 367–377. doi: 10.1093/scan/nss146
Robinson, T. E., and Berridge, K. C. (1993). The neural basis of drug craving: an incentive-sensitization theory of addiction. Brain Res. Brain Res. Rev. 18, 247–291. doi: 10.1016/0165-0173(93)90013-P
Rømer Thomsen, K., Whybrow, P. C., and Kringelbach, M. L. (2015). Reconceptualizing anhedonia: novel perspectives on balancing the pleasure networks in the human brain. Front. Behav. Neurosci. 9:49. doi: 10.3389/fnbeh.2015.00049
Sallows, G. O., and Graupner, T. D. (2005). Intensive behavioral treatment for children with autism: four-year outcome and predictors. Am. J. Ment. Retard. 110, 417–438. doi: 10.1352/0895-8017(2005)110[417:IBTFCW]2.0.CO;2
Salmond, C. H., de Haan, M., Friston, K. J., Gadian, D. G., and Vargha-Khadem, F. (2003). Investigating individual differences in brain abnormalities in autism. Philos. Trans. R. Soc. Lond. B Biol. Sci. 358, 405–413. doi: 10.1098/rstb.2002.1210
Sasson, N. J., Dichter, G. S., and Bodfish, J. W. (2012). Affective responses by adults with autism are reduced to social images but elevated to images related to circumscribed interests. PLoS ONE 7:e42457. doi: 10.1371/journal.pone.0042457
Schade Powers, A., Coburn-Litvak, P., and Evinger, C. (2010). Conditioned eyelid movement is not a blink. J. Neurophysiol. 103, 641–647. doi: 10.1152/jn.00631.2009
Schipul, S. E., Williams, D. L., Keller, T. A., Minshew, N. J., and Just, M. A. (2012). Distinctive neural processes during learning in autism. Cereb. Cortex 22, 937–950. doi: 10.1093/cercor/bhr162
Schreibman, L., Dawson, G., Stahmer, A. C., Landa, R., Rogers, S. J., McGee, G. G., et al. (2015). Naturalistic developmental behavioral interventions: empirically validated treatments for autism spectrum disorder. J. Autism Dev. Disord. 45, 2411–2428. doi: 10.1007/s10803-015-2407-8
Schreibman, L., Stahmer, A. C., Barlett, V. C., and Dufek, S. (2009). Brief report: toward refinement of a predictive behavioral profile for treatment outcome in children with autism. Res. Autism Spectr. Disord. 3, 163–172. doi: 10.1016/j.rasd.2008.04.008.
Schumann, C. M., Bauman, M. D., and Amaral, D. G. (2011). Abnormal structure or function of the amygdala is a common component of neurodevelopmental disorders. Neuropsychologia 49, 745–759. doi: 10.1016/j.neuropsychologia.2010.09.028
Scott-Van Zeeland, A. A., Dapretto, M., Ghahremani, D. G., Poldrack, R. A., and Bookheimer, S. Y. (2010). Reward processing in autism. Autism Res. 3, 53–67. doi: 10.1002/aur.122
Sears, L. L., Finn, P. R., and Steinmetz, J. E. (1994). Abnormal classical eye-blink conditioning in autism. J. Autism Dev. Disord. 24, 737–751. doi: 10.1007/BF02172283
Seeman, P. (2002). Atypical antipsychotics: mechanism of action. Can. J. Psychiatry 47, 27–38. doi: 10.1177/070674370204700106
Sepeta, L., Tsuchiya, N., Davies, M. S., Sigman, M., Bookheimer, S. Y., and Dapretto, M. (2012). Abnormal social reward processing in autism as indexed by pupillary responses to happy faces. J. Neurodev. Disord. 4:17. doi: 10.1186/1866-1955-4-17
Sherer, M. R., and Schreibman, L. (2005). Individual behavioral profiles and predictors of treatment effectiveness for children with autism. J. Consult. Clin. Psychol. 73, 525–538. doi: 10.1037/0022-006X.73.3.525
Sims, T. B., Van Reekum, C. M., Johnstone, T., and Chakrabarti, B. (2012). How reward modulates mimicry: EMG evidence of greater facial mimicry of more rewarding happy faces. Psychophysiology 49, 998–1004. doi: 10.1111/j.1469-8986.2012.01377.x
Smith, T., Groen, A. D., and Wynn, J. W. (2000). Randomized trial of intensive early intervention for children with pervasive developmental disorder. Am. J. Ment. Retard. 105, 269–285. doi: 10.1352/0895-8017(2000)105<0269:RTkoOIEI>2.0.CO;2
Solomon, M., Frank, M. J., Ragland, J. D., Smith, A. C., Niendam, T. A., Lesh, T. A., et al. (2015). Feedback-driven trial-by-trial learning in autism spectrum disorders. Am. J. Psychiatry 172, 173–181. doi: 10.1176/appi.ajp.2014.14010036
South, M., Chamberlain, P. D., Wigham, S., Newton, T., Le Couteur, A., McConachie, H., et al. (2014). Enhanced decision making and risk avoidance in high-functioning autism spectrum disorder. Neuropsychology 28, 222–228. doi: 10.1037/neu0000016
South, M., Larson, M. J., White, S. E., Dana, J., and Crowley, M. J. (2011). Better fear conditioning is associated with reduced symptom severity in autism spectrum disorders. Autism Res. 4, 412–421. doi: 10.1002/aur.221
Stanfield, A. C., McIntosh, A. M., Spencer, M. D., Philip, R., Gaur, S., and Lawrie, S. M. (2008). Towards a neuroanatomy of autism: a systematic review and meta-analysis of structural magnetic resonance imaging studies. Eur. Psychiatry 23, 289–299. doi: 10.1016/j.eurpsy.2007.05.006
Steinmetz, J. E. (2000). Brain substrates of classical eyeblink conditioning: a highly localized but also distributed system. Behav. Brain Res. 110, 13–24. doi: 10.1016/S0166-4328(99)00181-3
Sullivan, K., Stone, W. L., and Dawson, G. (2014). Potential neural mechanisms underlying the effectiveness of early intervention for children with autism spectrum disorder. Res. Dev. Disabil. 35, 2921–2932. doi: 10.1016/j.ridd.2014.07.027
Thürling, M., Kahl, F., Maderwald, S., Stefanescu, R. M., Schlamann, M., Boele, H.-J., et al. (2015). Cerebellar cortex and cerebellar nuclei are concomitantly activated during eyeblink conditioning: a 7T fMRI study in humans. J. Neurosci. 35, 1228–1239. doi: 10.1523/JNEUROSCI.2492-14.2015
Tottenham, N., and Sheridan, M. A. (2009). A review of adversity, the amygdala and the hippocampus: a consideration of developmental timing. Front. Hum. Neurosci. 3:68. doi: 10.3389/neuro.09.068.2009
Trosclair-Lasserre, N. M., Lerman, D. C., Call, N. A., Addison, L. R., and Kodak, T. (2008). Reinforcement magnitude: an evaluation of preference and reinforcer efficacy. J. Appl. Behav. Anal. 41, 203–220. doi: 10.1901/jaba.2008.41-203
Virués-Ortega, J. (2010). Applied behavior analytic intervention for autism in early childhood: meta-analysis, meta-regression and dose-response meta-analysis of multiple outcomes. Clin. Psychol. Rev. 30, 387–399. doi: 10.1016/j.cpr.2010.01.008
Wang, S. S.-H., Kloth, A. D., and Badura, A. (2014). The cerebellum, sensitive periods, and autism. Neuron 83, 518–532. doi: 10.1016/j.neuron.2014.07.016
Wassum, K. M., and Izquierdo, A. (2015). The basolateral amygdala in reward learning and addiction. Neurosci. Biobehav. Rev. 57, 271–283. doi: 10.1016/j.neubiorev.2015.08.017
Wilbarger, J. L., McIntosh, D. N., and Winkielman, P. (2009). Startle modulation in autism: positive affective stimuli enhance startle response. Neuropsychologia 47, 1323–1331. doi: 10.1016/j.neuropsychologia.2009.01.025
Wolf, M., Risley, T., and Mees, H. (1963). Application of operant conditioning procedures to the behaviour problems of an autistic child. Behav. Res. Ther. 1, 305–312. doi: 10.1016/0005-7967(63)90045-7
Keywords: reinforcement learning, autism spectrum disorder, conditioning, reward, neuroimaging
Citation: Schuetze M, Rohr CS, Dewey D, McCrimmon A and Bray S (2017) Reinforcement Learning in Autism Spectrum Disorder. Front. Psychol. 8:2035. doi: 10.3389/fpsyg.2017.02035
Received: 07 June 2017; Accepted: 07 November 2017;
Published: 21 November 2017.
Edited by:
Natasha Kirkham, Cornell University, United StatesReviewed by:
Steven Stagg, Anglia Ruskin University, United KingdomTeresa Mitchell, University of Massachusetts Medical School, United States
Copyright © 2017 Schuetze, Rohr, Dewey, McCrimmon and Bray. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Manuela Schuetze, manuschuetze@gmail.com