Edited by: John D. Salamone, University of Connecticut, USA
Reviewed by: Sean B. Ostlund, University of California at Los Angeles, USA; Donita L. Robinson, University of North Carolina, USA; Elio Acquas, Università degli Studi di Cagliari, Italy
*Correspondence: Regina A. Mangieri, College of Pharmacy, The University of Texas at Austin, 2409 University Ave. Stop A1915, Austin, TX 78712, USA e-mail:
This article was submitted to the journal Frontiers in Behavioral Neuroscience.
†These authors have contributed equally to this work.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
We previously reported that, in male, Long Evans rats, instrumental lever pressing that had been reinforced during limited training under a variable interval (VI) schedule by oral self-administration of a 10% sucrose/10% ethanol (10S10E) solution was insensitive to devaluation of 10S10E. In contrast, lever pressing that had been reinforced under a variable ratio (VR) schedule, or by self-administration of 10% sucrose (10S) alone, was sensitive to outcome devaluation. The relative insensitivity to outcome devaluation indicated that seeking of 10S10E by the VI-trained rats had become an instrumental habit. In the present study we employed an alternative operational definition of an instrumental habit and compared the effect of reversing the action-outcome contingency on lever press performance by rats trained under the same experimental conditions. Male Long Evans rats received daily operant training, in which lever presses were reinforced by 10S10E or 10S, under VI or VR schedules. After nine sessions of VI or VR training, rats were tested over four sessions in which the instrumental contingency was changed so that a lever press would prevent reinforcer delivery for 120 s. We found that rats that had been trained to lever press for 10S10E under the VR schedule showed a greater change in lever pressing across testing sessions than those that had received 10S10E reinforcement under the VI schedule. There was no such interaction with reinforcement schedule for rats that had received only 10S reinforcement during training. These findings are consistent with those of our previous study, and provide further evidence that addition of ethanol to sucrose may promote habitual responding in an instrumental task.
While socially acceptable and pervasive, alcohol use is infamous for sometimes becoming a “bad habit.” Unlike goal-directed actions, which are instrumental behaviors sensitive to both the value of their outcome and the causal relationship between performance and outcome acquisition (i.e., the instrumental contingency), “habits” are contextually-elicited, well-practiced behaviors performed automatically (viz., with minimal cognitive effort) in a relatively rigid and inflexible fashion. Of particular concern is whether and under what circumstances drugs of abuse promote or accelerate behavioral automaticity, a question that can be investigated experimentally using operant self-administration paradigms. To characterize the “goal-directed” vs. “habitual” nature of instrumental behaviors, a number of investigators have employed experimental manipulations that reveal the degree to which the instrumental outcome itself is the stimulus for the behavior. For example, the canonical test for “habitual” instrumental behavior involves first, manipulating the value of the instrumental outcome (most commonly, reducing), and then observing the behavior in an operant session without reinforcer feedback (viz., in extinction). In this classical operational definition, “habitual” seeking behavior is that which shows insensitivity to outcome devaluation (Dickinson,
One method for inducing outcome devaluation is to use injections of lithium chloride (LiCl) to pair malaise with ingestion of the reinforcing substance (Adams and Dickinson,
One caveat to such an interpretation is that when the paired outcome is a mixed substance consisting of components with distinct psychopharmacological and sensory properties, such as a sweetened alcohol drinking solution, there is a degree of uncertainty regarding the specificity of devaluation that clouds interpretation. Specifically, LiCl pairing could devalue the rewarding taste of sucrose, the caloric contribution of sucrose and ethanol to reward, or the rewarding effect of ethanol intoxication, and it is difficult to parse out which component was devalued by the LiCl pairing procedure. In view of this uncertainty, an independent measure of the degree of habit formation that occurs with 10S and 10S10E solutions will be helpful to confirm or refute the interpretation of the previous experiment. Thus, in order to clarify interpretation of our previous findings, we employed here an assay for the nature of instrumental behavior that does not manipulate value of the outcome, but rather the instrumental contingency (Dickinson et al.,
In the first experiment presented here, we trained male Long-Evans rats to press a lever for the opportunity to orally self-administer an ethanol-sucrose drinking solution using the same limited training protocols as in our previous work (Mangieri et al.,
All experimental procedures were approved by the Institutional Animal Care and Use Committee of the University of Texas at Austin (current Animal Use Protocol #2011-00069) and performed as per the Guidelines for the Care and Use of Animals in Neuroscience issued by the National Academies. Naïve, male Long Evans rats (
Upon arrival, group housed rats (three per cage) were allowed 1 week of habituation to their new environment. During this week only, the experimenter handled rats daily (approximately 10 min per rat). Two days before commencing behavioral training, rats were separated into individual cages, remaining individually housed and receiving no additional handling, except as required for operant chamber entry and exit, throughout the experiment. A total of four cohorts were trained for these experiments: two 10S10E (18 months apart) and two 10S (1 month apart).
As in Mangieri et al. (
The drinking solutions, 10% sucrose (w/v) and 10% sucrose (w/v): 10% ethanol (v/v), were prepared approximately every 3 days from ultra-pure sucrose (MP Biomedicals, Solon, OH), 95% ethanol (AAPER Alcohol and Chemical Co., Shelbyville, KY), and distilled water, and stored at 4°C.
Water deprivation began 22 h prior to the first session in the operant chamber, after which water was available in the home cage for 2 h. During this session only, the sipper tube was inserted into the chamber for the entire 20-min duration. Lever presses were inconsequential, but 20 mL of 10S was available for
Upon successful acquisition of the lever-press response, rats were no longer water deprived, and began instrumental training, receiving one 20-min operant session per day, 4–5 days per week. All rats completed two sessions with 10S, followed by seven with 10S10E or 10S, for a total of nine training sessions. An ethanol dose of at least 0.3 g/kg on the last three training sessions was required
Following training, the flexibility of seeking behavior was tested using four 20-min operant sessions of an omission interval (OI) schedule of reinforcement: access to the drinking solution was granted every 120 s unless the rat pressed the lever, resetting the timer on reinforcer delivery. The selection of the 120 s interval was based in part on the finding that this interval was at least eight times greater than the longest median inter-response interval observed in any of the groups tested on the last day of training. Although it was possible to earn nine instances of drinking solution access per OI session, no rat earned the maximum on any of the four sessions. Breaks (days without an operant session) during the testing phase differed between 10S10E, but not 10S cohorts. For the groups trained to drink 10S10E there were 3 separate patterns for “days off” before and during the omission test sessions: (1) 1 day off before session three and 1 day off before session four (
Med-PC IV software recorded the occurrence and time of each lever press, insertion of the sipper tube (reinforcer delivery), and lickometer circuit completion (contact with the sipper tube) in the session. At the end of every operant session, the remaining volume of drinking solution was manually measured and recorded. A plastic tray placed under the bottle assembly collected leaked fluid, which was added to the volume remaining in the bottle. Estimates of ethanol consumption (g/kg) were corrected for spillage.
Graphical representations of data (created with Microsoft Excel and Adobe Illustrator) show group means for the indicated measure; error bars indicate the standard error of the mean (s.e.m.). Data were analyzed using SPSS (version 17.0, IBM) univariate or repeated measures general linear model procedures, as appropriate. Specifically, repeated measures ANOVA with both between-groups and within-subjects factors was used to analyze training and testing behavior across time. Significant (
Data from the 10S10E groups were analyzed separately from the 10S groups. 10S cohorts were trained a month apart and 19–20 months after the first 10S10E cohort, and 1–2 months after the second. Of the 26 rats used in this experiment, two were excluded from statistical analyses: one due to freezing behavior during the first test (OI) session, the other due to equipment malfunction during the third test (OI) session.
After the initial acquisition of operant lever pressing, rats were assigned to one of two groups, “10S10E VI” (
At the end of the training period, both groups were tested over four sessions with the omission contingency in effect (omission of lever pressing for 120 s earned one reinforcer). Body weights prior to the 1st testing session did not differ (mean ± s.e.m. grams:
In order to gain further insight as to how the two groups were affected by the introduction of the omission contingency, we analyzed presses and reinforcers (in 4-min bins) within just the first test session. As shown by Figure
Lever press conditioning and training proceeded exactly as in Experiment 1, with the exception that the drinking solution contained in the sipper tube remained 10S throughout the experiment. Operant lever pressing was conditioned in all animals in this experiment within 2 days (data not shown), after which they were divided into two groups, “10S VI” (
As in Experiment 1, the two groups were tested over four sessions, during which omission of lever pressing for 120 s was necessary to receive reinforcement. Body weights prior to the 1st testing session did not differ (mean ± s.e.m. grams:
Here we provide new evidence that ethanol can influence the transition from goal-directed to habitual expression of instrumental behavior after only 7 days of exposure during once daily operant self-administration sessions. We carried out two experiments using male Long Evans rats trained to press a lever for the opportunity to orally self-administer either ethanol-sucrose or sucrose alone under VI or VR reinforcement schedules. In both experiments, rats were first conditioned to lever press for 10S, but in the first experiment the solution changed to 10S10E after two training sessions. All other aspects of training were identical between the two experiments. In Experiment 1, the 10S10E VI and 10S10E VR groups showed no gross differences in terms of the number of lever presses or reinforcers received during training sessions. When assayed over four test sessions in which omission of lever pressing was required for the rat to receive reinforcement, the rats trained under the VR schedule showed a robust decrease in lever pressing across test sessions. In contrast, lever pressing by the 10S10E VI group changed very little across the test sessions. When we repeated this experiment with rats that received only 10S during training (i.e., that were never exposed to ethanol), the two training groups, 10S VI and 10S VR, showed a similar decrement in lever pressing across testing. We previously found that this same limited 10S10E VI, but not 10S10E VR, training protocol resulted in lever pressing that was insensitive to outcome devaluation produced by pairing LiCl-induced malaise with ethanol-induced intoxication (Mangieri et al.,
It should be noted that there are two general limitations of the present study that temper the conclusions that we can make. First, we are unable to make firm conclusions comparing the 10S10E groups and the 10S groups because the experiments were performed independently. Second, the numbers of subjects in both experiments are relatively small for behavioral experiments. It is possible that statistically significant interactions between group and session may be observed in both experiments with larger sample sizes. However, on the basis of the present results, we argue that a fundamental difference exists between the outcomes of the two experiments. Specifically, we were able to detect a significant interaction between group and session in the 10S10E groups, reflected in an effect size of 0.64 and an observed power of 0.84 for Experiment 1. On the other hand, the effect size was 0.44 and the observed power was 0.41 for Experiment 2. Therefore, we argue that, given the similar experiment sample sizes (13 vs. 11), a small effect size contributes to the lower observed power in the 10S experiment, and it would take a much larger sample size to detect a statistically significant group by session interaction, if any, when groups are trained on 10S. Clearly, this is not the case for the 10S10E experiment. Nonetheless, a full resolution of this issue would require a completely independent replication in which all factors were represented and controlled for at the same time.
Mindful of these caveats, we interpret our findings within the conceptual framework of dual decision-making/behavioral control systems. Put simply, these two, interacting, “goal-directed” and “habit,” systems learn and operate in parallel (Belin et al.,
We observed such an effect of reinforcement schedule in our prior work, and identified a point in training at which instrumental responding by 10S10E-seeking, VR-, but not VI-, trained rats was still sensitive to outcome devaluation. In the current study, we employed an alternative behavioral assay, and found responding by the two 10S10E groups to differ in sensitivity to contingency reversal. Together, these findings demonstrate that, in our experiments, when tested after limited training, seeking behavior established by ethanol-sucrose VR reinforcement remains “goal-directed” while seeking behavior established by ethanol-sucrose VI reinforcement does not. Importantly, inclusion of both VI- and VR-trained 10S groups in the current study shows that indeed, the VI vs. VR difference in ethanol-exposed rats does not appear in sucrose-only rats. Given identical training in the present two experiments, albeit smaller sample sizes in the 10S groups, we did not observe evidence that adaptation to the negative contingency was different between the 10S VI and 10S VR groups. Although we cannot directly analyze the effect of drinking solution in this study because cohorts were not intermixed during the two experiments, the fact that we did not observe an effect of instrumental training schedule on the adaptation of lever pressing behavior by the 10S groups suggests that self-administration of ethanol during training contributes to the difference in performance between the 10S10E groups during testing.
One possibility is that the effects of ethanol and the VI schedule during training were simply additive, pushing the “strength” of the habit system past some threshold necessary for achieving dominance. Indeed, ethanol exposure has been reported by a number of groups to promote habitual expression of instrumental behavior, which may reflect enhanced S-R mechanisms and/or impaired goal-directed control—depending on the experimental manipulation or assay (Ostlund et al.,
On the other hand, it has been demonstrated that even acute or short-term, non-contingent exposure to ethanol can impact instrumental performance by both rats (Ostlund et al.,
Ostlund et al.'s findings may help explain why we observe a difference in test performance between 10S10E, but not 10S, groups in the present study. We propose that behavioral adaptation under the 120 s OI schedule was influenced not only by the strength of S-R mechanisms, but also by the ability of animals to exert goal-directed control. To explain, two temporal parameters of an omission reinforcement schedule influence the rate at which instrumental responding decays during omission training: the inter-reinforcement interval (given no response occurs) and the response-reinforcement (penalty) interval (Uhl and Garcia,
Given the demanding nature of requiring lever press omission for 120 s, it is plausible that adaptation to contingency reversal was influenced both by the ability of contextual stimuli to promote automatic/inflexible initiation of lever pressing (viz., the degree to which behavior in the self-administration context is under control of the habit system) and by the ability to exert inhibitory control over the same behavioral impulse in order to obtain a desired outcome (viz., the degree to which the goal-directed system can re/assert itself in this context in order to maximize reward given contingency change). Thus, performance in our behavioral test likely captured conflict between the two behavioral control systems, whereby greater ability to exert goal-directed control over habitual impulses would promote, and conversely, compromised control would impair, adaptation to the negative instrumental contingency. Therefore, assuming that ethanol self-administration by 10S10E groups in our study also produced ethanol-context associations capable of impairing goal-directed control (like those observed by Ostlund et al.)—and to the extent that we can assume goal-directed control was equivalently impaired between these groups (on the basis that they consumed similar doses of ethanol during training)—we can argue that such an impairment at test, along with a differential amount of learning
To summarize, we found that behavioral adaptation to a negative instrumental contingency after limited instrumental training (nine sessions total) was influenced by the type of training reinforcement schedule (VI vs. VR) when male Long Evans rats had self-administered ethanol plus sucrose in the operant context. A parallel experiment with rats self-administering sucrose alone did not yield differences in behavioral adaptation due to the reinforcement schedule. In interpreting the results of the present study, a general enhancement of “S-R” learning—either as a result of training under a VI schedule or ethanol exposure—alone does not seem to provide the best account for our findings. On the other hand, the biasing toward formation of “S-R” associations, by training under a VI reinforcement schedule, coupled with compromised goal-directed control, by testing in an ethanol-associated context, is an attractive explanation for our finding that the 10S10E VI group alone was impaired in adapting to the omission instrumental contingency.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Research support was provided by awards R37AA011852 (Rueben A. Gonzales), F32AA018252 (Regina A. Mangieri) from the National Institute on Alcohol Abuse and Alcoholism, HRD-0703584 (Roberto U. Cofresí) from the National Science Foundation, and the University of Texas Undergraduate Research Fellowship Award (Roberto U. Cofresí). Regina A. Mangieri and Roberto U. Cofresí contributed equally to this work.